Category Archives: Web Archives

UK Web Archive mini-conference 2020

On Wednesday 19th November I attended the UK Web Archive (UKWA) mini-conference 2020, my first conference as a Graduate Trainee Digital Archivist. It was hosted by Jason Webber, Engagement Manager at the UKWA and, as normal in these COVID times, it was hosted on Zoom (my first ever Zoom experience!)

The conference started with an introduction and demonstration of the UKWA by Jason Webber. Starting in 2005 the UKWA’s mission is to collect the entire UK webspace, at least once per year, and preserve the websites for future generations. As part of my traineeship I have used the UKWA but it was interesting to hear about the other functions and collections it provides. Along with being able to browse different versions of UK websites it also includes over 100 curated collections on themes ranging from Food to Brexit to Online Enthusiast Communities in the UK. It also features the SHINE tool, which was developed as part of the ‘Big UK Data Arts and Humanities’ project and contains over 3.5 billion items which have been full-text indexed so that every word is searchable. It allows users to perform searches and trend analysis on subjects over a huge range of websites, all you need to use this tool is a bit a Python knowledge. My Python knowledge is a bit basic but Caio Mello, during his researcher talk, provided a useful link for online python tutorials aimed at historians to aid in their research.

In his talk, Caio Mello (School of Advanced Study, University of London) discussed how he used the SHINE tool as part of his work for the CLEOPATRA Project. He was specifically looking at the Olympic legacy of the 2012 Olympics, how it was defined and how the view of the legacy changed over time. He explained the process he used to extract the information and the ways the information can be used for analysis, visualisation and context. My background is in mathematics and the concept of ‘Big Data’ came up frequently during my studies so it was fascinating to see how it can be used in a research project and how the UKWA is enabling research to be conducted over such a wide range of subjects.

The next researcher talk by Liam Markey (University of Liverpool and the British Library) showed a different approach to using the UKWA for his research project into how Remembrance in 20th Century Britain has changed. He explained how he conducted an analysis of archived newspaper articles, using specific search terms, to identify articles that focused on commemoration which he could then use to examine how the attitudes changed over time. The UKWA enabled him to find websites that focused on the war and compare these with mainstream newspapers to see how these differ.

The Keynote speaker was Paul Gooding (University of Glasgow) and was about the use and users of Non-Print Legal Deposit Libraries. His research as part of the Digital Library Futures Project, with the Bodleian Libraries and Cambridge University Library as case study partners, looked at how Academic Deposit libraries were impacted by e-Legal Deposit. It was an interesting discussion around some of the issues of the system, such as balancing the commercial rights with access for users and how highly restrictive access conditions are at odds with more recent legislation, such as the provision for disabled users and 2014 copyright exception for data and text mining for non-commercial uses.

Being new to the digital archiving world, my first conference was a great introduction to web archiving and provided context to the work I am doing. Thank you to the organisers and speakers for giving me insight into a few of the different ways the web archive is used and I have come away with a greater understanding of the scope and importance of digital archiving (as well as a list of blog posts and tutorials to delve into!)

Some Useful Links:

https://www.webarchive.org.uk/

https://programminghistorian.org/

https://blogs.bl.uk/webarchive/2020/11/how-remembrance-day-has-changed.html

http://cleopatra-project.eu/

 

#WeMissiPRES: Preserving social media and boiling 1.04 x 10^16 kettles

This year the annual iPRES digital preservation conference was understandably postponed and in its place the community hosted a 3-day Zoom conference called #WeMissiPRES. As two of the Bodleian Libraries’ Graduate Trainee Digital Archivists, Simon and I were in attendance and blogged about our experiences. This post contains some of my highlights.

The conference kicked off with a keynote by Geert Lovink. Geert is the founding director of the Institute of Network Cultures and the author of several books on critical Internet studies. His talk was wide-ranging and covered topics from the rise of so-called ‘Zoom fatigue’ (I guarantee you know this feeling by now) to how social media platforms affect all aspects of contemporary life, often in negative ways. Geert highlighted the importance of preserving social media in order to allow future generations to be able to understand the present historical moment. However, this is a complicated area of digital preservation because archiving social media presents a host of ethical and technical challenges. For instance, how do we accurately capture the experience of using social media when the content displayed to you is largely dictated by an algorithm that is not made public for us to replicate?

After the keynote I attended a series of talks about the ARCHIVER project. João Fernandes from CERN explained that the goal of this project is to improve archiving and digital preservation services for scientific and research data. Preservation solutions for this type of data need to be cost-effective, scalable, and capable of ingesting amounts of data within the petabyte range. There were several further talks from companies who are submitting to the design phase of this project, including Matthew Addis from Arkivum. Matthew’s talk focused on the ways that digital preservation can be conducted on the industrial scale required to meet the brief and explained that Arkivum is collaborating with Google to achieve this, because Google’s cloud infrastructure can be leveraged for petabyte-scale storage. He also noted that while the marriage of preserved content with robust metadata is important in any digital preservation context, it is essential for repositories dealing with very complex scientific data.

In the afternoon I attended a range of talks that addressed new standards and technologies in digital preservation. Linas Cepinskas (Data Archiving and Networked Services (DANS)) spoke about a self-assessment tool for the FAIR principles, which is designed to assess whether data is Findable, Accessible, Interoperable and Reusable. Later, Barbara Sierman (DigitalPreservation.nl) and Ingrid Dillo (DANS) spoke about TRUST, a new set of guiding principles that are designed to map well with FAIR and assess the reliability of data repositories. Antonio Guillermo Martinez (LIBNOVA) gave a talk about his research into Artificial Intelligence and machine learning applied to digital preservation. Through case studies, he identified that AI is especially good at tasks such as anomaly detection and automatic metadata generation. However, he found that regardless of how well the AI performs, it needs to generate better explanations for its decisions, because it’s hard for human beings to build trust in automated decisions that we find opaque.

Paul Stokes from Jisc3C gave a talk on calculating the carbon costs of digital curation and unfortunately concluded that not much research has been done in this area. The need to improve the environmental sustainability of all human activity could not be more pressing and digital preservation is no exception, as approximately 3% of the world’s electricity is used by data centres. Paul also offered the statistic that enough power is consumed by data centres worldwide to boil 10,400,000,000,000,000 kettles – which is the most important digital preservation metric I can think of.

This conference was challenging and eye-opening because it gave me an insight into (complicated!) areas of digital preservation that I was not familiar with, particularly surrounding the challenges of preserving large quantities of scientific and research data. I’m very grateful to the speakers for sharing their research and to the organisers, who did a fantastic job of bringing the community together to bridge the gap between 2019 and 2021!

#WeMissiPRES: A Bridge from 2019 to 2021

Every year, the international digital preservation community meets for the iPRES conference, an opportunity for practitioners to exchange knowledge and showcase the latest developments in the field. With the 2020 conference unable to take place due to the global pandemic, digital preservation professionals instead gathered online for #WeMissiPRES to ensure that the global community remained connected. Our graduate trainee digital archivist Simon Mackley attended the first day of the event; in this blog post he reflects on some of the highlights of the talks and what they tell us about the state of the field.

How do you keep the global digital preservation community connected when international conferences are not possible? This was the challenge faced by the organisers of #WeMissIPres, a three-day online event hosted by the Digital Preservation Coalition. Conceived as a festival of digital preservation, the aim was not to try and replicate the regular iPRES conference in an online format, but instead to serve as a bridge for the digital preservation community, connecting the efforts of 2019 with the plans for 2021.

As might be expected, the impact of the pandemic loomed large in many of the talks. Caylin Smith (Cambridge University Library) and Sara Day Thomson (University of Edinburgh) for instance gave a fascinating paper on the challenge of rapidly collecting institutional responses to coronavirus, focusing on the development of new workflows and streamlined processes. The difficulties of working from home, the requirements of remote access to resources, and the need to move training online likewise proved to be recurrent themes throughout the day. As someone whose own experience of digital preservation has been heavily shaped by the pandemic (I began my traineeship at the start of lockdown!) it was really useful to hear how colleagues in other institutions have risen to these challenges.

I was also struck by the different ways in which responses to the crisis have strengthened digital preservation efforts. Lynn Bruce and Eve Wright (National Records of Scotland) noted for instance that the experience of the pandemic has led to increased appreciation of the value of web-archiving from stakeholders, as the need to capture rapidly-changing content has become more apparent. Similarly, Natalie Harrower (Digital Repository of Ireland) made the excellent point that the crisis had not only highlighted the urgent need for the sharing of medical research data, but also the need to preserve it: Coronavirus data may one day prove essential to fighting a future pandemic, and so there is therefore a moral imperative for us to ensure that it is preserved.

As our keynote speaker Geert Lovink (Institute of Network Cultures) reminded us, the events of the past year have been momentous quite apart from the pandemic, with issues such as the distorting impacts of social media on society, the climate emergency, and global demands for racial justice all having risen to the forefront of society. It was great therefore to see the role of digital preservation in these challenges being addressed in many of the panel sessions. A personal highlight for me was the presentation by Daniel Steinmeier (KB National Library of the Netherlands) on diversity and digital preservation. Steinmeier stressed that in order for diversity efforts to be successful, institutions needed to commit to continuing programmes of inclusion rather than one-off actions, with the communities concerned actively included in the archiving process.

So what challenges can we expect from the year ahead? Perhaps more than ever, this year this has been a difficult question to answer. Nonetheless, a key theme that struck me from many of the discussions was that the growing challenge of archiving social media platforms was matched only by the increasing need to preserve the content hosted on them. As Zefi Kavvadia (International Institute of Social History) noted, many social media platforms actively resist archiving; even when preservation is possible, curators are faced with a dilemma between capturing user experiences and capturing platform data. Navigating this challenge will surely be a major priority for the profession going forward.

While perhaps no substitute for meeting in person, #WeMissiPRES nonetheless succeeded in bringing the international digital preservation community together in a shared celebration of the progress being made in the field, successfully bridging the gap between 2019 and 2021, and laying the foundations for next year’s conference.

 

#WeMissiPRES was held online from 22nd-24th September 2020. For more information, and for recordings of the talks and panel sessions, see the event page on the DPC website.

Archiving web content related to the University of Oxford and the coronavirus pandemic

Since March 2020, the scope of collection development at the Bodleian Libraries’ Web Archive has expanded to also focus on the coronavirus pandemic: how the University of Oxford, and wider university community have reacted and responded to the rapidly changing global situation and government guidance. The Bodleian Libraries’ Web Archive team have endeavoured (and will keep working) to capture, quality assess and make publicly available records from the web relating to Oxford and the coronavirus pandemic. Preserving these ephemeral records is important. Just a few months into what is sure to be a long road, what do these records show?

Firstly, records from the Bodleian Libraries’ Web Archive can demonstrate how university divisions and departments are continually adjusting in order to facilitate core activities of learning and research. This could be by moving planned events online or organising and hosting new events relevant to the current climate:

Capture of http://pcmlp.socleg.ox.ac.uk/ 24 May 2020 available through the Bodleian Libraries’ Web Archive. Wayback URL https://wayback.archive-it.org/2502/20200524133907/https://pcmlp.socleg.ox.ac.uk/global-media-policy-seminar-series-victor-pickard-on-media-policy-in-a-time-of-crisis/

Captures of websites also provide an insight to the numerous collaborations of Oxford University with both the UK government and other institutions at this unprecedented time; that is, the role Oxford is playing and how that role is changing and adapting. Much of this can be seen in the ever evolving news pages of departmental websites, especially those within Medical Sciences division, such as the Nuffield Department of Population Health’s collaboration with UK Biobank for the government department of health and social care announced on 17 May 2020.

The web archive preserves records of how certain groups are contributing to coronavirus covid-19 research, front line work and reviewing things at an extremely  fast pace which the curators at Bodleian Libraries’ Web Archive can attempt to capture by crawling more frequently. One example of this is the Centre for Evidence Based Medicine’s Oxford Covid-19 Evidence Service – a platform for rapid data analysis and reviews which is currently updated with several articles daily. Comparing two screenshots of different captures of the site, seven weeks apart, show us the different themes of data being reviewed, and particularly how the ‘Most Viewed’ questions change (or indeed, don’t change) over time.

Capture of https://www.cebm.net/covid-19/ 14 April 2020 available through the Bodleian Libraries’ Web Archive. Wayback URL https://wayback.archive-it.org/org-467/20200414111731/https://www.cebm.net/covid-19/

Interestingly, the page location has slightly changed, the eagle-eyed among you may have spotted that the article reviews are now under /oxford-covid-19-evidence-service/, which is still in the web crawler’s scope.

Capture of https://www.cebm.net/covid-19/ 05 June 2020 available through the Bodleian Libraries’ Web Archive. Wayback url https://wayback.archive-it.org/org-467/20200605100737/https://www.cebm.net/oxford-covid-19-evidence-service/

We welcome recommendations for sites to archive; if you would like to nominate a website for inclusion in the Bodleian Libraries’ Web Archive you can do so here. Meanwhile, the work to capture institutional, departmental and individual responses at this time continues.

Old Ideas, New Technologies: Historical and Vintage Festivals in the UK Web Archive

Festivals are wonderful events that can often involve thousands of people, united by their shared love for a common activity or theme. The UK Web Archive seeks to capture, and record these often colourful and creative demonstrations of human culture and creativity.

Some Festivals are very large and documented, such as Glastonbury which often attracts over a 100,000 people. However, there are also a number of smaller and more specific festivals which are less well known outside of their local communities and networks, such as the Shelswell History Festival. However, the internet has helped level the playing field, and given these smaller festivals an opportunity to publicise their events far beyond the reaches of their traditional borders and boundaries. And this has allowed archivists such as myself to find and add these festivals to the UK Web Archive.

(The Festivals Icon on the UK Web Archive Website)

Historical and Vintage Festivals

One of the most personally intriguing parts of the UK Web Archive festivals collection for me is Historical and Vintage festivals. These festivals rarely attract the level of media attention that a high profile music festival featuring the world’s biggest pop stars would enjoy. However, the UK Web Archive, is about diversity, inclusivity, and finding value in all parts of society. People who attend, organise, and take part in historical and vintage festivals form part of a collective effort which often results in a website that helps chronicle their enthusiasm.

Thus far we have found forty eight different historical and vintage festivals that take place in the United Kingdom. These festivals are broad and varied, and celebrate a multitude of things. This includes Newport Rising which celebrates the 1839 Chartist rebellion, the Lupton House Festival of History which celebrates a historic house, and Frock Me! Which is a vintage fashion fair. Every single one of these festivals is unique and specific in their own way, but they do have something in common. They all celebrate history and the past, and are characterised by a charming sense of nostalgia and remembrance.

While the website is no substitute for attending in person, they often include:

• Basic information about the festival’s time, place, and theme.
• An array of photographs.
• Anecdotes about the events.
• Information about the festivals donors and supporters.
• And additional information, such as attendance policies and rules etc.

A notable feature of these websites is how they use relatively new technologies to organise events which celebrate old events, places, and themes. This indicates a fantastic synergy between the heritage sector, and modern technology.

Curating the UK Web Archive’s Mental Health Collection

The UK Web Archive’s Mental Health, Social Media and the Internet collection  seeks to document the changing conversation surrounding mental health, social media and the internet by capturing UK-based websites for posterity.

Developing the Mental Health Collection

As well as including webpages which highlight the negative impact of social media on mental health, the mental health collection also serves to document online initiatives including web pages and social media platforms which are changing the conversation and helping to tackle the stigma surrounding mental health. Examples of sites within the mental health collection include:

  • HeadsTogether, a mental health initiative setup and run by the Duke and Duchess of Cambridge and Prince Harry incorporating a campaign to change the dialogue surrounding mental health and raise funds to provide new mental health services.
  • The Mix provides an important source of mental health information and advice for people under twenty-five, tackling topics such as anxiety and depression, self-care and counselling.
  • The Mental Health First Aid England Instagram account (mhfaengland) serves to raise mental health literacy through a series of eye-catching posts including: photographs, drawings and info graphics which promote self-care and good mental health.
  • The Mental Health Foundation is a UK charity which aims to help people to understand, protect and sustain good mental health as seen through their online Twitter page

Help us grow our collections

The mental health collection can become a richer resource with your help. You could see your site suggestions preserved in the UK Web Archive by nominating them here.

Developing collections on Gender Equality at the UK Web Archive

The Gender Equality collection

The UK web archive Gender Equality collection and its themed subsections provide a rich insight into attitudes and approaches towards gender equality in contemporary UK society and culture. This was previously discussed in my last blog post about the collection, which you can read here.

Curating the collection

A great deal of the discussion and activity relating to gender equality occurs predominantly in an online space. This means that as a curator for the Gender Equality collection, the harvest is plenty! The type of content being collected by the UK Web Archive includes:

Of course there is some crossover, not only regarding the type of content but also within subsections of the gender equality collection.

This image is made available and reproduced by CC-BY-NC-SA 2.0. [https://creativecommons.org/licenses/by-nc-sa/2.0/legalcode]

Specifically, I find the event sites in the collection really interesting. As well as documenting that the event(s) even existed and happened in the first place, they can give us a snapshot of who organised the event, as well as who the intended audience were. Also, the collection exhibits the evolution of websites related to gender equality over time (which can be very speedy indeed when it comes to sites like twitter accounts!), and the changing priorities, trends, initiatives and more that can tell us about attitudes towards gender equality in the UK. These kinds of websites are being created by and engaged with by humans right now.

Nominate a website!

The endeavour of the UK Web Archive never stops – if you would like to help grow the Gender Equality collection (or indeed, any other collections) click here to nominate a website to save. Go on…whilst you’re at it, you can explore the UK Web Archive’s funky new interface!

 

Image reference: Workers Solidarity Movement (2012) March for Choice

 

Festivals in the UK Web Archive

Live events are funny things; can their spirit be captured or do you have to “be there to get it”? Personally I don’t think you can, so why are we archiving festival websites?

Running throughout the year, though most tend to be clustered around the short UK summer, festivals form a huge part of the UK’s contemporary cultural scene.  While it’s often the big music festivals that come to mind such as Glastonbury and Reading or perhaps the more local CAMRA sponsored beer and cider festivals; these days there is a festival for pretty much everything under the sun.

UK Web Archive topics and themes

In part this explosion of festivals from the very local and niche to the mainstream and brand sponsored has been helped by the internet. You can now find festivals dedicated to anything from bird watching to meat grilling to vintage motors.

With the number of tools and platforms available for website creation and event and bookings management and the rise of social media, it seems anyone with an idea can put on a festival. More importantly with increasing connectedness that the web gives us, the reach of these home grown festivals has become potentially global.

Of course most will remain small local events that go on until the organisers lose interest or money such as Blissfields in Winchester which had to cancel their 2018 event due to poor ticket sales. But some will make it big like Neverworld which started in 2006 in Lee Denny’s back garden while his parents were away for the week but now 10+ years on has sold out the 5000 capacity festival venue it has relocated to.

The UK Web Archive‘s Festivals collection attempts to capture the huge variety of UK festivals taking place each year and currently has around 1200 events being archived that are loosely categorised based around 15 common themes, though of course there is a great deal of crossover as they can be found combining themes such as:

In this collection of UK festivals sites, while we cannot capture the spirit of a live event we can still try to capture their transient nature. Here you can see their rise and fall, the photographs and comments left in their wake, and their impact on local communities over time. Hopefully these sites and their contents can still give future researchers a sometimes surprising and often candid snapshot of contemporary British culture.

Emily Chen

Oxford LibGuides: Web Archives

Web archives are becoming more and more prevalent and are being increasingly used for research purposes. They are fundamental to the preservation of our cultural heritage in the interconnected digital age. With the continuing collection development on the Bodleian Libraries Web Archive and the recent launch of the new UK Web Archive site, the web archiving team at the Bodleian have produced a new guide to web archives. The new Web Archives LibGuide includes useful information for anyone wanting to learn more about web archives.

It focuses on the following areas:

  • The Internet Archive, The UK Web Archive and the Bodleian Libraries Web Archive.
  • Other web archives.
  • Web archive use cases.
  • Web archive citation information.

Check out the new look for the Web Archives LibGuide.

 

 

Introducing the new UK Web Archive website

Until recently, if you wanted to search the vast UK Legal Deposit Web Archive (containing the whole UK Web space), then you would need to travel to the reading room of a UK Legal Deposit Library to see if what you needed was there. For the first time, the new UK Web Archive website offers:

  • The ability to search the Legal Deposit web archive from anywhere.
  • The ability to search the Legal Deposit web archive alongside the ‘Open’ UK Web Archive (15,000 or so publicly available websites collected since 2005).
  • The opportunity to browse over 100 curated collections on a wide range of topics.

Who is the UK Web Archive?
UKWA is a partnership of all the UK Legal Deposit Libraries – The British Library, National Library of Scotland, National Library of Wales, the Bodleian Libraries, Cambridge University Libraries, and Trinity College, Dublin. The Legal Deposit Web Archive is available in the reading rooms of all the Libraries.

How much is available now?
At the time of writing, everything that a human (curators and collaborators) has selected since 2005 is searchable. This constitutes many thousands of websites and millions of individual web pages. The huge yearly Legal Deposit domain crawls will be added over the coming year.

This includes over 100 curated collections of websites on a wide range of topics and themes. Recent collections curated by the Bodleian Libraries include:

Do the websites look and work as they did originally?
Yes and no. Every effort is made so that websites look how they did originally and internal links should work. However, for a variety of technical  issues many websites will look different or some elements may be missing. As a minimum, all of the text in the collection is searchable and most images should be there. Whilst we collect a considerable amount of video, much of this will not play back.

Is every UK website available?
We aim to collect every website made or owned by a UK resident, however, in reality it is extremely difficult to be comprehensive! Our annual Legal Deposit collections include every .uk (and .london, .scot, .wales and .cymru) plus any website on a server located in the UK. Of course, many websites are .com, .info etc. and on servers in other countries.

If you have or know of a UK website that should be in the archive we encourage you to nominate them via the website.

Another version of this post was first published on the UK Web Archive blog.