Category Archives: Digital archives

Update from Harriet Costelloe, former Graduate Trainee Digital Archivist 2014-2016

Digital archives, ,
Harriet Costelloe setting up a display for an Open Day at the University of Surrey, 2019
Harriet setting up a display for an Open Day at the University of Surrey, 2019

After completing my traineeship I took up a post as Digital Development Officer at The National Archives. I developed information resources, carried out research on issues affecting the archives sector, and ran workshops on understanding archives catalogues. In this role I also had project management training and contributed to the assessments of archives for The National Archives’ Accreditation programme. I then moved to be College Archivist at Royal Holloway, University of London. As the sole archivist, I managed the acquisition and preservation of, and access to, the university’s collections through developing policies and procedures for the service, cataloguing material, invigilating in the research room, accommodating group visits and giving public talks, and teaching within academic departments. I also oversaw a move into a new store and co-curated the first large-scale archives exhibition at the university.

I am now the Archivist (Public Services) at the University of Surrey with responsibility for managing the research room, running our outreach programme and delivering archives sessions to students. I also contribute to exhibitions work undertaken by the team and am undertaking a Graduate Certificate in Learning and Teaching to reflect my commitment to and interest in using archives in higher education teaching. I still look back very fondly at my traineeship and am grateful for the skills I learnt and the opportunities it has afforded.

Harriet Costelloe, Sep 2019

Old Ideas, New Technologies: Historical and Vintage Festivals in the UK Web Archive

21st century, Digital archives, Modern, Uncategorized, Web Archives

Festivals are wonderful events that can often involve thousands of people, united by their shared love for a common activity or theme. The UK Web Archive seeks to capture, and record these often colourful and creative demonstrations of human culture and creativity.

Some Festivals are very large and documented, such as Glastonbury which often attracts over a 100,000 people. However, there are also a number of smaller and more specific festivals which are less well known outside of their local communities and networks, such as the Shelswell History Festival. However, the internet has helped level the playing field, and given these smaller festivals an opportunity to publicise their events far beyond the reaches of their traditional borders and boundaries. And this has allowed archivists such as myself to find and add these festivals to the UK Web Archive.

(The Festivals Icon on the UK Web Archive Website)

Historical and Vintage Festivals

One of the most personally intriguing parts of the UK Web Archive festivals collection for me is Historical and Vintage festivals. These festivals rarely attract the level of media attention that a high profile music festival featuring the world’s biggest pop stars would enjoy. However, the UK Web Archive, is about diversity, inclusivity, and finding value in all parts of society. People who attend, organise, and take part in historical and vintage festivals form part of a collective effort which often results in a website that helps chronicle their enthusiasm.

Thus far we have found forty eight different historical and vintage festivals that take place in the United Kingdom. These festivals are broad and varied, and celebrate a multitude of things. This includes Newport Rising which celebrates the 1839 Chartist rebellion, the Lupton House Festival of History which celebrates a historic house, and Frock Me! Which is a vintage fashion fair. Every single one of these festivals is unique and specific in their own way, but they do have something in common. They all celebrate history and the past, and are characterised by a charming sense of nostalgia and remembrance.

While the website is no substitute for attending in person, they often include:

• Basic information about the festival’s time, place, and theme.
• An array of photographs.
• Anecdotes about the events.
• Information about the festivals donors and supporters.
• And additional information, such as attendance policies and rules etc.

A notable feature of these websites is how they use relatively new technologies to organise events which celebrate old events, places, and themes. This indicates a fantastic synergy between the heritage sector, and modern technology.

What’s it like to be a trainee? Hannah Jordan, Graduate Trainee Digital Archivist, 2019-2021

Digital archives, ,

I moved to Oxford and started working as a Graduate Trainee Digital Archivist in April 2019. I graduated in 2016 with a BA in History and English Literature and during my undergraduate studies I volunteered in a couple of different roles within the heritage sector to try and decide what career path would be the best fit for me. I decided that I was interested in working as an Archivist because it would give me the best of both worlds: working ‘behind-the-scenes’ engaging directly with historic documents, whilst still doing some public engagement and outreach work.

I came to this job with very little experience of archival work, but this hasn’t been a barrier at all because the traineeship is designed to introduce you to the basics and build on your understanding in a structured way. So far I’ve learned a range of transferable skills, including modifying XML files, arranging and cataloguing papers and ephemera, and capturing media-based archives, such as cassettes, CDs and DVDs. Once per week I also work on curating the Bodleian Libraries’ Web Archive. My work on the Web Archive has been particularly interesting, as it has been a gateway to learning about the challenges we face in trying to develop technologies and methodologies for curating and preserving the colossal amount of born-digital information that we generate every day.

Studying part-time with Aberystwyth in addition to working full-time as a trainee can sometimes be difficult, but the course readings are interesting and useful for contextualising the work that I do in my day-to-day job. Oxford has so many beautiful libraries to explore that even finding somewhere new to settle down and do my readings feels like a bit of an adventure!

Working as a trainee in Special Collections at the Bodleian Libraries is extremely rewarding. It gives me the opportunity to handle some fascinating and unique collections and to work with supportive colleagues who really love their work. I can’t wait to see what the next year has in store.

Hannah Jordan, May 2019

Online Enthusiast Communities in the UK Web Archive

21st century, Digital archives, Division, , ,

There is a saying that ‘variety is the spice of life’ and this is certainly true when you think of the types of hobbies and interests the UK public engages in. There are the hobbies we have all probably heard of such as train spotting or metal detecting and there are the more obscure ones such as Poohsticks or Hand Dryer appreciation.  Websites are a useful tool for enthusiasts to communicate and share their passion with the world. At the UK Web Archive (UKWA) the Online Enthusiast Communities  collection aims to:

‘Capture how UK based public forums are used to discuss hobbies and activities and serve as a place for enthusiasts to converse with others sharing similar interests.’

This collection includes such a diverse and wonderful selection of websites and forums. I can honestly say that curating this collection has truly been a joy – there are probably very few jobs that allow you to look at The Letter Box Study Group (a website about the history and development of British roadside letter boxes) as part of your tasks for the day.

Differences I have noticed

As a curator you get to explore lots of sites and you begin to notice differences and similarities between websites. It is interesting to see the variety in website design and levels of expertise and to me it feels like this is reflected in the websites that are archived.

I have noticed lots of online communities using a variety of website builders. The huge diversity in tools appear to have made it easier to create more professional looking sites with ease. Compared to older sites, you notice:

  • the increased use of images
  • cleaner feel
  • neutral backgrounds
  • minimal text
  • occasional e-commerce sections

However, it is nostalgic to see some of the older more ‘blocky’ sites, as I do remember the days of dial-up internet access and early web sites. To me, forums tend to have a similar feel and the designs does not deviate greatly from each other.

I have also found how often a website updates intriguing. Some are regularly updated whereas others appear to have been untouched for several years. This may reflect that many websites are run by volunteers balancing other commitments. Regularity of updates is an important factor as it will contribute to deciding how often we capture the site – it is the skill of a web archivist to judge this accordingly however these frequencies can be updated.

Some of my Favourite sites

One of the joys of curating this collection is that you get to experience sites that are really unique that you would not normally explore. I wanted to highlight a few of the sites that particularly caught my attention, specifically from the ‘Miscellaneous’ sub section as this is my personal favourite.

Pylon of the Month

Pylon of the month (February 2018) from Sweden. Image Credit: Kristin Allardh, 2018

This is a site dedicated to electricity pylons highlighting a monthly winner. These could include current pylons or historic images and entries can come from the UK and beyond. Images are usually accompanied by some interesting history or facts.

Modernist Britain

Odeon cinema Leicester, Leicestershire. Image Credit: Richard Coltman, 2010

This site is beautifully designed and celebrates modernist architecture in Britain. There are fifty illustrated images with accompanying information about the history of the buildings and photographs taken by Richard Coltman.

Cloud appreciation society

A Lenticular cloud. Image Credit: © José Ramón Sáez, 2019

This site was launched in 2005 with the aim of ‘bringing together people who love the sky’. It has an international membership with members submitting images from all over the world. They also run events, cloud related news and in 2019 they are contributing to the non-profit FogQuest project.

The online enthusiast community is also very witty, there are some fantastically named sites and forums such as:

  • Planet of the Vapes – a forum about vaping
  • DIYnot Forum – a forum about DIY
  • Frit-Happens! – an online community for glass blowing and glass crafting

Curating the online enthusiast collection has been incredibly enjoyable. Having to actively seek new sites has made me more aware of the variety of hobbies and diversity of interests the public engage in.

As this collection develops, more sites relating to the variety of hobbies and interests will be captured and persevered for future generations explore, enjoy and research. However, due to the size, complexity and technological challenges of archiving all UK websites, some may get missed or we just do not know about them . If there is a site that you think should be included then you can nominate it on the ‘Save a UK website‘ page of the UKWA.

Developing collections on Gender Equality at the UK Web Archive

21st century, Digital archives, Web Archives, , , ,

The Gender Equality collection

The UK web archive Gender Equality collection and its themed subsections provide a rich insight into attitudes and approaches towards gender equality in contemporary UK society and culture. This was previously discussed in my last blog post about the collection, which you can read here.

Curating the collection

A great deal of the discussion and activity relating to gender equality occurs predominantly in an online space. This means that as a curator for the Gender Equality collection, the harvest is plenty! The type of content being collected by the UK Web Archive includes:

Of course there is some crossover, not only regarding the type of content but also within subsections of the gender equality collection.

This image is made available and reproduced by CC-BY-NC-SA 2.0. [https://creativecommons.org/licenses/by-nc-sa/2.0/legalcode]
Specifically, I find the event sites in the collection really interesting. As well as documenting that the event(s) even existed and happened in the first place, they can give us a snapshot of who organised the event, as well as who the intended audience were. Also, the collection exhibits the evolution of websites related to gender equality over time (which can be very speedy indeed when it comes to sites like twitter accounts!), and the changing priorities, trends, initiatives and more that can tell us about attitudes towards gender equality in the UK. These kinds of websites are being created by and engaged with by humans right now.

Nominate a website!

The endeavour of the UK Web Archive never stops – if you would like to help grow the Gender Equality collection (or indeed, any other collections) click here to nominate a website to save. Go on…whilst you’re at it, you can explore the UK Web Archive’s funky new interface!

 

Image reference: Workers Solidarity Movement (2012) March for Choice

 

Festivals in the UK Web Archive

21st century, Digital archives, Humanities, Modern, Social Sciences, Uncategorized, Web Archives

Live events are funny things; can their spirit be captured or do you have to “be there to get it”? Personally I don’t think you can, so why are we archiving festival websites?

Running throughout the year, though most tend to be clustered around the short UK summer, festivals form a huge part of the UK’s contemporary cultural scene.  While it’s often the big music festivals that come to mind such as Glastonbury and Reading or perhaps the more local CAMRA sponsored beer and cider festivals; these days there is a festival for pretty much everything under the sun.

UK Web Archive topics and themes

In part this explosion of festivals from the very local and niche to the mainstream and brand sponsored has been helped by the internet. You can now find festivals dedicated to anything from bird watching to meat grilling to vintage motors.

With the number of tools and platforms available for website creation and event and bookings management and the rise of social media, it seems anyone with an idea can put on a festival. More importantly with increasing connectedness that the web gives us, the reach of these home grown festivals has become potentially global.

Of course most will remain small local events that go on until the organisers lose interest or money such as Blissfields in Winchester which had to cancel their 2018 event due to poor ticket sales. But some will make it big like Neverworld which started in 2006 in Lee Denny’s back garden while his parents were away for the week but now 10+ years on has sold out the 5000 capacity festival venue it has relocated to.

The UK Web Archive‘s Festivals collection attempts to capture the huge variety of UK festivals taking place each year and currently has around 1200 events being archived that are loosely categorised based around 15 common themes, though of course there is a great deal of crossover as they can be found combining themes such as:

In this collection of UK festivals sites, while we cannot capture the spirit of a live event we can still try to capture their transient nature. Here you can see their rise and fall, the photographs and comments left in their wake, and their impact on local communities over time. Hopefully these sites and their contents can still give future researchers a sometimes surprising and often candid snapshot of contemporary British culture.

Emily Chen

Oxford LibGuides: Web Archives

21st century, Digital archives, Web Archives, , , ,

Web archives are becoming more and more prevalent and are being increasingly used for research purposes. They are fundamental to the preservation of our cultural heritage in the interconnected digital age. With the continuing collection development on the Bodleian Libraries Web Archive and the recent launch of the new UK Web Archive site, the web archiving team at the Bodleian have produced a new guide to web archives. The new Web Archives LibGuide includes useful information for anyone wanting to learn more about web archives.

It focuses on the following areas:

  • The Internet Archive, The UK Web Archive and the Bodleian Libraries Web Archive.
  • Other web archives.
  • Web archive use cases.
  • Web archive citation information.

Check out the new look for the Web Archives LibGuide.

 

 

Introducing the new UK Web Archive website

21st century, Digital archives, Web Archives

Until recently, if you wanted to search the vast UK Legal Deposit Web Archive (containing the whole UK Web space), then you would need to travel to the reading room of a UK Legal Deposit Library to see if what you needed was there. For the first time, the new UK Web Archive website offers:

  • The ability to search the Legal Deposit web archive from anywhere.
  • The ability to search the Legal Deposit web archive alongside the ‘Open’ UK Web Archive (15,000 or so publicly available websites collected since 2005).
  • The opportunity to browse over 100 curated collections on a wide range of topics.

Who is the UK Web Archive?
UKWA is a partnership of all the UK Legal Deposit Libraries – The British Library, National Library of Scotland, National Library of Wales, the Bodleian Libraries, Cambridge University Libraries, and Trinity College, Dublin. The Legal Deposit Web Archive is available in the reading rooms of all the Libraries.

How much is available now?
At the time of writing, everything that a human (curators and collaborators) has selected since 2005 is searchable. This constitutes many thousands of websites and millions of individual web pages. The huge yearly Legal Deposit domain crawls will be added over the coming year.

This includes over 100 curated collections of websites on a wide range of topics and themes. Recent collections curated by the Bodleian Libraries include:

Do the websites look and work as they did originally?
Yes and no. Every effort is made so that websites look how they did originally and internal links should work. However, for a variety of technical  issues many websites will look different or some elements may be missing. As a minimum, all of the text in the collection is searchable and most images should be there. Whilst we collect a considerable amount of video, much of this will not play back.

Is every UK website available?
We aim to collect every website made or owned by a UK resident, however, in reality it is extremely difficult to be comprehensive! Our annual Legal Deposit collections include every .uk (and .london, .scot, .wales and .cymru) plus any website on a server located in the UK. Of course, many websites are .com, .info etc. and on servers in other countries.

If you have or know of a UK website that should be in the archive we encourage you to nominate them via the website.

Another version of this post was first published on the UK Web Archive blog.

Archives Unleashed – Vancouver Datathon

Digital archives, Event, Learning and teaching, Modern, Uncategorized, Web Archives, , , , ,

On the 1st-2nd of November 2018 I was lucky enough to attend the  Archives Unleashed Datathon Vancouver co-hosted by the Archives Unleashed Team and Simon Fraser University Library along with KEY (SFU Big Data Initiative). I was very thankful and appreciative of the generous travel grant from the Andrew W. Mellon Foundation that made this possible.

The SFU campus at the Habour Centre was an amazing venue for the Datathon and it was nice to be able to take in some views of the surrounding mountains.

https://twitter.com/ianmilligan1/status/1058403942554464256

About the Archives Unleashed Project

The Archives Unleashed Project is a three year project with a focus on making historical internet content easily accessible to scholars and researchers whose interests lay in exploring and researching both the recent past and contemporary history.

After a series of datathons held at a number of International institutions such as the British Library, University of Toronto, Library of Congress and the Internet Archive, the Archives Unleashed Team identified some key areas of development that would enable and help to deliver their aim of making petabytes of valuable web content accessible.

Key Areas of Development
  • Better analytics tools
  • Community infrastructure
  • Accessible web archival interfaces

By engaging and building a community, alongside developing web archive search and data analysis tools the project is successfully enabling a wide range of people including scholars, programmers, archivists and librarians to “access, share and investigate recent history since the early days of the World Wide Web.”

The project has a three-pronged approach
  1. Build a software toolkit (Archives Unleashed Toolkit)
  2. Deploy the toolkit in a cloud-based environment (Archives Unleashed Cloud)
  3. Build a cohesive user community that is sustainable and inclusive by bringing together the project team members with archivists, librarians and researchers (Datathons)
Archives Unleashed Toolkit

The Archives Unleashed Toolkit (AUT) is an open-source platform for analysing web archives with Apache Spark. I was really impressed by AUT due to its scalability, relative ease of use and the huge amount of analytical options it provides. It can work on a laptop (Mac OS, Linux or Windows), a powerful cluster or on a single-node server and if you wanted to, you could even use a Raspberry Pi to run AUT. The Toolkit allows for a number of search functions across the entirety of a web archive collection. You can filter collections by domain, URL pattern, date, languages and more. Create lists of URLs to return the top ten in a collection. Extract plain text files from HTML files in the ARC or WARC file and clean the data by removing ‘boilerplate’ content such as advertisements. Its also possible to use the Stanford Named Entity Recognizer (NER) to extract names of entities, locations, organisations and persons. I’m looking forward to seeing the possibilities of how this functionality is adapted to localised instances and controlled vocabularies – would it be possible to run a similar programme for automated tagging of web archive collections in the future? Maybe ingest a collection into ATK , run a NER and automatically tag up the data providing richer metadata for web archives and subsequent research.

Archives Unleashed Cloud

The Archives Unleashed Cloud (AUK) is a GUI based front end for working with AUT, it essentially provides an accessible interface for generating research derivatives from Web archive files (WARCS). With a few clicks users can ingest and sync Archive-it collections, analyse the collections, create network graphs and visualise connections and nodes. It is currently free to use and runs on AUK central servers.

My experience at the Vancouver Datathon

The datathons bring together a small group of 15-20 people of varied professional backgrounds and experience to work and experiment with the Archives Unleashed Toolkit and the Archives Unleashed Cloud. I really like that the team have chosen to minimise the numbers that attend because it created a close knit working group that was full of collaboration, knowledge and idea exchange. It was a relaxed, fun and friendly environment to work in.

Day One

After a quick coffee and light breakfast, the Datathon opened with introductory talks from project team members Ian Milligan (Principal Investigator), Nick Ruest (Co-Principal Investigator) and Samantha Fritz (Project Manager), relating to the project – its goals and outcomes, the toolkit, available datasets and event logistics.

Another quick coffee break and it was back to work – participants were asked to think about the datasets that interested them, techniques they might want to use and questions or themes they would like to explore and write these on sticky notes.

Once placed on the white board, teams naturally formed around datasets, themes and questions. The team I was in consisted of  Kathleen Reed and Ben O’Brien  and formed around a common interest in exploring the First Nations and Indigenous communities dataset.

Virtual Machines were kindly provided by Compute Canada and available for use throughout the Datathon to run AUT, datasets were preloaded onto these VMs and a number of derivative files had already been created. We spent some time brainstorming, sharing ideas and exploring datasets using a number of different tools. The day finished with some informative lightning talks about the work participants had been doing with web archives at their home institutions.

Day Two

On day two we continued to explore datasets by using the full text derivatives and running some NER and performing key word searches using the command line tool Grep. We also analysed the text using sentiment analysis with the Natural Language Toolkit. To help visualise the data, we took the new text files produced from the key word searches and uploaded them into Voyant tools. This helped by visualising links between words, creating a list of top terms and provides quantitative data such as how many times each word appears. It was here we found that the word ‘letter’ appeared quite frequently and we finalised the dataset we would be using – University of British Columbia – bc-hydro-site-c.

We hunted down the site and found it contained a number of letters from people about the BC Hydro Dam Project. The problem was that the letters were in a table and when extracted the data was not clean enough. Ben O’Brien came up with a clever extraction solution utilising the raw HTML files and some script magic. The data was then prepped for geocoding by Kathleen Reed to show the geographical spread of the letter writers, hot-spots and timeline, a useful way of looking at the issue from the perspective of engagement and the community.

Map of letter writers.

Time Lapse of locations of letter writers. 

At the end of day 2 each team had a chance to present their project to the other teams. You can view the presentation (Exploring Letters of protest for the BC Hydro Dam Site C) we prepared here, as well as the other team projects.

Why Web Archives Matter

How we preserve, collect, share and exchange cultural information has changed dramatically. The act of remembering at National Institutes and Libraries has altered greatly in terms of scope, speed and scale due to the web. The way in which we provide access to, use and engage with archival material has been disrupted. All current and future historians who want to study the periods after the 1990s will have to use web archives as a resource. Currently issues around accessibility and usability have lagged behind and many students and historians are not ready. Projects like Archives Unleashed will help to furnish and equip researchers, historians, students and the community with the necessary tools to combat these problems. I look forward to seeing the next steps the project takes.

Archives Unleashed are currently accepted submissions for the next Datathon in March 2019, I highly recommend it.

What’s it like to be a trainee? Marjolein Platjee, Graduate Trainee Digital Archivist, 2018-2020

Digital archives, ,

Having now worked as a trainee with the Bodleian Libraries for a little over a month, I can honestly say that the job was 100% worth the move from The Netherlands to the UK. As my predecessors have written, the job is incredibly varied, interesting and very rewarding.

Like the majority of my fellow trainees I do not have a background in libraries or archives. However, I do have a background in research and using archives from work on my PhD focussing on British Popular Literature. Whilst writing my dissertation I was also working as an Information and Process coordinator. In this role I managed a number of IT related projects, including the implementation of a knowledge management base. It was this project that got me interested in knowledge and records management. So when I stumbled upon this traineeship with the Bodleian it seemed the ideal combination of my love for technology and research. As it turns out, it is indeed.

Although I have only been working at the Bodleian for a short while I have already been given the opportunity to do and learn so much. I have already been taught how to manipulate XML files, how to archive websites and how to digitize cassette tapes and other media. I am currently also being trained to assist in the reading rooms and I have been assigned my very own cataloguing project. Working on the latter has been especially exciting and surprising, as next to documents and books I have also been cataloguing merchandise which includes such ‘exotic’ items as t-shirts, jackets, corkscrews – they come in five different colours and have even been engraved – and temporary tattoos.

The distance study at Aberystwyth really prepares me for the tasks that I face in my work and therewith helps me to gain a better understanding of my job, its importance and the history behind it. It does take some self-discipline to keep up with the course work whilst working 4.5 days a week, but if you manage your time wisely it really is doable.

I look forward to learning even more over the course of the coming two years, and I am sure I will look back upon my decision to apply for this job as one of the best ones I could have made.

Marjolein Platjee, Nov 2018