As part of our trainee programme, we’ve had some particularly inspiring sessions about ‘assessment’. You might be forgiven for wondering how the words ‘assessment’ and ‘inspiring’ go together. Happily, assessment in this context has nothing to do with exams or inspections; it’s about evaluating library life in a whole variety of ways.
With Dr Frankie Wilson, who leads the Bodleian Libraries’ Assessment Team, trainees explored how stats, surveys and feedback can inform improvements to how libraries do things. We saw how assessment can also demonstrate the impact of what libraries already provide, in a world where physical library spaces and library staff expertise may be undervalued. In a follow-up workshop, we had fun experimenting with research methods ourselves – going undercover in the street observing how a space functions, exchanging ‘break-up letters’ to companies have bitterly disappointed us and testing glasses that track your eye movements.
I’ve enjoyed being involved in research activities such as focus groups myself, and I quite often find myself thinking about how decisions can be made in an evidenced way. So I was intrigued to see more during a day-long ‘micro-internship’ with the Bodleian’s Assessment Team. Here’s how it went:
9:00 I arrive at the historic Clarendon Building where I’m meeting Rachel Childs, User Experience (UX) Librarian. Initially we have a slightly awkward wait to get into the building because some newly-weds are having official pictures taken in its photogenic archway. (The perils of working in a beautiful part of town…) Once inside, we get settled and say hello to colleagues Debra Hampton and Tom Pouncy.
We start the day with a discussion about ‘usability testing’ of a particular tool (for example, an online reading list system). Unlike software user testing, which might answer questions such as “Does this button do what it is designed to do?”, usability testing asks a broader question: “Does this tool offer an optimal experience, given what we can observe about users’ behaviour and expectations?” Usability testing implies multiple rounds of testing of each iteration of a tool over a number of weeks. Looking at the flowchart of the whole process with Rachel, and hearing that testers are often time-pressed student volunteers, I could see how important it would be to plan ahead with everyone involved.
10:00 Rachel gives me some background on User Experience research techniques in general, which represent a combination of anthropologists’ methods (‘ethnographish’ techniques) and Design Research processes. Rachel explains that it is important to mix and match your methods; we need both attitudinal and behavioural research to answer a specific question, so that we can see not just what impressions a library-user reports about doing something, but how they actually go about something in reality, too.
10:30 Next, Debra talks me through the gathering of quantitative data from libraries in the Bodleian network. This includes physical occupancy data (how many people are using each library, when), as well as ‘service catalogue’ data (how often particular library resources are used, etc.). She explains that data often needs to go through a cleaning process to weed out errors and for data protection reasons. She introduces me to the software tools used, and explains that individual libraries tend to have one designated stats person who takes responsibility for recording the relevant numbers and passing them on to the team. This makes me realise I have no idea who does this at my own library!
After talking with Debra I have time for a tea break and a quick visit to the Bodleian Publishing team in the same building, to say hello and admire their lovingly-produced books!
11:15 I’m now with Tom, Business Data Analyst, to learn about KPIs (Key Performance Indicators) and data visualisation. Tom has worked on developing KPIs for strategic projects in the Bodleian Libraries organisation. On paper, some aspects of the organisational strategy might sound quite difficult to quantify, such as ‘deliver an inclusive public engagement programme’. We look at the way in which broad goals filter down into programmes, then individual projects, with KPIs attached to define success in a measurable way. Tom shows me how Power Bi is used to bring together accessible graphics and breakdowns illustrating what’s been achieved so far.
12:00 I have a chat with Frankie around midday about the priorities and challenges of her role as Head of Assessment. “It’s my job to tell the truth,” she summarises. The Assessment Team sits outside of the main Bodleian divisions and maintains an independent, user-centred approach. Frankie and her colleagues will undertake projects across the Bodleian as a whole (an organisation-wide Reader Survey has recently taken place, for example). But they also support individual libraries with their own research ideas and sometimes work with external partners as well. Relationship-building and listening skills are therefore really important for understanding different perspectives and making influential recommendations; Frankie credits some of her previous library roles, e.g. as a subject liaison librarian, with helping her to develop these.
14:30 After lunch I am tasked with archiving some images from a ‘Cultural Probe’. Not as ominous as it sounds, this study was ‘probing’ into participants’ experience of library support for their doctoral studies. Each person was given a physical journal with a prompt on each date. It could be a direct question about how they located the books/articles they had been using that day. Or it could be an instruction to ‘take a picture of something that has surprised you today’ (hence there being images for me to process). The thinking is that, by asking people to express themselves in a range of ways, a cultural probe can get to insights that more conventional methods may not. To me, this study looks quite enjoyable for participants and useful to them personally as a reflective log of this time in their lives. For the Assessment Team, it provides a really rich source of feedback that could be analysed in a variety of ways.
15:15 Tom sets me a small challenge: interpreting data from the feedback forms of workshop attendees and picking out the parts that are relevant to a KPI we have seen this morning. Initially I am a little apprehensive about putting my GCSE Statistics into practice, but soon I’m converting the attendees’ satisfaction ratings into a visual format using Excel tools and deciding which style would be most effective for telling the story at a glance.
16:30 In the final part of the day, I help Rachel prepare handouts for the next session of a co-design project she is running with neurodivergent students. ‘Co-design’ is about tailoring a new initiative to its users as closely as possible by involving them in every step of its conception. The focus of Rachel’s project is left intentionally open-ended at the start: participants work together to define an area of the library experience that they may want to rethink, and go from there. I could see that this approach benefits from a lot of preparation and openness/responsiveness on all sides. It is also quite exciting; it makes complete sense that engaging the target demographic in the creation of a new service is going to enhance its relevance and effectiveness. From a participant perspective, I could also imagine this being an interesting and empowering learning process.
17:00ish I head home, my brain starting to digest everything we’ve done and discussed. I feel as if I’ve gained a tangible insight into the Assessment Team’s work, combining day-to-day monitoring of library performance with some quite experimental and innovative projects to influence library life for the better. Thanks to Rachel, Debra, Tom and Frankie for such an illuminating day!
