Testing the online catalogue: results of user testing

Back in the Spring of this year, we carried out two phases of user testing on our online catalogue, Borthcat.   The key results of the first phase of testing are described over on our Digital Archivist, Jen Mitcham's, blog, as well as some of the actions we were able to take prior to the public launch of Borthcat in April 2016. While the basic phase of testing allowed us to make some really practical and in some cases speedy changes to the catalogue interface, the second phase of testing really gave us insight into individual user interactions with the catalogue  to examine how users search for and retrieve information from our holdings. Whilst results of similar testing have been carried out with Access to Memory (AtoM: our web-based, open-source archival description software) in one or two other institutions in North America, at the time of testing the Borthwick was the first UK AtoM user to carry out such detailed work. The findings of both the first and second phase of testing formed the backbone of my Masters dissertation in Archives and Records Management and, as I'm graduating from the University of Dundee next week (!), it seemed like the perfect time to give you a report into the results from the more detailed tests we carried out.

The Borthcat homepage

Why did we do it?

Across the archives sector, online access to information is now pretty much routine and users often expect digital versions of finding aids and, ideally, digitised version of the documents that they can search and examine. In developing our own online interface, Borthcat, we wanted to make sure that not only was information available to users but (more crucially, in my opinion) that users were able to successfully find that information using the tools we had provided. Looking at our own user base from the statistics we capture, between March and April 2016 - just prior to the launch of Borthcat - there were over double the amount of remote enquiries (1614) to physical visits (731) to the searchroom. In the same period there were over 13,300 unique hits our online digital document repositories: Find My Past, the Cause Papers and the Archbishops' Registers. We wanted to make our catalogue as informative and accessible as possible, not least because a large proportion of both our current and future users are researchers who may never be able to physically visit us at all.

How did we do it?

Jen's blog explores the results of the first phase of testing that we carried out - recruiting our users through a mixture of social media channels and onsite advertising in the searchroom and asking them to complete a brief online questionnaire.

The second stage of testing asked participants to work through a series of set exercises using Borthcat while being observed. During these sessions, participants' screens were recorded using Screencastify to capture their mouse movements and the number of clicks they made.

We wanted to capture some qualitative data on user interactions to enrich the statistical information we'd obtained in the first round of testing. We used the Archival Metrics Toolkit (a fantastic resource!) to help design the exercises and tried to ameliorate the effects of such a controlled environment. Of course, we couldn't hope to fully replicate a researcher's independent enquiry but the results we obtained were interesting and gave us an insight into our users that we hadn't had before.

What did we find out?

Here are just a few of the main findings from the testing.

Limited use of hierarchical menu

Users heavily relied on using the free text search bar at the top of the Borthcat homepage to identify records. Only one participant in the second phase of testing used the hierarchical information available on the left-hand side of the screen, and another used free text searching as their sole retrieval technique throughout the test period. This could be for a mixture of reasons. The majority of our entries in Borthcat are at collection level, and so there are fewer hierarchical descriptions available currently (although the test exercises focused on those archives with full catalogues). Further, and as reflected in the basic phase of testing, many users have become familiar with a free-text search when using search engines like Google.

Overwhelming 'wall of text'

Users found the level of information available on each entry, and the amount of results returned for some searches, to be overwhelming in some cases. There was an overall idea, again in common with the basic testing, that users wanted Borthcat to be able to tailor information more specifically to their queries. I think that this is where the presence of an archivist or the staff in the searchroom who understand our holdings are the most valuable asset we can have; this situation would be more easily resolved for a researcher who was onsite and able to consult a staff member for advice. Where the researcher is remote and is searching for unfamiliar (or unknown!) material then it is vital that the catalogue presents information clear enough for them to make an informed choice.  Some users in the observed tests used keyboard techniques like CTRL+F to narrow down occurrences of specific terms within an archival description, although the majority didn't.

Typeahead search suggestions 

Understanding icons

A further usability issue to come out of the detailed testing was that the icons used in Borthcat's 'Typeahead' search - where potential results are generated as you type - are not defined in our customised iteration of AtoM. In an exercise designed to look at how users interacted with our subject-term listings, participants were asked to find out how many of our holdings contain diaries - a subject terms that has been linked across several separate archives. The majority of users did this by searching for the term ‘diaries’ in the free-text search box. When they did so they were confronted by several entries, all called 'Diaries', at item, file and sub-series level, all from separate archives, as well as a subject term entry for diaries in general. The archival entries are all marked with a 'description' icon and the subject entry is marked with an icon showing a label. For those of us working with Borthcat on a daily basis, it was simple to select the relevant entry and to continue our work but most participants in the study repeated the search several times in order to work through all the options before finding the entry they required. This allowed us to identify a way to improve the usability of Borthcat as well as giving us food for thought in how we construct the titles of our records. The linking of records through the use of different subject terms is one of the most interesting capabilities of AtoM - it allows connections to be made across archives in a way that would be very difficult to do using paper finding aids and can draw out unexpected links. Being able to see how users interact with this capability over the longer term will be very important in understanding how researchers can make the most of the information we hold.

A personal connection

It really came across during the testing that users value a personal connection to their research, either through searching for personal names or through bringing their own research contexts and knowledge to the way they search for information. Several users commented that they would really appreciate a feature that would allow them to collect all the records they found interesting in order to look at them again or to send them through to the searchroom for retrieval. This wasn't something I had expected, but is something that other archives do. A great example is the pinboard feature at the Marks and Spencer archive, Marks in Time.

Carrying out this exercise has been really helpful in understanding that what our users want from our records and what we think they want isn’t always the same thing. Involving our users directly in the development of Borthcat was also a fantastic opportunity to engage more with our audience on a project that will be of practical benefit. I must thank all of the participants who took part in each phase of testing; without their invaluable contribution of time in completing both the survey and the observed exercises, we would not have been able to gain the insights we have done into how our users retrieve information, and how they'd like to be able to retrieve information in the future. It is of vital importance for us and for other archive repositories to keep our users’ needs at the heart of their considerations when making archival information accessible online.

Lydia Dean

Archivist

The results of this user testing have been discussed in more detail in my MLitt dissertation ‘Access to Memory: Understanding how users of the Borthwick Institute search for online archival information’ through the University of Dundee. You can find out more about our work with AtoM through our blog and Jen's blog 'Digital Archiving at the Borthwick' .

A Festive Conservation Run-down of the Archbishops’ Registers Project

As the Archbishops’ Registers Revealed project is drawing to a close along with the year 2015, I wanted to offer a brief overview of my involvement in the project. It can be quite tricky for a conservator to accurately convey exactly what it is they do in the workshop. This blog certainly isn’t as catchy as the 12 Days of Christmas - but I hope that it provides some advent calendar-sized tasters of the work I have been doing.

12 Limp parchment volumes

12 limp parchment volumes required no work in preparation for digitisation.

There are some things that conservators can do to improve the digitisation process – cleaning, unfolding, repairing, etc – but there are also some things that we cannot improve. We can clean a surface, which will lighten the areas around the ink and make the ink stand out better, but we cannot replace abraded or faded ink. Consequently we do need to assess archives before a digitisation work plan is put in place, so that we know what we will need to tackle and how long it might take.

Abp Reg 11 is the volume that required the greatest number of treatments.

Abp Reg 11 with the highest number of treatments recorded

Within the 37 volumes that were treated but not disbound 610 treatments were documented in total. 127 of these treatments were undertaken within Abp Reg 11. Treatments ranged from dry cleaning the surface of folios or unfolding the corners of a folio, to removing a previous repair that was obscuring text or repairing the edge of a folio that had suffered loss and damage. We would only undertake treatment where either text had been obscured (by dirt or folds) or the area was vulnerable to further deterioration during handling. Without this guideline in place it would not have been possible to complete the treatments in time for the digitisation to take place!

10 volumes contained paper in need of treatment

such as this document

10 volumes required paper repairs.

The majority of the folios in the Archbishops’ Registers are parchment, but there are occasional paper inserts and modern paper endleaves in the volumes too. 33 of the 610 treatments mentioned above were on paper, but almost all of the others were on parchment. 9 (give or take) descriptive phrases for the metadata  that gave me a headache! 9 descriptive phrases for the metadata that created plenty of debate.

This is a very subjective number, which would certainly fluctuate depending on who you spoke to! I first became involved with the metadata when it became apparent that not all of the images could take their image number from a folio number. The Archbishops’ Registers are nothing if not inconsistent, and there were various hiccoughs to accommodate, as well as the structural features of each volume (and those thrown in from previous bindings). A lot of my time was spent deciding what information to include, what to leave out, and which terms best reflected what the end user would see in the image.

8 volumes requiring only minor treatments such as

 the dry cleaning shown here

8 volumes requiring minor work…

As opposed to 32 volumes requiring major work! In my initial assessments, ‘minor work’ refers to cleaning or small areas of flattening. ‘Major work’ includes larger areas to flatten and more invasive or time consuming treatments. A small local humidification with a non-aqueous solvent could be applied and dried within an hour or so, whereas the application of a repair would take a minimum of 3 days of treatment when drying time is taken into account. My workflow planning needed to take all of this information into account, so that I could ensure the photographer had a seamless flow of volumes to image and process.

7 spines over 10cm in width.

The Archbishops’ Registers vary in size, but the most

7 spines over 10cm in width

memorable volumes are the largest. 7 of the volumes have spines between 10 and 15cm wide. Several of these have also been bound with thick wooden boards, and consequently they are large, heavy and unwieldy to manoeuvre. This has made them challenging to handle safely during conservation and digitisation. In spite of this (or partly because of this?) these are some of my favourite Registers – most of the bindings still function well, and they have an undeniably weighty presence. I can’t help but think when I look at them that they must contain a formidable number of sheep!

6 hours of Conservation at the Summer Institute.

In the summer of 2015 we held a Summer Institute for 12 participants on the subject of the Archbishops’ Registers. Classes and workshops covered the history and context of the registers, reading and interpretation of the registers and the opportunity to develop a mini-research project. I was privileged to be asked to take the students for a whole day, and managed to pack in information and

6 hours of Conservation talks and activities with students  at the Summer Institute

6 hours of Conservation talks and activities with students
 at the Summer Institute

investigative tasks regarding the materials, tools and techniques with which the registers were created, as well as explaining and demonstrating the role that Conservation has played in this project and discussing some of the ethical implications and dilemmas we have been working with.

5 sheets of goldbeater’s skin

This is the material I have been using to support damaged and vulnerable areas of the parchment folios. Over the course of the project I have repaired over 100 parchment folios and each of these takes a minimum of 3 days to complete. When treating parchment it is important to keep moisture to a minimum; consequently the repairs are applied in stages so that they can dry in between applications of adhesive.

5 sheets of goldbeater’s skin used to repair damaged parchment such as the example above from Abp Reg 10 f.25               (left: before treatment; right: after treatment)

4.3 kg of magnetic restraint

4.3kg my favourite magnetic pull strength

I have been using magnets as a tool to restrain parchment when it is drying. I use a ferrosheet under the parchment folio, so that a magnet placed on top of the parchment will hold the parchment in place. I have experimented with various sizes and strengths of magnet, but my current favourite is a neodymium cylindrical magnet of 12mm diameter and 6mm height at a strength of N42 which gives a pull of 4.3kg!

3 volumes disbound

The decision to disbind any of the registers was not taken lightly. The process is very invasive and can risk damaging the register; loose leaves are more vulnerable to future deterioration than those in a binding; removing the binding alters the format of the register; and historical evidence can be lost during disbinding. On the other hand the bindings we were considering were not original bindings; they were very stiff, which obscured a significant proportion of text on the majority of folios; and the stiffness of the binding was also hindering the functionality of the volume. 3 registers have been disbound and digitised as loose leaves. A major concern for the New Year will be to discuss with the archivists whether these registers will be re-bound, and if so in what manner.

3 registers disbound

2 sheets of gelatine remaining, used for repairs

 and poultices such as the example above

2 sheets of gelatine remaining

I have been using gelatine as my main adhesive of choice for both paper and parchment repairs. I have also used gelatine to create poultices, which I have used for a number of treatments. Poultices allow a slow transfer of moisture. I have used them to soften the adhesive of previous repairs in order to remove them. I have also used poultices to remove paper guards from parchment inserts. Lastly, I have been using gelatine poultices to remove materials that have been adhered to the spines of the volumes I have disbound. Including the volumes that have been disbound, I have used poultices to treat 289 folios, and removed spine linings from 3 volumes. This has used 61 sheets of gelatine – with 2 sheets left over for the New Year.

1 happy conservator!

1 frazzled but very happy conservator

I have sincerely enjoyed working on this project. It has been a privilege to work on the Archbishop’s Registers, and a pleasure to work with such beautiful volumes. I look forward to seeing the images of all the registers available online in the not too distant future!

Catherine Dand, Project Conservator

Professional Perspectives: Ethics in Conservation and Archives

In August 2015 three members of Borthwick staff travelled across land and sea to present papers at the annual Archives and Records Association conference in Dublin. The theme of the conference was 'Challenges, Obligations, or Imperatives? The moral and legal role of the Record Keeper today'. The three sessions that our staff presented demonstrate a range of ethical concerns that exist within the worlds of archives and conservation, but also offer a glimpse into the inner workings of these two relatively little-known professions.

Screen shot from Tracy’s presentation ‘Why do you do what you do to me? 

Conservation Prioritisation and Collaboration in the Digitisation of the Retreat Archive’.

For more information on the project visit: www.york.ac.uk/borthwick/projects/retreat/

Tracy Wilcockson was the opening speaker for the conservation strand on the second day of the conference with her presentation ‘Why do you do what you do to me? Conservation Prioritisation and Collaboration in the Digitisation of the Retreat Archive’. Tracy’s talk addressed her role in preparing material for the digitisation of the Retreat archive, which is a part of the wider ‘The Asylum and Beyond’ project funded by the Wellcome Trust. The depth and breadth of Tracy’s knowledge of conservation theory and practice enabled her to discuss the nature of digitisation as ethically problematic, while still clearly explaining the practical implications and processes that have affected the digitisation of the Retreat Archive. I was particularly struck by the application of digitisation to the ‘balance triangle’ that she described from Chris Caple’s book ‘Conservation Skills’, which demonstrates how every conservation process can be expressed by a balance between preservation, investigation and revelation. The talk also highlighted several examples of collaborative responses to digitisation problems between the Conservation and Digitisation teams, such as the use of light boxes for the image capture of receipts, so that they do not need to undergo the time-consuming process of removal from the documents that they are attached to.

Tracy’s talk was very well received, and generated a number of enthusiastic and complimentary tweets from listeners. The presentation has also resulted in some excellent contacts with a number of other organisations running digitisation projects, and the potential for further collaboration. Colleagues are interested in further information regarding how the Borthwick project has been managed, equipment and techniques that have been used, problems that have arisen and how they have been overcome. It will be exciting to see where this goes next.

My presentation ‘The Archbishops’ Registers of York: A case study of ethical dilemmas in conservation and digitisation’ was directly after Tracy’s, and also used a digitisation project as a case study, this time to study the role of the conservator as an arbitrator of ethical problem-solving and decision-making. One of the points I addressed was the tension that can exist between the available research into materials (which may recommend against certain treatments) and the need to access or digitise an archive (which may need treatment to enable access).

As an example I talked about the ethics of removing creases from parchment using solvents. Although I only had time to briefly outline the technique I finally used in the project, I had set up a demonstration volume for the delegates in the adjoining room, along with a selection of magnets and magnetic material to try. In the coffee-break after the first session I chatted with a stream of

Working with magnets on the ‘York’s Archbishops’ Registers Revealed’ project. 

For more information visit: www.york.ac.uk/borthwick/projects/archbishops-registers/

colleagues, discussing the materials and the technique, as well as other applications. The magnets generated a flurry of discussion comparing the many different ways that they are currently being used in conservation workshops around the country: in exhibitions and displays; for restraint of parchment during treatment; wrapped in blotters for restraint and drying of local repairs; for construction of boxes and book rests; and one enterprising department are using them as very effective darts… I was pleased that a presentation that had aimed to highlight the importance of communication and collaboration had generated so much productive discussion.

After lunch the conservation, digital preservation and archive strands merged into a hot-pot of workshops, panels and break-out sessions. Gary Brannan, our Access Archivist, was running a workshop entitled ‘From Filth to the Future: Reviewing the ARA training offer’. The session was based heavily upon the ARA Northern Region’s 2013 Filth conference, and was designed to get delegates thinking about the role that ARA could - and perhaps, maybe even should - have in supporting members dealing with difficult, disturbing and legally dubious collections.

Flying home after a productive and inspiring day.

The exercises were based on real experiences sent in by ARA members and featured issues including I still sometimes find myself picturing the photograph from a Coroner’s notebook showing the image of a man who had been murdered by having his head nailed to a tree and Male reader requesting (repeatedly) access to 1950s photographs of schoolgirls in gym clothes. Delegates were asked to sort the issues provided into those which they felt they needed emotional support to deal with, and those which may be helped by practical advice and training. Some of the results were surprising - for instance, much unease at processing and making available content that may upset third parties, and a desire for training in dealing with requests from customers for embarrassing (but not legally exempt) data. However, the greatest need came in the desire for both training and support in dealing with content and imagery related to death and inquests.

Gary’s session received considerable interest and encouragement from delegates, and provided a supportive outlet for frank discussions in a profession whose members often work in small teams or in isolation. The subject matter and style of delivery really embodied the theme and aims of the conference as a whole: addressing relevant issues amongst our peers, sharing experiences and exploring practical solutions. Talking to Tracy and Gary on our journey home I was struck by how much we had taken away from the conference – new ideas and perspectives, new contacts – and for myself a renewed motivation for the job that I do and respect for the colleagues that I work alongside, both within the Borthwick and without. The overlapping worlds of archives and conservation might not be very well known – but they are passionately appreciated by those that know them well.

Catherine Dand, Conservator.