We’ve been running Primo as our new Library Search discovery system since the end of April so it’s been ‘live’ for just over four months.  Although it’s been a quieter time of year over the summer I thought it would be interesting to start to see what the analytics are saying about how Library Search is being used.

Internal click-throughs
Some analytics are provided by the supplier in the form of click-through statistics and there are some interesting figures that come out of those.  The majority of searches are ‘Basic searches’, some 85%.  Only about 11% of searches use Advanced search.  Advanced search isn’t offered against the Library Search box embedded into the home page of the library website but is offered next to the search box on the results page and on any subsequent search.  It’s probably slightly less than I might have expected as it seemed to be fairly frequently mentioned as being used regularly on our previous search tool.

About 17% of searches lead to users refining their search using the facets.  Refining the search using facets is something we are encouraging users to do, so that’s a figure we might want to see going up.  Interestingly only 13% navigated to the next page in a set of search results using the forward arrow, suggesting that users overwhelmingly expect to see what they want on the first page of results. (I’ve a slight suspicion about this figure as the interface presents links to pages 2-5 as well as the arrow – which goes to pages 6 onwards –  and I wonder if pages 2-5 are taken into account in the click-through figure).

Very few searches (0.5% of searches) led users to use the bX recommendations, despite this being in a prominent place on the page.  The ‘Did you mean’ prompt also seemed to have been used in 1% of searches.  The bookshelf feature ‘add to e-shelf’is used in about 2% of searches.

Web analytics
Browsers used pie chartLooking at web analytics shows that Chrome is the most popular browser, followed by Internet Exploer, Safari, and Firefox.

75% of traffic comes from Windows computers with 15% from Macintoshes.  There’s a similar amount of traffic from tablets to what we see on our main library website, with tablet traffic running at about 6.6% but mobile traffic is a bit lower at just under 4%.

Overall impressions
Devices using library search seem pretty much in line with traffic to other library websites.  There’s less mobile phone use but possibly that is because Primo isn’t particularly well-optimised for mobile devices and also maybe something to test with users whether they are all that interested in searching library discovery systems through mobile phones.

I’m not so surprised that basic search is used much more than advanced search.  It matches the expectations from the student research of a ‘google-like’ simple search box.  The data seems to suggest that users expect to find results that are relevant on page one and not go much further, something again to test with users ‘Are they getting what they want’.  Perhaps I’m not too surprised that the ‘recommender’ suggestions are not being used but it implies that having them at the top of the page might be taking up important space that could be used for something more useful to users.  Some interesting pointers about things to follow up in research and testing with users.

 

browzine magazine shelfWe’ve started using BrowZine (browzine.com) as a different way of offering access to online journals.  Up until recently there were iOS and Android app versions but they have now been joined by a desktop version.

BrowZine’s interesting as it tries to replicate the experience of browsing recent copies of journals in a physical library.  It links into the library authentication system and is fed with a list of library holdings.  There are also some open access materials in the system.

You can browse for journals by subject or search for specific journals and then view the table of contents for each journal issue and link straight through to the full-text of the articles in the journals.  In the app versions you can add journal titles to your personal bookshelf (a feature promised for the desktop version later this year) and also see when new articles have been added to your chosen journals (shown with the standard red circle against the journal on the iOS version).

A useful tool if there are a selection of journals that you need to keep up to date with.  Certainly the ease with which you can connect with the full-text contrasts markedly with some of the hoops that we seem to expect users to cope with in some other library systems.

OpenTree sample badgeOne of the interesting features of our new library game OpenTree for me is that it is possible to engage with it in a few different ways.  Although at one level it’s about a game, with points and badges for interacting with the game and with library content, resources and webpages.  It’s social so you can connect with other people and review and share resources.

But, as a user you can choose the extent that you want to share.  So you can choose to share your activity with all users in OpenTree, or restrict it so only your friends can see your activity, or choose to keep your activity private.  You can also choose whether or not things you highlight are made public.

So you’d wonder what value you’d get in using it if you make your activity entirely private.  But you can use it as a way of tracking which library resources you are using.  And you can organise them by tagging them and writing notes about them so you’ve got a record of the resources you used for a particular assignment.  You might want to keep your activity private if you’re writing a paper and don’t want to share your sources or if you aren’t so keen on social aspects.

If you share your activities with friends and maybe connect with people studying the same module as you, then you could see some value in sharing useful resources with fellow students you might not meet otherwise.  In a distance-learning institution with potentially hundreds of students studying your module, students might meet a few students in local tutorials or on module forums but might never connect with most people following the same pathway as themselves.

And some people will be happy to share, will want to get engaged with all the social aspects and the gaming aspects of OpenTree.  It will be really interesting to see how users get to grips with OpenTree and what they make of it and to hear how people are using it.

It will particularly be interesting to see how our users engagement with it might differ from versions at bricks-and-mortar Universities at Huddersfield, Glasgow and Manchester.  OpenTree’s focus is online and digital so doesn’t include loans and library visits, and our users are often older, studying part-time and not campus-based.

Subject leaderboard screenshotIn early feedback, we’re already seeing a sense that some of the game aspects, such as the Subject leaderboard is of less interest than expected.  Maybe that reflects students being focused around outcomes much more, although research seems to suggest (Tomlinson 2014 ‘Exploring the impact of policy changes on students’ attitudes and approaches to learning in higher education’ HEA) that this isn’t just a factor for part-time and distance-learning students as a result of increased university fees and student loans.  It might also be that because we haven’t gone for an individual leaderboard that there’s less personal investment, or just that users aren’t so sure what it represents.

 

 

OpenTree badge examplesOne of the projects that we’ve been working on as part of our Library Futures programme has been a product called OpenTree.  OpenTree is based on the Librarygame software from a small development team at ‘Running in the Halls’.  Librarygame adds gaming and social aspects to student engagement with library services.

Librarygame was developed originally as Lemontree for Huddersfield University (https://library.hud.ac.uk/lemontree/) and then updated and adopted as librarytree and BookedIn for Glasgow and Manchester Universities respectively (https://librarytree.gla.ac.uk/ and https://bookedin.manchester.ac.uk/).

Being originally based around engagement with physcial libraries and taking data from library loans from the library management system, or from physical library visits, via building access logs, the basic game model had to change a bit for a distance-learning University where students don’t visit the University library or borrow books.

OuOpenTree screenshotr main engagement with students is their use of online library resources and library websites.  Fortunately most of our resource access goes through EZProxy so we were able to find a way of allowing users who sign-up to the game to get points for the resources they access.  We’ve also been able to add javascript tracking onto a couple of our websites to give students points for accessing those sites.

OpenTree gives users points for accessing resources and points build up into levels in the game.  Activities such as making friends, reviewing, tagging and sharing resources also get you badges in the game.  We’ve also added in a Challenges section to highlight activities to encourage users to try out different things, trying Being Digital, for example.

Because it lists library resources you’ve been accessing I’ve already been finding it useful as a way of organising and remembering library resources I’ve been using, so we’re hopeful that students will also find it useful and really get into the social aspects.

OpenTree launches to students in the autumn but is up-and-running in beta now.  A video introducing OpenTree is on YouTube at: https://www.youtube.com/watch?v=yeSU0FwVNvU

We’re really looking forward to seeing how students get on with OpenTree and already have a few thoughts about enhancements and developments, and no doubt other ideas will come up once more people start using it.

 

 

 

 

 

I noticed this morning a blog post on the Wellcome Library plans to build a cloud-based digital library platform, ‘Moving the Wellcome Library to the cloud‘  It’s a fascinating piece of news.  The Wellcome Library’s amibition and scale, talking about having over 30m digitised pages by 2018 and about building a platform that could potentially be made use of by others is interesting to see.

As we’ve seen with Library Management Systems, cloud-based systems are becoming commonplace but where digital libraries seem to be concerned, most of them are operated as locally hosted systems.   The article also talks about the use of IIIF (International Image Interoperability Framework)  which is something for digital libraries to take notice of.  It also flags some developments to Wellcome’s media player to create a new Universal Viewer to handle video, audio and other material.  Given how tricky we’ve found getting accesible media players it will be interesting to keep an eye on these developments.

Mention of APIs, commodity services and APIs are also in scope.  Something definitely to watch for the future.

So we’re slowly emerging from our recent LMS project and a bit of time to stop and reflect, partly at least to get project documentation for lessons learned and suchlike written up and the project closed down.  We’ve moved from Voyager, SFX and EBSCO Discovery across to Alma and Primo.  We went from a project kick off meeting towards the end of September 2014 to being live on Alma at the end of January 2015 and live on Primo at the end of April.

So time for a few reflections about some of the things to think about from this implementation.  I’d worked out the other day that it has been the fifth LMS procurement/implementation process I’ve been involved with, and doing different roles and similar roles in each of them.  For this one I started as part of the project team but ended leading the implementation stage.

Reflection One
Tidy up your data before you start your implementation.  Particularly your bibliographic data but if you can other data too.  You might not be able to do so if you are on an older system as you might not have the tools to sort out some of the issues.  But the less rubbish you can take over to your nice new system the less sorting out you’ve got to do on the new system.  And when you are testing your initial conversion too much rubbish makes it harder to see the ‘wood for the trees’, in other words work out what are problems that you need to fix by changing the way the data converts and what is just a consequence of poor data. With bibliographic data the game has changed, you are now trying to match your data with a massive bibliographic knowledge base.

Reflection Two
It might be ideal to plan to go live with both LMS and Discovery at the same time but it’s hard to do.  The two streams often need the same technical resources at the same time.  Timescales are tight to get everything sorted in time.  We decided that we needed to give users more notice of the changes to the Discovery system and make sure there was a changeover period by running in parallel.

Reflection Three
You can move quickly.  We took about four months from the startup meeting to being live on Alma but it means that you have a very compressed schedule.  Suppliers have a well-rehearsed approach and project plan but it’s designed as a generic approach.  There’s some flexibility but it’s deliberately a generic tried-and-tested approach.  You have to be prepared to be flexible and push things through as quickly as possible.  There isn’t much time for lots of consultation about decisions, which leads to…

Reflection Four
As much as possible, get your decisions about changes in policies and new approaches made before you start.  Or at least make sure that the people on the project team can get decisions made quickly (or make them themselves) and can identify from the large numbers of documents, guidance and spreadsheets to work through, what the key decisions you need to make will be.

Reflection Five
Get the experts who know about each of the elements of your LMS/Discovery systems involved with the project team.  There’s a balance between having too many and too few people on your project team but you need people who know about your policies, processes, practices and workflows, your metadata (and about metadata in general in a lot of detail to configure normalisation, FRBR’ization etc etc), who know about your technology and how to configure authentication and CSS.  Your project team is vital to your chances of delivering.

Reflection Six
Think about your workflows and document them.  Reflect on them as you go through your training.  LMS workflows have some flexibility but you still end up going through the workflows used by the system.  Whatever workflows you start with you will no doubt end up changing or modifying them once you are live.

Reflection Seven
Training.  Documentation is good.  Training videos are useful and have the advantage of being able to be used whenever people have time.  But you still need a blended approach, staff can’t watch hours of videos, and you need to give people training about how your policies and practices will be implemented in the new LMS.  So be prepared to run face to face sessions for staff.

Reflection Eight
Regular software updates.  Alma gets monthly software updates.  Moving from a system that was relatively static we wondered about how disruptive it would be.  Advice from other Libraries was that it wasn’t a problem.  And it doesn’t seem to be.  There are new updated user guides and help in the system and the changes happen over the weekend when we aren’t using the LMS.

Reflection Nine
It’s Software as a Service so it’s all different.  I think we were used to Discovery being provided this way so that’s less of a change.  The LMS was run previously by our corporate IT department so in some senses it’s just moved from one provider to another.  We’ve a bit less control and flexibility to do stuff with it but OK, and on the other hand we’ve more powerful tools and APIs.

Refelection Ten
Analytics is good and a powerful tool but build up your knowledge and expertise to get the best out of it.  We’ve reduced our reports and do a smaller number than we’d thought we need.  Scheduled reports and widgets and dashboards are really useful and we’re pretty much scratching the surface of what we can do.  Access to the community reports that others have done is pretty useful especially when you are starting.

Refelection Eleven
Contacts with other users are really useful.  Sessions talking to other customers, User Group meetings and the mailing lists have all been really valuable.  An active user community is a vital asset for products not just the open source ones.

and finally, Reflection Twelve
We ran a separate strand to do some user research with students into what users wanted from library search.  This was really invaluable as it gave us evidence to help in the procurement stage, but particularly it helped shape the decisions made about how to setup Primo.  We’ve been able to say: this is what library users want and we have the evidence about it.  And that has been really important in being able to challenge thinking based on what us librarians think users want (or what we think they should want).

So, twelve reflections about the last few months.  Interesting, enlightening, enjoyable, frustrating at times, and tiring.  But worthwhile, achievable and something that is allowing us to move away from a set of mainly legacy systems, not well-integrated, not so easy to manage to a set of systems that are better integrated, have better tools and perhaps as important have a better platform to build from.

Absence from blogging over the last few months feels very much like some form of winter hibernation but it’s mainly been a case of not having too much time for reflection in the middle of a library management system implementation.  We haven’t quite finished yet but are a long way through the proceBeadnell wadersss and have been live on a cloud-based LMS for just over a month. So I can try to put together some early thoughts about the process and experience.

Time

I worked on our project proposal around Christmas 2012 for a project we termed Library Futures that included a library management system and discovery procurement and implementation.  But that wasn’t really the start of the process.  We’d spent a bit of time looking at what our needs were and working with some consultants to get a better idea of the best options for us.  I’d also had some involvement with the Jisc LMS Change project, all of which helped us to understand what was out there and what our options were.  So that takes us back into 2011 and maybe a bit earlier.  And a lot of the thinking was about the best timing for changing systems as the LMS market was in the early stages of the ‘Software as a Service’ reinvention and products were (and maybe still are) at an early stage.  So by my rough calculation that’s a couple of years in the planning, followed by a year to secure approval, followed by an eighteen month or so procurement and implementation stage.  It takes a long time and a lot of effort, and the final stage of implementation isn’t the most time-consuming part.

Process
In the procurement stage we went the full EU tender route and for our requirements catalogue (specification) made extensive use of the LibTechRFP exemplars http://libtechrfp.wikispaces.com/ not just the UK Core Specification but also the examples for the Library Services Platform, Electronic Resources Management and Search and Discovery.  And we also needed to add in our own requirements and cut out features aimed more at a traditional ‘physical’ university.  It ended up with quite a large and detailed catalogue of requirements.  But I’ve always felt that to be important for library systems as the detail is vital (and not just because the successful tender response forms part of our contract).  Library management systems have to cover a lot of functions and it’s important to get the detail to understand what using that system will mean for you in practice.  Interesting to me though was to find some of our search requirements already getting reused in another systems requirement document in the institution.
Tools
I’m always on the lookout for useful new tools for projects, website and so on.  So it was good to see a tool like Basecamp being used by the supplier we chose. It isn’t a free tool (other than for an initial period) but it worked well as a way of sharing files and having the sort of discussions that you need when going through the implementation process.  I felt the to do list feature worked a bit less well.  As a communication tool it worked neatly without being too formal or time-consuming.  We’ve ended up using it on two different projects with two entirely different suppliers so it is obviously doing something right.

Other thoughts
Final thoughts for the moment are about the range of skills needed in a team putting in an LMS.  Some obvious ones such as systems and IT knowledge, procurement and project management, and for libraries obvious areas such as knowledge of the library acquisitions, cataloguing/metadata and circulation processes.  But also ones that can get overlooked around training expertise, administrative support, decision making, business analysis and data quality.  And above all some determination and team spirit to get through an immense to do list.

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 11,000 times in 2014. If it were a concert at Sydney Opera House, it would take about 4 sold-out performances for that many people to see it.

Click here to see the complete report.

At the end of November I was at a different sort of conference to the ones I normally get to attend.  This one, Design4learning was held at the OU in Milton Keynes, but was a more general education conference.  Described as “The Conference aims to advance the understanding and application of blended learning, design4learning and learning analytics ” Design4learning covered topics such as MOOCs, elearning, learning design and learning analytics.

There were a useful series of presentations at the conference and several of them are available from the conference website.   We’d put together a poster for the conference talking about the work we’ve started to do in the library on ‘library analytics’ – entitled ‘Learning Analytics – exploring the value of library data and it was good to talk to a few non-library people about the wealth of data that libraries capture and how that can contribute to the institutional picture of learning analyticPoster for Design4learning conferences.

Our poster covered some of the exploration that we’ve been doing, mainly with online resource usage from our EZProxy logfiles.  In some cases we’ve been able to join that data with demographic and other data from surveys to start to look in a very small way at patterns of online library use.

Design4learning conference poster v3

The poster also highlighted the range of data that libraries capture and the sorts of questions that could be asked and potentially answered.  It also flagged up the leading-edge work by projects such as Huddersfield’s Library Impact Data Project and the work of the Jisc Lamp project.

An interesting conference and an opportunity to talk with a different group of people about the potential of library data.

Photograph of office buildings at Holborn Circus

Holborn Circus – I was struck by the different angles of the buildings

Themes

For me two big themes came to mind after this year’s Future of Technology in Education Conference (FOTE). Firstly, around creativity, innovation and co-creation; and secondly about how fundamental data and analytics is becoming.

Creativity, innovation and co-creation

Several of the speakers talked about innovation and creativity.  Dave Coplin talked of the value of Minecraft and Project Spark and the need to create space for creativity, while Bethany Koby showed us examples of some of the maker kits ‘Technology Will Save Us’ are creating.

Others talked of ‘flipping the classroom’ and learning from students as well as co-creation and it was interesting in the Tech start-up pitchfest that a lot of the ideas were student-created tools, some working in the area of collaborative learning.

Data and analytics

The second big trend for me was about analytics and data.  I was particularly interested to see how many of the tools and apps being pitched at the conference had an underlying layer of analytics.  Evaloop which was working in the area of student feedback, Knodium – a space for student collaboration, Reframed.tv – offering interaction and sharing tools for video content, Unitu – an issues tracking tool and MyCQs – a learning tool, all seemed to make extensive use of data and analytics, while Fluency included teaching analytics skills.  It is interesting to see how many app developers have learnt the lessons of Amazon and Google of the value of the underlying data.

Final thoughts and what didn’t come up at the conference

I didn’t hear the acronymn MOOC at all – slightly surprising as it was certainly a big theme of last year’s conference.  Has the MOOC bubble passed? or is it just embedded into the mainstream of education?  Similarly Learning Analytics (as a specific theme).  Certainly analytics and data was mentioned (as I’ve noted above) but of Learning Analytics – not a mention, maybe it’s embedded into HE practice now?

Final thoughts on FOTE.  A different focus to previous years but still with some really good sessions and the usual parallel social media back-channels full of interesting conversations. Given that most people arrived with at least one mobile device, power sockets to recharge them were in rather short supply.

Twitter posts

Categories

Calendar

September 2015
M T W T F S S
« Aug    
 123456
78910111213
14151617181920
21222324252627
282930  

Creative Commons License

Follow

Get every new post delivered to your Inbox.

Join 46 other followers