On Tuesday this week we did a short lunchtime session for library and other OU staff on the work we did last year on the RISE activity data project. So I did a short presentation on what we did in the project, and Liz (@LizMallett) covered the user evaluation and feedback. We also had a presentation by Will Woods (@willwoods) from IET on the University’s work around Learning Analytics.  Learning Analytics has now become an important project for the university and it is interesting to see how this moves forward in the next few months.   There is a short blog post on the event on the RISE blog here that includes embedded links to the presentations on RISE.  

Moving forward with Activity Data
Since RISE finished we’ve been looking at ways of embedding some of the recommendation ideas into our mainstream services. We’ve still been routinely adding EZProxy data into the RISE database.  At the moment we are moving the RISE prototype search interface and the Google gadget across to a new web server as we are closing down the old library website. That should keep the search prototype running for a bit more time. It’s also a chance to tweak the code and sort out any bits that have degraded. 

Our website developer (@beersoft) has been building some new features based on the ideas around using activity data. The live library website already displays dynamic lists of resources at a title level in the library resources section on the website http://www8.open.ac.uk/library/library-resources.

One of the prototypes takes the standard resource lists (which are at a title level) and shows the most recently viewed articles from those journals, using the data from the RISE database. The screenshot shows one of the current prototypes.  So users would not only see the relevant journal title (with a link at the title level), but would also see the most recently viewed articles from that journal.  For users that are logged in it would also be feasible to show the articles viewed by people on their course, or even their own recently viewed articles.

We’ve been starting to think about how best to present these new ideas on the website as we want to gauge user reactions to them  Thinking at the moment is that we want to keep them separate from the ‘production’ spec services, so would have them in a separate ‘innovation’ or ‘beta’ space.  I quite like the Cornell CULLABS or the Harvard Library Innovation Lab as a model to follow.

http://www.flickr.com/photos/alancleaver/

The proposition
I’ve spent the last couple of days at a fascinating JISC/SCONUL workshop, ‘The Squeezed Middle? Exploring the future of Library Systems.  ‘The Squeezed Middle’ referring to the concentration of attention in recent months on electronic resource management (in the SCONUL Shared Services and Knowledge Base + activities) and Discovery Systems (such as Summon, EDS and Primo), that has rather taken the focus away from other library systems, notably the Library Management System.  In part, it was explained, this was deliberate, as developments in open source LMS (such as Kuali OLE and Evergreen) and developments of new systems such as Alma from ExLibris that look at unifying print, electronic and digital resource management, have been (and still are) in development and there needs to be some maturity.  But we are now starting to see these developments moving on and open source starting to be adopted (by Staffordshire University library for example).  So the time is right to start to focus on these systems afresh.

The workshop
Punctuating the workshop were a series of deliberately provocative and challenging ’visions’ of the future library of 2020 and a video from Lorcan Dempsey.  [Paul Walk has blogged his here.] Against this background we looked at several topics such as collections, space, systems and expertise around the library systems domain.  Overnight we looked at a series of sixty-odd themes and activities and followed that up today looking at prioritisation and value of those activities to try to understand what might be some priority tasks.

Reflections
A few things came to mind for me during and after the workshop.  Firstly, there maybe isn’t a clear definition of the boundaries of this space and really no common view of what aspects of print/electronic/digital processes and collections we are scoping and addressing.  It also struck me that a lot of the issues, concerns and priorities were about data rather than systems or processes.  So they included topics such as licenses for ebooks, open bibliographic metadata, passing data to institutional finance systems and activity data for example.  I do find it particularly interesting that despite the effort that goes into the data that libraries consume, there are some really big tasks to address to flow data around our systems without duplication or unnecessary activity.  (Incidentally, there’s a concept used in Customer Care, termed ‘Unnecessary contact’ and there used to be a National Indicator NI14 where local government had to reduce unnecessary contact.  In other words reduce the instances where customers have to contact you for further clarification. So you aim to deal with the issue at first point of contact.  I start to wonder whether there’s a similar concept that we might apply to libraries when we carry out extra processing and cataloguing instead of taking ‘shelf-ready’ books and downloaded bibliographic records – unneccessary refinements maybe?)

I also found it interesting how the topic of reading list solutions came up as a hot issue.  It’s a particular interest to me given involvement in the OU’s TELSTAR reference management project. The Reading-List-Solutions JISCMail list has been busy in the last week talking about the various systems (often developed in house).  And it was really fascinating to see how such a fundamental and time-consuming part of our daily work hasn’t really been solved, let alone integrated completely into the procurement and discovery workflow. Although I know that there’s some significant complexity there I find that particularly strange that it hasn’t been a feature built into the LMS.

Final thoughts or library systems of the future

http://www.flickr.com/photos/treborscholz/

It seems to me that there are some general principles that you could think about for future library systems in this space.  And I suppose I’m thinking beyond the next generation of systems such as Alma.  And these may be completely of-the-wall ideas.  But there are few things that come to mind as we move towards 2020. So what might a 2020 LMS look like?

> the systems are component’ised (think Drupal CMS), so both libraries and users can choose which components they use.  And they are largely about flowing data, workflow and process rather than about storing data.

> users control their own profiles (and data) – we (institutions) give them a ‘key’ to access collections we have paid for (so authentication is at a network level or with aggregators?)

> catalogues are distributed – linked data uses the most appropriate vocabularies, most not even run by libraries – local elements are added at the time you choose to procure – there is no ‘catalogue record’ as such but a collection of descriptive elements – you choose where you get your records from, but you don’t download them to ‘your’ lms database

> discovery interface is at the choice of the user – collections are packaged/streamed? and contributed to the aggregators

> rather than a model where libraries buy licensed content and then run systems for their users to access that content – so all institutions largely duplicate their systems – the content owners/aggregators provide the access maybe? as they already start to do with discovery systems?

> there is a ‘rump’ of an LMS database that is your audit trail of transactions and holdings (but with network-level unique IDs that link to descriptive data held at the network level), statistics are held in the cloud (JUSP+++),

> so we contribute our special digital and electronic collections – either to national scale repositories or to open discovery systems?

Maybe not very realistic and fanciful, but something that is a world away from the monolithic LMS that even the open source and new generation systems seem to be building.

All round it was a really good and enjoyable workshop and I’m glad I had the opportunity to go.  I hope the stuff we’ve done helps to inform the future thinking and directions.  Thanks to SCONUL and JISC for organising/funding it and to Ben Showers and David Kay.

http://www.flickr.com/photos/34053291@N05/4482738948/sizes/m/in/photostream/

STELLAR is particularly going to be looking at trying to understand the value of those legacy learning materials by talking to the academics who have been involved in creating those materials.   There are quite a few reasons why older course materials may still have value, they might be able to be reused in new courses on the basis that reusing old materials might be less costly than creating new materials.  They might have value in being able to be transformed into Open Educational Resources.  Or, for example, they might have value in being good historical examples of styles of teaching and learning.  So STELLAR will be exploring different types and models of expressing the value of those materials.

Finding out about the value that is placed on these materials can also be an important factor when trying to understand which materials to preserve as a priority, or where you should expend your resources, and we’d hope that STELLAR would help to inform HE policies as institutions build up increasing amounts of digital learning materials.

Transformation
As part of STELLAR we will be taking some digital legacy learning material and transforming it into linked data (with some help from our friends in KMi). This gives us the opportunity to connect old course materials into the OU’s data.open.ac.uk ecosystem by linking to existing datasets on current courses and OER material in OpenLearn.  By transforming the content in this way we can then explore whether making it more discoverable changes the value proposition, makes the content more likely to be reused or opens up other possibilities.  It should be an interesting project and one that I’m looking forward to, as there are going to be a lot of opportunties to build up my understanding of these issues and aspects.

There’s quite a lot of interesting stuff in the site and also in the LibraryCloud site at www.librarycloud.org. One of the things that particularly interested me (from experiences with the RISE Activity Data project) was the section about data privacy and anonymisation, as a key requirement always has to be that with any dataset where the aspiration is for open release, it must be prepared in a way that ensures that users are unable to be identified individually.

The checkout visualisation is also a neat way of showing that sort of data in a nice clear fashion.  The feature that lets you sort the data by different schools is useful and slightly brings to mind one of the MOSAIC competition entries that used a graph-type visualisation that allowed you to navigate through library use data.  It did amuse me though that ‘Headphones’ appears twice in the top ten with different numbers.   The perils of libraries using their Library Management Systems to loan all sorts of other things!

A history of data
I found the blog post on the Sumerians to be particularly interesting.   The starting point is the contention that their greatest achievement was the invention of data and there are some good examples of how the written language was used to record who owned what (or who owed what to whom).  I like the comparison made between the ‘threats of supernatural retribribution’  being used to protect the integrity of the data with modern warnings over video copying, both being ‘ways of forcefully expressing society’s norms, rather than a credible threat of punishment’

It find it interesting how often we seem to find that early examples of writing often turn out to be lists, in other words data rather than stories.  Another example that comes to mind are the Vindolanda tablets.  These are from the Roman period and found during excavations at a roman fort in Northern England.

http://vindolanda.csad.ox.ac.uk/

“… for dining pair(s) of blankets … paenulae, white (?) … from an outfit: paenulae … and a laena and a (?) … for dining loose robe(s) … under-paenula(e) … vests … from Tranquillus
under-paenula(e) … [[from Tranquillus]]
from Brocchus tunics … half-belted (?) … tunics for dining (?) … (Back, 2nd hand?) … branches (?), number … a vase …
with a handle rings with stones (?) …”

http://vindolanda.csad.oxon.ac/TV-II-196  (Tab. Vindol. II 196)

I also thought the comments making a comparion between instructions for interpreting omens and predicting the future from data to be really interesting.  A great deal is often made of the importance of ‘facts and data’ and it has long seemed to me that the critical factor isn’t the data that you have, but how you interpret it and what decisions you make.  And it often seems to me that the interpretation of data and decision making is a much less scientific exercise.

Part two covering the Harvard Library Innovation Laboratory to follow in the next blog post.

TAPoRware seems to be exactly the sort of text comparison tool that I was looking for.  Developed at the University of Alberta, TAPoR (Text Analysis Portal For Research), has a range of HTML, XML and Plain text tools that allow you to analyse words, find patterns and look for data within text for example.  So I’ve been playing around with the Comparator tool to compare some of the lists of 100 search terms used in the website, federated and catalogue searches.

The comparator tool lets you compare two sets of data at a time and you can upload your set of data as a text file from a local file.  For some reason it wouldn’t accept an excel file but it will display the results as either html or as a tab-delimited file.  The comparator tool goes through and provides some data about how many words there are and how many are unique or appear multiple times.  Then it provides a list of words that are common or unique to either file.

The tool only lets you compare two files at a time, ideally I’d have liked to compare three files.  It also compares the words individually, whereas most of the search terms included in my files are actually search phrases.    So I’ve had to run three comparisons to compare each file with the other two.  The table below summarises the comparisons and shows what percentage of terms are common or unique to each file of search terms.

If I understand correctly then the implication is that there is more in common between the search terms for the catalogue and federated search than between federated search and the website.  When I looked at the search terms originally there were around 45% that had been used across three of the search boxes and website search terms did seem to differ slightly from the federated and catalogue searches. That seems to be borne out by the text comparator that shows the website search data as having less common words.

TAPoR looks like a useful tool, although I’ve barely scratched the surface of what it can do.  Now we’ve changed our website to just a single discovery search there’s some further work we can maybe do to analyse the terms that people are using now to compare with what they used to use on the tabbed search system.

That made me wonder about the topics that I’ve been blogging about and whether there was any particular pattern to the topics.  Taking out the more generic post categories such as libraries and reflections gives the set of words that make up the wordle on the right.   So I’ve most frequently blogged about the website, not surprising as a lot of my work revolves around the new library website that we’ve built over the last couple of years.

The next most frequent topics have been about ipads and activity data, followed by analytics and digital libraries.   I’ve also blogged a bit about kindles and ebooks but only a couple of times about linked data or discovery systems which slightly surprised me as both have featured quite a lot in my work.   On reflection, I’ve also written a few posts about search, which includes some elements of comment on discovery systems, but largely has been about how library search is presented within library websites rather than about discovery systems per se.

Blog post statistics
Having looked at the topics I’ve written about I thought I’d also look at which topics got most views to see if there were any patterns.  My feeling is that posts about search and ipads seem to get most views.  But when I investigated the wordpress statistics in a bit more detail I realised pretty quickly that they weren’t really going to provide an answer.  Although they do show in the list the number of views each post received, the biggest number of views have been of the home page.  So people going directly to the home page to view the latest blog post aren’t going to show up in the statistics for a particular blog post unfortunately.

It’s always going to be the case I suppose that a lot of the views of a new blog post are going to be from the home page of the blog, and there would generally be several blog posts on the home page.  But it does make it difficult to work out which are the most popular topics.   WordPress.com’s statistics for their free blogs give you the basic idea of traffic to the blog but aren’t in any way an equivalent to something like Google Analytics.  Unfortunately you aren’t able to add Google Analytics to a wordpress.com hosted blog.

WordPress blog statistics do show you search terms used to access your blog and where users are being referred from.  The most popular search terms I’m seeing are: refworks api, ipad screen, library search, mendeley ios5 (a bit of a strange combination) and shared services.  The list of search terms does throw up some really strange mispellings of search terms though: lidray search; libariy surch; libreary serch; library shearch; and library surch, all appear in the list.  Google’s ‘did you mean’ feature must be doing its stuff.

Referrers are quite an interesting set of statistics.  Unsurprisingly search engines, largely Google, provide most of the referrrals, but I was surprised that a largely text-based blog had nearly as many referrals from Google Images as Google search.   Next along was twitter, not too surprising as I generally tweet any new blog posts and benefit from a few retweets by people.  There’s quite a wide range of referrers which was quite a surprise and even some from facebook which is interesting as I know I’ve not promoted any blog post from facebook.

Overall, the wordpress.com statistics give you a reasonable indication of how many visits you are getting for your blog and some information about how people are finding your blog and where they are coming from.  If you want the full features of an analytics package you’d have to move to an alternative or paid-for blog host, but the statistics are OK for a free blog host.

Student search behaviour is something that is of great interest to me as I work at a University that delivers courses at a distance so library search is one of the main ways that students interact with our library.  We’ve grappled with the challenge of how we present library search for a while and I’ve blogged about it before a couple of times, most recently here.

So it is really good to see Andrew’s thoughts and research into library search.  It’s interesting to read about the rise of the secretive ‘algorithmic culture’ that he describes as it really starts to explain the trust that users invest in search engines like Google and the implications that this has for library search systems.  We’ve all recognised the impact that Google has on student expectations and Andrew clearly identifies the simplicity and single search box and simple keyword as being something that libraries have been trying to mimic.  Given that library resources have rather less internal coherence (e.g. the typical federated search systems) than Google’s search index then maybe it’s not surprising that the record is mixed.

The figures Andrew reports clearly show students using library search systems as they would Google which leads to problems with too little or too many search results appearing.  That is a problem that is all too familar to users of the new generation Discovery systems such as Ebsco Discovery and Summon.  As Andrew points out these systems also use relevance ranking algorithms that they can be quite proprietary about.

I suppose I’m not surprised that students largely aren’t using what librarians would consider to be the most appropriate search tool for their particular enquiry.  They use what they have had success with in the past.  At undergraduate level at least I’m not surprised that students don’t have the knowledge of which is the most appropriate database to use.  That’s a skill that librarians have had to master and although we all do a lot to try to get this type of domain search information across it clearly doesn’t get through.  But perhaps the concentration of effort on ‘one-stop’ type discovery searches is obscuring that message?

Andrew also covers students skills in evaluating (0r not evaluating) the quality of results and the self-perpetuating loop of trusting results listed on the first page.  Certainly the examples of students deciding that because their search didn’t turn up any results ‘then the information must not exist and they should give upon the topic’ are familar.

A really fascinating and useful paper and piece of research into student search behaviour and something I look forward to hearing more about.

“The user is not broken in that our job is to fulfill the user’s needs, and the user’s needs are, while not always well-defined, possible to meet, or understood by either side, valid — so accusing the user of Doing It Wrong is counterproductive to our goals and needs, and should be avoided. “

It’s something that came out clearly in our usability testing for our new library website and I blogged about it back in October here, talking about the history of that particular meme, so it’s interesting to see it come up again in connection with the search features on the Barnes and Noble website.

I do find it intriguing that it’s not an uncommon reaction among librarians to these sort of examples of user behaviour.  It seems to range from a comment along the lines of ‘well, they aren’t searching ‘properly’ to ‘well, if I could just have 30 minutes with them I could show them how to get much better search results’. 

In part I think it comes from wanting to help people have a better search experience, a user training aspect, which is a role that librarians have developed.  So there’s a willingness to provide training to users which tends to translate into thinking that problems can be solved by training, guidance and support. Rather than by focusing on getting the search experience right so users don’t need training.

But I wonder also whether there’s an aspect that librarians are so used to dealing with poor user interfaces in the typical search systems that they’ve had to master over the years.  So they are more ‘tolerant’ of poor search systems and expect to come up with effective search strategies.  So they forget that most users just want to search in a simple way, and really want a search that is just good enough.  They don’t necessarily want to construct the perfect search strategy.  Users have learnt that on search engines you can usually get some relevant results and expect other systems to be the same.