RISE and the JISC Activity Data Programme

Wednesday saw an early start to travel to the startup meeting for the JISC Activity Data programme, so two of the RISE team were in Birmingham.  It was a good opportunity to talk to the other projects in the strand and find out what they are doing. Some of the projects we were reasonably familiar with such as the UCIAD project in KMi at the OU and the Library Impact Data project being run by Huddersfield, but others we didn’t know too much about, such as the SALT project at John Rylands and Mimas and the AEIOU project in Wales. 

Rather just getting each project to outline the work they were doing the Programme meeting took a different approach, so we each did a short presentation about the hypothesis that is at the root of each of the projects in the Activity Data strand.  In our case our hypothesis is that

“Recommendations Improve the Search experience in new generation discovery solutions.” 

We then had several sessions where we talked about aspects such as IPR, technical challenges and how we might sell the business case for using activity data to senior managers.  We also heard from the synthesis project about how they planned to pull together the work of the projects.  The technical discussion was particularly useful as it brought to the surface all sorts of issues around technical challenges, solutions and IP issues and approaches.   Overall it was a really interesting and useful session.

One of the interesting consequences of having a hypothesis at the heart of a project is that it puts the focus very much on the investigative and experimental purpose of the project.  We are not trying to build a long term solution in the six month project, but we are trying to investigate a particular aspect.  In many ways Recommendations Improve the Search Experience in my mind at least, always has a question mark against it. We don’t know that it does help students, but we want to try to shed some light on how behaviour changes and what users think about the value.  To me that is quite interesting because it takes the focus of the project away from the technical challenges of actually building the stuff and focuses on what you can do with it, what difference does it make and how you make that evaluation.

In the RISE project itself we’ve been making really good progress.  We’ve agreed the database structure and Paul, our developer has built it and parsed all the EZProxy log files into it.  We’ve log files going back to last December so have a reasonable amount of data to play with.  We are really fortunate that Paul has some experience of recommendation systems so has come up with quite a few ideas about what we can do.  We will blog some details about the technical approach on the RISE blog www.open.ac.uk/blogs/rise in the near future but in general we are trying to create several levels of recommendations.

• Level one is recommendations based on connections between searchers ‘people on your course looked at these articles’
• Level two creates recommendations based on relationships between documents.  ‘people who looked at this also looked at that’
• Level three is subject-based recommendations.  ‘if you liked this you might like that’

The next stages are to build the search interface prototypes and finalise the user metrics we want to collect to be able to determine any changes in user behaviour.  We plan to explore an AB testing model by giving users different versions of the search interface and seeing how their behaviour changes.  We’ve also got some work to start to build a Google Gadget version of the EDS search.  So we have a fair bit to be getting on with in RISE but it’s good to be able to spend some time with looking at the wealth of largely unexploited data that we have in our systems.