You are currently browsing the tag archive for the ‘User Activity Data’ tag.
Wednesday saw an early start to travel to the startup meeting for the JISC Activity Data programme, so two of the RISE team were in Birmingham. It was a good opportunity to talk to the other projects in the strand and find out what they are doing. Some of the projects we were reasonably familiar with such as the UCIAD project in KMi at the OU and the Library Impact Data project being run by Huddersfield, but others we didn’t know too much about, such as the SALT project at John Rylands and Mimas and the AEIOU project in Wales.
Rather just getting each project to outline the work they were doing the Programme meeting took a different approach, so we each did a short presentation about the hypothesis that is at the root of each of the projects in the Activity Data strand. In our case our hypothesis is that
“Recommendations Improve the Search experience in new generation discovery solutions.”
We then had several sessions where we talked about aspects such as IPR, technical challenges and how we might sell the business case for using activity data to senior managers. We also heard from the synthesis project about how they planned to pull together the work of the projects. The technical discussion was particularly useful as it brought to the surface all sorts of issues around technical challenges, solutions and IP issues and approaches. Overall it was a really interesting and useful session.
One of the interesting consequences of having a hypothesis at the heart of a project is that it puts the focus very much on the investigative and experimental purpose of the project. We are not trying to build a long term solution in the six month project, but we are trying to investigate a particular aspect. In many ways Recommendations Improve the Search Experience in my mind at least, always has a question mark against it. We don’t know that it does help students, but we want to try to shed some light on how behaviour changes and what users think about the value. To me that is quite interesting because it takes the focus of the project away from the technical challenges of actually building the stuff and focuses on what you can do with it, what difference does it make and how you make that evaluation.
In the RISE project itself we’ve been making really good progress. We’ve agreed the database structure and Paul, our developer has built it and parsed all the EZProxy log files into it. We’ve log files going back to last December so have a reasonable amount of data to play with. We are really fortunate that Paul has some experience of recommendation systems so has come up with quite a few ideas about what we can do. We will blog some details about the technical approach on the RISE blog www.open.ac.uk/blogs/rise in the near future but in general we are trying to create several levels of recommendations.
- Level one is recommendations based on connections between searchers ‘people on your course looked at these articles’
- Level two creates recommendations based on relationships between documents. ‘people who looked at this also looked at that’
- Level three is subject-based recommendations. ‘if you liked this you might like that’
The next stages are to build the search interface prototypes and finalise the user metrics we want to collect to be able to determine any changes in user behaviour. We plan to explore an AB testing model by giving users different versions of the search interface and seeing how their behaviour changes. We’ve also got some work to start to build a Google Gadget version of the EDS search. So we have a fair bit to be getting on with in RISE but it’s good to be able to spend some time with looking at the wealth of largely unexploited data that we have in our systems.
New activity data project
I blogged back in November on some reflections on preparing and submitting a funding bid. Well, we’ve recently received word that we are one of eight projects to be funded as part of the JISC programme and we’re really pleased to be successful. It’s a fairly short six month project but is working in an area that I find particularly interesting, that of user activity data and how it can (and should be used) to improve services.
Our new project RISE is part of the JISC Activity Data programme and the projects cover quite a wide range of data from VLEs, library systems and other corporate systems so it will be really interesting to see what comes out of them. We’re going to be looking at data from our ezproxy system which we use as the primary method of getting students and staff access to licenced library resources. As most of our use is remote (unsurprisingly as I work at a distance learning institution) it’s a critical system for us. We’ve recently put in place a new aggregated search system (Ebsco Discovery Solution) and we are going to explore how we can use our ezproxy search data to provide recommendations to searchers and see what difference it makes to their behaviour. Using data to provide recommendations are an everyday part of commercial life and taken for granted if you’re a regular user of services such as Amazon. But libraries (with notable exceptions such as Huddersfield) have been strangely reluctant to exploit the rich store of activity data that they can collect to improve the user experience.
RISE gives us the opportunity to see what we can do with the data in terms of making recommendations, what the challenges are, what we can’t do with it, and whether it is of interest to anyone else. It’s going to be an interesting few month. RISE will shortly have it’s own project blog and twitter tag #ourise and I’m aiming to blog some personal reflections and notes on how we get on here.