UKOLN
Raising Awareness

"A centre of excellence in digital information management, providing advice and services to the library, information and cultural heritage communities."

UKOLN is based at the University of Bath.

EASTER (Evaluating Automated Subject Tools for Enhancing Retrieval)

Background

EASTER study was a JISC-funded project. Further information is available on the JISC Web site.

Overview

Start date: 1 April 2009
End date: 1 September 2010
Funding: JISC Information Environment Programme 2009-11

The purpose of the project is to:

  • Test and evaluate existing tools for automated subject metadata generation;
  • Better understand what is possible and what the limitations of current solutions are; and,
  • Make subsequent recommendations for services employing subject metadata in the JISC community.

The rationale is based on the following arguments:

  • Subject metadata are most important in resource discovery, yet most expensive to produce manually. There is a huge challenge facing UK HE digital collections, institutional repositories, and aggregators of institutional repository content, as to how to provide high quality subject metadata for increasing numbers of digital information at reasonable costs;
  • Subject metadata are much more difficult to generate automatically especially in comparison to formal metadata such as file type, title, etc.;
  • Due to the high cost of evaluation, automated subject metadata tools are rarely tested in live environments of use.

Selected tools for automated subject metadata generation will be tested in two contexts: by Intute cataloguers in the cataloguing workflow; and by end-users of Intute who search for information in Intute as part of their research, learning, and information management. The approach will comprise the following steps:

  • First, a methodology for evaluating the tools will be developed;
  • Selected tools will be evaluated using a created ‘gold standard’;
  • The best tool(s) for the purposes of Intute will be implemented into a demonstrator that will feed its results into the cataloguing workflow; and,
  • A task-based end-user retrieval study will be conducted to determine the contribution of automatically assigned terms and manually assigned terms, each alone and in combination, to retrieval success (retrieving relevant documents) and failure (missing relevant documents and retrieving irrelevant documents).

Project partners

Non-funded supporting partners are:

Archive of Project Web Site

An archive of the project Web site is available on the Internet Archive. On 21 March 2013 the project Web site had been crawled 14 times, dating back to 21 July 2009.