This report provides an overview of the eLib programme, at the end of the first year of funding. It is based on the annual reports of thirty-six projects, the reporting period in most cases covering the first nine to twelve months of the project lifecycle. Five projects included in the study had been operational for less than six months.
The main sections of the report synthesise the data provided by projects on four topic areas: activities and progress, learning from the process of implementation, interim evaluation results and future developments. A concluding section comments on apparent strengths and weaknesses in the start up and early implementation phase of the programme.
For most projects, the main activities during the first year have centred on getting the project established: appointing staff, putting in place management structures and processes, developing the technical and legal infrastructure, preparing a work plan, promoting the project and gathering information from users.
Projects in each of the domain areas have undertaken similar sets of activities, although the focus and priority between them reflects such factors as maturity of the project, its positioning along an experimental-implementation continuum, the use of stable or advanced technologies, and the nature of the intended outputs.
The eLib projects have an impressive array of outputs from their first year’s activities. Many projects have developed demonstrators and prototypes that are either used as a talking point to elicit initial user feedback and to raise awareness, or which are currently being field tested as part of a more systematic evaluation. These include, for example, the first issues of on-line journals; the launch of databases with resource description numbers in excess of original targets; the piloting of electronic document delivery services and products in multiple test sites; the production and field testing of electronic course packs; and the design and delivery of training workshops and materials and awareness-raising events.
There have been several different kinds of learning in eLib. At the broadest level, involvement in eLib has allowed project participants to develop a better understanding of the organisational, technical and social nature of the innovation process. For some individuals, this has been a new awareness whereas for others it represents a deeper understanding of the uncertainties that are inherent in any innovative systems development.
At a generic level across the programme, many different lessons have been learned about setting up and managing a project, about the environment in which projects are operating, and about users and the context of use. Some of the more common lessons reported by projects include:
• the unanticipated difficulties of managing a consortium project over dispersed sites and the consequent need for the project manager to take an active role in building cohesion between the partners
• the importance of revisiting the vision or image of the project from time to time and being willing to temper the grand ideas with more realistic and pragmatic considerations
• similarly, acceptance of the need for flexibility and change, reflected in a workplan that is capable of being redrawn or modified to meet changing circumstances
• the importance of early demos or rapid prototypes for hands-on experience and early feedback from users
• the need to build user involvement, and to foster user ‘ownership’ of the innovation, into all stages of the development process
At the more specific level of individual programme areas, projects also report many shared lessons that are detailed in the body of the report.
Evaluation has commonly been undertaken as an integral part of the development process, entailing a variety of methods for the collection or systematic structured feedback from different user groups. The range of methods in use included on-line questionnaires, usage logs, observations of use, surveys, focus group discussions and informal feedback from users in training workshops.
For most projects, it is too early in the lifecycle to report on outcomes, effects and impacts. Even so, a number were able to report on interim evaluation findings from field trials and piloting activities. This was particularly the case in the areas of On-Demand Publishing and Access to Networked Resources.
With few exceptions, projects did not anticipate any significant change in direction or refocusing of the core concept or vision of their project. The common view was that continuous feedback from users provided an adequate means of adapting to changing user requirements in a dynamic IT environment. That said, the rapid pace of technology development had already overtaken some projects, prompting them to leap-frog over existing technologies to new Web-based solutions.
Envisaging future scenarios for the uptake of the project outputs was not a familiar activity to most project managers, and the majority had yet to develop their business case. We would expect to see the options for exploitation and a sustainable future explored much more systematically in the second year annual reports.
The report draws attention to the strengths and weaknesses of the development and implementation phases of projects during their first year. Among the issues considered are:
• the value of ‘learning by discovery and trial-and-error’
• the need to promote ‘scenarios of use’ as well as specific products and services
• the initial project emphasis on user reactions and user acceptance and the need to consider the wider criterion of usefulness in the later stages of projects
• the diverse and multiple methods used for evaluation and the development of on-line data capture methods, tools and protocols
• the general lack (with one or two notable exceptions) of a user-centred development approach whereby users are involved as active participants in the various stages of design, development and implementation
The Electronic Libraries Programme (eLib) was funded
by the Joint Information Systems Committee (JISC)
Page version: 1;
Web page maintained by
UKOLN Systems Team
and hosted by
UKOLN
- feedback to
systems@ukoln.ac.uk
.
Site last revised on: Tuesday, 24-Nov-1998 14:21:03 UTC
DC Metadata