Prepared by

The Tavistock Institute

Evaluation Development and Review Unit

Most eLib projects are now engaging with real implementation issues in a diverse set of organisational contexts. Projects are beginning to have effects in the programme’s external environment:

• eLib projects have an increasingly high profile in wider technical debates and are making a strong contribution to the development of international standards

• eLib projects have helped stimulate many companies within the publishing industry into taking positions on electronic information and into addressing and clarifying their perspectives on intellectual property rights, and on charging structures

• eLib projects are creating growing synergies with other initiatives including major EU R&D initiatives

• eLib projects are fostering networking and dialogue across the HE and the electronic information systems in all manner of unanticipated ways

Significant improvements can be observed in project management and development practices due to widespread adoption of iterative development cycle approaches, supported by formative evaluation. Project potentials have been further enhanced by

• building flexibility into requirements specifications, and by

• cross project borrowings and synergies, and linked supporting studies and other research.

Projects continue to encounter a number of obstacles to their progress, although these are not so pronounced or widely shared as last year:

• Some projects continue to struggle with the consequences of having formulated initial technical ‘solutions’ without clarity about the nature of the problem (or for whom), nor about the downstream consequences for uptake and embedding in user practices.

• Dialogue with key external communities (most notably publishers) continues in some cases to be fragmentary and weak.

• Continuing staffing problems, driven by a buoyant market for technical labour and the inability of projects to offer anything other than short term contracts.

• Continuing difficulties with team working in spatially dispersed partnerships, although often practical solutions are being found through, for example, concentrating development tasks on one site, and by working hard to improve communications.

As projects move in their second year to testing their projects and services with real users, a number of challenges in the external environment to the sustainability of projects are becoming clearer:

• The rapidly changing market environment creates uncertainty for projects about the future demand for their products/services, and perhaps emphasises the inherent weaknesses of HE projects in terms of entrepreneurial and capital resources.

• Copyright clearance problems create very significant and fundamental challenges for projects in areas like on-demand publishing and electronic reserve.

• There is an inevitable need for broader, complementary changes in attitudes and behaviour among librarians, academics and students, which projects themselves are rarely in the best position to influence.

• Overall, issues of training and awareness, and wider cultural change now appear more pressing than ever.

• nature and boundedness of the discipline or field

• skills/experience/expertise of staff

• synergy with other projects/developments

• nature of the material (grey, ephemeral, graphic etc.)

• stage reached in project lifecycle

During the second year, most of the ANR projects moved beyond the uncertainties and difficulties of the start-up phase and began to consolidate their position. At least three of the projects felt they had moved to a stage of becoming an operational service rather than a pilot demonstrator.

Successful strategies for service development have included a greater emphasis on engagement with user communities, especially as regards building partnerships in the supply and use of subject based resources, as well as additional emphasis on training and awareness and promotional activities.

ANR projects are beginning to clarify possible ‘exit strategies’ with most foreseeing some mixture of subscription income, commercial alliances, sale of related products and continuing core funding.

The ODP and ESC projects continue to confront significant operational challenges most notably:

• turning technologies of delivery which are available in principle into practical economic technical systems

• overcoming the myriad difficulties connected with copyright clearance, protection and charges

• reconciling the necessary functional limitations of ODP and ESC systems with the working practices and needs of course providers/teachers

Nevertheless, the ODP projects (which are in the main longer established than the ESC projects) have made great strides in clarifying the operational requirements of workable systems, and produced a significant body of evaluation results. However, it now appears that the capital investment required for ongoing systems may be beyond the capacity of any one project or small set of institutions. The longer term future of ODP/ESC initiatives in HE will be probably be primarily shaped by the current rapid changes in the attitudes of publishers towards electronic publishing.

The EDD projects are among the most ambitious of the eLib projects, both technically and in terms of service and organisation. As the EDD projects move into their second year, some projects have developed strong collaborative links with commercial partners and closer convergence and inter-operability has emerged between individual projects.

The EDD projects have uncovered the relative unreadiness of publishers when faced with the practical technical task of supporting electronic document delivery, and the projects themselves have begun to emerge as resources for other HE journal access initiatives. Nevertheless the future of the EDD projects remains uncertain, and they themselves raise questions about the appropriate role of universities in areas of complex technical development where the level of investment and associated risks are more suited to commercial software and systems companies.

Electronic Journals (EJs) has been the most innovative and speculative area of the eLib programme, and an enormous amount both technologically (concerning the actual technical form of EJs) and conceptually (as regards new modes of scholarly communication) has been achieved since these projects began their work. As is their function as demonstration projects these projects tell us much about what can be done now, what scope there is for building on to current capacity and the obstacles to exploiting current capacity.

There are a range of different but interrelated development areas which these projects are producing significant learning:

• Changes in practice in scholarly communication and its implications for scholarly communities depends to a critical extent on the degree to which individual disciplines already operate with a sound base of common IT skills; the various EJ projects have been influencing changes in practice yet continue to be constrained by current practice in various ways.

• Providing additional information to that normally contained in text and providing for more interaction will require building new platforms and environments with software which is affordable in a HE context; a number of EJ projects have engaged in developments in this area, providing useful learning for others similarly engaged.

• Electronic communication poses particular challenges to developing processes and procedures for quality control, copyright and ownership. Subscription/registration and cost recovery issues involve substantial interests for disciplines and print publishers, two of the EJ projects have focused particularly on this area, while the others have touched on it more tangentially.

• Increasing use of electronic scholarly communication depends on the development of a range of stimuli, such as effective citation and acceptance of electronic publications in research assessment exercises, as well as on timely training, advice and support, ways forward in these regards are offered to the wider community by the EJ projects.

• The recording and sharing of metadata which reflects different standards and quality control levels, particularly as regards ‘grey’ literature, will require some convergence between the area of EJs and other areas (such as ANR), a number of EJ projects are exploring aspects of this potential convergence.

Overall, it may be observed that the EJ projects are now achieving the status of less ‘pilot projects’ (representing lines of development which may be found wanting and abandoned) as ‘path finding’ projects whose learning is likely to influence a coming generation of EJ initiatives.

The three digitisation projects have successfully overcome or defined ways to overcome the inherent technical challenges in digitisation. Now as they approach the end of JISC funding, questions of sustainability are being raised both in relation to the level of demand and the basis of cost recovery. Towards the end of the reporting period, projects have been giving much greater emphasis to investigating the needs of potential users as a guide to better defining future service offers.

Having addressed the considerable technical demands of establishing specific imaging services, and the development of workflow strategies around the image acquisition process, the imaging projects have been more closely concerned this year with the use of their resources in practice by differing types of end users.

In their second year the T&A projects have continued with their existing programme of activities employing both cascade (train the trainers) and direct mass training approaches including;

• ongoing production and dissemination of training materials, including regular updating

• the delivery of workshops, seminars and on-line training

• ongoing (formative) evaluation and quality assurance

• the launch of new services

Generally, as their knowledge of trainee needs has increased the training offered is becoming more tailored and diverse. Demand for training continues to outstrip supply and the T&A projects are emerging as an important organisational capacity within HE quite distinct from their role within eLib itself.

As we found in the first overview of annual reports, projects provided a wealth of data about their activities, progress, achievements and findings. It has been difficult to do justice to the material, and our synthesis is no substitute for closer examination of the individual annual reports. In addition, projects have produced many other deliverables or outputs which amplify different aspects of their work, and these provide valuable information for those looking for exemplars of good practice, details on technical choices, baseline data and relevant market research, and manuals and procedures that encapsulate learning that has often been painfully, and painstakingly, acquired.

The pro-forma schema for the presentation of 1997 annual reports varied in only minor ways from the one used in 1996. Projects were requested to structure their reports around four key areas, each with a set of open-ended questions to be addressed (see Appendix 1 for the amended 1997 annual report format). These were:

• Activities and progress
• Learning from the process of implementation
• Interim evaluation results
• Future development

For our analysis of this year’s reports, however, we have adopted a different framework that is more appropriate to the lifecycle stage of the majority of eLib projects. Whereas our principal focus last year was on the generic activities associated with the start up phase, this year we have concentrated more on the individual domain areas. As projects have moved to a more stable, mature phase they have a great deal more to say about the experiences of implementation and the initial findings from their evaluations, both of which are largely domain based. The issues, problems and challenges pertinent to each domain are also becoming more apparent.

We have therefore eschewed a common format as the basis for a synthetic overview of each domain area in favour of a more pluralistic one. We have something to say about each of the four areas above, but we have not presented the findings of our analysis exclusively within this framework, nor is there complete consistency in the headings used for the different domain reports.

There are two main sections to this report. The first section draws together the themes and issues that run across the programme as a whole or are shared by a number of domain areas. Thus we report on the contribution of projects to achieving eLib programme goals, the important factors which are shaping the projects and enhancing their potential, and the obstacles to progress and longer term sustainability.

The second section provides a domain by domain overview. We try both to convey areas of commonality and difference between the projects, and also to highlight what seem to be the key themes, issues and findings for each domain. In the main, we have identified projects by name as this makes it easier for the reader to follow up a point of interest by reading the relevant project annual report.

The eLib projects are ranged across a spectrum from piloting actions in tested environments through to the delivery of services in real world contexts. The second year of the eLib programme has seen substantial movement along this spectrum with many projects now engaging with the real issues of implementation in different organisational contexts. As projects have become more stable and mature, so they have been able to direct project energies to making real progress towards their goals rather than being pre-occupied with internal project management and fire-fighting on technical glitches. As one project commented, ‘our progress has generated confidence within project teams, leading to a shared sense of mission and a will to overcome the problems which inevitably existed initially’.

First, the annual reports indicate a number of areas where eLib projects are playing a high profile in debates and have been able to contribute to the development of international standards. One notable area is the development of (Dublin Core) metadata standards and conventions for the Internet. Some projects are taking a strong lead in this, but nearly all projects were able to report on actions taken to embed metadata in their electronic resource materials.

Second, the eLib projects have served to mobilise many companies within the publishing industry into taking a position on electronic information and making more transparent their copyright practices and defining charges. Although inconsistency is still rife and there has been some reversal of more flexible attitudes, the fact that many more publishers now have developed, or are developing, policies in this area is welcomed by eLib projects. At a programme-wide level, it is now more apparent where HE libraries are best placed to negotiate at an institutional level and where there is potential leverage in looking to a centralist or consortium solution.

Third we see many examples where eLib projects have found new synergies with other developments, to the mutual advantage of both. Fruitful partnerships have been forged with EU funded projects in parallel areas of development, both on the technical side but also in the incorporation of standards, protocols and quality assurance procedures. An eLib project experimenting with digitisation of a leading journal in the design field, for example, was able to take advantage of an EU technology development project to test the delivery of quality text and images on high speed networks across Europe. Some projects have enjoyed similarly productive relationships with commercial sector companies, engaging in co-learning through joint production. A collaborative venture between an ANR project and two other agencies, for example, has led to the development of a shared social science Thesaurus, not previously configured in the project proposal.

Fourth, the eLib projects have fostered networking and dialogue across the system in all manner of unanticipated ways. Numerous projects have made effective use of the Web, not only as a means for dissemination their various outputs, but more importantly as a tool for stimulating debate, sharing information and learning. Whilst there is much still to learn about what makes for effective communication, it is noteworthy that reaching out to the wider HE library community is a goal shared by many, not just those in the Training and Awareness domain. Lis-discussions have been set up on such topics as metadata debates, strategies for copyright clearance, networked learner support and the use of EU documentation in European Studies courses.

The potential and relevance of projects has been enhanced by the widespread adoption of an iterative development cycle, drawing on formative evaluation. This has provided the basis for continuous improvement of the various products and services under development. We commented in the 1996 Synthesis Report on examples of good practice in this area (as well as some of the limitations of the approach taken by most projects), and the second round projects have largely followed suit.

Two further ways in which projects have enhanced the relevance and usefulness of their products and services and increased their exploitation potential have been 1) through building flexibility into the requirements specification, and 2) through cross-project borrowing and synergy, and links to supporting studies and other research.

We observed many instances where projects were taking advantage of new technologies to increase their capacity for exploitation. First and foremost, there has been widespread take up of Web technology in response to the growing universality of access to the Web. This has in many cases meant abandoning the original technical specification to take advantage of a networked version, or developing the two in tandem.

Another example is the common adoption of an open systems architecture. In an evolving and maturing IT market, such a toolkit approach makes it much easier to incorporate new features or to validate to new standards as well as opening up more market opportunities for a differentiated product. Various projects have been experimenting in this area, including making provision for changing the software base whilst maintaining existing data. Many of the software packages being developed anticipate the emergence of new solutions to problematic areas, such as for example charging systems, and can readily incorporate such changes.

Projects provided many examples of the benefits of being part of a wider domain area and programme, particularly where cross-project borrowing of outputs, technical solutions, strategies or methodologies, has added value to their own project.

Some projects have also benefited enormously from being located within a ‘centre of excellence’ bringing together a concentration of related projects. SOSIG, for example, is located within the Institute for Learning Research Technology which is co-hosting four other data-based social science research services. All benefit from the sharing of expertise, hardware and software as well as from interoperability - so providing a ‘virtual resource centre’ or one-stop shop for all social science research data.

The contribution made by supporting studies and other forms of linked research is also noteworthy. Projects that were grounded on a good research base, were informed by research undertaken in parallel, or which had adopted an action research methodology commented favourably on the contribution that research made to their understanding of the innovation process and the way in which it had informed their strategies and interventions. A good many projects learned to their cost that ‘user feedback’ downstream in the development path is no substitute for a good knowledge of the user population and the context in which the innovation is being introduced, at the start of the design process. Exemplar models of good practice in this respect include the ACORN project which was underpinned by in-depth user surveys and profiles, and SOSIG and OMNI which both benefited from a parallel research study of the effect of subject based gateways on the working patterns of end users and their changing use of the Internet.

Elib projects encountered a number of obstacles to their continuing progress in the second year, although not so pronounced or widely shared as those which dominated the first year of the project lifecycle.

The main obstacles identified by projects in their annual reports originated from three different sources:

• inherent in the design process;
• internal to project organisation and management;
• residing in the external environment.

Several projects have realised, in the course of their second year, that it was a mistake to have focused their project on a technical ‘solution’ without being clear either about the nature of the problem (or for whom), or about the downstream implications for its uptake and embedding in user practices. The design process was the classic, now discredited, linear one with implementation seen as a last stage, rather than as part of the first critical stage of analysing the future context of use.

Such a realisation was most clearly articulated by a project in the On Demand Publishing domain:

A second shortcoming in project design (inferred by us from reading the annual reports) which impeded progress was the lack of an adequate mechanism or procedure to support effective dialogue with publishers as stakeholders. There were notable exceptions here (ACORN for example, which set up a Participating Publishers Seminar) but our impression from reading the annual reports is that most projects dealt with publishers on an individual, case by case basis, rather than engaging with them as part of a broad process of dialogue.

For many projects, the main setback to their progress in the second year stemmed either from staff turnover and the time taken to fill a vacant position, or to an inability to recruit someone with the requisite skills. In cases where staff mobility involved the prime driver of the project, it was even more likely that the project would experience a set back. Staffing difficulties were a main difficulty in the projects’ first year (1996 Synthesis Report), and it would appear that the market for staff with the necessary skills is still very buoyant, with little incentive for people to be in short-term contracts unless the rewards are very high.

Several projects also experienced ongoing difficulty with geographic dispersal, especially where the technical task of systems development and integration was distributed between partners. This was resolved in some cases by consolidating development tasks and relocating them in one site, and in others by working hard to improve communications. Whereas this was a widely shared difficulty in the first year, by the second year many project team members had however become more familiar with a mode of collaborative, dispersed working. A comment in one of the annual reports well illustrates this point:

As projects increasingly move to testing their products and services in the ‘real world’ and have to think more about the post-funding stage, so they have become more aware of impediments or barriers in the external environment. Their evaluations are also highlighting the difficulties of implementation, especially cultural and organisational ones that were unanticipated.

First, various of the domain reports highlight the changing market environment in which eLib projects are functioning, creating uncertainty about the future demand for their products and services. Key actors include commercial companies, the publishing industry and database vendors and their direct entry into the networked information market will inevitably impact on the niche market environments in which eLib projects are operating. Elib projects will need to provide value-added services to the HE community if they are not able to compete on price, but it is unclear whether individual HE libraries will have the resources to buy many of the services on offer.

Second, for several domains, copyright clearance remains a significant bottleneck, requiring a different solution from the institution based approach taken by projects.

As we noted above, publishers are becoming more organised and many now have policies on copyright for electronic materials in place, but the net effect just in the lifetime of projects has been a step backwards. For the majority of projects, the request was limited to the duration of the trial with a narrowly defined group of students. The refusal rate for permissions has gone up, the charges in many cases have increased, and a substantial number of requests for copyright clearance have gone unanswered. Copyright clearance can thus incur high labour costs with no return. However, any consideration of the copyright issue needs to begin with the conclusions of the important supporting staff on On Demand Publishing produced in this period by a consortium of ODP projects.

Third, attitudes and behaviours among professional librarians, academics and students continue in many instances to be a barrier to the effective embedding of eLib products and services. Librarians ‘on the ground’ are reluctant about passing on the costs of electronic resource materials to students, academics are as yet unwilling to change well-established habits of preparing reading lists ‘just-in-time’ (but not ‘soon enough’ for the lead-time required for electronic delivery), and many students (and staff) still lack the confidence or the skills for effective use of networked information retrieval. On a more serious level, academics have concerns about questions of equity and access in the use of electronic materials as well as about the impact on student’s search strategies until there is a critical mass of material available on line. Librarians for their part are not convinced that in some areas of development, such as electronic short loan, their work load will decrease rather than increase because of the need to run parallel print-based and electronic systems.

One or two projects expressed hope that the Dearing Committee Report would provide a more hospitable climate for eLib project activities in the Training and Awareness domain, particularly with respect to the ongoing professional development of librarians and academic teaching staff.

PROJECTS: ADAM: Art, Design, Architecture & Media Information Gateway; BIZ/ED: Business Education on the Internet; CAIN: Conflict Archive on the Internet; CATRIONA II: Management of Electronic Information Resources; EEVL: Edinburgh Engineering Virtual Library; IHR-Info: Information Server for History;; OMNI: Organising Medical Networked Information; ROADS: Resource Organisation and Discovery in Subject-based services; RUDI: Resources for Urban Design Information; SOSIG: Social Science Information Gateway

The nine ANR projects are diverse in nature. The main group of seven projects provide subject-based gateways in a number of disciplinary areas and fields of study, predominantly utilising the ROADS software. ROADS is a distinctive project, set up to develop tools, methods and protocols that enable subject specialists to build and maintain information gateways. Catriona II also stands alone as an ANR project, its concern being the management and creation of institutional and departmental electronic resources in Scottish universities, and associated infrastructure and policy issues.

Given the synthesising and comparative aspect of this Report, the focus in this domain area is very largely on the subject-based gateways. These projects are primarily concerned about adding value to information through a process of selection, description, indexing and cataloguing, and through provision of ancillary services.

Our analysis of the second year annual reports reveals quite significant differences between projects in their achievements and the extent to which they have potential sustainable futures beyond the life of the eLib program. The reasons for this would seem to be many and varied, and can only be teased out of the reports. Our reading of the reports suggests the following to be important:

• nature and boundedness of the discipline or field
• skills/experience/expertise of staff
• synergy with other projects/developments
• nature of the material (grey, ephemeral, graphic etc.)
• homogeneity of the user base
• stage reached in the project lifecycle

During the second year, most of the ANR projects moved beyond the uncertainties and difficulties of the start-up phase and began to consolidate their position. At least three of the projects felt they had moved to a stage of becoming an operational service rather than a pilot demonstrator. We discerned a broadly comparable set of activities among the main group of subject-based gateway projects, although the emphasis given to each reflected their idiosyncratic natures and development path.

These were:

• ongoing development and enhancement of services

• training and awareness raising
• promotional activities including web site design
• evaluation

The achievement of projects in developing the subject-based gateways to information resources is evident from the table below which summarises the range of resource descriptions and services offered by three different ANR projects.

Expanding the number of resource descriptions in the database is a central activity of all subject-based ANR projects, achieved with differing degrees of success. Projects reported on and sometimes reviewed the effectiveness of different strategies for reaching their targets. Alongside more formal strategies aimed at mobilising users as information providers, projects also emphasised the importance of ongoing ad hoc activity in following up opportunities from discussions at conferences or meetings, or via visits to the project from other organisations. Factors limiting the growth rate of content acquisition and resource discovery were in one instance attributed to an inadequate level of staffing to follow up potential information sources and contacts, and in another to the extensive reallocation of the Resource Officers’ time to standards development work and associated quality assurance procedures.

Most projects have also devoted considerable efforts to enhancing their interfaces by fostering improved search possibilities.

Several projects have continued to invest significant amounts of time and effort in the development and delivery of training materials whilst one or two others have relied mainly on the Netskills project to raise awareness and develop competencies in use of the Internet. Biz/ed has effectively built on materials developed in Sosig to produce training material relevant to the project. Some projects do not have a training component at all, either because of the nature of the project or as a reflection of the stage reached in their development.

Among the developments in the way training is managed and delivered, the following are noteworthy:

• a shift to a reactive strategy as subject librarians and others increasingly request training, together with a proactive strategy in reaching out to new audiences
• more collaboration with other projects and related organisations in the provision of joint training, allowing for shared expertise and resources as well as more coherent provision to the end user community
• concern with quality assurance of training documentation, including in one project the development of standards to ensure consistency of style and approach
• a new emphasis on downloadable on-line materials, also available in parallel print form
• the introduction of new topics, including for example the use of metadata

Training events continue to be a main source of user feedback on ANR demonstrator services, captured through various evaluation instruments. The data indicate that workshops not only encourage participants to use the services more but also to ‘spread the word’ to colleagues and other users.

Evaluation activities vary across projects, but most commonly effort has been directed towards user feedback to inform continuous improvement of the demonstrator service and the analysis of usage data gathered through log files. In several cases, evaluation forms have been simplified and on-line surveys abandoned in favour of paper-based questionnaires. ROADS is also making use of customised defect tracking and enhancement tracking software which will highlight those areas of most concern to end users.

Although project evaluators are cautious about the use of raw usage figures, trend figures are seen as a more reliable indicator of performance or usefulness. The four projects presenting usage data reported a steadily upward trend, with fluctuations broadly following the cycle of the academic year. Peak usage figures included an increase in genuine user accesses to 120,000 per month for SOSIG, and to 11,000 per week for EEVL.

The more fruitful evaluations have a longitudinal dimension that allows changing patterns of Internet usage among workshop participants to be mapped. By gathering baseline data on participants’ profile of computer and Internet usage prior to he training workshops, two of the projects with a shared evaluator have been able to assess how the workshop has changed participants’ use of the Internet as well as to explore the effect of subject based information gateways, over time, on the working patterns of end users.

Descriptive analyses of the data suggest that attendees at SOSIG and OMNI training workshops have significantly increased their use of subject-based services since attending the workshop. They have also increased their use of other Internet services. This increase in use exists regardless of the different levels of reported ability and use before attending the training.

The picture which emerges from follow up interviews with participants is that most do not plan a search beforehand, use only one word, less often a phrase, and only few people use Boolean connectors or truncations. Searches are most often based on starting from a known address and develop as they go along by modifying or adding words, using synonyms or proper names. The addition of the Thesaurus to the interface of several projects is expected to assist those who use a single search term.

A main development reported by several projects related to the integration into the service of a Thesaurus to assist users searching the database to refine their search terms:

• a collaborative venture with IBSS and the Data Archive has led to the development of a shared social science Thesaurus with a Web interface (SOSIG)
• OMNI has incorporated an experimental version of an interface to the OMNI data using the UMLS (Unified Medical Language System) for thesaurus searching
• EEVL proposes to investigate the implementation of a Thesaurus to the main database

• Biz/ed have developed a more focused search engine based on the free Harvest software to allow actual text or catalogued resources to be searched.

ANR projects have also implemented classification schemes to allow more efficient indexing and searching:

• Resource discovery and cataloguing carried out by ADAM has ensured that records are indexed with controlled terminology from the Art and Architecture Thesaurus and the Union List of Artist’s Names, and classified using the Dewey Decimal Classification system.
• The selection of Universal Decimal Classification numbers used by SOSIG has now been extended to cover a wider selection of social science subjects. This will allow for more specific browsing and should help manage the increase in the number of resources.

Several of the ANR projects have played a high profile in the metadata debate and in the development of the Dublin Core (CD), a metadata standard which attempts to provide a common core set of elements which can be applied to a variety of information sources. Metadata, in the form of resource descriptions entered and indexed in a database, can then be browsed or searched by end-users seeking to locate high quality Internet resources in relevant subject areas. Although projects are following down the DC metadata path, one project questioned aspects of the usefulness of DC metadata in the context of its database service whilst another is maintaining contact in parallel with developers of the Data Documentation Initiative which potentially offers a different international standard to describe social science datasets.

• The ROADs software (in its default configuration) is conditionally Dublin Core compliant and also provides additional elements to handle administrative metadata not included in DC. Work on ROADS Harvesting supported the development of an experimental Dublin Core (DC) metadata generator, DC-dot, which will retrieve Web pages and automatically generate formatted Dublin Core suitable for embedding in the headers of HTML data. Further work on interpretability and standards development allows different metadata formats to be inter-converted.
• The ADAM project was among the first to embed DC metadata within the HTML conversions of project documentation.
• SOSIG is encouraging information providers to use DC when creating resources which may then be included in the SOSIG database
• EEVL pages produced conventionally now have DC core data attached and a script has been produced that would allow delivery of the EEVL template information as metadata. However, the feeling of the project is that the quality of service that can be offered by building descriptions of resources is unlikely to be reached on a system using hidden metadata.
• RUDI - metadata is being used to label basic HTML documents with, for example, subject keywords, author names, and place names. Each of these metadata fields can be used to focus a search so that the user can be much more confident of receiving a useful set of results. The degree of relevance in this case is instilled in the data by subject specialists, rather than relying on the vagaries of the ranking algorithms built into most search engines.

In parallel, additional efforts have been made to assure the quality of information resources:

• New work has been undertaken to formalise the SOSIG selection procedure. The SOSIG researchers co-authored the report ‘Selection Criteria for Quality Controlled Information Gateways’ and used this research to develop a Scope Policy and a set of Selection Criteria for SOSIG. These documents make explicit the methods for assuring a high quality controlled gateway, and can be used to guide new staff and subject librarians selecting resources for the Gateway.
• OMNI set up an Advisory Group for Evaluation Criteria which has continued to inform the project’s strategy in the expansion of its database of resources. A web base for the AGEC was created, giving integrated access to OMNI’s evaluation criteria, information about the group, and online versions of published articles. The existence of an independent advisory group is a clear demonstration to users of the seriousness and care taken in evaluating resources for OMNI.
• ADAM has formulated a cataloguing strategy, covering such aspects of the record creation process as: the collections policy, guidelines for resource collection, the use of controlled terminology resources, the adoption of a classification system, and any cataloguing rules used

• CAIN has enlisted the help of academics in supervising the identification and selection of material for inclusion. The resource currently contains not only bibliographic records help by the libraries of the partner institutions but also the personal bibliographies and databases of researchers active in the field.

The ROADS team has been responsible for much of the early deployment and concrete syntax specification for the Dublin Core, in addition to hosting the main DC working group forum. This illustrates that the UK community of librarians and other information professionals can play a leading role in the development of metadata standards and conventions for the Internet.

As tools have become available for validating to later standards, projects have taken advantage of them. Careful attention to detail in OMNI, for example, has ensured that it is possible to use the OMNI service from both the latest version of Netscape and text-based browser such as Lynx.

Projects encountered a number of difficulties in the second year of operation, although these were not dissimilar from those reported in the first year annual reports. The main areas of difficulty reported by ANR projects concerned technical matters, staff and project management, and copyright.

Some critical lessons were learned by ROADS users as by ROADS itself Although many of the difficulties encountered by subject gateway project based on ROADS were experienced as technical shortcomings of the ROADS software, their origin was often socio-technical in nature and stemmed from inherent tensions in the development process. The conclusion drawn by one project that it was a mistake to base the service around a piece of software under development may be a valid one, but at the same time ROADS is also right in its analysis of the collaborative/symbiotic relationship which existed between it and the ANR services and the points in the development cycle where consensus broke down. ROADS sees in retrospect some possible strategies that might have dealt with differing expectations and conflicting requirements.

Several projects continued in the second year to be disrupted by staffing problems, with key staff leaving the project bereft of necessary leadership or technical expertise. The withdrawal of one of the partners slowed progress in one consortium, whilst another found that significant changes to the staffing arrangements have meant that roles, working practices and communication protocols all had to be reallocated, redesigned and relearnt. A few projects experienced ongoing staff management difficulties arising from the geographically dispersed nature of team members and the shortcomings that stem from having to rely on formal reporting structures to monitor progress and take corrective action rather than on direct first hand observation.

Copyright clearance was a problem for CAIN, relating to the particular type of grey literature included within its resource materials. Offsetting this difficulty has been a positive response from publishers to the use of substantial sections of text, including chapters from books, accessible from the gateway.

A number of the ANR reports commented on their perceptions of the changing market environment for subject-based gateways and the possible future for their project within this context. Among the developments noted were:

• the appearance of a growing number of free versions of datasets (e.g. Medline) on the Internet which were not foreseen. Despite the superior provision of services enjoyed by many in the HE/research community via locally networked CD-ROM products (such as OVID and Silverplatter), there appears to be a great deal of interest in these Web offerings.
• Library automation and database vendors are beginning to view networked subject based services as a potential market segment and it is fairly safe to assume that most OPACs will soon have the ability to have a Web front and direct links to online resources (if they do not already have them).
• closer collaboration between ANR projects, commercial database providers and others. The extent to which Subject Gateways are able to include publishers’ web sites for individual journals is seen as important for their longer term success.

The ANR projects envisage different futures for themselves, tempered in part by how they see the longer term market environment. None of the projects has a well developed business plan, although they are beginning to identify what might be the key elements. Two projects have set income targets for the coming year.

The main sources of ongoing financial support, which would sustain projects to varying degrees, include: subscription income; revenue from commercial alliances for on-line services; income from products such as publications and events, consultancy and advertising; and central funding from JISC and relevant Research Councils and professional bodies. Most projects envisage operating in a mixed economy.

ROADS sees the potential to operate in a commercial environment beyond the ANR information gateways, by extending its toolkit and development services to a wider set of clients both in the UK and beyond. Within the market domain of subject gateways, ROAD maintains there is definitely a place for a free (or at least low cost) toolkit. It is likely that development will continue on an unfunded basis e.g. under GNU Public License and/or supported by charging for support.

EEVL has also given consideration to producing a more generic version for use with other services, but the project is aware of other systems such as ROADS and Isearch and other commercial database access routes now possible through the Web. Conversely, EEVL believes that if other systems offer features which cannot be incorporated into the EEVL software, it would be possible for EEVL to change its software base whilst maintaining its existing data. This has already been demonstrated experimentally by converting the data to operate with the beta release of ROADS v.1.

PROJECTS. Edbank; On-Demand Publishing in the Humanities (was ‘Only Connect’); SCOPE: Scottish On Demand Publishing Enterprise; ERIMS: Electronic Readings in Management Studies; Eurotext: A Collaborative Resource Bank of Learning Materials on Europe

Within the ODP domain, seven eLib projects have been set up. One of these has already been completed whilst another only began in 1997 and so is reporting mainly on start-up activities. This synthesis of ODP project reports focuses for the most part on the four projects for which second year annual reports are available.

The OPD projects are engaging with a similar set of issues although the underpinning strategy varies from one to another. Two projects are primarily subject based, another is cross-institutional, a third is document source-based and a fourth might be described as system based (a Scottish consortium of 13 members). They range from those that are still grappling with technical issues to those that are mainly concerned with implementation. The projects are experimenting with different systems for supplying materials electronically, assessing the effectiveness of supply-side approaches to generating a bank of resource materials, testing users’ preferences and exploring the benefits from cross-institutional collaboration.

At the broadest level, the ODP projects raise a number of key questions about the viability of ODP, taking account not only of how publishers are responding to a changing electronic publishing environment but also of user (students and academics) behaviours and attitudes. To anticipate what seems to be emerging as the key finding from this group of projects, the costs of delivering ODP appear prohibitively expensive for a small-scale operation. To achieve economies of scale, with respect to the copyright clearance process as well as to digitisation, some centralised model would seem to be the way forward.

Whilst ODP projects have undertaken many of the same activities, their main preoccupations during the second year reflect how far they are along the path to implementing a demonstrator or full service in a real user environment.

The various activities reported on by projects include:

• the different stages involved in materials/resource development for online delivery

• developing systems for copyright management
• implementing field trials
• evaluation
• promotional work

Several of the ODP projects are faced with converting hard copy materials into a digitised form. These include printed texts such as EU documentation, lecturers’ notes, and other course materials with a range of text, graphics and numerics. The scanning process involves a number of steps - scanning documents, re-formatting, spell-checking and proof-reading. This process has proved immensely time-consuming (and therefore expensive) as well as posing various technical challenges.

There is debate about what are the most appropriate database formats, with different approaches evident among the eLib projects. Projects seem to be equally divided between a preference for HTML and for PDF files, each having its own advantages and limitations. HTML for example, offers the opportunity to create hot links to WWW sites - an important consideration for updating database references. It is a much more expensive process however than mounting materials as simple PDF files. There is a view that PDF is emerging as the de facto standard for full-text documents on the Internet and it would appear that publishers prefer PDF and may therefore be more likely to give permission for their material to be used in this format.

Obtaining permission to digitise material belonging to a third party and make it available for electronic access and/or distribution has been one of the biggest and most common problems eLib projects have had to face. Electronic clearances can be very time consuming and expensive and many of the projects under-estimated the amount of time and money that copyright clearances can take. (Experiences and lessons concerning copyright matters are taken up below).

Actions taken by project to facilitate copyright clearance include:

• development of model agreements for use with publishers (with a view to the longer term, not just the short term life of the project)
• secondment into the project team of a Copyright Officer
• installation of a software system Permissi for development of an institutional database on copyright. The system uses templates to create permission request letters and clearance forms, and as well records all applications made for use of third party material.
• ongoing liaison with publishers, providing convincing evidence of on-line data security

ODP projects have taken different strategies to coursepack and database development. Those consistent with a demand driven model include:

• the compilation of full-text resources in a focused subject area, based on a selective combination of reading lists, short loan collections and lecturers’ own source material. Students may be guided to these resources via reading lists - either from the library system or from a reading list on a web server, or via personal communication from librarians or teaching staff.
• the consolidation of a readings database in the Management Studies area containing a large corpus of bibliographic records and scanned literature, based on shared Reading Lists from participating institutions

On the supply side, strategies have included:

• the mounting on a server of existing Open Learning Foundation materials, digitally re-formatted as materials available for integration into existing courses
• providing access to electronic resources in a full-text database comprising materials scanned in, including new books from ‘core’ publishers, electronic journals, and documents (such as EU documentation)

All the ODP projects have made attempts to integrate database materials into library catalogue systems, whether through their OPACs on the Web or a reading list on a Web server. One project is working to build in search facilities to support document retrieval on a subject basis in addition to the reading list mode of access. Linking in this way has presented a serious technical challenge and has to date defeated the institutional partners in most projects.

Encouragement to eLib projects to embed Dublin Core metadata in electronic resource materials has been taken seriously by the ODP projects, with most reporting that this presented few difficulties.

Strategies for copyright management adopted by the ODP projects include both educational and technological approaches.

• On the educational side, one project organised training sessions for course managers, aimed at increasing their understanding of rights and copyright and how these impact on their work.

• Other projects have pursued the technological approach to copyright management and control, mainly through the development of security systems on the one hand and billing and tracking systems on the other.

The SCOPE project for example has been working on the development of a document delivery system incorporating five levels of security for network transmission and on-line delivery. The software system is cross-platform and will not require libraries to invest in additional hardware. The On Demand Publishing in the Humanities team has adopted a four layer strategy based on IP addresses, access passwords to the LAN and course material, and simple stenography with HTML pages encoded with user ID. EDBANK controls access to the computer network through the central administration system. Publishers have made it clear that they consider IP address restrictions to be very important especially when delivering materials overseas.

ODP projects have experimented with a number of different subject areas including Management Studies, Education, Sociology, Economic, Engineering, Nursing and European Studies. In several cases, the intention has been to collaborate at a cross-institutional level both in the development of a shared database of published (and in-house) bibliographic

material as well in the sharing of teaching materials that have been put together by individual lecturers. Other projects have adopted a more localised approach, offering the facility of customising a set of resource materials that can be integrated with other sources.

An ongoing task for all project teams has been gathering reading list information, and ensuring that these have been updated to take account of new developments in the subject and new publications.

Various strategies have been adopted for increasing academic involvement in the coursepack/database development process. In Eurotext, for example, academics involved in teaching European modules were invited to participate in an Academic network. A closed list was set up on Mailbase, with a view to involving network members in identifying the key documents in their specific areas of expertise or to supply copies of their reading lists.

Some projects have found it necessary to adjust their strategy. In one institution, the preferred approach has been to liaise closely with individual academics, whilst in another the preferred approach has been to invite academics to joint the project team so that are not merely a client of the service but have a voice in its development.

The ODP projects have given prominence to evaluation, both formative and summative. Among the methods used are: focus groups with students and librarians, questionnaires and on-line evaluation. As a domain area, the ODP projects are oriented more towards implementation and the evaluations reflect the projects’ concerns with assessing impacts and effects - in areas such as the acceptability of online delivery of materials, changing patterns of library use, students’ searching strategies and use of materials, substitution effects (electronic resources for sales of texts) and so forth.

The ODP second year annual reports make little mention of promotional work, other than in a negative sense. One of the projects, reflecting on its experiences, noted that the need for liaison, marketing and publicity skills which are essential for a new project’s development were overlooked.

In common with the first year projects (as noted in the 1996 Synthesis Report, the new project in this domain, EDBANK, has been active in profiling itself and generating interest in its activities.

Notable achievements of the ODP projects can be seen in the following Table:

Some of the main project findings from evaluation studies are presented in the Table below. These are reported as individual project findings, although in places we have indicated where they seem to be common to other ODP projects on the one hand, or to suggest significant differences between projects on the other.

A key issue for the ODP domain concerns the behaviours of publishers in the maturing electronic publishing environment. The SCOPE report has observed that ‘mobilisation in this area is evident in that many more publishers now have developed or are developing policies on licensing their copyright materials for electronic use in academic libraries’.

Unfortunately, this has not meant a more favourable environment to support the growth of ODP. Among emergent difficulties noted by ODD project are:

• an increasing tendency for publishers to refuse permission to publish
• an unwillingness to grant further permissions for use of materials beyond the agreed licence rights
• increases in the rate charged for copyright materials
• a shift away from transaction-based payments

Some publishers it seems are keen to make decisions about licensing with reference to the forecast effect on sales of the course text at the institution for which an extract is selected.

Another development impinging on projects such as ERIMS which are building up a database of bibliographic resources is the launch by big commercial publishers such as Blackwells and Elsevier Science of their own systems for delivering journals in electronic format. In the long term, this is likely to mean that books will be more pertinent to the requirements of a system such as ERIMS.

At the broadest strategic level, the development of a more mature market for EP materials, as evidenced by the mobilisation of the publishers, suggests the need for libraries/institutions in turn to negotiate from a position of strength in numbers. It is instructive that SCOPE, negotiating on behalf of 13 Scottish consortium HEIs, has found the terms it agreed with publishers to be increasingly weakened in this new environment. The proposal for a centralised body to deal with copyright clearance would seem to be consistent with the experiences of ODP projects in this area.

Some of the projects in the ODP domain saw their project in primarily technical terms. It has become increasingly evident to these projects, as they moved progressively towards implementation and uptake of their coursepack materials, that the challenge was above all one of organisational and cultural change. Most importantly, the lack of a strong end-user involvement at all stages of the development process (a common feature of projects which we noted in the 1996 Synthesis Report) has proved a major inhibitor to project success.

PROJECTS. ACORN: Access to COurse Readings via Networks; ERCOMS: Electronic Reserve Copyright Management System; PATRON: Performing Arts Teaching Resources Online

Five projects were funded by eLib within this domain, most of them starting up in the second phase. Thus the annual reports tend to reflect the issues that characterise the start-up and early implementation stages of a project life-cycle and which were clearly documented in the first Synthesis of Annual Reports. In this section, our analysis is based on three projects - of the remaining two, one had finished and the annual report of another was not available.

Each of the projects in this domain has some unique or special feature. Thus ACORN has involved a key publishers as project partner, enabling the project to explore the role of a third party agent in the rights management process. PATRON is unusual in dealing with multi-media in the performing arts area, raising not only additional technical issues pertaining to audio and visual but also to the more complex rights issues. ERCOMS’ aim is to try and ease the copyright negotiation process by building a generic electronic copyright management system for electronic reserve systems, to be tested in two partner institutions.

Aside from the technical challenges of systems development and integration, copyright is the main bottleneck for the electronic short loan projects (in common with other domains such as ODP and EDD). Managers have to negotiate with publishers for electronic use of copyright works and publishers are not prepared to give permission unless some means of controlled access and tracking of the use of their resources on the institutional network is in place. Whilst tracking copyright ownership has always been quite problematic, it is heightened in the electronic age. The projects’ experiences in this regard give some indication of the downstream implications of publishers’ present attitudes and requirements.

The three ESL projects have been engaged in a broadly similar set of activities relating to:

• user requirements assessment and related market research
• systems development/integration
• copyright clearance
• building up the electronic collection
• publicity
• evaluation

Most of their time has been taken up with the first three on this list, and our overview of project activities addresses these priorities. At a more systemic level, PATRON is an excellent example of a user-centred development process, with focus group discussion proceeding in parallel with development of the user interface and underpinned by a multi-stranded evaluation. This is noteworthy in view of our observation in the 1996 Synthesis Report of a widespread lack of awareness of user-centred design (and not merely user feedback).

In seeing user requirements assessment as a key activity, the three projects have laid a strong base for the eventual uptake of their product and services. Examples of the techniques adopted for identifying user requirements, and the functionalities of the system, are given in the Table overleaf.

Rights clearance is particularly complex for PATRON, given that each of the media has a different set of associated rights reflecting different industry and historical backgrounds. For example, the concept of fair dealing is not widely accepted, and there are key intermediary bodies namely the rights collecting bodies. This project has negotiated its way through the minefield of rights through discussion of a proposed course of action (as recommended by professor Charles Oppenheim) with relevant industry bodies, rights associations and the publishers itemising the works concerned. PATRON has sought to establish agreement wherever possible with industry bodies, so eliminating arduous and time-consuming negotiations with individual rights holders. The approach adopted has been to suggest that publishers become participants in the pilot, offering them experience of multi-media and rights management with a well-defined group of users for a specified time.

ACORN has devoted a great deal of time pursuing copyright permissions from publishers and other copyright owners. An output of its work has been A Manual of procedures on gaining copyright clearance for journal articles, and the entire permissions seeking process has been analysed in detail in the Phase One Permissions Report.

On a technical level, the ESL projects have experimented with different solutions to copyright management, although there is ready acknowledgement that these are at a very early stage of development:.

• ACORN designed and developed its own generic system for the delivery and management of electronic journal articles. The system has five levels of security in order to protect the copyright and moral rights within the documents.

• PATRON expected to incorporate early versions of encryption and watermarking into IBM’s digital Library system, even though not yet available commercially. However, concerns at the level of security with the IBM system and the lack of encryption has prompted the project to investigate the use of ‘Multimedia Protection Protocol’ for audio from Fraunhofer Institute.

• ERCOMS is developing a cost-effective solution for copyright management which will provide controlled access to copyright material, track usage of documents, handle charges and payment by users, provide usage feedback for publishers, and enable libraries to evaluate usage of the electronic reserve system over a period of time. Document security will not be incorporated in the first prototype although the technical partner in the project intends to investigate watermarking and encryption techniques in a subsequent phase.

The ERCOMS package makes provision, among other things, for different approaches to ‘defining charges’ such as licensing or metering. In addition to these simple models, the ERCOMS annual report identifies a number of other common business models that could be the basis for a charging system. Ongoing development is expected to make it possible to include within the ‘defining charges software module’ more creative business models as they emerge.

The findings that are emerging from the early experiences of the ELS projects relate broadly to three areas:

• experience with copyright clearance
• cultural change issues
• demand and usefulness

The three ESL projects have had varying success in gaining copyright clearance for the material to be included in the electronic reserve collection. It is noteworthy that difficulties were experienced, even when it was established that clearance was sought for the duration of the trial only and for access to a limited number of students. As noted above, two or the projects actively sought publisher involvement in the trials as an opportunity for both parties to explore the issues in a safe, testbed environment.

The Table below summarises the experiences of the three projects, each of them dealing with a range of copyright (and non-copyright) materials. In the case of ACORN and ERCOMS, these included books and journal articles (primarily text but also figures and graphs) in a range of disciplinary areas in two different universities; in PATRON’s case, the material included audio and visual (dance).

The ACORN project is very firmly grounded in an understanding of current user behaviours (both students and academics) as these relate to short loan collections, and the changes that are required to accommodate the concept of electronic collections.

• the present practice among academics of updating their reading lists immediately before (or after) the start of semester will need to change in order to accommodate the lead in time required for copyright clearance and digitisation

• the publishing culture is dynamic and diverse - tracking copyright ownership is onerous and made difficult by the failure of publishers to provide up to date contact information (of 84 publishers approached, the contact information for 45 of them was incorrect); there was no consensus on charges. This suggests the need for a central clearing house if the benefits of electronic short loan collections are to be experienced by libraries

• library staff are concerned that an electronic service may have to run in parallel to a paper collection thus increasing rather than decreasing work load; the proportion of short loan material currently available electronically is only a small fraction of the total and thus scalability in terms of the stress it would place on the university IT structure is a concern.

Only ACORN is currently in a position to report on user feedback on the service provided in a real institutional context. Usage statistics, shortly after launch, revealed:

• just over one-quarter of target students actually used the service
• the most popular access to the articles was via the library, despite far greater availability of services machines elsewhere
• the third most popular use of the service was for wider reading

The three ESL projects see different futures for themselves, reflecting their different market niches.

ERCOMS has an elaborated business plan that identifies possible options for the commercialisation of its system based on projections of software sales and incomes. It sees the most appropriate exit strategy to explore a commercial partnership with a supplier experienced in library and publisher software markets.

PATRON believes the potential exists for applying its interface in other multimedia areas such as linguistics where audio and text are used concurrently. It is currently preparing an exit strategy and business plan.

ACORN highlights the problem of scalability and believes the sustainability of the service in its current form would depend on a reduction in digitisation costs, perhaps through consortia rates, and the reduction in permission seeking costs perhaps via a central clearing house.

PROJECTS. EDDIS: Electronic Document Delivery; SEREN: Sharing of Educational Resources in an Electronic Network in Wales; Infobike: Browsing and Reading Access

Five projects were funded by eLib in the EDD domain, and this overview deals with the annual reports of three of them. One project, JEDDS, was completed earlier and did not furnish a second year annual report whilst the report of another project was still forthcoming.

The EDD projects are among the most ambitious of the eLib projects, both technically but also in service and organisational terms. EDD represents a very real challenge and the way forward for some projects lies in strong collaboration with commercial partners rather than a solo effort.

A main lesson to come out from the EDD projects has been the need for closer convergence and inter-operability between the individual projects in order to achieve a real value-added service. This has been recognised at an eLib programme level, and a measure of rationalisation and convergence has already occurred.

At a more strategic level, the experience of the EDD projects raises questions about what is the appropriate role of universities in areas of complex technical development where the level of investment and associated risks might more appropriately be undertaken by commercial software companies. It is also significant to note the overlap, and possible competition, between eLib and EU funded projects and the desirability of gaining synergy between these, as is now the outcome for EDDIS following its merging with DALI.

The realignments or redirections which have occurred have been welcomed by the projects as a fruitful way forward, notwithstanding some loss of autonomy or disempowerment for the university as partner in the case of EDDIS. A shared view is that the tasks taken on by projects were more complex than had been imagined, and that closer cooperation and convergence had the potential to benefit all parties. The decision to merge the Infobike and Higher Education Electronic Journals Access Service projects into a single project is expected to deliver clear cost savings as well as clarifying the roles of both services.

6.3 Achievements

The achievements reported by the EDD projects are mainly technical, reflecting the efforts that have been put into this area of project development. They include:

• the successful delivery of articles, identified from a search of the bibliographic materials in the BIDS ISI service, in their entirety to the user’s desktop. The linkage is now part of the generic BIDS Web Interface and the line to the electronic content will be provided from more and more search interfaces (Infobike)

• ISO/LL implementation - among the first example of practical implementation in the UK (EDDIS)

• collaboration with JEDDS for document transmission such that the technical requirements and planning have been worked out between the two commercial companies (FDI and RGL) and the two project plans aligned to schedule delivery of the JEDDS model for testing by EDDIS in 1997 (EDDIS)

• the delivery of the SEREN software, involving a shift from a PC solution to a web-based one.

For SEREN, the main challenge confronting the project is organisational. The complexity of the institution building task was not appreciated at the outset and the imbalance in project activities towards the technical has left the project in a vulnerable position in terms of uptake of the service and its embedding in institutional cultures. No contact was made with the broadly based user group of participating institutions (beyond the lead partners) from the time the proposal was submitted until Spring 1997 and so involvement and contribution has been significantly lacking.

Infobike also faces difficulties in the delivery of electronic content, although from publishers rather than consortium members. This it attributes to the immaturity of the market, whereby the only publishers geared up to supply bibliographic header material, article content and subscription content relating to articles are those who already have their own electronic content delivery systems. The vast majority of publishers are having great difficulty in providing this information. Even Blackwell Publishers, with over 100 journals and a competent IT department have taken several months to load 200 issues of their journals.

As a result, Infobike has had to expend unexpectedly large amounts of development effort in massaging electronic content data, particularly subscriptions data, into the necessary formats for JournalsOnline leading to delays in providing a critical mass of electronic content within the market place.

For EDDIS, the addition of a commercial partner (FDI) to the EDDIS Board , although generally seen as a good thing, has introduced complications for business planning. Whilst the EDDIS product has been loosely spoken of a ‘free’ to the HE community, it will likely require not insignificant sums to sustain it as a durable product in a price sensitive market.

Both EDDIS and SEREN have reported difficulties with Z39.50 protocol and targets. In the latter case, the problems are technical and relate to the server, whilst for the former the difficulties stem from the unwillingness and incapability of third parties to provide Z39.50 targets. The availability of such targets is essential is EDDIS is to provide useful service, but is outside consortium control.

Two projects have undertaken evaluations as part of an iterative development cycle and the third plans to undertaken acceptance testing of the software and in-use evaluation of the product.

Evaluation findings from focus groups discussions in SEREN highlighted the problems that the project faces in not having consulted users more fully at the outset, Users expressed uncertainty about what its added value would be and concern at possible negative indirect effects as a result of introducing competition with BLDSC.

In the case of Infobike, the evaluation indicated user satisfaction with the navigation interface and document delivery, but negative comments on presentation, content and web presence. These findings have prompted a ‘makeover’ to the JournalsOnline interface that provides a more attractive ‘look and feel’ and more ‘punch’.

The future for the EDD projects is uncertain, given their stage of development and the significant shifts in direction which have occurred to date. Whilst the business case has been explored in one instance, the annual report notes that there is no requirement from eLib that projects should be self-supporting operating after the project, less so a profitable business.

SEREN is exploring the possibility of integration with the public library network, so providing an ILL mechanism for Wales. The funding model for this is unclear however, and no business case has been developed. Some possible spin-offs from the SEREN software suggest possibilities for a document distribution service around site libraries, and the adaptation of the ordering side of the system to provide a management system for LAMDA.

Infobike has not looked to the future beyond delivering Phase 2 which provides third party access to journals online through standard interfaces, thereby facilitating connection between JournalsOnline and services wishing to use it as a document supplier.

EDDIS is developing a business model that attempts to reconcile eLib’s requirement for free software, FDI’s requirement for a return on investment and the consortium’s requirement to be self-supporting. A business operation could be based around support, maintenance and customer services, and the elements of a marketing plan are being explored .

PROJECTS. CLIC: A parallel electronic version of an established journal - Chemical Communications; Internet Archaeology: an international electronic journals for archaeology; PPT: Parallel Publishing for Transactions; The Superjournal project: Development and testing of infrastructures and tools for Electronic Journal Publishing; Electronic support for Scholarly Communication; Electronic Seminars in History; Electronic Reviews in History; DeLiberations on Teaching and Learning in Higher Education; NewsAgent for libraries: a personalised current awareness service for library and information staff; JILT: The Journal of Information, Law and Technology; Open Journal: The integration of electronic journals with networked information resources; Sociological Research Online:

Electronic Journals (EJs) has been the most innovative and speculative area of the eLib programme, and an enormous amount both technologically (concerning the actual technical form of EJs) and conceptually (as regards new modes of scholarly communication) has been achieved since these projects began their work. As is their function as demonstration projects these projects tell us much about what can be done now, what scope there is for building on to current capacity and the obstacles to exploiting current capacity.

It is difficult to do justice in such a short synthesis to the range of project learning and interested readers are strongly directed to the individual project annual reports (available on the Web). However it is worth noting here the sheer range of different but interrelated development areas which these projects are producing significant learning:

• Changes in practice in scholarly communication and its implications for scholarly communities depends to a critical extent on the degree to which individual disciplines already operate with a sound base of common IT skills; the various EJ projects have been influencing changes in practice yet continue to be constrained by current practice in various ways.

• Providing additional information to that normally contained in text and providing for more interaction will require building new platforms and environments with software which is affordable in a HE context; a number of EJ projects have engaged in developments in this area, providing useful learning for others similarly engaged.

• Electronic communication poses particular challenges to developing processes and procedures for quality control, copyright and ownership. Subscription/registration and cost recovery issues involve substantial interests for disciplines and print publishers, two of the EJ projects have focused particularly on this area, while the others have touched on it more tangentially.

• Increasing use of electronic scholarly communication depends on the development of a range of stimuli, such as effective citation and acceptance of electronic publications in research assessment exercises, as well as on timely training, advice and support, ways forward in these regards are offered to the wider community by the EJ projects.

• The recording and sharing of metadata which reflects different standards and quality control levels, particularly as regards ‘grey’ literature, will require some convergence between the area of EJs and other areas (such as ANR), a number of EJ projects are exploring aspects of this potential convergence.

Overall, it may be observed that the EJ projects are now achieving the status of less ‘pilot projects’ (representing lines of development which may be found wanting and abandoned) as ‘path finding’ projects whose learning is likely to influence a coming generation of EJ initiatives.

All the electronic journal projects are engaged in a continuous struggle to reconcile different formats, languages, protocols and procedures in a context where every achievement may be immediately superseded by developments elsewhere (for example Java to name but one). Among the projects, some aim to make use only of established technologies and software to provide on-line journals only. Some seek to develop environments and applications to provide both electronic and print versions, which involves choosing between, upgrading, adjusting and linking different software applications. It is useful to focus on a sub-set of the EJ projects under review to point up some of the technical complexity involved here:

Typically EJ projects only have limited resources for managing editors (e.g. Internet Archaeology has 1.5 days per week). This means a considerable burden has to off loaded on to authors (for example conversion costs) and unremunerated editors. More generally electronic journals challenge the ways in which tasks and roles have been allocated in the past, and seem to tend towards creating new (more demanding) standards for authors. This can be seen most clearly in the electronic journals which parallel print journals, where the question of who is responsible for tagging or marking up electronic copy is often a matter of dispute, but is in fact true for all electronic journals. Certainly the development of standard specifications for submissions has been a particular focus of a number of projects.

It is a commonplace that new journals, and all the more so electronic journals, have a problem in attracting ‘good’ authors (everybody likes to publish in the most prestigious disciplinary journals which are normally the longest established). However this is by no means the experience within the eLib programme: While JILT reports that submissions were at first slow in coming, both Sociological Research Online and Internet Archaeology report considerable success in attracting good quality submissions.

In the case of the two latter journals it is worth observing that in the case of Sociological Research Online, UK sociology has had for some time a pressing need for at least one more mainstream journal given the current scale of activity in the discipline, while in the case of Internet Archaeology the opportunity to effectively publish multi-dimensional material is being offered to the profession for the first time, emphasising the point that electronic journals need an additional selling point beyond simply that they are ‘on-line’. Yet it must also be said that all the electronic journals have had difficulties in persuading authors to include new multi-media elements - even CLIC operating from perhaps the highest skill base of all has expressed disappointment with the low take-up of the text enhancement features available to their authors.

One factor that continues to inhibit authors, highlighted by a number of projects, is the failure of the Research Assessment Exercise to adequately acknowledge electronic publication. The low status given to electronic publication in the RAE is also to be found at the disciplinary level, for example, the Chemical Abstract Society refuses to include abstracts from the electronic version of Chemical Communications until they have been published in the parallel paper version.

When we turn to the actual readership of the electronic journals, the raw figures reported suggest a very significant programme success: To give a flavour of this, 3,100 readers from 100 different countries have registered with Sociological Research Online and no less than 291 Web sites provide links to JILT.

In order to capitalise on this initial success in attracting readers most EJ projects are now attempting to ‘add-value’ to the offer to readers through providing more and more authoritative data content; enhancing text with additional multi- or hyper-media elements; providing greater flexibility in searching and browsing; and by improving turn around times to more speedily communicate research results.

However it is generally acknowledged that really attractive offers to readers still requires solutions to two more fundamental problems:

In the more interactive journals such as Formations and Deliberations where the traditional distinction between authors and readers are being broken down, readers appear to be uncomfortable with this challenge to traditional forms - ‘journals’ are something which you read, mail lists are somewhere where you discuss - or sometimes simply to existing ‘social’ patterns of communication. For example, while members of a mail list (ISL) were happy to use Deliberations as a source of reference and even as a source of new ideas to discuss, they continued to use the mailbase rather than Deliberations for that discussion.

Interactive journals also raise a raft of specific technical issues: (such as) selecting between users to provide targeted communications (Reviews in History) or end user configurations (Newsagent). Generally, the technical and organisational backdrop to projects continuously shapes the projects’ projected product offer and the characteristics of that product, inevitably projects are as much shaped by available technical capacity and competence as by editors insights or users’ expressed wishes. In one extreme example, key software features identified by the project design/editorial team remain unmet due to the absence of resource in the (separate) organisation charged with technical development.

In many other projects a multi-site division of labour has proved frustrating as the full complexity of the task in hand has had to be faced, as summed up in the following observation:

There is a degree of consensus reflected in the annual reports of the Electronic Journal projects about the necessary preconditions if real progress is to be made with exploiting the new technologies in innovative scholarly communication practices. These preconditions include;

• appropriate computer environments with suitable software tools,

• understanding of the tools and training and support in using them,

• real incentives in terms of key forms of scholarly recognition which effect status, funding and careers,

• a willingness to move among the current guardians of quality of content,

• investment by publishers in the necessary procedures and protocols, and

• convergence with other developments in on-line resources.

While progress can be made (and is being made) in each of these areas, overall progress requires progress in all areas simultaneously. What is difficult for UK HE is that so much depends on developments beyond UK HE, in commercial software systems development, among publishers and in the international academic community. Nevertheless, the ferment of activity around EJs in eLib is likely to effect these external actors and CEI may wish to ensure that UK HE continues to provide a pathfinding role in this area. In any case, some attempt to relate eLib activities in this area to wider developments would be a very useful component of the envisaged eventual final summative evaluation of eLib.

PROJECTS: Internet library of Early Journals; DIAD: Digitisation in Art and Design; Higher Education Digitisation Service

The three digitisation projects are grappling with a number of issues in common, although each is also seeking to resolve difficulties or challenges that are specific to its unique focus. Two of the projects are primarily oriented towards digitising a back run of journals, the one concerned with the design field and its attendant problems of high image quality and uncertainty about where copyright ownership rests, the other with archival material and the particular technical challenge of digitising material without dismembering bound volumes. The third project offers a digitisation service to clients in both the academic and commercial communities where it must in the longer term compete with commercial digitisation companies. At they approach the end of JISC funding, the projects in this domain raise central questions about sustainability, both in relation to the level of demand and the basis for cost recovery. The core concept of the services project, as a provider of customised or bespoke value-added services to the HE community, is also severely challenged in real market environment.

Among the main activities undertaken by the digitisation projects, are the following:

• digitisation and OCR work
• copyright clearance
• interface design and end user feedback
• service delivery

Although originally seen as a sequential set of discrete activities, projects now perceive these as inter-related and iterative. Some activities clearly have a more technical focus, but there is also recognition of the socio- dimension to what might previously have been seen as an exclusively technical matter.

For the project in the Design field, this technical side of the project has been largely unproblematic despite having to deal with the higher incidence of colour in the later years of the journal back run. The second project, however, has faced continuing difficulties into the second year of the project stemming in the main from the inadequacy of the technology for dealing with grey-scale provision. Although promised an upgrade of the IMAX interface card that would provide full grey scale capability, the new card has been further delayed.

Much effort by this second project has been devoted to maximising image quality, through progressive procedural refinements, made possible by increasing operator skill in using the technology. The learning from this process has been captured in two reports, and a draft procedural manual has been prepared.

Another area where the second project has encountered difficulties and set backs has been in OCR processing. OCR throughput takes longer than the initial scanning process and is currently the bottleneck in the production process. Experience to date indicates that whilst OCRing will be feasible with some titles, the added value will be much lower with others and not possible at all in certain cases.

The service oriented project is heavily orientated towards the digitisation of textual content and reported no difficulty in this area of its operations.

Securing the copyright clearance of articles and images in the digitised journals has been the major pre-occupation for the project in the design field. The checking process is a time-consuming and labour intensive process, entailing a variety of search strategies for tracking down copyright holders. In many instances, particularly in the case of photographs featured in the journal, it is not always clear where copyright ownership resides.

The service project has similarly found copyright clearance to figure large as an issue for its client base. Although not initially part of the services provided to clients, it has emerged as one of the key elements of a consultancy service that would need to be on offer to clients.

End-users in the two journal/archival projects have played a role in the iterative design process, although primarily through providing feedback on prototypes rather than as active partners at different stages of the development process. Areas where users have had an influence include interface design, imaging methodology, indexing and the criteria of acceptable OCR quality.

It is noteworthy that none of the projects in the digitisation domain appears to have begun with a market assessment or needs analysis. Rather, having undertaken the technical work with some end user input, projects are now looking to establish through focus group discussion in different test sites what level of demand might exist for the services on offer.

The digitisation services project offers a range of customised valued added services to support the conversion of high volumes of learning and scholarly materials into electronic format. These include consultancy and advice, pre-production testing to define the quality and accepted standards to be agreed with the client together with costings, and initial work in identifying the copyright owners of the material.

Some synergy in the design field is also resulting from links with the EU project MAID (Multimedia Assets for Industrial Design). It has developed a technological solution for networked delivery and the DIAD project was tested on high speed networks across Europe with considerable success. Both DIAD and MAID have benefited from sharing developments, due to similarities in technology, content and scope.

The cost of digitisation is an important issue although none of the projects is yet in the position to provide information of this kind.

This issue of cost is absolutely central to the HEDS project with its client orientation. Through focus group discussions, it has begun to identify the cost elements throughout the lifetime of the digitisation process. These include the cost requirements for different types of documents, the costs of accessibility (including also security of information) and preservation, as well as ongoing costs associated with future systems development and the migration of technology.

In seeking to extend its digitisation service and copyright clearance to images as well as text, HEDS is potentially confronted with a vast array of documents which it must deal with on a bespoke individual basis. To deal with the problem HEDS would need to provide some sort of pricing ‘menu’ such that material of a similar nature requiring similar treatment could be batched together.

Publishers’ attitudes towards copyright raised a number of important issues with regard to digitisation. Publishers clearly are concerned about the potential loss of revenue which could follow from making information freely available. Publishing on the Internet rather than on platforms that make it easier to limit access to certain sub-groups of the educational community is thus an area of critical concern. Stringent security measures, whilst favoured by publishers, are seen by the user community as discouraging access and inhibiting cultural change.

The two journal projects are at an early stage of formulating an exit strategy, including the viability of a commercial outcome. In the case of DIAD, one short term option is to link more closely with the ADAM gateway or to deposit the total database with VADS or AHDS.

The ILEJ project is exploring options for development beyond the period of present project funding in three key areas:

• provision of continuing access to the HE community to material which has been digitised in the project;
• a major expansion of ILEJ based on the expansion of the extensive 18th and 19th Century journal literature, providing a UK corpus comparable to that of JSTOR.

• continuing use of equipment purchased for the project, particularly for microfilm collections.

The costing data on the present experiment would provide the starting point for identifying the substantial resources that will be required for any expansion.

The HEDS project is able to compete with commercial companies because it presently benefits from substantially reduced tariffs through JISC funding. Even so, in the eyes of its clients there is confusion about the similarity between some services offered by HEDS and those offered by other services like TASI and VADS. These would need to be resolved through the project marketing effort and by forming closer links with other services. In the longer term, the HEDS business case would need to rest on the value added services it is able to provide to the HE community, beyond what the commercial sector offers. It is unclear at present whether its pricing structure could be competitive in the context of offering a bespoke service for the diverse digitisation needs of the academic community. In any event, it sees advantages to forming a strategic alliance with an established commercial copyright clearance service.

PROJECTS. DIGIMAP: National on-line Access to Ordinance Survey Digital Map Data; HELIX: Higher Education Libraries Image Exchange; MIDRIB: Medical Images: Digitised Reference Information Bank

The small group of three Images projects were funded in the second phase of eLib. One of these, HELIX, was only in its first year during this reporting period. As such it was encountering many of the start-up issues commonly faced by projects, which were largely covered in last year’s Synthesis of Annual Reports. Helix aims to build a ‘general image database’ which can be delivered free at point of use to the UK HE community. However, many of the technical and business challenges involved in this are being developed in the associated ELISE project, as outlined further below under Activities. This report focuses mainly on the work of the other two project, which involve greater technical complexity in digitising their databases and which also bear responsibility for resolving copyright and demand issues. MIDRIB deals with a comprehensive database of medical images for use in teaching and learning, available nationally via the Internet; Digimap aims to provide staff and students with access to Ordnance Survey digital map data that is both timely and cost effective. Both are concerned with how these resources may be used in practice and to best effect, with differing types of end user in mind.

Although the two projects have commonalities with other electronic documents, their main challenge has been dealing with the complexity of the technical aspects that stem from the very particular characteristics of their databases, and with related issues such as copyright and models of data supply. In HELIX these issues are being addressed more directly by ELISE II with which HELIX is associated.

Both Digimap and Midrib projects have come to a better appreciation of the importance of seeing a specific project innovation in the context of the setting in which it is to be used. Whilst their preoccupation has largely been with the technical side in the first year of operation, they recognise the need to refocus attention on how the resource may be used in practice.

MIDRIB is following an essentially linear process of research, development and production. The main effort in the first year has been on research, leading on to development, while production lies ahead in the second year of the project.

Among the challenges to which the project must find solutions are: the sheer potential size of the resource, its distributed nature, the multiplicity of image databases to be incorporated, user registration and access control, online review, and the multimedia nature of the resource (video, sound, hypermedia as well as still images).

Principal areas of activity have been:

• technical development
• digitisation and development of workflow strategies around the image acquisition process
• the development of a demonstration version of a CD interface for certain sets of images for use in teaching
• publicity
• expansion of the image collections available on the MIDRIB database
• cataloguing and indexing
• development of documentation on copyright and patient consent
• trials to investigate the system security aspects

HELIX is largely a content-building project with delivery systems being developed in the associated EU-funded project ELISE (Electronic Library Image Service for Europe) also located at De Montfort. Such technical work as is necessary in HELIX is being carried out in accordance with models developed in ELISE II, the successor to the ELISE project. The content building encompasses a ‘general image database’ of 45,000 images, 15,000 from each of its three sites plus an additional database module specific to the Social and Political History of Britain from 1859 to the present day.

Decisions around digitisation strategies, identifying suitable materials and beginning the process of digitisation and indexing as well as preparing for the additional module have been the main activities in the first year. The technical challenges have been minor and successfully resolved, though there has been some delay. HELIX has also been confronted by a need to reconcile different goal aims and processes of academic and commercial partners.

The technical and data issues which had to be overcome by the project stem from a mismatch between the way in which OS data is structured and user needs. Whereas user groups desire maps to be digital facsimiles of OS paper products, OS does not include that level of symbol specification with the digital data they supply and do not wish to provide this (possibly for copyright reasons). Second, each OS product is structured in a slightly different way, making it difficult to manage in an automated way as well as confusing users when they try to use the data. While it would be technically possible to edit/restructure the data, this is not feasible given the resources available. Nor would Digimap wish to deviate from a policy of minimum change to the underlying data provided by the OS. Thirdly, the service presents an OS product view and requires users to be familiar with OS digital products. Users however want a geography based view which is simple to use and hides questions of scale and product from the user. This represents an intellectual and design challenge, currently beyond the resources of the project.

The main challenges faced by the project have to do with managing the expectations of different potential user groups; dealing with digital map data which is subject to constant revision; and reorienting the user interface in the prototype to allow for retrieval by geographic and topic content rather than publisher’s product.

Digimap is characterised by a strong user-led development approach, employing an action research methodology. Its principal areas of activity during the first year underline this concern with building ownership of the project with participants in the test sites, and with enabling the central project team to learn about and understand the anticipated structure of demand and the local infrastructure at participating sites.

• MIDRIB has now in alpha test phase a highly flexible search engine, developed as a collection of server-side JAVA modules which provide a uniform, hierarchical interface to multiple databases, and uses locally designed and coded multithreaded searching. This enables the system to simultaneously search available resources and to explore these as each search completes, without forcing the user to wait for all threads to report their results. This offers a multitasking functionality rarely seen elsewhere on the WWW. MIDRIB believes the benefits of such interoperability to users of medical and other resources to be enormous.

• The project has been very successful in building up the image collections in a broad spread of medical areas. The number of high quality images useful for education in most collections is about 10% to 30% of the total collection.

• Digimap has developed a service demonstrator which has been through four major iterations, as prototype versions, each benefiting from feedback from users and representing a qualitative improvement.

• The development of a partnership with the Ordnance Survey (OS) has afforded a unique opportunity for Digimap to act as an ‘on-line research resource laboratory’ in which various data supply models can be explored. As a result of the partnership the project now has sufficient data to allow an effective evaluation of usage of OS data sets. In comparison with what was available at the outset, the quantity of data offered now meets users’ potential needs.
• HELIX’s association with ELISE and the Knowledge Gallery enhances the value of the project by providing access to ELISE developments to a wider audience and developing cohesion between the different databases.

Ethical issues are paramount when considering the distribution of medical images. An electronic environment complicates further what is already a difficult area. In the light of expert clarification of some important ethical issues, MIDRIB withdrew the documentation it had prepared (guidelines to donors and clinicians, patient information sheet and patient consent form) and which had been cleared by the relevant ethical committees, and is beginning again.

Within such an environment, registration and system security are therefore extremely important. This has posed a potentially enormous task for MIDRIB with a possible 800,000 users who must be verified as legitimate viewers of the resource prior to registration. The NISS model of user registration is currently being explored, and several separate trials are under way to investigate further the system security aspects and to investigate performance limitations.

Quality control and assurance, both of the images and the associated metadata is key. The MIDRIB collection system provides QA phases for digitisation, image registration in the database, and catalogue and indexing processes. This is supplemented by the facility for on-line review and comment by visitors to the site. ( a form of online peer review) and subsequent analysis by the team.

A key issue for Digimap, in common with some other eLib projects, has been whether to continue to develop bespoke software for the link to the WWW or to buy into the (less satisfactory) technology now on offer. The current commercial technology, offered by GIS vendors, offers neither the flexibility nor the capabilities that are available within the JAVA-based software developed by the project.

Both MIDRIB and Digimap reported management and organisational difficulties, relating among other things to the availability of staff with the requisite profile, and the unanticipated demands that good management makes on a dispersed, multi-partner project. HELIX has also suffered from staff changes but are confident that they now have a stable team.

In the case of MIDRIB, a key partner pulled out from the project consortium which necessitated a certain amount of restructuring but which also consolidated activities to a more limited number of sites within easy geographic reach. Digimap gained from establishing a partnership with OS (and also adding another trial site partner), after an initial period in which it was considerably hampered by JISC-OS negotiations over the supply of data to HEIs.

Evaluation to date has taken the form in both MIDRIB and Digimap of an ongoing iterative process throughout the project, involving such techniques as questionnaires, focus groups and individual interviews. The results have been fed back into the research and development processes. HELIX planned to begin its evaluation activities in October 1997.

Both Digimap and MIDRIB face uncertainty about the future. In the case of MIDRIB, there are legal complications following the withdrawal of one of the partners whilst for Digimap the uncertainty relates to outcomes from the JISC-OS negotiations, technical developments by commercial and academic competitors in the field, and the role of other developments in this area.

Digimap is however able to go further in exploring a number of options which include the possibility of collaboration with the British Library, the use of Digimap in public libraries, and further collaboration with Glasgow University Computing Sciences Department .

HELIX has already undertaken to make the images delivered available to the UK HE community free at point of use for four years beyond the end of the project and at the time of writing were confident that the framework being developed by ELISE II would be in place within the necessary time frame.

PROJECTS. Ariadne: A parallel Web and print newsletter for librarians and information scientists; EduLib: Educational Development for Higher Education Library Staff; Netlinks: Networked Learner Support; Netskills: Network Skills Training for Users of the Electronic Library; TAPin: Training and Awareness Programme in networks

The Training and Awareness projects demonstrate a number of different strategies for promoting and sustaining cultural and organisational change as well as for fostering the professional development of librarians in a new information age. The analysis which follows is based on five of the T&A projects, the annual reports of the remaining two projects not yet being available.

The strategies of two projects are largely predicated on the idea of cascading, with training and support focused on key individuals who are expected to be agents of cultural change and promoters of professional skills development. In both cases, the methods adopted include primarily face-to-face training events, supported by Web-based resource materials. Another two projects have adopted a strategy aimed at making a mass impact through delivering large numbers of workshops and seminars or through the development of networked training opportunities, aimed mainly at individuals in library/information services. Both projects make use of networked learning resources, one almost exclusively so. The fifth project, an authoritative parallel publication (print and Web versions), uses the media to inform, to stimulate debate and to strengthen the quality of Internet information services and resources of use to the profession.

A key finding for three T&A projects has been the limitations of its strategy for achieving the desired changes, coupled with an increased awareness of the importance of institutional factors and the more immediate organisational context as constraints on what can be achieved.

Two T&A projects, Netskills and EduLib, are the subject of an external impact evaluation study undertaken by IMPEL 2, which is seeking to establish how far the projects have been successful in consolidating and disseminating the professional development gains that have accrued to the individual participants so that they are more fully embedded in an institutional context.

Perhaps more than any of the other domains, the Training and Awareness projects have identified areas of complementary strength and made ready use of the training materials, workshop programs and other resources in support of their own strategies.

In the Table below, we summarise what are the distinctive approaches or strategies of the four T&A training-focused projects during the second year, including their target group(s), the methods used, the support services on offer, and the main outputs.

During the second year, most T&A projects have moved from laying the groundwork including training needs analysis, institutional audits, course design, preparation of materials, and so on, to the delivery of a workshop programme and other training support services. The main activities reported by the various training-focused projects were:

• ongoing production and dissemination of training materials, including regular updating
• the delivery of workshops, seminars and on-line training
• ongoing (formative) evaluation and quality assurance
• the launch of new services
• project review

Most projects continued to be involved in developing and updating training and support materials, reflecting both the constantly changing world of IT and the consequent need to include new developments in networked information, and also a recognition of the need to make provision for new target audiences (e.g. for those taking on a training role, or individuals wanting intermediate or advanced skills in this area).

The training materials have in all cases being developed by an extensive and continual cycle of development, delivery, feedback from workshop participants and trainers, and subsequent modification. In some projects, this development took place in parallel with training staff getting up to speed and delivering training; in others, there was a clearly defined pilot stage before the materials were used with the intended target audience.

The training materials and support resources produced by the T&A projects can be grouped as follows:

• presentations and workshop materials for trainers to deliver
• self-paced learning materials including how-to-guides and interactive tutorials
• sites with collections of useful resources for network trainers
• back up resource materials: case studies of current practice; pedagogic models; perspectives on change and innovation

Much of the second year of the T&A projects has been spent in delivering training in different sites across the UK in a face-to-face teaching mode. The workshops and seminars have generally been conducted on a regional basis so allowing reasonable access by participants across the HE system.

Activities range from training sessions open to all comers on a self-selecting basis, to events that are targeted at a specific audience such as information managers or subject specialists, to a full program of workshops mounted weekly over a 9-week period for library nominees.

The scale of training activity varies widely, reflecting the different strategies adopted by the T&A projects. Netskills, for instance has delivered more than 180 workshops and seminars to 6,000 people, giving a total of 18,000 hours of training. EduLib involved 136 librarians in its extended workshop program, with the expectation that the participants would in turn cascade their knowledge and skills to colleagues and academics back in their institutions.

One project reported a demand driven shift away from the general introduction course, to subject tailored introductory courses, authoring courses and other specialist courses.

Another found that the academics identified as ‘champions’ in their subject area were not interested in generic training modules put together by librarians, but instead wanted individual, tailored sessions that were responsive to their needs at that particular time. They had little time for discovery and exploration, being primarily interested in core resources.

NetLinkS has designed and piloted the interface for a distance learning, networked professional development course on NLS. The course will run for 16 weeks from September-February and can accommodate up to 40 participants. Other resource materials for use in training are mounted on the project’s Web site.

The T&A projects have all devoted considerable resources to formative evaluation as well as laying the basis for a summative evaluation. Ariadne, the publication project, has used focus groups to good effect along with a ‘snapshot telephone survey’ of its professional librarian readership. Data from access logs has yielded useful information for several projects, especially when used in conjunction with other qualitative data.

Course level evaluation is a common activity, based on feedback forms and an on-line variant. Participants’ and trainers’ views on the workshop have generally been gathered at the time of the event, and in one or two projects there has been a follow-up request for evaluative information a month or so later. Netskills for instance made provision for on-line discussion with and between participants following the workshops. The scale of formative evaluation activity can be demanding on participants no less than on evaluators, as EduLib has found. It has used NUDIST software to analyse more than 1000 evaluation proformas that have been returned from participants at its workshop programme. Whilst formative evaluation is very important as a means of continuous improvement, there is a point at which the collection of such data ceases to provide a significant return on effort.

Other evaluative activities have focused more on the input side. Netskills developed extensive but practical quality assurance procedures so as to maintain the high quality of its products given the number of trainers producing and delivering materials in a ‘mass training’ context. The QA work, presented in a document that has been revised and updated in the light of experience and changing requirements, covers planning and designing materials, proof reading and house, and piloting. In addition, the project sought feedback on its course materials from external trainers and Advisory Group members, as a further input into the QA procedures and to the materials themselves.

Efforts have also been made to look at the impacts of the training and awareness activities on longer term attitudes, behaviours and practices at an individual and institutional level. IMPEL2 is charged with this task in the case of two of the T&A projects, working in collaboration with the project evaluators. TAPin on the other hand has adopted a different approach to impact evaluation, based on a change process model. The research team, in the course of gathering evaluative data in case studies of the five partner institutions, has sought to engage participant librarians in dialogue and discussion about the changing culture of the library and their role in this process as change agents and opinion leaders

Among the new services on offer by two T&A projects are a Network Education and Training Electronic Gateway (NETEG) that was publicly launched in May 1997 after a trial period, and a moderated Mailbase discussion list as the central platform for the development of an on-line community of interest around NLS which was established in September 1996.

NETEG provides support for trainers who are delivering their own workshops and courses in the use of the network through giving access to a collection of network training materials and other resources. As well as building up the database of resources, the project team undertook a reviewing process to establish the quality and consistency of resources and their descriptions. One output of this activity has been the development of quality assurance procedures for entering and evaluating resources.

The NIs-Forum Discussion List, which has to data attracted 925 members, now hosts a programme of ‘mini-conferences’ in addition to the more usual open format for unstructured exchange of information and ideas. Four mini conferences have been held so far, facilitated by invited discussion leaders, and attracting UK and international contributors from library, IT and teaching staff with an interest in NLS.

In addition to the evaluation findings reported by projects that relate to uptake of project materials and services and to participant feedback on aspects of course delivery, most of the annual reports also draw out some key findings or raise issues that are important for the project’s future direction.

One set of findings from different projects relates to the changing role of the librarian:

• Our project is raising interesting questions about the extent to which it is possible for librarians who a) are not expert educationalists but b) have an interest in teaching and learning, can communicate and disseminate expertise among colleagues. This process clearly places considerable demands upon them and it is evident that their activity is more effective if they can work together in small teams, in addition they do need the support of the materials which EduLib is producing.

• Librarians need to recognise the important role they play in changing the culture, of not only their library and information services, but of their institutions as a whole. They are gateways to the wider information available to academics and seldom recognise their roles as agents for change.

• What is needed is a cross-disciplinary element. Academic and learner support staff with various different specialisms (e.g. librarians, IT and educational technology staff) need to be encouraged to work together on projects which take an integrated approach to development but which enable allocation of leadership roles to whichever services or departments are the most appropriate in circumstances and for local purposes.

Another set of findings raises questions about the likely longer term impact of T&A activities on institutional behaviours:

• The data suggest that the participants gain more from the courses in terms of their personal career development than impacting on their immediate work. The follow-up interviews may reveal a change in attitude here, when participants have actually taken their new skills back into the workplace

• The survey reveals an element of ‘informal’ cascading of the new skills to colleagues. Some institutions have an expectation that if one person attends a Netskills course they can teach all their colleagues, whilst other institutions send all of their staff to the appropriate workshop. It is hoped that follow-up interviews will reveal how participants are teaching colleagues their new skills and how much this is driven by institutional pressures.

• Our experience has underlined the importance of sensitivity to local institutional contexts in the provision of external support for change, and highlighted questions relating to the role of senior management and library services’ staff in instigating institution-wide change

• Where campus networks and desktop access was advanced, TAPin activity fitted easily into the institution. Even so it seemed these Libraries still have much to do to update their image and access ethos, and to rethink the importance of their roles in value added mediation in a network literate organisation. Where network were newer, the impact of TAPin appeared greater. These Libraries seemed more committed to use the TAPin model and to work at helping their own library staff and academics exploit the networks.

One project among the T&A group reflected on the importance of a ‘reflective culture’ among the team as an essential ingredient for innovation projects such as eLib, and the integration of that learning into ongoing development:

• One important aspect of the project has been the building and maintaining of a highly effective team. This has been extremely successful even with staff changes. An internal training programme for new staff, peer training sessions on new technologies for staff members (often held in collaboration with the Mailbase Team) and a sharing of the liaison effort by reporting on conferences, interesting journal articles or items on mailing lists, has proved an effective way to work…The integration of our information systems with the practical application of existing technology has improved our effectiveness as a team. The human and organisational issues involved here are providing useful experience which will feed into our future courses. We recommend the use of a project team ‘Intranet’ and the use of external process-consultants to other projects.

The valuing of professional exchange was highlighted by one project as an important finding for eLib:

• There was one quite overwhelming response which came from nearly all our participants. It was that our courses provided the opportunity for librarians to meet together in a collegial atmosphere and exchange experiences and practices. It is an interesting reflection upon the considerable and continuous demands placed upon librarians that they have to get away from their own institutions in order to find the circumstances in which they can engage in professional exchanges.

In a number of annual reports, it was evident that projects had taken time to stand back from their activities and reflect broadly on their experiences. In at least three projects, this process of reflection had led projects to question the effectiveness of their overall strategy for achieving their individual project goals. In two cases, projects had come to realise the limitations of an individual-centred approach when seen in the broader context of the institutional changes needed to support a wholesale shift to a networked environment. In another, the project saw clearly that to move from pilot status to a real marketplace would require a partnership with a publisher.

• ‘The strategy we adopted to target individuals in library/information services to undertake coordinator and change agent roles in relation to NLS development in their institutions has met with limited success. It is clear that without significant support at the level of strategic planning and the time to take on such roles, it may well be unrealistic in many institutional contexts to expect these staff to become proactive. Nor is it necessarily the case that information service staff are the mot suitable staff to take on coordination of an area of development in which there is such a diversity of stakeholder interests and perspectives.

• ‘Members of the Development Team [did not] , as previously envisaged, operate as individuals responsible for disseminating educational expertise within regional consortia, they [worked instead] as members of small regional teams. This [was] for a number of reasons, namely: to take full advantage of the team spirit and cooperation which has been developed among members during the second phase of the Project; to make the most effective use of different members’ alternative strengths and expertise, to promote further training and develop confidence among the group; and to ensure that there is ‘cover’ if any members of the team are ill or move to other posts.

• ‘In broad terms we have learnt enough to realise that enthusiasm and dedication together with editorial ability and solid, pragmatic administration, is enough successfully to launch a good publishing idea. In order to develop that idea into a vehicle for income, however, attracting advertising and achieving a wide readership in a real marketplace, there are many benefits in working with a professional publisher. As a very small single-publication enterprise, we can never achieve the cost-benefits which are available to publishers through production economies of scale, nor maximise the advertising potential, nor market seriously in the way a publisher could with a full knowledge of the industry. As we move into our exit phase, therefore, one of the chief lessons we have learned is that we need professional assistance.’

The T&A projects have a number of cross-project linkages, most of which are seen as beneficial to the parties involved. This can be at the level of Steering Group membership, as is the case with two projects which are represented on the SG of another project. More commonly, however, the T&A projects have made effective use of one another’s materials or training courses for their own participant/contributor group. NetLinkS and TAPin, for instance, have cooperated on training and future collaboration is planned with Netskills.

One project reported that it had not found the other T&A projects within eLib as useful as id did in year one. The management team felt that the training provision on offer did not take sufficient account of advances in IT, nor was it well suited to those wanting intermediate or advanced skills in working with networked information.

The projects involved with IMPEL2 found this to be a valued relationship, both adding an independent perspective on the impact of the T&A projects as well as relieving projects of the burden of a summative evaluation. The relationship was a collaborative one, and seen as mutually beneficial.

The T&A projects are at different points along the path of considering their future sustainability or the appropriate steps in an exit strategy. Some are actively pursuing options as part of putting together a business plan; others expect to give serious consideration to the project’s future in the coming period, and one project makes no mention at all of the post-funding period.

Among the options for a sustainable future under consideration by the various projects are the following:

• differentiating between the project’s exploitable products and services and locating different sources of funding or support for each e.g. recovery of costs for the on-line course through course fees; paid subscription for the Website services; offering a consultancy service in the process of managing cultural change

• generate income through expanding into new markets (FE, public libraries, health) for the project’s training products and services

• partnership with a publishing company that can assist ARIADNE with a redesign and relaunch, and possibly with advertising, subscription management, marketing, production and distribution

• further funding by JISC as part of its commitment to support ongoing national services where appropriate

When we consider on the content of the eLib second year annual project reports, it is particularly striking how we are now beginning to see some of the benefits anticipated in early FIGIT decisions actually emerging in practice.

We see, for example, the formative impact of project management workshops, evaluation workshops/clinics, supporting studies, and the demands of annual reporting itself, on project organisation and behaviour. This is particularly so in the greater emphasis given by projects now to user centred design approaches, good project management, and decision making informed by evaluation.

We may reflect that a) there will be wider benefits for HE as a whole as the learning and organisational skills first acquired in the context of the eLib projects is applied eventually in other projects and situations, particularly if these improved standards of operation are reinforced by explicit stipulations (by JISC and by individual HEIs) when funding future activities.

And b) that while a great deal of individual and organisational learning has taken place simply by virtue of the space for practical learning created by the eLib programme, much of the improvement in project organisation observed in the second year of the programme has also taken place by design. (This, of course, was key to FIGIT’s overall programme management strategy - provide projects with funding for activities but also provide the support necessary for undertaking those activities successfully.) Hopefully, we will now see the further beneficial effects of a reinforced emphasis on organisational issues in the selection and support of projects in Phase III of eLib.

We can also observe through the project reports actual mobilisation and catalytic effects of the programme - on researchers, publishers, EU programmes, and individual disciplines and HEIs. And the strategy of broad engagement through a large number of projects adopted by FIGIT, allied to the role of the Programme Director and Assistant Director and various FIGIT (now CEI) members as ‘missionaries’ for the programme, as well of the extensive round of workshops and conferences arising from, inspired by or drawing input from eLib, seem to have cumulatively created a ‘buzz’ and climate of expectation (not least in JISC and the HEFCs themselves) which is beginning to effect HE at many different levels from policy making to actual library provision and even, to some extent, research and teaching practice.

However, it be wise not to take these gains for granted nor to simply consider them to be the inevitable but somehow un-planable effects of an large initiative like eLib - Other HE initiatives in the past have struggle to achieve such like broader effects, and some reflection on why and how particular eLib elements have ‘worked’ (and others have not) and the implications for future initiatives is probably now timely.

It should also be noted that eLib has not managed to provide ‘solutions’ for a number of problems which threaten to become perennial in JISC programmes. These are:

Staffing: The short term contracts associated with project based funding inevitably create problems of staff recruitment and retention: JISC might usefully consider how creating more continuity of employment through better secondment arrangements, bundling of projects, or rolling funding of ‘core’ teams (e.g. UKLON, Netskills), might ease some of the human resource problems in programmes like eLib.

Exit strategies: It is difficult for individual projects, often with limited funding, short timescales and demanding ‘deliverables’ to generate effective exit strategies in the context of a difficult and rapidly changing external environment: There is a need to move beyond merely calling for business plans and exit strategies to more strategic interventions which provide evolutionary strategies for areas (ANR, EDD, ODP and so on) as a whole.

‘Cultural change’: Every attempted innovation brings in it’s train the need for cultural change: The arrangements (CALT, JISC Assist etc.) made by JISC in the last year to address the need for a greater concentration on training, awareness and cultural change issues, will be important in addressing this challenge.

The ANR projects have accumulated a mass of practical experience on providing subject specific Web resources, covering issues such as user needs, suitable interfaces and technical strategies as well as providing quality assured access to a significant mass of actual information.

However, the future of individual ANR projects will in part depend on broader JISC/CEI developments around subject based resources and broader decisions to be made around developing the distributed national electronic resource. While there is a need for JISC/CEI to take some decisions across the board about ANR projects, it would be to misunderstand their nature as subject based services not to also relate any further core funding to the particular nature and needs of their disciplines especially as regards convergence and complementarity with other disciplinary initiatives.

We have observed that the future of individual ODP and ESC projects will largely depend on broader developments in electronic publishing beyond their immediate control yet which they themselves have been catalysts for. Beyond these catalytic roles and their important contribution to increasing knowledge in this area, the most important result of these projects may well be something they paid least attention to at their outset, namely, clarifying the requirements and increasing the preparedness to undertake the necessary changes/’re-engineering’ in library and teaching practices that will be required if the potential of electronic publishing is to be effectively exploited in HE contexts. Hopefully elements of this learning will be seen to be applied directly in some of the eLib Phase III hybrid library projects.

Even more strikingly than in the case of ODP/ESL, the EDD projects have moved on our knowledge of and understanding about the technical, economic and organisational requirements for such new services to emerge. As with the case of ODP/ESL, it is by no means certain that actual sustainable EDD services will emerge within UK HE, but it is reasonable to argue that the actual event of sustainable EDD services to UK HE has in any case been significantly brought forward by the pathfinding activities of these eLib projects. The paradox of the EDD projects, as well as that of the ODP projects, is that the emergence of commercial competition is likely to make HE based services difficult to sustain, but if such competition is called forth this can only be seen as a eLib programme success.

5. Electronic Journals, Pre-prints and Quality Assurance

We have observed that there is a degree of consensus reflected in the annual reports of the Electronic Journal projects about the necessary preconditions if real progress is to be made with exploiting the new technologies in innovative scholarly communication practices. These preconditions include;

• appropriate computer environments with suitable software tools,
• understanding of the tools and training and support in using them,
• real incentives in terms of key forms of scholarly recognition which effect status, funding and careers,
• a willingness to move among the current guardians of quality of content,
• investment by publishers in the necessary procedures and protocols, and
• convergence with other developments in on-line resources.

While progress can be made (and is being made) in each of these areas, overall progress requires progress in all areas simultaneously. What is difficult for UK HE is that so much depends on developments beyond UK HE, in commercial software systems development, among publishers and in the international academic community. Nevertheless, the ferment of activity around EJs in eLib is likely to effect these external actors and CEI may wish to ensure that UK HE continues to provide a pathfinding role in this area. In any case, some attempt to relate eLib activities in this area to wider developments would be a very useful component of the envisaged eventual final summative evaluation of eLib.

6. Digitisation

The core concept of the digitisation projects as niche providers of customised or bespoke value-added services to the HE community is likely to be severely challenged in a real market environment. However, the need for digitisation services within HE will persist until time as competitive commercial services emerge and publishers begin to digitise their own back catalogues. A steady supply of ‘business’ from projects within HE will be important to the continuance of the digitisation services.

7. Images

It would be panglossian to imagine that the diverse imagining needs of academic researchers will ever be fully met by commercial offerings alone. The eLib imaging projects provide us with useful prototypes of future HE based specialist imaging services, and regardless of their own individual sustainability offer valuable examples for other services that will be needed within HE.

8. Training and Awareness

We have observed that the various eLib T&A projects have emerged as important resources to UK HE in their own right. There are clearly opportunities here to continue to exploit this capacity beyond the lifetime of the eLib programme by commissioning additional targeted efforts in other associated areas.

As innovative projects from which others are expecting to learn, it is important that eLib projects provide information and knowledge that will be speedily accessible to the wider community. This annual reporting structure is one way of ensuring that the lessons emerging from monitoring and evaluating the progress and success of the project are recorded, systematised and disseminated.

The framework elaborated below is intended to provide eLib in its overall management role with a consistent and coherent set of data from all projects about activities and progress, the process of implementation, reflections on what has been learned and revised understandings and expectations about the project innovation. Regular reporting of this kind is also useful for project self-evaluation and reflexivity among the partners about what is being learned.

The framework takes the form of a set of leading questions that are relatively unstructured. In the first year, we sought to elicit 'soft' data from projects, informed by your formative evaluation and experiences of implementation. In this second and subsequent years, reporting should include more systematic feedback from trials or demonstrators, evidence of outcomes and effects, and clearer plans for post-project exploitation .

The annual reporting structure proposed here is intended to dovetail with management agreements made with eLib about regular reporting. In preparing an annual report, we would ask you to report for the period subsequent to that covered by last year’s report and up to the last major milestone as specified in your own project plan.

Projects are requested to follow the format below when preparing their annual report. We would also ask you to append any relevant documentation (evaluation reports, business plans, publicity materials etc.) when forwarding your report to eLib.

This section is concerned with activities and progress in relation to your proposal and contract with eLib. It should identify what has been achieved since the start of the project/since the last report.

This section of the report is concerned with differences between what was planned and what actually occurred, and the reasons for any such changes to plan. Please tell us about any difficulties you have encountered as well as unexpected opportunities that may have opened up. Projects are encouraged to report on all forms of change including 'learning from failure'.

Projects will be collecting systematic, structured feedback at key stages in the project lifecycle, as outlined in the Guidelines for eLib Project Evaluation and in line with your own project evaluation plan agreed with FIGIT (as was). These data are likely to be both formative (informing ongoing development and decision-making in the project) as well as summative (providing evidence of effects and longer term impacts).

Some possible facets or indicators which you might consider for each of these broad criteria are listed in Table 1 in the Guidelines on Evaluation. These should be supplemented by project's own operationalisation of evaluation criteria, relevant to their particular situation and view of what is important.

Electronic Journals: Effects on scholarly practice.

Access to Network Resources: Contribution to and emerging role in subject information retrieval infrastructure.

On Demand Publishing: Emerging models of HE library provision particularly as regards the provision of teaching resources.

Electronic Document Delivery: The elements of the business case for operational services.

Training and Awareness: Indications of impact on behaviour.

In this section you should set out any ideas you might have about future developments. It would be helpful to identify any changes in planned direction as well as any changes in proposed management and staffing arrangements.