eLib Phases I & II
Format for 3rd Year Project Annual Reporting (August 1999)
Introduction
As innovative projects from which others are expecting to learn, it is important that eLib projects provide information and knowledge that will be speedily accessible to the wider community. This annual reporting structure is one way of ensuring that the lessons emerging from monitoring and evaluating the progress and success of the project are recorded, systematised and disseminated.
The framework elaborated below is intended to provide eLib in its overall management role with a consistent and coherent set of data from all projects about activities and progress, the process of implementation, reflections on what has been learned and revised understandings and expectations about the project innovation. Regular reporting of this kind is also useful for project self-evaluation and reflexivity among the partners about what is being learned.
The framework takes the form of a set of leading questions that are relatively unstructured. In the first year, we sought to elicit 'soft' data from projects, informed by your formative evaluation and experiences of implementation. In this third year, reporting should include more systematic feedback from trials or demonstrators, evidence of outcomes and effects, and clearer plans for post-project exploitation .
The annual reporting structure proposed here is intended to dovetail with management agreements made with eLib about regular reporting. In preparing an annual report, we would ask you to report for the period subsequent to that covered by last year’s report and up to the last major milestone as specified in your own project plan.
Projects are requested to follow the format below when preparing their annual report. We would also ask you to append any relevant documentation (evaluation reports, business plans, publicity materials etc.) when forwarding your report to eLib.
1. Activities and Progress
This section is concerned with activities and progress in relation to your proposal and contract with eLib. It should identify what has been achieved since the start of the project/since the last report.
Questions to address:
- What have been the major activities undertaken by the project?
- What have been the effects of any changes you may have made to the project plan in the light of the experience of the second year of the project and indicated in last year’s annual report?
- What have been your main objectives or targets during the period and how far have you been able to meet them?
- What outputs have you produced from your activities (such as prototypes, models, demonstrator services, actual services, events, reports etc.)? Please quantify dimensions where possible.
- Do you have any particular successes to report?
2. Learning from the process of implementation
This section of the report is concerned with differences between what was planned and what actually occurred, and the reasons for any such changes to plan. Please tell us about any difficulties you have encountered as well as unexpected opportunities that may have opened up. Projects are encouraged to report on all forms of change including 'learning from failure'.
Check list of questions:
- Have you encountered any difficulties in managing the project and carrying out your activities (e.g. staffing problems, technical delays, increased costs, involving relevant users)?
- What influence, if any, (positive or negative) have other projects, the programme as a whole, or the programme office had on your project?
- What changes have you made to your plan (aims, objectives, staffing, activities, etc.) in the light of your experiences?
- What are the reasons for these changes?
- Has your project thrown up any unanticipated outcomes or unexpected opportunities and how have you taken account of these?
- What have you learned from your experiences of innovation and development?
Do you now have a different understanding of what you are trying to achieve, or the nature of the innovation?
3. Evaluation results
Projects will be collecting systematic, structured feedback at key stages in the project lifecycle, as outlined in the Guidelines for eLib Project Evaluation and in line with your own project evaluation plan agreed with FIGIT (as was). These data are likely to be both formative (informing ongoing development and decision-making in the project) as well as summative (providing evidence of effects and longer term impacts).
Please report on the findings which are emerging from your evaluation activities, commenting in particular on any general outcomes, effects and impacts. (Please also note and explain here any difficulties or delays you have experienced in carrying out your evaluation activities.)
Your third year annual report should focus where ever possible on the results of trials and pilots and address all relevant issues including
- technical
- organisational
- economic
- scholarly, and
- learning and teaching
Actual evaluation reports should be appended.
In keeping with the eLib programme's overall evaluation preoccupations, you might also usefully comment on the project outcomes and effects in relation to the following:
- mobilisation
- cultural change
- cost-effectiveness/value-added
- sustainability
- demand/usefulness/performance
- future scenarios
- contribution to overall project goals
Some possible facets or indicators which you might consider for each of these broad criteria are listed in Table 1 in the Guidelines on Evaluation. These should be supplemented by project's own operationalisation of evaluation criteria, relevant to their particular situation and view of what is important.
As you are approaching the end of your trials and pilots, feedback on the following area specific issues is particularly requested:
Electronic Journals: Effects on scholarly practice.
Access to Network Resources: Contribution to and emerging role in subject information retrieval infrastructure.
On Demand Publishing: Emerging models of HE library provision particularly as regards the provision of teaching resources.
Electronic Document Delivery: The elements of the business case for operational services.
Training and Awareness: Indications of impact on behaviour.
4. Future development
In this section you should set out any ideas you might have about future developments. It would be helpful to identify any changes in planned direction as well as any changes in proposed management and staffing arrangements.
Questions to be addressed:
- What are the main objectives for the next reporting period?
- What, if any, changes in overall direction are proposed?
- How do you now envisage the future scenario for the product or services you are developing, beyond the project timeframe?
Projects which are ending soon should present here, as appropriate;
- Post-project plans of the participating institutions, where possible expressed in terms of business plans/ specifications of follow-on projects or activities.
- Overall conclusions about the results and implications of the project.
Your exit strategy should be fully explained.
Business plans should be carefully related to the evaluative evidence you have accumulated in the course of the project.
If you do not intend to continue the project beyond the funding period please explain what has been gained and what scope there is for future action.
If you have any queries about the Annual Reporting process or format please feel free to contact John Kelleher at
The Tavistock Institute
Evaluation Development and Review Unit
30 Tabernacle Street
London EC2A 4DD
phone: 0171 417 0407
fax: 0171 417 0567
email: j.kelleher@tavinstitute.org