As innovative projects from which others are expecting to learn, it is important that eLib projects provide information and knowledge that will be speedily accessible to the wider community. This final reporting structure is one way of ensuring that the lessons that have emerged from monitoring and evaluating the progress and success of the project are recorded, systematised and disseminated.
The framework elaborated below is intended to provide eLib in its overall management role with a consistent and coherent set of data from all projects about activities and results, the process of implementation, reflections on what has been learned and revised understandings and expectations about the project innovation. Reporting of this kind is also useful for project self-evaluation and reflexivity among the partners about what has been learned.
The framework takes the form of a set of leading questions that are relatively unstructured. In previous years, we sought to elicit `soft' data from projects, informed by your formative evaluation and experiences of implementation. In this final report, reporting should include more systematic feedback from trials or demonstrators, evidence of outcomes and effects, and (if appropriate) clear plans for post-project exploitation.
The final reporting structure proposed here is intended to dovetail with management agreements made with eLib about regular reporting. In preparing the final report, we would ask you to report for the period subsequent to that covered by your last annual report and up to the end of the project and on the project overall. We would ask in particular that you reflect on the `lessons' coming out of the project which can be usefully applied elsewhere.
Projects are requested to follow the format below when preparing their final report. We would also ask you to append any relevant documentation (evaluation reports, business plans, publicity materials etc.) when forwarding your report to eLib.
This section is concerned with activities and progress in relation to your proposal and contract with eLib. It should identify what has been achieved since the last report and summarise achievements overall.
Questions to address:
- What have been the major activities undertaken by the project?
- What have been the effects of any changes you may have made to the project plan in the light of the experience of previous years of the project and indicated in the last annual report?
- What have been your main objectives or targets and how far have you been able to meet them?
- What outputs have you produced from your activities (such as prototypes, models, demonstrator services, actual services, events, reports, dissemination etc.)? Please quantify dimensions where possible.
- Do you have any particular successes to report?
This section of the report is concerned with differences between what was planned and what actually occurred, and the reasons for any such changes to plan. Please tell us about any difficulties you have encountered as well as unexpected opportunities that may have opened up. Projects are encouraged to report on all forms of change, including `learning from failure'.
Check list of questions:
- Have you encountered any difficulties in managing the project and carrying out your activities (e.g. staffing problems, technical delays, increased costs, involving relevant users)?
- What influence, if any, (positive or negative) have other projects, the programme as a whole, or the programme office had on your project?
- What changes have you made to your plan (aims, objectives, staffing, activities, etc.) in the light of your experiences?
- What are the reasons for these changes?
- Has your project thrown up any unanticipated outcomes or unexpected opportunities and how have you taken account of these?
- What have you learned from your experiences of innovation and development?
- Do you now have a different understanding of what you have been trying to achieve, or the nature of the innovation?
Projects will have collected systematic, structured feedback at key stages in the project lifecycle, as outlined in the Guidelines for eLib Project Evaluation and in line with your own project evaluation plan agreed with eLib/JISC. These data are likely to be both formative (informing ongoing development and decision-making in the project) as well as summative (providing evidence of effects and longer term impacts).
Please report on the findings which have emerged from your evaluation activities, commenting in particular on any general outcomes, effects and impacts. (Please also note and explain here any difficulties or delays you have experienced in carrying out your evaluation activities.)
Your final report should focus wherever possible on the results of trials and pilots and address all relevant issues including
- technical
- organisational
- economic
- scholarly, and
- learning and teaching
Actual evaluation reports should be appended.
In keeping with the eLib programme's overall evaluation preoccupations, you might also usefully comment on the project outcomes and effects in relation to the following:
- mobilisation
- cultural change
- cost-effectiveness/value-added
- sustainability
- demand/usefulness/performance
- future scenarios
- contribution to overall project goals
Some possible facets or indicators which you might consider for each of these broad criteria are listed in Table 1 in the Guidelines on Evaluation. These should be supplemented by project's own operationalisation of evaluation criteria, relevant to their particular situation and view of what is important.
In this section you should set out any ideas you might have about future developments whether expressed as a business plan, exit strategy or plans for a follow on project.
Questions to be addressed:
- How do you now envisage the future scenario for the product or services you are developing, beyond the project timeframe?
You should present here, as appropriate;
- Post-project plans of the participating institutions, where possible expressed in terms of business plans/ specifications of follow-on projects or activities.
- Overall conclusions about the results and implications of the project.
Your exit strategy should be fully explained. Business plans should be carefully related to the evaluative evidence you have accumulated in the course of the project.
If you do not intend to continue the project beyond the funding period please explain what has been gained and what scope there is for future action.
Question to be addressed:
- Are there any issues emerging from the project which merit further investigation by JISC or other bodies?
You should give a clear account of the project finances indicating where the project budgets have been spent. Any difficulties that have arisen in managing the funding of the project should also be discussed.
In this section you should summarise any major `lessons' that have emerged from the project and which you feel could be applied elsewhere. These may or may not have been mentioned in other sections of the report. They may include conclusions about particular technologies, views on management issues, or comments on service requirements.
Questions to be addressed:
- What lessons have emerged from the development work of the project (technical, social etc.)?
- What lessons have emerged associated with the management of the project (connected with methodologies, communication, planning etc.)?
- In what ways has the eLib Programme Office had an impact on the project? Would you recommend any changes in the way the Programme Office has dealt with the project or strand?
- In what ways can lessons from the project contribute to the development of the DNER?
- How can other institutions not involved in the project be assisted in taking up its lessons?
If you have any queries about the final reporting process or format please contact
Stephen Pinfield
Hallward Library
University Park
University of Nottingham
Nottingham
NG7 2RD
Tel 0115 951 5109
Fax 0115 951 4558
Email stephen.pinfield@nottingham.ac.uk
Please send one copy of the report to Stephen Pinfield and one copy to Lorcan Dempsey at the DNER Office.