QA Focus Case Study Documents: Print All - General
[ Home Page - Surveys - Documents - Presentations - Search ]
This page is for printing out the case studies on the general areas, not covered elsewhere. Note that some of the internal links may not work.
The RDN's Subject Portals Project
(SPP) is funded under the
JISC's DNER Development Programme.
There were two proposals, SAD I
(Subject Access to the DNER) and SAD II.
The original SAD I proposal was part of a closed JISC DNER call, 'Enhancing JISC
Services to take part in the DNER'. The SAD II proposal was successful under
the JISC 5/99 call, 'Enhancing the DNER for Teaching and Learning'.
The original project proposals are available [1].
The aim of the project is to improve the functionality of five of the RDN hub sites to develop them into subject portals. Subject portals are filters of Web content that present end users with a tailored view of the Web within a particular subject area. In order to design software tools that simultaneously satisfy the needs of a variety of different sites and make it easier for institutional portals to embed our services in the future, we are designing a series of Web "portlets". One portlet will be built for each of the key portal functions required, focussing initially on authorisation and authentication (account management); cross-searching; and user profiling; but including eventually a range of "additional services" such as news feeds, jobs information, and details of courses and conferences. The project is committed to using open source software wherever possible.
The hub sites involved in the SPP are EEVL (based at Heriot Watt University, Edinburgh), SOSIG (University of Bristol), HUMBUL (University of Oxford); BIOME (University of Nottingham) and PSIGate (University of Manchester). The project is managed from UKOLN based at the University of Bath, and the technical development is led from ILRT at the University of Bristol.
The fact that the SPP partners are geographically dispersed has posed a number of challenges. Since the objective of the SPP is the enhancement of the existing hub sites, hub representatives have naturally wished to be closely involved, both on the technical and on the content management sides of the project. At the last count, 38 people are involved in the project, devoting to it varying percentages of their time. But this means that physical meetings are difficult to organise and costly: since work began in December 1999 on the SAD II project, only two full project meetings have been held, with another planned for the beginning of 2003. Smaller physical meetings have been held by the technical developers at ILRT and the five hubs, but these again are extremely time-consuming.
We also faced the problem that many of the project partners had never worked together before. Not only was this a challenge on a social level, it was also likely to prove difficult to find where the skills and experience (and software preferences) of the developers overlapped, and at the beginning of 2002, the then project manager Julie Stuckes commissioned a skills audit to discover the range and extent of these skills and where the disparities lay. It was also likely to be hard to keep track at the project centre of the different development activities taking place in order to produce a single product, and to reduce the risk of duplicating effort, or worse, producing incompatible work. We also thought moreover that it was desirable to develop a method of describing the technical work involved in the project in a way easily understood by the content managers and non-technical people outside of the project.
We tackled the problem of communication across the project by the use of a project JISCmail mailing list [2]. The list is archived on the private version of the SPP Web site [3] where other internal documents are also posted.
The developers have their own list (spp-dev@dev.portal.ac.uk) and their own private Web site [4] which is stored in a versioning system (CVS - Concurrent Version System [5]) which gives any authenticated user the ability to update the site remotely.
In addition the developers hold weekly live chat meetings using IRC (Internet Relay Chat [6]) software (as shown in Figure 1), the transcripts of which are logged and archived on the developers' Web site.
Using IRC means the developers are able to keep each other informed of their activities in a relaxed and informal manner; this has aided closer working relationships.
Figure 1: Example IRC Session
As well as holding the developers' Web site, CVS also contains the project's source code and build environment. This takes the form of a central repository into and out of which developers check code remotely, ensuring that their local development environments are kept in step. A Web interface also provides the option of browsing the code, as well as reviewing change histories. Automatic e-mail notification alerts the developers to updates checked into the CVS repository, and all changes are also logged. This has proved an essential tool when co-ordinating distributed code development.
The other part of the software development infrastructure is providing a build environment that takes care of standard tasks, allowing the team members to concentrate on their coding. Using a combination of open-source tools (e.g. ant [7] and junit [8]) a system has been created that allows the developer to build their code automatically, run tests against it, and then configure and deploy it into their test server. As well as this, the build system will also check for new versions of third-party packages used by the project, updating them automatically if necessary. This system is also managed by current project down from the central repository, build, configure and deploy it, having it running in a matter of minutes.
Because of the widely dispersed team, the difference in software preferences and the mixed technical ability across the project, we looked around for a design process that would best record and standardise our requirements. UML (Unified Modelling Language [9]) is now a widely accepted standard for object oriented modeling, and we chose it because we felt it produced a design that is clear and precise, so making it easy to understand for technical and non-technical minds alike. UML gave us a means to visualise and integrate use cases, integration diagrams and class models. Moreover using UML modelling tools, it was possible to generate code from the model or update the model whenever the code was further developed.
Figure 2: Example UML Diagram
Finding UML software that had all the features needed was a problem: there are plenty of products available but none quite met all our requirements, especially when it came to synchronising the work being done by different authors. Eventually we opted to use the ICONIX process [10]. This is a simplified approach to UML modeling, which uses a core subset of diagrams. This enabled us to move from use cases to code quickly and efficiently using a minimum number of steps, thus giving the technical side of the project a manageable coding cycle.
Additional funding was obtained from the JISC in order to bring one of the authors of the ICONIX process (Doug Rosenberg) over from California to run a three day UML training course. Although this course was specially designed for SPP, places were offered to other 5/99 projects in order to promote wider use of this methodology across the JISC community. Unfortunately, despite early interest, no other project was represented at the training, although Andy Powell, the technical co-ordinator for the RDN, attended the course. Additional funding was also received from the JISC to purchase licences for Rational Rose [11], which we had identified as the most effective software available to produce the design diagrams
Finally, to provide greater structure to the project, a timetable of activities produced using MS Project is posted on the private project Web site and is kept continually up to date. A message is posted to the project mailing list to alert partners of any major changes to the timetable.
It would have been sensible for us to have adopted a process for software development at an earlier stage in the project: it was perhaps a need that we could have anticipated during the SAD I project phase. Also, it is worth noting from our experiences that getting the communications and technical support infrastructure in place is a job in itself, and should be built into the initial planning stage of any large and dispersed project.
Electronic communication is still no substitute for face-to-face meetings so the SPP development team continue to try to meet as regularly as possible. Time is inevitably a major problem wherever project partners have other work commitments: all the project partners based at the hub sites have to juggle SPP work which is for the project as a whole, with that which relates particularly to their own hub's adoption of the project's outcomes. Increasingly, as the project develops, less work will be required from the project "centre" and more at the hubs, leading to an eventual handover of the subject portal developments to the hubs for future management.
It is our plan to make use of UML diagrams in the final project documentation to describe the design and development process. They will offer a detailed explanation of our decision making throughout the project and will give future projects an insight into our methodology. Andy Powell was also so impressed with UML that he is planning to use it across development work for the RDN in the future.
The future development of the SPP beyond the end of the project is likely to be led by the technical development partners, for instance in the continued development of the portlets to enable them to be installed into alternative open source software platforms to make the technology as compatible with existing systems as possible. It is therefore greatly to the benefit of the project that they have become such an effective and close working team.
Ruth Martin, SPP Project Manager
UKOLN
University of Bath
Bath
BA2 7AY
Email: r.martin@ukoln.ac.uk
Jasper Tredgold, SPP Technical Co-ordinator
ILRT
University of Bristol
10 Berkeley Square
Bristol
BS8 1HH
Email: jasper.tredgold@bris.ac.uk
QA Focus is a distributed project, with team members based in UKOLN (University of Bath) and the AHDS (Kings College, London). In addition one of the QA Focus team members is a remote worker. In order to provide effective communications for the team members there is a need for an effective communications infrastructure. This case study describes the communications infrastructure which is employed.
The distributed nature of the QA Focus team means that a good communications infrastructure is an essential part of working practice. The communications tools chosen for use need to be both efficient and easy to maintain, as well as being freely available.
The QA Focus communications infrastructure has been built around a number of separate but complimentary tools.
One of the first communication mechanisms established was a shared file space. MyYahoo [1] is a highly customisable, shared repository. The site provides a number of services including news, bookmarks, maps, calendar and email. The briefcase area allows online storage of files that can then be accessed from anywhere, either by visiting the site or by clicking on links to items. Any type of file can be stored providing they fall within certain content and size guidelines provided by Yahoo. Using MyYahoo the QA Focus team can manage files from work, home or any other location. Yahoo currently provides 30MB worth of free space.
Figure 1: The MyYahoo briefcase
A Yahoogroups mailing list [2] called qa-focus-team was also set up for internal QA Focus use. Yahoo encourage the use of user profiles allowing all their communication methods to be linked together. On setting up your own email account or a MyYahoo Web site you create a profile, this profile can then be assigned an email. Setting up the mailing list involved selecting a Yahoo! Groups Category and deciding on a group email address. Members are then enrolled or invited to join the group. Each individual member can then be configured, allowing them to post, receive and/or receive a copy of all messages to their usual daytime email addresses. The list is maintained and customised by the list owner and lists can be set up for public use or private use only. The main advantage of using the list is the creation of a comprehensive archive. This means that all email information is in an open space and not only held in one person's email box, whom may be on holiday or have changed jobs.
A Blog is a Web log held on the Internet which is updated frequently. Blogging has taken off in recent years and there is now a variety of free software allowing you to set up your own blog without any programming skills. One of the most famous blogs is blogger.com [3]. In order to record activities, ideas etc a blog was set up by the QA Focus team using Movabletype [4], a decentralised, Web based personal publishing system. The blog is currently only accessible internally and is used as a record of activities carried out during the week. These summaries will help with keeping note of work carried out and compiling reports. It is hoped that at a later date the blog will be open for external viewing.
Figure 2: The QA Focus Blog
The QA Focus team have also experimented with forms of instant messaging. These are services that provide users with instantaneous contact with other Internet users. The main advantages of instant messaging are that you can carry out a real-time conversation while involved with other tasks. There is a much higher level of synchronicity than that achieved with an e-mail conversation, so it is useful for high priority work that needs group input.
At one stage the Yahoo IM tool was used. However due to the limitations of this tool members of the QA Focus team agree to move to use of the MSN Messenger tool. This is now used for regular virtual meetings with our remote worker and across the team as a whole. In addition the software is used for short-term tasks for which email is not required, such as arranging meetings.
Setting up the various communication tools is fairly straightforward but can be time-consuming. The real problem is getting users or members of a team to actually use the tools. The core QA Focus team only consists of three people so encouraging use has not been that much of an issue, but occasionally you do find yourself slipping back into old ways of working and using solely email.
Having a good communications infrastructure is key when working in a team, especially when members are distributed remotely. The important factor in establishing use is to document procedures and use the tools diligently at the start so use becomes second nature.
The nature of QA Focus means that all experience and experimentation in a Web related area is always useful and gives us knowledge of both the problems and success areas. However if we had to repeat the process then maybe we would spend more time investigating the different tools available and document their advantages and disadvantages. Unfortunately as most people working on a project will know there is never enough time for research as anyone would like.
Brian Kelly
UKOLN
University of Bath
BATH
Tel: 01225 383943
Minor changes made to details of IM tools by Brian Kelly, 23 February 2003.
The Exploit Interactive e-journal [1] was funded by the European Commission's Telematics For Libraries Programme. Seven issues were published from May 1999 and October 2000.
The original proposal outlined a technical architecture for the e-journal. In addition the workplan included a mid-term technical review which would provide an opportunity to evaluate the technical architecture and provide an opportunity to implement enhancements which were felty necessary or desirable. This document summaries the developments which were made.
A summary of the mid-project technical developments to Exploit Interactive where published as an article in the e-journal [2]. The main developments included:
During the lifetime of the e-journal a number of the externally-hosted services became unavailable.
The project workplan allowed us to review the technical architecture of the e-journal. This allowed us flexibility to enhance the project deliverable. We will seek to ensure that future project proposals allow such flexibility.
Brian Kelly
UKOLN
University of Bath
BATH
BA2 7AY
Tel: 01225 383943
Email: B.kelly@ukoln.ac.uk
http://www.ukoln.ac.uk/
For QA Focus use.
The aim of the Healthier Nation [1] project is to identify suitable learning materials from JISC-funded resource databases and/or content repositories and re-purpose a sample of the material as learning chunks to support Health and Social Care curriculum delivery (at FE and HE Levels. The project has specifically focused on "the Big Four" diseases affecting Scotland: Cancer, Coronary Heart Disease, Stroke and Mental Illness.
The first Strand of the project has been concerned with research and evaluation. The objectives of strand 1 have been to identify and evaluate relevant materials on the "big 4" diseases by:
Academic experts from each partner institution prepared a 'mapping grid' to assist the research team to identify relevant resources. The grid included information on key subject areas, specific keywords and exclusions, courses which could use the material (including the level) and any relevant learning outcomes.
As the emphasis for the academic staff was on finding resources that could be used in teaching situations at FE and initial HE levels - possibly with some element of re-purposing - the research team concentrated on the relevant subject gateways, rather than bibliographic sources and indexes.
To provide a structured framework for the evaluation of the learning material, resource sheets were used to record relevant details. The sheets ensured that the evaluation criteria used by all partner institutions were consistent.
The researcher team evaluated all the learning material on their content (clarity, authority, bias, level) and their style and functionality. Copyright details were also recorded for future re-purposing. Restricted vocabularies were used whenever possible to assist metadata tagging of learning objects. Resource sheets were then passed to academic staff to evaluate their appropriateness for teaching and to indicate how they could be used (delivery type, mapped to level/course, any re-purposing that would be required).
The intention was to carry out accessibility evaluations on a selection of the resources during this part of the project. A key issue that has affected this work has been the lack of agreed criteria for accessibility evaluations. One of the project partners, RNC, has been working with TechDis to develop a model for evaluating resources and the accessibility evaluation of the resources will now be carried out at a later stage of the project.
Following evaluation, the materials were:
The research team had difficulty in retrieving relevant material for the project using the search options in RDN subject gateways. Whenever two subject terms were combined the number of hits was drastically reduced. The search term "Heart" for example, retrieved 312 sites in BIOME; by adding a second term "Heart physiology" this was reduced to 8 sites. Search terminology was often restricted to key areas only, e.g. neoplasm, then the researchers trawled through the numerous hits to find materials – a lengthy process, but the only way to ensure that useful material had not been missed.
Searching under the sub-headings that had been provided by the academic staff produced few or in some cases no hits. A BIOME search for "ACE inhibitors" only retrieved 1 site. To provide enough material for the future strands of the project, Google was also used to locate materials for both mental illness and coronary heart disease/stroke.
On average only one in 10 of the resources located were passed to the academic staff for evaluation. The majority were too advanced, predominantly text based and therefore had no advantages over a textbook, or did not cover the project's subject areas (particularly the sub-headings/keywords).
Over 500 resources were evaluated by academic staff, but only 46% made it to the final repository of resources, or as a supplementary teaching resource. The main reasons for the rejection of the remainder were that the material was:
Academic staff felt that, while some of the resources were excellent (particularly for cancer), in general the resources have not been as good as expected and there were not enough graphic or interactive materials for re-purposing. Mental health resources were geared towards the layperson and had a heavy emphasis on organisations.
Most of the resources went through a secondary evaluation stage to ensure that comments made by FE academic staff were applicable for HE and vice versa. In the secondary evaluation, there was general agreement between the academics in FE and HE about the usefulness of the resources. Although some materials were either too high or too low a level, others were rejected because of their similarity or due to problems of access.
All of the academics involved in the project, felt that they would use alternative sources to locate material. Google was their preferred option as it gave access to relevant material more easily than the subject gateways and has the advantage of advanced search strategies, including searches for images, applying multiple search terms, restricting searches by country of origin.
Addressing points 3 and 4 above should significantly reduce the time tutors would have to spend on searching for resources.
Heather Sanderson
Project Manager
Email: heather@kerson1.freeserve.co.uk
This document is based on one of the Research reports available from the Healthy Nation Web site. For further information please see these reports.
This case study describes the experiences of the INHALE project [1] (which ran from September 2000- March 2003) and the subsequent INFORMS project [2] (which ran from October 2002-August 2003).
The INHALE Project (one of the JISC 5/99 Programme projects) had a number of aims:
The outcomes from the project were:
At the outset of the project the vision of what was required was set out clearly in the project plan and this was closely adhered to throughout.
During the first year the project team successfully created a standalone set of information skills units for students on Nursing and Health courses. The JISC's technical guidelines on interoperability and accessibility guided the Web developer in the creation of the online resources. The new information skills materials using the JISC's DNER resources as well as freely available Web resources were tested within pilot modules in the School of Health Sciences at Huddersfield. Evaluation reports from these were written. These evaluations fed into the continuous "product" development.
During the second year (September 2001- September 2002) additional information skills units were created and some of the initial units were customised. Some of the new resources were based around subscription information databases and were cascaded out for use within the partner institutions, Leeds Metropolitan University and the University of Central Lancashire.
Meanwhile at Huddersfield the resources were being embedded at different levels within Blackboard in new pilot modules.
Running parallel to the delivery of the resources within modules was the continuing development of the INHALE "database". The "database" was seen as the key to enable customisation of the initial set of INHALE materials and the generation of new units by all the partner institutions. This required the disaggregation of all the original materials into objects. Fortunately, from the outset, the vision was that the end result would be a database of learning "objects" and all the materials were created with this concept to the fore. Thus the disassembly was not as onerous a task as it may seem.
Dissemination of the project's learning and outcomes began early in the project and had two key strands. The first was to involve stakeholders in the delivery of information skills within the institutions. Workshops and meetings were held internally and attended by academic teaching staff, librarians, learning technologists, computing service staff and learning and teaching advisors. The second strand was to disseminate to the wider UK HE and FE community and various events were held beginning with an event that was to be repeated "E-Resources for E-Courses".
By July 2002 interest in the use of the INHALE resources had grown. In September 2002, the submission of a proposal for a project within the DiVLE Programme to continue the work of the INHALE project was successful. The new project was named INFORMS and from October 2002 to March 2003, the INHALE project and INFORMS projects ran concurrently. During this time the University of Loughborough and the University of Oxford, (the new INFORMS project partners), were able to test the transferability and viability of all the INHALE project materials and models as well as inputting new ideas for developing the resources.
By the end of the INHALE project in March 2002 there were over 200 units within the new database and a number of new institutions were also testing and using the database.
At this point an exit strategy was written for the INHALE project. The project team felt that there was a possible "market" for the INHALE/INFORMS information skills database within the HE/FE community. However the JISC Programme Managers considered that the database of units required more "market testing" within the HE/FE community. To some extent the INFORMS project has allowed the team to begin the process of market testing.
The INFORMS project officially completed in August 2003 and there are now over 400 units in the INFORMS database and 17 institutions have portfolios of units across the range of subjects studies across the HE/FE community. Usage of the resources can be tracked via a Web log analysis tool developed in house that is linked to the database.
Librarians (and some academic teaching staff) institutions are creating their own online, interactive innovative information skills teaching and learning resources without any knowledge of Web authoring. The database allows instant editing and updating, it automatically produces accessible and printable versions of the units. The 400 plus units in the database are shared across all the participating institutions. Units copied across institutions are tracked via an audit trail. A User Guide, Editorial Policy and Conditions of Use Agreement are all essential documents that have been produced to support users of the database.
There was some initial hold-up in getting the project started and by the time the Project Co-ordinator joined the team in January 2001, the project was approximately 2-3 months behind in writing the initial units, rolling out the baseline evaluation and writing the evaluation instruments. Delivery to the students in the first pilot module was set for mid-February 2001 and this deadline was met.
Manchester Metropolitan University pulled out of the project in June 2001 when they were successful with obtaining funding from the JISC for the BIG BLUE Project.
The Project Director left to take up another post at the University of Central Lancashire in September 2001. The loss of the Director's role as the stakeholder for the project within the Library Management Team and amongst the Academic Librarians had a detrimental effect on the uptake of the resources across the institution that is still being addressed.
The loss of someone else with information skills expertise to bounce off ideas and to provide another point of view on the project's development, as well as mutual support, has been a problem for the subsequent Project Director.
However the move by the Project Director to the University of Central Lancashire was beneficial as UCLan was invited to take the place of Manchester Metropolitan Library on the project and the input from that institution was invaluable.
In November 2001, two months after the demise of the Project Director the project's Web developer was recruited to an internal position in the library. This could have proved disastrous but in fact a new Web developer was recruited from the interviewees for the internal post and began work on the project only 10 days after the original developer had moved.
The new Project Director encountered internal political problems that have constantly hampered the uptake of the resources.
In September 2001 the University of Huddersfield experienced a severe problem with the load on its network. The project was unable to continue development on integrating video and audio into the resources.
The problem with the LAN traffic had a knock-on effect. The central service managing the Blackboard resources plus the learning and teaching support for this was re-organised. Key stakeholders in this support area within the University left so the necessary key personnel to champion the uptake of the INHALE resources in Blackboard were lost. Eventually some new posts have been created.
The Project Director mis-judged the demands that running the two projects (INHALE & INFORMS) during the period October 2002 to March 2003 would make.
The JISC only require projects to make their Web sites available to the rest of HE/FE for 3 years after the end of the project. Thus if a resource has a potential for further uptake and development then the project will need to produce a strategy to enable this.
The University of Huddersfield is not in a position to fund user support for the database. The institution is still in the early days of recovery after its re-organisation of the technical and teaching and learning support infrastructure for Blackboard.
The INFORMS (INHALE) project team have been pursuing a number of possible strategies:
The INFORMS Project team think that there is a commercial potential for the INFORMS software beyond the HE/FE sector and have been successful in a bid for funding to investigate and pursue this further over the next 12 months via a University of Huddersfield Commercial Fellowship.
It is planned that any profit will eventually be used to provide support for the INFORMS database. (Staffed support for HE/FE users of the INFORMS database, support of the Web server hosting the database, support to implement new developments).
The location of the INFORMS resources within an Information Skills Portal alongside the VTS, Big Blue and the Resource Guides, etc. would be an ideal scenario and one that has been suggested already by the Big Blue project.
Both the Open University and Sheffield Hallam have products that may benefit from the technical developments of the INHALE/INFORMS projects.
If demand for portfolios in the database grows then the capacity of the Web server at Huddersfield will be over-reached. So one possible strategy could be to move the database to either EDINA or MIMAS.
Mirroring the database at Edina has been explored and this may be possible in 12 months if the return from the commercialisation of the software is sufficient.
It may be possible to give away the software to HE/FE institutions to run on their own servers and develop should they wish to do so. The main disadvantage of this is the loss of the shared resources.
A case has been put to the JISC, the reply has been that additional evidence of a need must be gathered through "market testing".
For the time being the new INFORMS (Commercial) Project is the route being taken by the ex Project Director of INHALE/INFORMS to create supportive funding in the long term for the INFORMS database of information skills teaching and learning resources. The new INFORMS (Commercial) project began officially on 1st October 2003 and will run for 12 months. One of its first successes has been to secure a place at a reception in the House of Commons being held by the Set for Britain group who are promoting start-up, spin-out, spin-off commercialisation of UK University research. At the reception we will be delivering a poster presentation for the MPs, Peers and various other attendees of the proposed commercialisation of the INHALE/INFORMS software.
The TimeWeb (Time Series Data on the Web) project was a joint project between Biz/ed [1] at the Institute for Learning and Research Technology [2] at the University of Bristol and the JISC/ESRC supported MIMAS service [3] at Manchester Computing at the University of Manchester. The central aim of the project was to develop the key national and international macro-economic time series data banks, such as the OECD Main Economic Indicators, held at MIMAS into a major learning and teaching resource.
The key deliverables of the TimeWeb project were:
The TimeWeb learning and teaching materials and the TimeWeb Explorer were successfully launched into service on Thursday, 14th February 2002 [5]. Through the use of shared style sheets and a common design, movement between the learning and teaching materials developed by Biz/ed and the TimeWeb Explorer Web site developed at MIMAS appeared seamless to the user. Thus Timeweb provided an integrated package of both data and learning and teaching materials.
This case study describes the approaches adopted by Biz/ed and MIMAS to deliver the TimeWeb project deliverables to users and also to embed those deliverables into a service environment to facilitate long term maintenance and support.
In order for the JISC to be successful in its stated aim of enhancing JISC services for learning and teaching, it was imperative that the deliverables from the TimeWeb project were released to users and embedded in a service environment. Both MIMAS and Biz/ed fully understood the importance of releasing the deliverables and promoting their long term use in order to maximise JISC's investment.
In the original project plan, it was intended that the release of prototype interfaces and learning and teaching materials for user testing and evaluation would take place at various stages during the development phase. The objective was that final release of the TimeWeb Explorer and the associated learning and teaching materials would coincide with the end of the project. Once the project ended it was anticipated that the ongoing support and maintenance of the TimeWeb Explorer and learning and teaching materials would be absorbed by the existing MIMAS and Biz/ed service infrastructures.
At the time, these aims were felt to be realistic as both MIMAS and ILRT had considerable experience in transferring project deliverables into services. Whilst MIMAS and Biz/ed successfully achieved the objective of releasing the deliverables into service at the end of the project, the long term support and maintenance has proved more problematic than originally anticipated.
The TimeWeb team encountered a range of problems which had to be overcome in order to achieve the twin objectives of releasing the project deliverables to users and also to embed these deliverables in a service environment to facilitate long term maintenance and support. The following is a summary of the problems encountered and how the Biz/ed and MIMAS teams overcame them:
MIMAS encountered a range of technical problems that needed to be overcome before the TimeWeb Explorer could be officially released to users. To avoid the normal problems associated with the long term support and maintenance of software developed 'in house' MIMAS decided to use a proprietary solution for the development of the Web based interfaces to the time series databanks. The selected solution was SAS AppDev Studio [6] which had been developed by the SAS Institute [7]. The intention was to use the visual programming environment provided by SAS to build a lightweight Java based interface to the time series databanks.
Whilst Java facilitated the development a sophisticated and interactive interface it also resulted in a series of major development problems which had to be resolved. For example, the Java sandbox security model typically does not allow data files to be written to the server or client, an essential step for data downloads. Such development problems were compounded as the TimeWeb Explorer was one of the most advanced projects ever written with SAS AppDev studio, and SAS themselves were limited in the technical help they could provide. The additional staff effort required to resolve the unanticipated technical problems significantly held up development work and prevented MIMAS from releasing the interface for user testing until towards the end of the project. It also resulted in MIMAS shelving plans for the more advanced user interface.
When the TimeWeb Explorer was released for initial user testing a number of unanticipated deployment problems were encountered which caused significant delays. Firstly, the use of the applet required users to install a particular version of the Sun Java Plug-in (Sun's newer releases of the plug-in are unfortunately not backward compatible with earlier versions). AppDev Studio tends to lag behind the latest version of the plug-in produced by Sun and, moreover, different versions of the plug-in could not co-exist on the same PC. This created problems for users unable to install software on their PC due to network restrictions, or for cluster users where the latest version of the plug-in had already been installed. Much work went into finding the best compromise, resulting in a parallel version of Timeweb that ran on later versions of the plug-in also being created. A second deployment problem resulted from the many variations amongst user systems (such as operating system, browser version, download permissions, cache settings or network connection), all of which had some influence on the operation of the TimeWeb Explorer. All these deployment problems had to be fully investigated and documented to allow a wide range of users as possible to use the Timeweb Explorer reliably. Resolution of these technical problems required significant additional development effort towards the end of the project which further delayed the release of the TimeWeb Explorer into service.
Before the TimeWeb Explorer was released to users as a new service, it was necessary to embed it within the existing MIMAS Macro-Economic Time Series Databank Service. As the OECD MEI was updated monthly it was necessary to establish data loading procedures which existing support staff could use. As part of the service integration, it was also necessary to implement and test the access management system required to restrict access to authorised users as required under the terms and conditions of the OECD data redistribution agreement.
It was also necessary to develop a range of support and promotional materials to coincide with the release of the TimeWeb Explorer. MIMAS launched the Explorer alongside an accompanying Web site containing help pages, detailed information on running requirements and links to the metadata for the OECD MEI databank. In addition to email announcements sent out to various lists, a TimeWeb Explorer factcard [8] and an A3 TimeWeb publicity poster were produced and widely distributed. The creation of these publicity materials required assistance from other support staff within MIMAS. In addition, it was also necessary to provide training to MIMAS Helpdesk staff to enable them to deal with initial queries relating to the use of the TimeWeb Explorer.
Having transitioned the TimeWeb Explorer into a supported MIMAS service it soon became apparent that additional effort was required for both on-going maintenance and development of the interface. For example, additional software engineering effort would be required to respond to user feedback/bug reporting and - more importantly - to extend the TimeWeb Explorer interface to provide access to other time series databanks. The loss of dedicated software engineering effort at the end of the project - due to the absence of continuation funding - made the on-going maintenance and development of the interface very problematic.
When the TimeWeb project started in 2000, there were no proprietary systems available that could have been used to provide the required flexible Web-based access to aggregate time series. By the time the project had ended, the Beyond 20/20 Web Data Server (WDS) [9] had emerged as a standard tool for the publication and dissemination of international time series databanks over the Web and was starting to be used by many of the world's largest international and national governmental organisations, such as OECD and the Office for National Statistics (ONS). Not only did the Beyond 20/20 WDS offer the required functionality, it could also be used to import data in a range of different formats. More significantly, the WDS runs in a standard Web browser (IE 4.01/Netscape 4.5 and above) with Javascript enabled thus avoiding the problems associated with Java plug-ins which had been encountered with the TimeWeb Explorer.
In 2002/2003, the MIMAS Macro-economic Time Series Data Service underwent a major transformation as part of the establishment of the new ESRC/JISC funded Economic and Social Data Service (ESDS) [10]. In January 2003, the new ESDS International Data Service [11] based at MIMAS was launched. In order to provide flexible Web-based access to a much larger portfolio of international time series databanks statistics produced by organisations, such as the International Monetary Fund, and to minimise in-house interface development overheads, a strategic decision was taken to standardise on the Beyond 20/02 WDS interface. As a result, an internal project team was set up to plan and oversee the transition from the TimeWeb Explorer to Beyond 20/20 WDS. The project team benefited considerably from the lessons learnt when introducing the TimeWeb Explorer interface into service and the transition to Beyond 20/20 was completed in April 2003.
One of the most significant problems faced in the creation of the learning materials was the sheer breadth of potential data handling skills that exist. There is a wide variety of contexts and qualifications that involve data skills. The Biz/ed team was aware that whilst the Higher Education market was the chief target, the materials would have maximum effectiveness if they addressed other audiences. It follows that supporting the needs of different users is difficult when the user base can be drawn from such a variety of backgrounds.
The main problem faced by the Biz/ed team was in relation to the need for sample data to support the learning and teaching materials under development. This need having been identified, it was necessary to source the datasets and agree terms for their release by the data provider. In this case it was felt appropriate that UK data would be sampled. UK National Statistics were approached in order to gain their approval for a small number of datasets to be held within the TimeWeb suite of learning and teaching materials.
During the period of negotiations with National Statistics there was a change in policy at Governmental level which had the effect of removing all barriers to the use of official data, on the proviso that commercial benefit was not to be obtained. As Biz/ed is a free educational service, this did not pose a problem. However, getting hold of the data codes for the sample datasets added extra delays in being able to finally release the TimeWeb learning and teaching materials.
In preparation for TimeWeb moving into service, it was recognised that the maintenance of up-to-date data was crucial. This involved technical work in creating scripts to run out the data from National Statistics. This occurs on an annual basis. However problems continue to emerge as the codes applied to the data by National Statistics appear to be changed on every update. Thus, on-going maintenance continues to be an issue.
As a non-JISC service at the time of the project, the materials were placed within Biz/ed as a stand-alone resource. Given that Biz/ed became a JISC service in late 2002, there are now issues around the integration of the TimeWeb resource into the service and how they are maintained
One of the key things to come out of the project was how difficult it was to respond to emerging standards and changing requirements both during the development phase and once deliverables have been transferred into a service environment. For example, since the completion of the TimeWeb project, learning objects have emerged as a major theme in e-learning. Migrating the TimeWeb materials to a learning object model and ensuring compliance with new metadata standards (e.g. IEEE LOM) so that that they are reusable and form part of a true resource discovery environment would be a major undertaking which would require additional funding. However, it is very difficult to respond to new funding opportunities, such as X4L [12], when teams and associated expertise have dispersed.
We believe that TimeWeb would have benefited from closer examination of possible project exit strategies at various points during the project. When the project finished in February 2002 there was very little guidance from JISC about future directions. An optimal solution would have been for the project partners - in their roles as service providers - to seek continuation funding for the materials to be updated and the data interface to be maintained. For instance, the sample datasets used within the learning materials could have been adapted to reflect changing interests and events. Whilst we demonstrated successfully that project deliverables could be delivered into service through existing service providers it was clear that additional resources were going to be required for long term support and maintenance. As a project, we should have been more proactive at an earlier stage in terms of making a case to JISC for additional funding.
The detailed planning of the transfer of project deliverables into service was left until towards the end of the project. It would have been better to start the planning at a much earlier stage. It would have also have been advisable to have defined the transfer of deliverables to service as a separate work package in the original project plan. This work package would have needed to be kept under review during the course of the project to reflect changes and developments. However, it was clear from our experience that we had underestimated the amount of software engineering effort required to transfer 'project quality' software to 'service quality'. We also underestimated the amount of additional work that would have to be provided by other support staff to assist with the transfer to service.
Whilst Java held out the promise of developing a sophisticated and interactive interface to time series that would meet the needs of researchers and students alike, we had not fully anticipated the technical problems that would arise. Had we been aware of the pitfalls of the Java route, we would have probably adopted a simpler and more robust database driven approach to delivering time series data across the Web. Rather than trying to fully exploit leading edge technology we should have focused on a less challenging software solution that would have been easier to transfer into service and subsequently maintain.
Whilst the TimeWeb Explorer had a limited service life and was eventually replaced by a commercial system, this does not mean that it was a failure. During the year in service it resulted in a significant increase in the use of the OCED MEI - much of it for teaching and learning. Developing the TimeWeb Explorer gave MIMAS invaluable insights into what was required to deliver international macro-economic time series via an interface that was suitable for both researchers and students. Therefore, TimeWeb has played an important role in the establishment of ESDS International as a major new UK academic data service.
Keith Cole
Deputy Director/Services Manager
MIMAS
Manchester Computing
University of Manchester
Tel: 0161 275 6066
Email: Keith.Cole@man.ac.uk
Andy Hargrave
Biz/ed Research Officer
Institute for Learning and Research Technology
University of Bristol
Tel: 0117 9287124
Email: Andy.Hargrave@bristol.ac.uk
[ Home Page - Surveys - Documents - Presentations - Search ]
Web page by Brian Kelly of UKOLN.
Last Modified 17-November-2003
Email comments to webmaster@ukoln.ac.uk
QA Focus Comments
The SPP project (initially known as SAD I and then SAD II) was funded by the JISC's 5/99 programme.
Brian Kelly, QA Focus, 4 November 2002
Citation Details
Managing a Distributed Development Project: The Subject Portals Project, Martin, R. and Tredgold, J., QA Focus case study 03, UKOLN,
<http://www.ukoln.ac.uk/qa-focus/documents/case-studies/case-study-03/>
First published 4th November 2002.
Changes