QA Focus Briefing Documents: Print All - Service Deployment
[ Home Page - Surveys - Documents - Presentations - Search ]
This page is for printing out briefing papers on service deployment. Note that some of the internal links may not work.
This document provides top tips which can help to ensure that project deliverables can be deployed into a service environment with the minimum of difficulties.
Provide a description of the technical architecture of aspects of your project which are intended for deployment into service. The description will be helpful for the service provider. In addition it can help the funders in gaining an appreciation of the technical approaches being taken by projects across a digital library programme as well as being of value to your project team (especially if staff leave).
You should ensure that your technical infrastructure which is capable of implementing your policies. For example, if you wish to make use of XHTML on your Web site you are unlikely to be able to achieve this if you are using Microsoft Word as your authoring tool.
If you are making use of any new standards or unusual technologies you should document this, and explain the reasons for your choice. This could include use of emerging standards (e.g. SVG, SMIL), use of Content Management Systems, etc.
Give some thought to where your deliverables will be deployed. This could be by a JISC Service, within your institution, within other institutions or elsewhere.
You should seek to make contact with the service provider for your deliverables. You should seek to gain an understanding of their requirements (e.g. see [1] [2]). In addition it can help if the service provider is aware of your work and any special requirements associated with your project.
The service provider will need to ensure that there are no legal barriers to the deployment of your deliverables. This can include clarifying copyright, IPR and accessibility issues.
You should ensure that you provide installation documentation which should list dependencies on other software and cover any security or performance issues. As well as the installation documentation you should also provide user documentation which can help the service provide support for end users.
Although it can be helpful of your project team is in a position to provide advice to the service provider after the end of the project, the project team should also be willing to relinquish control over the project if, for example, the service provider needs to make changes to your deliverables.
Learn from the experiences of others. For example, read the case studies which provide various examples of porting systems into a service environment [3] [4].
Be willing to share your experiences. For example, consider writing a case study for QA Focus [5] [5].
Projects deliverables are normally expected to be deployed in a service environment. The deliverables could be passed on to an existing JISC service provider. In some cases, however, a project may evolve into a service. This document outlines some of the issues that need to be considered to facilitate such a transition.
If evolving to a service is not relevant to your project, the issues services need to address when deploying your project deliverables may still be of interest.
Hosting of your project deliverables is one of the first issues to be considered. A prototype service may be developed on in-house equipment and in an environment which may not be appropriate for long term production. Issues to consider include:
Your service may require regular updates of the raw data which the service is delivering to users. Issues to consider when moving into a production environment include:
The JISC supports a range of subject-specific gateway services. Decide which gateway, if any, your service fits into. The subject matter of your service may span more than one area and therefore need to be incorporated in more than one gateway.
Review the RDN [1] and the services within it and see where a description and link to your service may fit. Arrange for your service to be made visible. The more links that are established to your service, the more likely it is to become visible to search engines such as Google and the more successful it is likely to be in terms of awareness and take-up.
When an experimental or development system is turned into a production service, there are a number of copyright, licensing and other legal issues that need to be carefully considered.
Does your service contain any material that is subject to copyright or IPR legislation? This could include such things as images, artwork, extracts from publications, sound or movie clips and so on. If it does, you will need to get permission before you can 'publish' your site.
Have you considered how accessible your service is to those with special needs or disabilities? There are now legal obligations that need to be taken into account before releasing a new system.
The JISC TechDis service [2] provides information on how to make your Web site conform. Also consult the appropriate QA Focus document on Accessibility Testing [3] and the JISC Legal Information Service [4] for a range of advice on issues such as Data Protection, Freedom of Information, Disability and the Law, Intellectual Property and much else.
As soon as you have a reliable release date, publicise the fact on relevant JISCMail and other lists. Keep people informed as to the progress of the new service as launch day approaches.
As soon as delays appear inevitable, let people know, even if a revised date hasn't been fixed. This will help front-line staff, who will have to support your service, decide on their own local information strategy.
The move of an experimental or development service into a full production service provides a 'hook' for raising its profile. Things to consider include:
Consider the kind of support and publicity materials that are appropriate for your service. Examples include:
Think about the target audience for the material. You may want to produce different versions for users from different backgrounds and experience. Consider which items may be worth printing (as opposed to being made available on the Web). For example posters and flyers are useful for distribution at events such as conferences and exhibitions. Review what other JISC services have done and discuss their experiences with them.
You should also seek advice from the JISC's Communications and Marketing Team [5] who maintain a register of key events and are able to help with such things as preparing and issuing press releases.
Once your service is in production there will be a requirement to improve or update the service and to fix problems. User feedback on suggested service improvements or errors should be gathered through a contact publicised on the service's Web site.
Presentations and demonstrations provide forums for discussion and constructive criticism. Find out if there is an existing user group who will extend their remit to cover your service.
When changes are identified and implemented, ensure that the change is publicised well in advance. Unless the change is an important bug fix, try to make the changes infrequently, preferably to coincide with term-breaks.
Check if your service will come under the remit of the JISC's Monitoring Unit [6]. If it does, you will need to agree a service level definition with them. Typically you will also need to:
For some projects, it will be clear from the start that the intention is to transition the project into an end-user service, either hosted by the project itself, or by another host such as a national data centre.
Other projects may have the potential for development into a production service, but without this being a declared aim of the project.
In both cases, it is sensible to think carefully about how the system might fit into a service environment at the planning and design stage, to avoid costly re-engineering and retro-fitting of features later on.
The software regime that may seem most appropriate for an experimental development environment may not be the best choice when running a large-scale end-user service. Issues to think about include:
A key factor in the success of any project is careful preparation and planning. If you intend your project to develop into an end-user production service, it is worth spending time and effort in the early stages of the project testing your ideas and designs. It is easier to rewrite a specification document than to re-engineer a software product.
Depending on the nature of the project, some of the following may be worth considering:
Controlling access to your service may not be an issue when it is in an experimental or development phase, but will become an important consideration if it is released into service.
Some issues to review include:
When your project reaches the stage of being turned into an production service with large numbers of users, consideration will need to be given to issues which are less important during the development phase.
It is helpful to be aware of these at an early stage in the planning and design of the project to avoid difficult problems later. Some things you should think about include:
It is to be expected that a Web-based user service will require maintenance, revision and updating during its lifetime. There may be requests for new features, or for modifications to the way existing facilities work.
Bear in mind that the people doing this work may not be the original project team that created the service. It is important that the end-products are designed and structured in such a way as to allow parts of the system to be modified and updated by others who are less familiar with the system without unexpected consequences.
Therefore, when starting to develop a new system:
The start of your project will involve a great deal of planning and work scheduling. If you will be developing software, this is also the best time to consider and plan for its long-term future and viability. Decisions on software development made in the early stages of a project are important as they will often govern the options open to you for deployment beyond the life of the project. Although some choices may be influenced by the current technical environment of your institution, early consideration of a range of deployment issues will allow the possibility of a greater number of hosting options at the end of project, so ensuring continued existence of the software you have developed, and long-term access to it.
Careful choices will also reduce the cost of the work required for deployment, and allow you to minimize the portion of your budget that you have to allocate to the Service Provider.
If possible, software should be developed on the same platform that will eventually be used for service delivery. Microsoft Windows and Unix (especially Solaris) servers are the main options.
Porting software developed on one platform to another may not be straightforward, even if the chosen application software is claimed to run on both platforms. Proven technical solutions are preferred - do you have examples where your chosen application software has been used on both platforms?
If software licenses are required by the Service Provider, these must be available at a cost within the service budget. Be aware of licensing conditions: a Service Provider may require a commercial license if a charge is to be made for the service, whereas the project may be able to use an educational license. The cost of the various types of license may vary.
Care is also needed when choosing software that is free at the point of use to project staff, such as a University site licence for a commercial database system. Even though the project itself may incur no additional costs, licences could be prohibitively expensive for the Service Provider.
Consider the use of open source software [1] to avoid most licence problems! Good quality open source software can greatly reduce the cost of software development. Developers should be aware, however, that some 'open source' software is poorly written, inadequately documented, and entirely unsupported. Be aware that the costs of ongoing software maintenance, often undertaken by staff outside the original project, should also be factored in.
Good programming practice and documentation is very important. Well-written and structured software with comprehensive documentation will ease transition to a service environment and aid the work of the Service Provider [2]. It is better for a project to recruit a good engineer used to working in a professional development environment, than to recruit purely on the basis of specific technical skills. Also, try to code in languages commonly adopted in your application area: for example Java or Perl for Web programming. You can write Web applications in Fortran, but don't.
If possible a modular architecture is best. It will maximise the options for transfer to a Service Provider and also any future development. For example, if one application were used for a Web user interface and another for a database back end then, provided these communicate using open standards (Z39.50, standard SQL, for example), Web Services might be added to the service at a future date. A service built with a fully integrated single package of components that use proprietary native protocols might have to be scrapped and rebuilt to satisfy even fairly minor new requirements.
Use of Open Standards [3] should ensure portability, but there will still need to be technical structures supporting their use and deployment, whether in a project or service environment. You will need to document all the technical layers that need to be reproduced by the Service Provider in order for your software to run. Open standards can also give flexibility; for example the project and the service provider do not necessarily need to use the same SQL database, provided the standard is followed.
Be aware of your intended user base. Ensure that any user interface developed during the project has been through usability tests and allow time for any feedback to be incorporated into the final design. A well-designed interface will mean less support calls for the Service Provider.
When designing your user interface remember that there are legal requirements to fulfil with regard to disability access which the Service Provider will need to be satisfied are met. The JISC TechDis [4] service provides information and advice. You may wish to consider provision of user documentation and training documentation in support of the service, which the Service Provider could use and make available.
Comprehensive error reporting should be a feature of the deployed application. This will aid the Service Provider in identifying and solving problems. You should consider building comprehensive error reporting mechanisms into your software from the beginning, along with various mechanisms for escalating the severity of reported errors that threaten the viability of the service. These may range from simply logging errors to a file, through to emailing key personnel.
Services must be monitored. It should be possible to use a simple HTTP request (or equivalent for non-Web interfaces) to test the service is available and running, without requiring a multi-step process (such as log in, initiate session and run a search).
Logging is crucial for services, especially subscription services where customers need to monitor usage to assess value for money. Project COUNTER [5] defines best practice in this area. If project staff are still available, the Service Provider will then be able to provide you with logging information and potentially seek your input on future activity and development.
Authentication and authorisation should be flexible since requirements are subject to change. Enable the service provider to execute an external script, or at least write their own module or object, rather than embedding the authentication mechanism in the user interface code.
Where the product makes use of external middleware services (an example being for OpenURL support), ensure these are totally configurable by the service provider. Configuration files are good; but the ability to add modules or objects for these features is better.
Although not a technical consideration, it is important and worth emphasising that the Service Provider will require that all copyright and IPR issues be clarified. Where software has been developed, does the institution at which project staff work have any IPR guidelines that must be followed? What provision is needed to allow the Service Provider to make changes to the software? Is a formal agreement between the project institution and the Service Provider needed?
If you have identified where your software could be hosted then make early contact with the Service Provider to discuss costs and any constraints that may arise in deployment.
The Service Provider will need to be confident that your application will be stable, will scale, and will perform acceptably in response to user demand. If this is not the case then the application will eventually bottleneck and tie up machine resources unproductively which will lead to unresponsiveness.
You should ensure that the application is stress tested by an appropriate number of users issuing a representative number of service requests. There are also several tools available to stress test an application, but a prerequisite to this step is that the project team should be aware of their intended user base and the anticipated number of users and requests. The project team should also be aware of project and service machine architectures as divergence in architecture will affect the viability of any stress testing metrics generated. The Service Provider will want estimates of memory and processor use scaled by the number of simultaneous users. Performance and scalability will remain unresolved issues unless the project software can be tested in a service environment. If this is the case it is especially important to stick to proven technical solutions. You should discuss stress-testing results with your Service Provider as soon as possible.
Adopting best practices is a good start to ensuring that your application will be stable. The discipline of producing well-written and properly documented code is one safeguard against the generation of bugs within code.
If there are likely to be service updates you will need to consider the procedures involved and detail how the new data will be made available and incorporated into the service. Service Providers will generally wish to store two copies of databases that require updates; one being used for service with the other instance being used for updates and testing. Updated databases will also require frequent backups whilst static data requires only one copy of the database without regular backups. Consider splitting large data sets into separate segments: a portion that is static (for example archive data added prior to 2001) and a smaller portion that is updated (data added since 2001). Also, aim to keep data and application software as separate as possible. Again, this will aid a backup regime in a service environment.
You should anticipate that the Service Provider may need to make changes to your software. This may be due to possible technical conflicts with other services hosted by the Service Provider, or may be due to their implementation policy or house style. Again, early contact with a possible Service Provider will highlight these issues and help avoid potential difficulties. Also consider if project staff will be available for referrals of errors or omissions in functionality. If not, you will need to allow the Service Provider to make changes to your software.
If further development of the software beyond the project is feasible you should agree a development schedule and a timetable for transfer to production, as provision of a continued and stable service will be of prime importance to the Service Provider. Major changes to a user interface will also have implications for support and user documentation. If no continued development is planned, the Service Provider may still wish to introduce bug fixes or new versions of any software you have used. Again, good documentation and well-documented code will ensure that problems are minimised.
You should consider under what circumstances your software should be withdrawn and cease to be made available through a Service Provider. If you would expect to be involved in the decision to withdraw the service then contact with project personnel will need to be maintained, or you will need to provide guidance at time of transfer to service about the possible lifetime of the hosting agreement.
Allow time before the end of the project to work with the Service Provider. The availability and expertise of project staff will influence the success of moving to service deployment and ultimately the associated costs.
A complete handover of the software without good contact with the project team and without support may well cause problems and will also take longer. This is particularly true if the application contains technologies unfamiliar to the Service Provider. The project team should be prepared to assist the Service Provider in areas where it has specialist expertise and, if possible, factor in continued access to project personnel beyond the end of the project.
Complete and full documentation detailing the necessary steps for installation and deployment of your software and the service architecture, will aid an optimum transition to hosting by a Service Provider. The Service Provider may not have exactly the same understanding and skill set as the project team itself, and will require explicit instructions. Alternatively, the Service Provider may request help from the project team in identifying a particular aspect of the service architecture that could be replaced with a preferred and known component. Deploying technologies that are unfamiliar to the Service Provider will reduce their responsiveness and effectiveness in handling problems with the application.
Consideration should be given to development of a test bed and test scripts that will allow the Service Provider to confirm correct operation of your software once installed in the service environment.
Backup and disaster recovery procedures are the responsibility of the Service Provider; do not waste project time on defining specific procedures for the service (but do, of course, back up your project work for your own benefit).
Project activity is by its nature about exploring possibilities to develop new service functionality, and you may choose, or need, to utilise emerging tools and technologies. This approach to development and the software it produces may not fit comfortably with the desire of the Service Provider. A Service Provider wants software of service quality based on known solutions that ensures good use of resources and sustainability in a service environment. It is recognised that these opposing drives may be inevitable and that projects must be allowed to explore new technologies and methods, even at the expense of placing additional demands on Service Providers to resolve the problems of deployment.
If relatively immature technologies are being used it is very important that modular development procedures are used as much as possible. Where software has been developed in a modular fashion it will often be relatively straightforward to replace individual components; for example, to change to a different database application or servlet container. During the development process this means competing technologies, which may be at different stages of maturity, can be benchmarked against each other. At the deployment stage it means the option that provides best 'service quality' can be adopted.
Whatever choice of software environment is made, it is always wise to follow best practise by producing well-written and documented code.
It is worth stressing the benefits of contacting possible Service Providers to explore options at the start of a project: they too may be considering future strategy and it is possible that both your and their plans might benefit from co-operation.
Digital repositories are often thought of primarily as a computer system, consisting of hardware, software and networks, but they are more than this. Digital repositories are organisations similar in purpose to libraries or archives and, just as it does for these organisations, quality assurance should form an integral part of the work of a digital repository.
A digital repository should:
A digital repository intent on long-term retention of its holdings should conform to the Reference Model for an Open Archival Information System (OAIS) [1].
Useful information is available in the QA Focus Briefing papers on "From Project To Production Service" [2] and "Planning An End User Service" [3].
Quality assurance can be incorporated into the work of a digital repository through the establishment of formal (but not necessarily complex) policies and procedures.
The CEDARS project suggested that collections management policies should cover: selection, acquisition, organisation, storage, access (user registration and authentication, delivery of masters versions), de-selection, and preservation. Policies developed to cover these topics should be subject to internal and external review as part of a formal approval process. Policies should be reviewed at regular intervals.
Policies should be written to conform to the requirements of relevant legislation, notably the Data Protection Act, 1998.
The day-to-day operation of the repository should be connected to its overall policy framework through the development of procedures. Procedures should be:
Digital repositories need to make use of a wide range of standards and best practices for data creation, metadata creation, data storage, transmission and for many other areas. Many of these topics are discussed in more detail in other QA Focus documents. Selection of technical standards should take particular account of the guidance in QA Focus briefing papers on "Matrix for Selection of Standards" [4] and "Top Tips For Selecting Open Source Software" [5].
A digital repository should operate within a clear legal framework that establishes the rights and responsibilities of the repository, its depositors and its users. A formal agreement should be established between each depositor and the repository, by way of a signed licence form or other technique (unavoidable online licence agreement).
This agreement should limit the liability of the repository (e.g. where a depositor does not have copyright), while conferring the repository with rights to manage or withdraw content. Otherwise, the depositor's rights should be protected and any limits on the service provided by the repository should be made clear (such as limits on how long data will be stored, and whether migration or other preservation actions will be undertaken).
[ Home Page - Surveys - Documents - Presentations - Search ]
Web page by Brian Kelly of UKOLN.
Last Modified 30-June-2004
Email comments to webmaster@ukoln.ac.uk