Search Facilities For Your Web Site

Background

Web sites which contain more than a handful of pages should provide a search facility. This is important for several reasons:

• End users will not be aware of the size of your Web site, and so will wish to use a search tool to explore it.
• End users will not necessary understand the browse structure of your Web site.
• Search engines are one of the most widely used tools by end users.

Approaches To Providing Search Facilities

The two main approaches to the provision of search engines on a Web site are to host a search engine locally or to make use of an externally-hosted search engine.

Local Search Engine

The traditional approach is to install search engine software locally. The software may be open source (such as ht://Dig [1]) or licensed software (such as Inktomi [2]). It should be noted that the search engine software does not have to be installed on the same system as the Web server. This means that you are not constrained to using the same operating system environment for your search engine as your Web server.

Because the search engine software can hosted separately from the main Web server it may be possible to make use of an existing search engine service within the organisation which can be extended to index a new Web site.

Externally-Hosted Search Engines

An alternative approach is to allow a third party to index your Web site. There are a number of companies which provide such services. Some of these services are free: they may be funded by advertising revenue. Such services include Google [3], Atomz [4] and FreeFind [5].

Pros And Cons

Using a locally-installed search engine gives you control over the software. You can control the resources to be indexed and those to be excluded, the indexing frequency, the user interface, etc. However such control may have a price: you may need to have technical expertise in order to install, configure and maintain the software.

Using an externally-hosted search engine can remove the need for technical expertise: installing an externally-hosted search engine typically requires simply completing a Web form and then adding some HTML code to your Web site. However this ease-of-use has its disadvantages: typically you will lose the control over the resources to be indexed, the indexing frequency, the user interfaces, etc. In addition there is the dependency on a third party, and the dangers of a loss of service if the organisation changes its usage conditions, goes out of business, etc.

Trends

Surveys of search facilities used on UK University Web sites have been carried out since 1998 [6]. This provides information not only on the search engines tools used, but also to spot trends.

Since the surveys began the most widely used tool has been ht://Dig - an open source product. In recent years the licensed product Inktomi has shown a growth in usage. Interestingly, use of home-grown software and specialist products has decreased - search engine software appears now to be a commodity product.

Another interesting trend appears to be in the provision of two search facilities; a locally-hosted search engine and a remote one - e.g. see the University of Lancaster [7].

References

1. ht://Dig,
   <http://www.htdig.org/>
2. Inktomi,
   <http://www.inktomi.com/>
3. Google,
   <http://www.google.com/>
4. Atomz,
   <http://www.atomz.com/>
5. FreeFind,
   <http://www.freefind.com/>
6. Surveys of Search Engines on UK University Web Sites,
   <http://www.ukoln.ac.uk/web-focus/surveys/uk-he-search-engines/>
7. University of Lancaster Search Page,
   <http://www.lancs.ac.uk/search.htm>