Caching helps reach Net edge
Shawn P. McCarthy
Government Web sites are starting to 'cache on,' and that's a good thing.
Caching is one of the tricks developed by Internet portals and search engines to speed page loading and take stress off main servers. Government sites have experimented with regional caching and now are moving into it in a bigger way.
The Census Bureau, for example, turned to a caching service to channel the flood of extra traffic for the release of Census 2000 results. Caching considerably reduced the workload of bureau servers at www.census.gov
Caching services use dozens or hundreds of regional servers spread across the Internet to distribute a customer site's graphics, and sometimes its text and multimedia files, too. Government agencies, however, tend to retain control over their data files on their own servers.
A user who requests a page from a cached site's uniform resource locator receives part or all of the page from a cache server instead.
The cache server stays current by comparing the date and time stamps on its files to the date and time stamps on the originals. When the original files change, the cache server grabs fresh copies, either at a set time or on demand.
It can cost tens of thousands of dollars a month to hire a caching service to distribute your content in this way. But it might be less expensive and more efficient than trying to handle the distribution yourself. If demand increases, you'd have to buy more servers and Internet bandwidth and distribute the load over several machines administered by your own staff.
Plus, caching services let an agency focus more on data gathering, analysis and presentation, and less on system upgrades.
Caching services have run into financial trouble as stock prices dropped and earnings failed to take off. The services have not become the darlings of the Net as some thought they would, mainly because their subscriber base of dot-com companies has shrunk rapidly.
Now that the government is taking its first steps into caching, it could help keep alive a technology that's important to the speed of the Net.
The major caching players are Akamai Technologies Inc. of Cambridge, Mass., Edgix Corp. of New York and iBeam Broadcasting Corp. of Sunnyvale, Calif., which focuses on multimedia delivery.
These companies use edgy words and trademarks to reflect delivery of services close to the Internet's edge. So just where is this edge? In theory, it's the end user's client machine.
But in terms of pipelines, the Net edge is at, or close to, citywide points of presence for T1 lines and dial-up services, Internet cable head ends and digital subscriber line hubs.
Caching services reach these distribution points via land lines and satellite feeds. There is often a master cache that gathers Web content in response to local cache points' requests.
If your agency might soon have to drop a bundle on bandwidth, servers and storage to meet increasing demand for Web information, take time to investigate caching first. It could be a long-term investment that gets you a bit closer to the edge, so you can provide better public service.Shawn P. McCarthy designs products for a Web search engine provider. E-mail him at [email protected].