Bots & search tools
Bots & search tools<@VM>Comb the Web for sweetheart deals using automated information-gathering sites
These tools can scour the Web
for what you needBy Drew Robb
Special to GCN
Online purchasing, in theory, can deliver the highest-quality products at the best available price because online shopping puts the entire market at your fingertips. The problem is that the Internet contains approximately 575,000 times as many pages as the Los Angeles Business-to-Business Yellow Pages. Your fingertips would have to do a lot of walking to get through them all.
That's where bots come in.
A robot, or bot for short, is a program that acts as an agent for a user or another program, or simulates a human activity.
Robotic software generally is believed to have originated with Eliza, an artificial intelligence program developed by Massachusetts Institute of Technology professor Joseph Weizenbaum in 1966 to study natural language communication between humans and machines. Eliza was programmed to act like a psychoanalyst, answering questions with further questions, and it reportedly could be just as frustrating to talk to as a real shrink.

Bots have come a long way. They provide dozens of different online services, assisting not only with purchasing but also with such things as finding news on specific subjects or keeping track of companies within a specific, regulated industry. Some could be of particular use to government information technology shops and other offices:
Newsbots. Newsbots continuously monitor online news services and notify a user when articles meeting programmed criteria appear. Users specify both the types of news being sought and the sources to be watched to tailor the news to their exact needs.
A programmer can easily set one up to keep the IT department up to date on computer news, one to inform the public relations department of agency press coverage and another to track relevant legislative developments.
Shopbots. Although shopbots are often promoted as last-minute shopping tools, they can be harnessed effectively by government purchasing departments to locate the best price on a Sun Microsystems Ultra 60 Model 2450 server, for example. Some shopbots track the price of an item over time, with graphs showing if the price is going up or down.
Shopbots also can be set to notify you by
e-mail when a product is available at the price you specify, check if the item is available on auction sites, perform automatic price comparisons with other suppliers and compile ratings on the technical support provided by online suppliers.
Because each shopbot specializes in specific types of products, an office might use separate shopbots for computer purchases, travel arrangements and other purchases of services or products. At the very least, they can make it easier to get the requisite three quotes for every purchase order.
Spiders or Web crawlers. Spiders are robots that follow hyperlinks and make copies of the linked pages to create directories that can be used by search engines.
In maintaining a site for an agency or department'whether it's an intranet, extranet or Internet site'a spider can be used to index your own Web site, as well as those of other specified agencies or organizations, to keep the site's search engine completely up to date.
For example, the Occupational Safety and Health Administration works closely with the National Institute of Occupational Safety and Health, and its search engine pulls up documents from both organizations' sites.
Update bots. These monitor various sites and inform the user when information has been updated. Depending on the type of bot selected, the user can be notified by an e-mail message, through a customized Web page or by accessing a file that tracks all changes since it was last accessed.
For example, an update bot can track new information posted by major companies or trade groups of the industries an agency regulates. Similarly, in managing a centralized Web site to which people in different offices can post data, the update bot monitors the Web site for changes to post on the 'What's New' page.
Data management bots. There are several types of data management bots. One stores personal information and automatically inputs it into online forms so they don't have to be filled out manually each time. Besides saving time for one's own staff, the forms could also be offered by an agency's site to improve the public's interface with government.
Other data management bots take documents and mine them to create keyword lists or to generate summaries. These keywords and summaries are then used by search engines or internal document storage and processing systems. This enables users to readily locate data without requiring someone to manually summarize documents.
Intelligent service agents. Intelligent service agents act like human employees to answer frequently asked questions. They are a step up from voice mail information systems in that people can ask questions verbally and get answers to those exact questions.
Although these bots cannot handle all questions, they can reduce the number of calls and e-mail messages agency personnel have to answer.
Bot design bots. Although hundreds of bots are readily available, there also are bots that assist in creating robots or intelligent agents to meet exact needs, which can save programmers time.
There are two excellent online reference sites on bots, explaining how to get them and how they can simplify tasks.
The BotSpot, at
www.botspot.com, contains articles explaining the different types of bots and how they work, as well as reviews of several hundred of them.
Webcrawler has a database containing technical data about more than 200 Web robots at
info.webcrawler.com/mak/projects/robots/active.html.
With the Internet now approaching 1 billion pages, it is impossible for one person to keep track of what is happening even in one sector of it.
Using robots and intelligent agents, however, makes it possible to get the most out of the vast amounts of data available.
We are still a few years away from Jetsons-like robots that clean the house and do the laundry, but software robots are at least available to help with procurement and other routine tasks.
Drew Robb of Tujunga, Calif., writes about information technology.
Site |
Type of site |
What it does |
Product specialty |
Active Buyers Guide www.activebuyersguide.com |
Intelligent shopping agent |
Teaches a user about the features available in a particular product category and then asks a series of questions to determine which product best fits |
General consumer and business |
Auction Octopus www.auctionoctopus.com |
Auction search |
Searches auction sites in real time |
General consumer and business |
Auction Rover www.auctionrover.com |
Auction search |
Searches auction sites and provides links to 850 specialized online auctions |
General consumer and business |
Auction Watchers www.auctionwatchers.com |
Auction search |
Searches online auction sites for computer items |
Computer products only |
Bidder's Edge www.biddersedge.com |
Auction search |
Searches 77 auction sites for specified items |
General consumer and business |
Buyer's Index www.buyersindex.com |
Comparison |
Indexes items from 16,000 Internet and conventional retailers; has print catalog information online |
General consumer and business |
CNET Shopper www.shopper.cnet.com |
Computer products shopper |
Has product listings and price comparisons for computer products |
Computer products only |
Consumer World www.consumerworld.org/ pages/shopping.htm |
Consumer information site |
Provides product information and price comparison |
General consumer and business |
eSmarts www.esmarts.com |
Consumer-oriented buying site |
Provides product and company information and links to suppliers |
General consumer, including banks and long-distance phone service |
Frictionless Value Comparison Engine www.frictionless.com |
Intelligent shopping agent |
Has 150 product categories and data on features of 3 million products; takes user through a series of questions to determine the best product for the specified application |
General consumer and business |
InfoSpace Compare Price vo.infospace.com |
Product search |
Lists companies that offer a specified product |
General consumer and business |
Jango www.jango.com |
Product search |
Has pull-down menus to specify the type of product you want; offers option of looking for product reviews or product prices |
General consumer and business |
mySimon www.mySimon.com |
Comparison shopper |
Does price comparisons, tracks prices over time |
General consumer, office and computer products |
Online Shopping Directory www.samizdat.com/ shopping.html |
Directory |
Provides a broad directory of sites with links to several hundred shopping, software and reference sites |
Consumer, business, government and education products |
PriceScan www.pricescan.com |
Comparison shopping |
Searches Web sites and print ads for prices; has pull-down menus for selecting products |
General consumer, office and computer products |
R U Sure Shopping Agent www.rusure.com |
Price comparison plug-in |
Searches supported sites for the best price on an item |
General consumer, office and computer products |
StreetPrices.com www.streetprices.com |
Comparison shopping |
Conducts product searches among 250 online merchants; produces a directory of products and prices |
General consumer and business |
TechBargains www.techbargains.com |
Product recommendation |
Provides technical articles and product comparisons; will recommend products and explain why, and provides links to suppliers |
Computer products |
TechShopper www.techweb.com/shopper |
Product recommendation |
Provides pull-down menus to give product specs and identifies products that meet those specs |
Computer products |