Researcher charts Net growth
Lucent techie spends his days mapping Internet's intricate traffic patterns
This color-coded map of paths
By William Jackson
NATO's recent bombing campaign against Yugoslavia literally changed the map of the Internet.
'Something dramatically cut back on what we could get through to after May 6 or 7,' said William R. Cheswick, who has undertaken the task of mapping the Internet at the Bell Labs research arm of Lucent Technologies Inc. of Murray Hills, N.J.
Cheswick, who spoke at the recent Black Hat Briefings in Las Vegas, said his aim is to put a face on the amorphous collection of networks that make up the Internet. Each day he gathers 100M to 200M of data, tracing pathways through 100,000 routers to try to make sense of the way the Net is growing and changing.
He began paying daily attention to Yugoslavian networks from the third day of the bombing. 'Hey, if they're going to have a war, I want to watch on the Internet,' he said.
Although routers in the Balkan nation started becoming unavailable when the bombs began falling, he has not yet analyzed the data for the effects of the bombing. Cheswick and his partner, Carnegie Mellon University graduate student Hal Burch, still are in the data-gathering stage.
The database, updated daily, appears on the Web at www.cs.bell labs.com/~ches/map/db.gz. Cheswick said he hopes people will look at, manipulate and play with the 9M of compressed raw data of trace-route paths in text files.
Cheswick and Burch began the mapping project last summer. The charts they produced from the trace-route data resemble lichenlike trees. Nodes are not arranged geographically.Their own rules
'Our programs jostle the nodes around according to half a dozen simple rules, simulating springs and repelling forces,' Cheswick said. 'A typical layout run takes 20 hours on a 400-MHz Pentium system.'
He assigns colors to the charts to show geographic locations, Internet service providers, top-level domains and other features.
No one chart shows the entire Internet, Cheswick said, and it is 'my Internet. It may not be yours.' His goal is not to produce a road map but rather to show patterns of growth and development over time, and maybe to produce a good poster or T-shirt.
Cheswick said he scans about 10 percent of the Net daily and does a full scan monthly using trace-routelike path probes. An automated program sends small user datagram protocol packets to random ports. Dropped UDP packets are not re-sent, and the packets' time-to-live value drops with each hop. When time is up, packet deaths are reported back, and the reports reveal paths between routers without bothering working hosts.
''By last September, the project had a database on 61,000 routers, a number that has since grown to 100,000. About 30 sites noticed the pinging within the first six months, Cheswick said. A few requested that it be stopped.
'Military sites noticed immediately,' he said. Now, he warns the Defense Information Systems Agency and the Computer Emergency Response Team before pinging them.
Even so, they enjoy the project, he said. As a civilian, he can gather information the government might not be able to get. 'It's OK for me to ping Finland,' he said, 'but it might be an act of war if the U.S. government did.'
Diagramming a tree with 100,000 nodes
is quite difficult, but the results are visually striking. 'The early results looked like a peacock smashed into a windshield,' Cheswick said.