Los Alamos builds time machine to the way the Web was

The Internet is constantly shifting, with many websites changing their content and appearance almost daily. Such a protean environment poses a problem for archivists or anyone interested in seeing what a home page looked like at a specific date and year. But a new application and standard could potentially turn Web browsers and other software into time machines.

A team of computer scientists at the Los Alamos Research Library in New Mexico and Old Dominion University (ODU) in Virginia has written a technical specification that embeds the concept of time into Internet applications. The specification is part of the team’s proposed information framework, known as Memento, which creates an application that allows the version control of Web pages, databases and other online information sources.

Work on Memento originated from collaborative work by Los Alamos and ODU on digital preservation methods to create long-term data repositories, said Herbert Van de Sompel, leader of the Los Alamos team. The interest in archiving Web pages grew out of this initial work.

There are actually many Web archives, often part of a national museum or archive such as the Library of Congress or the British Museum, Van de Sompel said. But to find old Web pages, users must know where to look. Memento uses a protocol that allows searches across all of these archives, he said.

The Library of Congress’ Internet Archive began in the 1990s. Much of the archiving technology currently in use relies on Web crawling software to take snapshots of Web pages for storage. Most Internet archives typically only archive/search sites in their own countries, Van de Sompel said.

Content management programs such as Wikipedia also record and store versions of a page over time. Memento allows users to access a page as it appeared on a specific date. There are also other ways now available to collect and record Web pages as they appeared on a specific date, but there is no simple, one-step process to look up old pages. “There is a sizable portion of the Web of the past that is available. But accessing it is cumbersome and that’s where Memento comes in,” he said.

Memento does not search at the archive level. Instead it works by time stamping a page version that allows it to be referenced at a later date. Van de Sompel said that one application could be in content management and version control systems such as Wikipedia by using a universal resource identifier, or URI. Universal resource locators, or URLs, are a subset of the URI protocols, which include http:// and ftp:// formats. Memento can also search through dates using a slider graphic. For example, a user selects a newspaper Web site and moves the slider back to a specific date, and Memento will call up the archived page.

By entering a site’s URI into Memento, users can search multiple archives via an HTTP-based search tool. The technology automatically searches all of the Internet’s Web archives and directs the user to the archived copy, no matter its location. This is a considerable advantage over current searches, in which users must know the locations of the archives to access their data. “You have to consult each of [the archives] through search,” Van de Sompel said.

One of the Memento team’s goals is to make the time-searching capability a standard Internet protocol. However, before it becomes widely accepted, the researchers have developed a plug in that can be used on the Firefox Web browser. There is also a mobile version for the Android operating system that is under development, Van de Sompel said.

Memento has many possible government applications for archiving and storage. For example, in the Netherlands, Van de Sompel said many municipalities are actively archiving their Web presences because they anticipate legislation will soon require them to do so. This process has attracted a number of companies that are helping Dutch town and city governments archive their sites.

In the United Kingdom, a law requires government Web pages to have active hyperlinks. Often, old links go dead as a site is updated and data changes. Memento could provide a solution to this by allowing administrators to track back to a time when a link was last active.

Link data also changes over time,  Los Alamos computer scientist Robert Sanderson said. Sites such as data.gov.uk, which list statistical and economic data such as national gross domestic product, change their information over time. So the links for a nation’s GDP would change over the years. A tool such as Memento would allow researchers to locate that specific data, Sanderson said.

In the United States, the Citability group promotes accessibility to public Web sites and archived data. “This is a perfect playing field for Memento,” Van de Sompel said. On the commercial side, he said a large company in the Washington D.C. metro area is talking to the Los Alamos team because it is very interested in archiving its internal Web sites, which include classified and nonclassified data.

To ensure that that the capability gets more attention outside of the world of archivists, Los Alamos is working on a large-scale collaborative effort with the International Internet Preservation Consortium on a project to integrate all of the consortium’s archives. The Los Alamos team is one year into the process. Van de Sompel said.

However, he cautioned that even if it is adopted as a standard, it does not mean that it will be widely used or accepted. Despite these challenges, ultimately Van de Sompel would like to see Memento or a similar archival software tool automatically accessible through popular Web browsers or websites such as Twitter or Wikipedia.