[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Archiving websites



Martin Firestein wrote:
> I was wondering what questions, decisions, and/or factors are involved in
> archiving information on websites.

I would recommend that you search through the archives to the Archives list for past discussions of
this issue:
http://listserv.muohio.edu/archives/archives.html

You could try a few different search terms including just typing in the phrase "archiving websites"
(without using quotes):
http://listserv.muohio.edu/SCRIPTS/WA.EXE?S2=archives&q=archiving+websites&s=&f=&a=&b=

These past postings raise many of the major issues.  If you could provide more information about what
you might be planning, there are probably more specific considerations to discuss.

I would contend that the technicalities of making copies of pages actually constitute some of the
easiest parts of the puzzle.  There are a number of tools available for capturing web content, many of
which have been used in recent years by projects aiming at long-term preservation.  I've put together
a list of many of them:
http://www-personal.si.umich.edu/~calz/ermlinks/cap_web.htm

If you're looking for examples of large-scale Web capture projects, you may want to check out some of
the following:

Internet Archive
(A lot, though not all, of the Internet since 1996.)
http://www.archive.org/

September 11 Web Archive
(One of several projects aimed specifically at documenting the events surrounding September 11).
http://september11.archive.org/

Kulturarw3
(Capture of the Swedish web.  This project ran into some legal hurdles related to providing access to
the content, but the government very recently gave them the green light to share the captured pages.)
http://www.kb.se/kw3/ENG/Default.htm

Preserving and Accessing Networked DOcumentary Resources of Australia (PANDORA)
http://www.nla.gov.au/policy/plan/pandora.html

Lots of Copies Keep Stuff Safe (LOCKSS)
(Deals with online publications, but provides an interesting low-cost approach to providing reliable
copies of the pages.)
http://lockss.stanford.edu/

Google Cache
(Google keeps copies of pages for about 30 days, which often allows visitors to see the content of
sites that have been recently removed or whose servers or down or overloaded.  Not an archive, in the
sense of long-term presrvation, but it certainly proves that capture can take place on a massive
scale.)
http://www.google.com/

As with the collection of any other materials, you'll have to consider things like selection,
appraisal, retention scheduling (when appropriate), long-term preservation and metadata (descriptive,
discovery, administrative).  You may also want to consider making specific arrangements with the
producers of the Web content, particularly if there's a lot of dynamic content that's being served
from databases.  None of these issues is insurmountable, by any means, but each is worth considering.

If you're interested in how to manage documents and records that are contained within Web sites, you
may want to check out some of the references at:
http://www-personal.si.umich.edu/~calz/ermlinks/webcon.htm

=========================================================
 Cal Lee
 University of Michigan             Phone: 734-647-0505
 School of Information
 http://www-personal.si.umich.edu/~calz/
   "To the extent that archival organizations are concerned
    with current record practices and the content of future
    holdings, they cannot long delay plans for appraising,
    accessioning, and servicing this new [computer] media
    of communication."
                                  - Meyer Fishbein, 1970

A posting from the Archives & Archivists LISTSERV List!

To subscribe or unsubscribe, send e-mail to listserv@listserv.muohio.edu
      In body of message:  SUB ARCHIVES firstname lastname
                    *or*:  UNSUB ARCHIVES
To post a message, send e-mail to archives@listserv.muohio.edu

Or to do *anything* (and enjoy doing it!), use the web interface at
     http://listserv.muohio.edu/archives/archives.html

Problems?  Send e-mail to Robert F Schmidt <rschmidt@lib.muohio.edu>