SUBJECT "SEARCHING" From: jik@athena.mit.edu (Jonathan I. Kamens) Date: 14 Feb 1993 06:01:30 GMT Organization: Massachusetts Institute of Technology [While this is focused on finding source code it gives some good general searching information. Souce code sites deleted. DFP] I. Table of contents II. Introduction III. How do you find sources? IV. Things not do do V. Searching techniques in detail 1. Usenet source newsgroups 2. The "archie" database 3. Tom Czarnik's anonymous ftp directory 4. The "ftpable-readmes" WAIS database 5. Comp.archives 6. Comp.sources.wanted 7. Prospero 8. Mail server queries VI. File retrieval techniques in detail 1. By anonymous ftp 2. By anonymous UUCP 3. By mail a. Hrc mail archive b. Uunet mail archive c. BITFTP d. Ftpmail e. AT&T's netlib archive f. Periodic posting archives g. Trickle mail servers h. Other mail servers VII. Credits ---------------------------------------------------------------------- II. Introduction This posting discusses the resources available to people who are looking for source code. Please read it before posting source code requests to comp.sources.wanted, alt.sources.wanted or any other newsgroup. III. How do you find sources? 1. "I know the name of the program I want, but I don't know where to get it." a. Check the indices for the various main Usenet source newsgroups. b. Check with archie. c. Check Tom Czarnik's anonymous ftp directory. d. Check comp.archives. e. Check the "ftpable-readmes" WAIS database. f. Post to comp.sources.wanted. g. If you are looking for an MS-DOS program, check pd1:simlist.arc on wsmr-simtel20.army.mil or /pc/INDEX.ZIP on garbo.uwasa.fi, available via anonymous ftp (see the section below about how to retrieve a file via anonymous ftp). 5. "I am looking for source code that does ," where is something algorithmic or mathematical that is commonly solved with computers. Check AT&T's "netlib" archive (which is accessible via E-mail as described below, or via anonymous ftp to research.att.com {username "netlib" instead of "anonymous"}), and if that doesn't work, proceed as in (3) above. 9. "I'm looking for the source code for Unix." Most implementations of Unix contain source code that is, at least to some extent, proprietary and not freely redistributable. If you are looking for the source code to a particular Unix utility, you may have better luck looking for a public-domain reimplementation of that utility, using (1) or (3) above. Furthermore, the Free Software Foundation (which is dedicated to the goal of making high quality free software, including a complete Unix-compatible software system called GNU, available for everyone) may distribute a freely redistributable version of the utility, protected by the GNU Public License (Tom Czarnik's list mentions many anonymous ftp archive sites for FSF software; furthermore, the hrc mail server archives some FSF software). *** There are two freely redistributable implementations of Unix. The first is 386BSD, which is based on BSD Unix sources, and the second is Linux. Both of these run on 386-class machines. For more information about them, see the comp.unix.bsd and comp.os.linux newsgroups, or look for 386BSD and Linux in the source archives using the instructions above. To find anonymous ftp archives of Usenet newsgroups, check Tom Czarnik's anonymous ftp directory (see below). The most well-known Usenet newsgroup archive is probably ftp.uu.net, which archives comp.sources.3b1, comp.sources.amiga, comp.sources.games, comp.sources.misc, comp.sources.reviewed, comp.sources.sun, comp.sources.unix, and comp.sources.x, among other things. Another large Usenet archive site is wuarchive.wustl.edu (128.252.135.4), which archives alt.sources, comp.sources.mac, and comp.sources.apple2, in addition to most of the newsgroups archived on ftp.uu.net. A very large European anonymous ftp site is nic.funet.fi (128.214.6.100), which archives Usenet newsgroups and mirrors several foreign specialized ftp servers. Other large European archive sites are mcsun.eu.net (192.16.202.1) and inria.inria.fr (192.93.2.1). If you are in Europe, you should look on one of these sites for things you need before trying sites elsewhere. The mail archive server on hrc.UUCP (see below) archives alt.sources, comp.sources.unix, comp.sources.games, comp.sources.misc, comp.sources.x, and comp.sources.sun, among other things. 2. The "archie" database "Archie" is a database of anonymous ftp sites and their contents. The software for it was written by the "Archie Group" (Peter Deutsch, Alan Emtage, Bill Heelan, and Mike Parker) at McGill University in Montreal, Canada, and they maintain the database as well. "Archie" keeps track of the entire contents of a very large number of anonymous ftp sites, and allows you to search for files on those sites using various different kinds of filename searches. Archie also has a software description database (with contents similar to the output of the "whatis" command under Unix), on which you can do keyword searches. The following machines are currently running and advertising the archie service: Host name Internet address Country ------------------- ---------------- -------------- archie.ans.net 147.225.1.2 United States archie.au 139.130.4.6 Australia New Zealand archie.doc.ic.ac.uk 146.169.11.3 United Kingdom archie.funet.fi 128.214.109.110 Finland archie.mcgill.ca 132.206.2.3 Canada archie.ncu.edu.tw 140.115.19.24 Taiwan archie.rutgers.edu 128.6.18.15 North America archie.sura.net 128.167.254.179 United States archie.unl.edu 129.93.1.14 North America archie.wide.ad.jp 133.4.3.6 Japan cs.huji.ac.il 132.65.6.5 Israel Archie sites may soon be available elsewhere as well. You can access archie at most of these machines in one of three different ways: 1. Telnet or rlogin to the machine with username "archie" and no password. 2. Send mail to archie@machine (e.g. archie@archie.au). Send a message with "help" in the body to find out more. 3. Use one of the Prospero-based archie clients. The site in Japan, archie.wide.ad.jp, supports only the third method. The third method listed is the preferred one, because it puts less of a load on the archie servers and is faster than the other two methods (its one drawback is that it does not yet provide an interface to the archie "whatis" database). If you already have Prospero installed at your site (see below), then you can search archie by vcd'ing to the directory /archive-sites/archie/regex and using "vls" with the regular expression for which you want to search. Alternatively, you can get one of the stand-alone archie clients that does not require all of Prospero in order to run. The clients currently available include a C version with a text interface, a C version with an X interface, and a perl version with a text interface. They are available for anonymous ftp in /archie/clients or /pub/archie/clients at each of the archie sites listed above. When using archie, you should connect to the site from the list above that is closest to you, network-wise. If you would like more information about archie, you can write to archie-l@cs.mcgill.ca. (Note to Janet/PSS users -- the United Kingdom archie site is accessible on the Janet host doc.ic.ac.uk [000005102000]. Connect to it and specify "archie" as the host name and "archie" as the username.) 3. Tom Czarnik's anonymous ftp directory Tom Czarnik maintains an extensive list of anonymous ftp sites and overviews of their contents. The list is posted monthly to comp.misc, comp.sources.wanted and alt.sources.wanted. If it has expired at your site, see the documentation below about retrieving postings from the periodic posting archive. 4. The "ftpable-readmes" WAIS database Vincent Cate maintains a WAIS database of README files for various packages on anonymous ftp sites all over the Internet. The database is called "readmes", on port 210 of the host alex.sp.cs.cmu.edu [128.2.209.13]. For more information, connect to the WAIS server on that host and search for README in the "INFO" database in order to retrieve the entire source for the ftpable-readmes database, or retrieve it from think.com's directory-of-servers WAIS database. For more information about WAIS, retrieve the file /wais/README from the anonymous ftp server think.com. 5. Comp.archives The comp.archives newsgroup, moderated by Adam J. Richter , contains announcements of archive sites and their contents. If you cannot find what you're looking for in the comp.archives postings available at your site at any given time, then you can read the newsgroup for a while and watch for new postings that are of interest to you, or you can try to find an archive site that archives the postings in comp.archives (e.g. wuarchive.wustl.edu, cs.dal.ca). Furthermore, comp.archives is accessible via WAIS, in the database named "comp.archives" on port 9000 on talon.ucs.orst.edu. If you don't know what WAIS is, don't worry about it. 7. Prospero If you are a Unix site on the Internet, you can use the Prospero system (whose author is Clifford Neuman) to search through archives of various sorts all over the Internet, and to retrieve files once you have found them. Prospero uses a virtual filesystem which allows you to transparently view directories and retrieve files. There is some overlap between Prospero and other resources mentioned in this document; for example, both archie and the periodic posting archives on pit-manager.mit.edu are accessible via Prospero. The Prospero software is available in /pub/prospero.tar.Z on cs.washington.edu; the user software may already be installed at your site, and if not, you can get it and install it yourself. For more information about Prospero, send mail to info-prospero@isi.edu. 8. Mail server queries Anonymous ftp is most effectively used only for retrieving files and not for searching for them, since it is a file transfer protocol and not much more than that. However, many (if not all) of the mail archive servers which allow file retrieval by electronic mail provide a more functional (albeit slower) interface than ftp which allows you to query the servers to find out what they have available on them. Therefore, if you find it necessary to use mail archive servers to get files, take advantage of the indexing and search features available on the servers. The features of individual servers can not be documented here, because there are too many different servers running too many different kinds of software, but the instructions below do explain how to ask the servers for help and find out what commands they support. ------------------------------ VI. File retrieval techniques in detail 1. By anonymous ftp If you are on a site that is connected to the Internet and allows its users to ftp out to other Internet sites, then you have anonymous ftp access. The usual procedure for using anonymous ftp is to type the command "ftp machine-name", where "machine-name" is the name of the machine to which you want to connect, and then to use "anonymous" as the username and "user@host" (i.e. your E-mail address) as the password when you are prompted for it by ftp. Type "help" inside ftp to get a list of commands, and/or read the man page for ftp, or any other documentation about it available at your site for more information. If they don't answer your question, then ask someone at your site for help. If you don't have Internet and ftp access, then you can use an ftp-mail server such as Princeton's BITFTP or ftpmail to retrieve files from anonymous ftp archives. However, you should only use an ftp-mail server when the same files are not available from a dedicated mail archive server. See the BITFTP and ftpmail instructions below. 2. By anonymous UUCP There are various UUCP sites on the net that publish their modem telephone numbers and a public username and password that can be used to transfer files from the sites via UUCP. For more information about doing this, see the documentation for the "uucp" command on your system. One place to find information about anonymous UUCP archives is the Nixpub listing of public access Unix sites, maintained by Phil Eschallier and posted in comp.misc and alt.bbs. If that posting has expired at your site, you can get copies of it from the Periodic posting archive described below. 3. By mail a. Hrc mail archive As mentioned above, the UUCP host "hrc" maintains an archive of Usenet source newsgroups and other things. To find out how to use it, send mail to "hrc!archives", or "archives@hrc.UUCP", or whatever format your site uses to send mail to the user "archives" on the machine "hrc", and include the following commands in the body of the message: send path
send help You should replace
with a path from hrc to you. If you know a prominent UUCP site that has you in its path, then your path can start at that site, e.g. "uunet!mit-athena!jik" (PLEASE don't use that address, or I'll get the responses to your requests!). If you're on a site that has a name registered with the Internet name servers, you can use "uunet!athena.mit.edu!jik". b. Uunet mail archive Uunet.uu.net provides E-mail access to the sources it archives (see the discussion about ftp.uu.net above for some idea of what is available). The address to which to send requests is "netlib@uunet.uu.net". You can send a message containing "help" in its body for more information. Note, however, that this service does not provide access to all of UUNET's archived files, so you may have to use an ftp-by-mail service instead to get to some of them. c. BITFTP The BITFTP server, run by Princeton University, allows people on the BITNET to retrieve via E-mail files at anonymous ftp archive sites. To find out how to use it, send mail to "bitftp@pucc.bitnet" with "help" in the body. The BITFTP server will reject requests from non-BITNET addresses, so if you're not on the BITNET and you need to do anonymous ftp retrieval via E-mail, you should use ftpmail (see below). d. Ftpmail Ftpmail servers allow you to retrieve via E-mail files at anonymous ftp archive sites. Two ftpmail servers currently available are "ftpmail@decwrl.dec.com" and "ftpmail@src.doc.ic.ac.uk". To find out how to use an ftpmail server, send mail to it with "help" in the body. If you're on the BITNET, please use the BITFTP server (see above) rather than ftpmail. NOTE: Don't do something bogus like send your requests both to BITFTP and ftpmail, or to multiple ftpmail servers, hoping that one of them will get back to you first. Choose one server for your request, and use it. e. AT&T's netlib archive AT&T's "netlib" archive is the repository for a large body of source code and other material, much of it mathematical, algorithmic or scientific in nature. The archive is accessible via anonymous ftp to research.att.com, or via electronic mail. To find out how to use the mail server, send mail to "netlib@research.att.com" with "help" in the body of the message. f. Periodic posting archive As mentioned above, The machine pit-manager.mit.edu (18.172.1.27) maintains an archive of periodic Usenet postings. You can access it via anonymous ftp, or via mail server. To find out how to use the mail server, send a message to "mail-server@pit-manager.mit.edu" with Subject "help". The archive on pit-manager is also accessible via WAIS. If you don't know what that is, don't worry about it; if you do know what it is, you can search through periodic Usenet postings by connecting to the "usenet" WAIS database on pit-manager.mit.edu. g. Trickle mail servers A BITnet mail server package called "Trickle" is supported at a number of different BITnet sites all over the world: Austria: trickle@awiwuw11.bitnet Denmark: trickle@dktc11.bitnet Belgium: trickle@banufs11.bitnet France: trickle@frmop11.bitnet Germany: trickle@dtuzdv1.bitnet Italy: trickle@imipoli.bitnet Netherlands: trickle@hearn.bitnet Spain: trickle@eb0ub011.bitnet Turkey: trickle@trearn.bitnet Israel: trickle@taunivm.bitnet These archives contain files of all sorts from a number of different major anonymous ftp archive sites, including wsmr-simtel20.army.mil, ftp.uu.net, export.lcs.mit.edu and sauna.hut.fi. They are a good way for people on the BITnet to get access to archives. To find out how to use Trickle send a mail message to the Trickle server closest to you with "/HELP" in the body of the message and an empty Subject: line. h. Other mail servers There are other mail servers besides the ones listed above. If you want to find out more about a server, send a message to it with "help" in the body and see what it sends back. The following is a list [edited] of some of the available services. Others are listed in Scott Yanoff's "Updated Internet Services list" posting, which appears regularly in alt.internet.services, comp.misc, biz.comp.services, alt.bbs.internet, and news.answers. archive-server@bcm.tmc.edu UUCP maps, source-code for BCM WHOIS database, NFS and PC-NFS information and source-code, Unisys U-series information and source code, other stuff archive-server@germany.eu.net Archives of MS-DOS, Amiga, and Apple newsgroups, Internet RFCs, other stuff statlib@lib.stat.cmu.edu Lots of statistical software (also accessible via anonymous ftp to lib.stat.cmu.edu -- username statlib) -- Jonathan Kamens jik@MIT.Edu Aktis, Inc. Moderator, *.answers