[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

National Archives Assembly resolution re disposition of Census 2000 image files



A member of the National Archives Assembly has requested that I post the Assembly's resolution re disposition of Census 2000 image files.

     THE NATIONAL ARCHIVES ASSEMBLY
                  The National Archives at College Park
          8601 Adelphi Road, College Park, MD  20740-6001




               Resolution Regarding the Disposition of the Census 2000 Image Files

     WHEREAS the Assembly recognizes the sole authority of the Archivist to determine the disposition
     of Federal records and defers to his judgment on how to balance archival decisions against the
     realities of external pressures and considerations, and
     WHEREAS the personal information with enduring value provided in response to the 2000
     Decennial Census, the Individual Census Record File, has been scheduled for transfer to NARA for
     permanent retention, and
     WHEREAS the Assembly has concerns about the Volume implications of accessioning
     approximately 700 million images either in digital form or on microfilm, and
     WHEREAS the Assembly has concerns about the Volume implications of accessioning
     approximately 160 Terabytes of redundant data which could potentially translate into more than two
     million tape cartridges, and
     WHEREAS the Assembly has concerns about the Budgetary Implications of preserving this
     redundant data, specifically the inability to determine the exact cost of archival retention because no
     one has ever attempted to preserve and maintain a collection of this magnitude, and
     WHEREAS the Assembly has concerns about the Interspersion of Valueless Material within the
     redundant images because an estimated two-thirds of the images (or as many as 500 million) are of
     forms containing no respondent data, and
     WHEREAS the Assembly has concerns about the misunderstanding of the issue of Marginalia,
              and,
     WHEREAS the Assembly has concerns about the misunderstanding of the issue of Handwriting
     Analysis, and
     WHEREAS the Assembly has concerns that the normal procedures involving stakeholder review and
     comment were not followed in the processing of N1-029-00-004.  Therefore, the Assembly does not
     believe the Archivist was able to make a fully informed decision.

THEREFORE BE IT RESOLVED that the Executive Board of the National Archives Assembly urges the
Archivist to revisit the appraisal decision making the digital image files from Census 2000 a permanent
record.

In accordance with the Constitution of the National Archives Assembly, this resolution was adopted by a
simple majority of the members of the Executive Board of the National Archives Assembly on July 20,
2000.



BACKGROUND

The National Archives Assembly is an organization of present and former NARA employees who support
the development of NARA as the leading archival institution in the world. The Assembly provides a forum
for employee communication on NARA policies and programs and serves to convey its members' views to
the Archivist of the United States.  The National Archives Assembly Executive Board has adopted this
resolution to express serious reservations about a recent decision concerning the disposition of Census
2000 long and short form images.  The information contained in this document identifies specific concerns
related to the appraisal of the census image files as a permanent record.

Beginning in 1994, thirteen NARA staff collaborated with representatives of the Census Bureau at the
 front end  of the records life cycle to develop requirements for and identify the best products from the
2000 Census that would meet the needs of the genealogical, social science, and other research
communities.  In its report of March 1, 1995, titled  Preserving Census 2000 Records: A Report of the
Census 2000 Working Group of the National Archives and Records Administration  NARA recommended
the creation of an electronic file containing individual responses to the census in a format that specifically
meets NARA s transfer and preservation requirements, as a desired  product  from the 2000 Census.
Computer output microfilm of the images was identified as a potentially desirable product, but not
required because the Census Bureau was not planning to create microfilm in the course of its conducting
Census 2000.  The Census Bureau worked from that point on under the impres sion that they were
meeting the needs of NARA and future generations of researchers by budgeting for, preparing, and
eventually arranging for transfer an electronic file known as the Individual Census Record File (ICRF) and
other permanent records.

All required data collected on the Census forms, both handwritten and check box, are captured in the
ICRF using sophisticated Optical Character Recognition/Optical Mark Recognition technologies operating
at over 99% accuracy rates.  Incomplete or inaccurate responses are verified by Census Bureau staff and
entered directly into the ICRF.  The ICRF is scheduled as a permanent record on N1-029-00-001, item 3.
NARA appraised the ICRF as permanent because it is the final checked, edited and verified permanent
record of individual responses to Census 2000 and because it reflects the basic record upon which the
Bureau of the Census will base its tabulations of the 2000 Census.  The ICRF is also the record citizens
will use for the protection of their legal rights and interests through the Census Age Search program, until
the records are open in 2072.

Schedule N1-029-00-001 also included items covering the paper forms (item 1) on which the data was
originally collected and the image files (item 2). The schedule listed both items as temporary.  The item
describing the images was included to illustrate that they are essentially an intermediate processing stage in
the compilation of the final checked, edited and verified individual Census 2000 data captured in the
ICRF.  Intermediate processing files have been disposable under GRS 20 since 1972.  NARA requested
that the Census Bureau withdraw the item to allow NARA to complete a full and detailed appraisal.  The
availability of this schedule was announced in the Federal Register of December 13, 1999.  NARA
received several comments urging the retention of the images.

On December 7, 1999, NARA received a new schedule, N1-029-00-002, from the Bureau of the Census
proposing the images of the Census 2000 for disposal after 10 years. A team of NARA archivists reviewed
this proposed disposition and, after extensive analysis, recommended temporary retention of the records
based on the following appraisal criteria: the information in the records was already scheduled as
permanent in the ICRF, the existence of populations not covered or incompletely covered by the scanned
images (i.e. group quarters and other populations will not have associated images), the interspersion of
blank pages containing valueless information (estimated to be 2/3 of all images), and the lack of an index
or the ability to retrieve images by personal identifier. The complete original appraisal of the Census 2000
images is contained in N1-029-00-002.

On March 6, 2000, NARA announced the availability of the schedule and appraisal of the images in the
Federal Register and also directly solicited comments from a number of genealogical, social science, and
public policy groups.  The comment period closed on May 22, 2000.  In response to both schedules
(N1-029-00-001 and N1-029-00-002), NARA received comments from total of 13 unique entities,
including two members of congress (Congressmen Jim Turner and Henry Waxman).  Most of these
parties urged the retention of the images primarily because images of original schedules from all previous
censuses are available, because images of Census 2000 forms will contain marginalia not otherwise
captured in the ICRF, and to allow for future analysis of the handwriting included on the forms.

As part of its detailed analysis of the image files, the appraisal report on N1-029-00-002 addressed the
comments received in response to N1-029-00-001.  The report states that   it appeared to the appraisal
team that most commenters presumed that the scanned image files were similar to, or would serve as a
replacement for, the microfilm census records that NARA has appraised as permanently valuable for
previous decennial censuses    Apparently, the commenters did not understand that the recordkeeping
system developed for the 2000 census was qualitatively different than the systems used for earlier
censuses.  The permanent records of individual responses to Census 2000 are in the ICRF, not the image
file.

However, NARA responded to public comment by reversing its appraisal of the image files with this
justification:

     Based on our analysis of the public comments, and following internal NARA review and informal
     discussions with staff at the Census Bureau and at the General Accounting Office (GAO), we
     recommend that the image files be retained permanently and transferred to NARA when they are ten
     years old.  The scanned image files will contain information that will not be duplicated in the
     Individual Census Record File (ICRF)   such as marginal comments.  While estimates of the actual
     number of marginal comments that might be found on the questionnaires vary widely, our
     discussions suggest that as many a [sic] 1 form in 1,000 may have [sic] contain such comments.
     We also confirmed with the Census Bureau and GAO that it is technically feasible to develop an
     [sic] computer system that is capable of linking the scanned images to a unique Housing Unit
     Identification Number, and further by person, address, and other geographic coding.  While these
     links curre ntly do not exist, based on conversations with NH by John Constance [NARA s
     Congressional Affairs officer], it is our opinion that the index files necessary for researchers to
     access the scanned images can be developed at the appropriate time.  We will also explore other
     ways to work cooperatively with the Census Bureau to develop the index links and resolve any
     technical issues relating to the scanned image files prior to their transfer to NARA.

The appraisal for the images is dated May 18, 2000.  The schedule was logged on May 24, 2000, and
assigned job number N1-029-00-004.  The appraisal dossier contains the records schedule, the NA Form
13133, a transmittal memo for the appraisal report, a two page appraisal report, summaries of all of the
public comments which were received, and a copy of a letter dated May 17, 2000 from the Assistant
Archivist, Office of Records Services-Washington, D.C.  to Congressman Henry Waxman.  This purpose
of the letter is to inform Mr. Waxman that the Assistant Archivist is planning to recommend permanent
retention of the image files to the Archivist.  This letter also states that  We will coordinate all of the
details of the disposition language and our final recommendations to the Archivist with David McMillan of
the Committee staff.   On June 7, 2000, the Archivist of the United States signed records disposition
schedule N1-029-00-004, authorizing the p ermanent retention of the image files from the 2000 Census.


SUPPORTING INFORMATION FOR THE RESOLUTION

VOLUME AND COSTS: The disposition of the records created and used to document the Census 2000 is an
important issue for NARA.  One of the series of records, which have generated enormous interest, are the scanned
images of the long and short forms that were returned by individual households.  On June 7, 2000, the Archivist of
the United States signed records disposition schedule N1-029-00-004, authorizing the permanent retention of the
image files from the 2000 Census.  The Census Bureau estimates there are more than 700 million images. NARA s
current holdings are estimated at 4 billion documents --- this decision alone essentially increases the volume of
NARA s total holdings by 17.5%.

The only approved options currently available for the transfer of electronic records, including scanned images, are
9-track tape, 3480-class tape cartridge and CD-ROM (ref. 36 CFR 1228).   Given that recent Census estimates
suggest the volume of images to be approximately 160 terabytes, and that CD-ROMs hold the greatest volume of
electronic records files (approximately 600 megabytes), NARA should expect to receive approximately 320,000
CD-ROMs.  Opting for either 9-track, 6250 bpi magnetic tape or 3480-class tape cartridge would result in over 1
million transfer media.  NARA s current electronic records preservation procedures call for creating separate
master and backup copies onto 3480-class tape cartridge.  Therefore, under NARA s current preservation
procedures, NARA will have to procure over 2 million 3480-class tape cartridges.

If microfilm is identified as the desired transfer or preservation media, the cost to create such microfilm should be
considered.  The cost may reasonably approach $.10 per image for one copy, or $70 million.  No organization has
ever faced microfilming at such huge volumes, so no one can be certain of the exact costs or the potential impact on
the environment from the generation of such huge amounts of chemical byproducts.  If NARA requires that the
Census Bureau meet 36 CFR 1230 (requiring a silver halide original and diazo backup), then the costs could
potentially double and NARA, if the microfilm is created with 1,000 images per reel for example, would face
accessioning 1.4 million reels.

The original appraisal included an estimated volume of Census 2000 scanned images of approximately 60
terabytes.  NARA s imaging experts were asked to provide the estimated costs for maintaining that volume.  The
response preliminarily estimated that it would cost  $5.3 million to $10.5 million per year ($53 million to $105
million for the first 10 years) for this amount of data.   Given that the Bureau of the Census has now revised this
estimate upwards to 160 terabytes, and if we assume the costs to increase proportionately to volume, NARA could
expect the annual cost to maintain the Census 2000 images at $14.31 million to $28.4 million per year ($143.1
million to $284 million for the first 10 years) for this amount of data.  The cost for currently acceptable preservation
media alone is approximately $8 million at $4.00 per 3480-class tape cartridge.  However, a caveat must be placed
on these cost figures.  Specifically, no archival repository has e ver faced accessioning and preserving such a large
volume of images and costs are based on projects of a limited scope.  We do know however that costs of some order
of magnitude not normally associated with the archival preservation of electronic records will be associated with the
permanent preservation and migration of the image files.

INTERSPERSION OF VALUELESS MATERIAL: The Bureau of the Census necessarily image processes every
page of every form returned to it regardless of whether the return is completely filled out.  In other words, the 2000
Census long form is approximately 20 pages in length with space for six respondents.  If there is only one
respondent in the household, all pages in the long form will be image scanned into the system.  The Census 2000
image processing system necessarily requires that all pages be imaged in order to extract the data from the forms.
The Bureau of the Census estimates that 2/3 of the images contain no information.  Therefore, NARA can expect to
have among it archival holdings almost 500 million images of blank forms.

MARGINALIA & HANDWRITING ANALYSIS:  One of the arguments included in the public comments is that
the Census 2000 forms will contain a great deal of valuable marginalia and will be necessary for future researchers
to conduct handwriting analysis.  The Bureau of the Census was unable to provide NARA with specific figures
based on previous censuses or on the Census 2000 Dress Rehearsal to indicate the extent to which marginalia is on
the forms.  There is no evidence for the statement that 1 in 1,000 forms contain marginalia.  The earliest Census in
which respondents had an opportunity to write marginalia on census forms was 1970, the first  mail out, mail back
census [the 1960 Census was a  mail out-enumerator pickup  census].  No entity has reviewed or analyzed the
1970 or later censuses for such marginalia.  In addition, the Bureau of the Census estimates that the percent of
forms directly returned by respondents to be 66% of the total; cen sus field enumerators would gather the
remaining responses.  Therefore, marginal comments would only be possible on 2/3 of the forms.

Another argument included in the public comments is that the images need to be retained for future handwriting
analysis.  In addition to the fact that only 66% of the forms were potentially filled out by the respondent, there is no
opportunity to write on the form other than in block printing. Graphology, or handwriting analysis, is the study of
handwriting shapes and patterns to determine the personality and behavior of the writer.   Given the paucity of
written items on the census forms and that only block printing is allowed, and that the specific respondent to the
questions is never identified, it is questionable whether the public s comments should have received as much weight
as they apparently did.

DOCUMENTATION: NARA s normal appraisal procedures include provisions for the circulation of new
disposition schedules and appraisal reports to various stakeholder units for review and comment.  These
procedures allow for the creation and preservation of adequate and proper documentation regarding appraisal
decisions, in keeping with basic tenets of the Federal Records Act (44 USC 3101). This appraisal was not circulated
to any NW or NR custodial unit for review or comment.  Therefore, the official record does not contain any
objections or concerns from the custodial units.

NARA makes its appraisal judgments based on the content, context and structure of records and whether records
have enduring informational or evidential value.  Several factors weigh in on such decisions including the records'
uniqueness, completeness, and volume.  Other factors influencing the appraisal decision are of a technical nature
and include hardware and software dependencies, media, data compression, significant cost or resources impacts,
or other technical issues.  The dossier for N1-029-00-004 does not contain either a formal or informal technical
analysis of the Census 2000 images.  The official record also does not address budgetary implications, interspersion
of valueless material, or a substantive analysis and verification of concerns expressed in the public comments.
Because these issues are not addressed in the appraisal dossier and, therefore were not brought to the Archivist s
attention, he could not consider them.


Join 18 million Eudora users by signing up for a free Eudora Web-Mail account at http://www.eudoramail.com

A posting from the Archives & Archivists LISTSERV List!

To subscribe or unsubscribe, send e-mail to listserv@listserv.muohio.edu
      In body of message:  SUB ARCHIVES firstname lastname
                    *or*:  UNSUB ARCHIVES
To post a message, send e-mail to archives@listserv.muohio.edu

Or to do *anything* (and enjoy doing it!), use the web interface at
     http://listserv.muohio.edu/archives/archives.html

Problems?  Send e-mail to Robert F Schmidt <rschmidt@lib.muohio.edu>