[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: XML Encapsulation



I appreciate Phil's thoughtful comments and agree with many of them.  I
would still take issue with some, though...

> The efforts to archive electronic records have been going on for decades
> with little practical success.  The work of committees to develop
> guidelines such as OAIS are commendable and may provide useful solutions.
>  However, I don't see the role of an archives is to compensate for the
> failures of the industry to provide practical and enduring formats for
> electronic information.  We should insure that the files are captured
> and held in original form as best as is possible.  If researchers and
> the general public have difficulty in accessing the objects over time,
> it is a societal (and commercial) issue.

I agree with this in principle, though I do think there are cases when
the principle must be broken.  I think the line between native formats
and standardized ones tends to be quite gray.  Organizations and vendors
themselves expend a lot of resources on migration, screen scraping,
restructuring of database tables to accomodate for new business rules,
etc.  Any data that lives for very long is bound to undergo a number of
changes.  I don't think it's very realistic to assume that process will
suddenly end once an archivist takes custody of them.  There's clearly a
difference between the amount of change made to data that's in active
use and data that resides in an official static (well, sort of static)
record. And I do think there could be hope in the use of emulation for
components of some systems.

But I feel very strongly that we have to BECOME part of the industry,
not just continuously react to it.  We don't need to all learn java and
lock ourselves away in rooms writing code.  But we do need to know
enough about information technology to articulate (or work without
others to articulate) our needs in terms of system specifications, data
models, business rules and the various other tools that IT managers and
software developers understand.  Computers can do some amazing things,
if someone tells them to in the right way.  But if we're not there
telling those people what we think they should do, it's unlikely that
they'll start caring about preservation and long-term authenticity
issues as much as we do.

> The archival community has watched the technology industry embrace and then
> discard formats for the past forty years without presenting an effective,
> unified challenge to the vendors.  It is also apparent that there is not a
> compelling, economic driver to force the technolgy industry to resolve the
> format issue.  Having said that, I must declare that I am optimistic that
> the industry will address it.  Money talks and is influencing priorities in
> information storage, especially regarding the records of commercial
> transactions.  If the archivists place the responsibility where it belongs
> (on the industry) and resist the temptation to "work around" format and
> platform obsolescence or solve it for them, then I believe real progress
> will occur.

I think it's worth reiterating that if we took this position, we'd be
the only ones in the industry doing so.  If two companies merge, or two
divisions of a government agencies decide to cooperate on a data
warehouse project, the IT staff have to "work around" like crazy.  The
industry doesn't create any free-standing, monolithic systems (except
for some quite tiny ones that are unlikely to be holding a lot of
interesting records).  Even those complying to every conceivable
industry standard can have numerous interoperability problems.

Regardless of whether an archival repository takes custody of records at
some point or they're just retained over a long period within a given
organization, system incompatibilities will constantly emerge.

We should exert much more pressure on vendors to meet our needs.  But,
like everyone else who manages complex information systems, we can't
rely on them to meet all of them.

> 1. If our national defense depended upon somebody reading a Commodore
> floppy, chances are we would figure out a way to do it; even fifty years
> from now.  The issue is not that media and formats become obsolete, we
> abandon the effort to read them.  It just becomes a cost - benefit matter.
> It is very important to save the digital files on currently popular media
> and replicate the contents to future media as needed.  At least then the
> challenge is reduced to interpreting the bits within.

That cost-benefit is very important.  Tapes can be from the bottom of
the ocean and read, tapes that have been written over dozens of times
can often be read.  Software can even be reverse engineered from
scratch, but all of this takes a lot of resources.

I must admit that I don't understand the sentence: "The issue is not
that media and formats become obsolete, we abandon the effort to read
them."  Aren't those the exact same thing?  I don't think anyone would
argue that modern technologist are inherently unable to ever build
another UNIVAC if they wanted to.  The point is that they don't
(usually) want to.  When the IT industry is all about keeping compatible
with what people are using now, I can't see the value of knowing that
someone could always go back to an earlier technology if she really
wanted to (and had all of the technical expertise to build it all on her
own).

Those developing and maintaing systems make a large portion of their
decisions based on cost-benefit analysis and risk analysis.  And I can't
see a computer industry in any forseeable future that decides to embrace
all versions of all formats, protocols and operating systems that have
ever been in use.  I still thinks it's up to us to work with them to try
to pick the most stable, open options and then deal with all the
headaches that will enivitably result from the industry often refusing
to agree with our predictions.

> The several cases cited in the Annex to the OAIS standards document all declare
> that the files were not converted but kept in their native format.  Why
> would a significant portion of the document promote transformation to a more
> readable format, but then fail to cite cases where that happened?

Sounds like some case studies waiting to happen.

> 4.  Converting information content through a non-reversible transformation
> to an ASCII or even Unicode set has merit when the contents can be generalized
> without serious information loss.  However, wholesale application of conversion
> to ensure readability in an open format seems dangerous, and perhaps damaging
> to evidential value.

In many cases, this could be true.  It's worth noting, though that
transformation to a standard format need not be something as obvious as
converting a Word document to flat ASCII.  The metadata and
functionality that's important can often be reflected in the new
standard format.  If not, then the transformation probably wasn't done
right or this is a case where saving the native format is vital due the
difficulty of moving its essential characteristics (content, context and
structure) into another format.

>  What distinguishes metaPackager is that it is a real application doing real
> things.  Not just another theory written up in a white paper or dressed up
> in a reference model document.

I would agree that a relatively easy to use, off-the-shelf product is a
welcome advance.  Many of the tools being used now are great for
Perl/PHP or Java hackers and XML gurus but not so great for the rest of
us.

Though it seems to me that there are numerous cases in which object
encapsulated metadata is happening without metaPackager.  One place to
read about them is
http://www.xmlhack.com/
though there are many others.  Numerous implementations of
object-oriented programming and data management make use of the concept.

Just one recent interesting advance (based more on extraction than pure
encapsulation) can be found at
http://www.xmlhack.com/read.php?item=578

> 7.  It is appropriate to be concerned about proprietary utilities to open wrapped objects.
> However, the world seems to have accepted such formats as .pdf and .zip
> (owned by Adobe and Unisys).  I expect many vendors will, likewise, support the
> encapsulation algorithm.

I hope so.  Please let me know when they do.

> 8.  Finally, I share Cal's concern about the metadata conundrum.  There is never enough
> of the right metadata to suit everyone.  However, I would rather have 12 or 15 common
> attributes used well than 150 attributes left empty due to the rigor required to assign
> values during composition of the object.  The value of a collection, whether it be
> historical or scientific in nature, is measured by the contribution it makes to society.
> If it isn't used, due to its complexity or the challenge of using its finding aids, then
> its contribution will be jeopardized.

This is obviously an important tradeoff.  I think it's important to
note, though, that many of these attributes can potentially be captured
automatically.  Demanding hand keying of 150 attributes is obviously
unreasonable.  But requiring that the "last modified" date and the
appropriate security permissions be retained along with a document
probably isn't.

=======================================================
Cal Lee
Electronic Records Project Archivist
Kansas State Historical Society
Phone: 785-272-8681, ext. 280         Fax: 785-272-8682
http://da.state.ks.us/itab/erc/
http://www.kshs.org/archives/recmgt.htm
                  "Obsolete power corrupts obsoletely."
                                           - Ted Nelson

A posting from the Archives & Archivists LISTSERV List!

To subscribe or unsubscribe, send e-mail to listserv@listserv.muohio.edu
      In body of message:  SUB ARCHIVES firstname lastname
                    *or*:  UNSUB ARCHIVES
To post a message, send e-mail to archives@listserv.muohio.edu

Or to do *anything* (and enjoy doing it!), use the web interface at
     http://listserv.muohio.edu/archives/archives.html

Problems?  Send e-mail to Robert F Schmidt <rschmidt@lib.muohio.edu>