Child pages
  • Preservation Policies
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 19 Next »

RLG "Trusted Digital Repository" Checklist

The most important document related to preservation is RLG's Audit Checklist for the Certification of Trusted Digital Repositories.

Items on RLG's checklist of immediate interest. For the most part, these items relate to our use of HPSS:

  • 3.6 Repository commits to define, collect, track, and provide, on demand, its information
    integrity measurements.
  • B1.1 Repository identifies properties it will preserve for each class of digital object.
  • B1.5 Repository obtains sufficient physical control over the digital objects to preserve
    them. (The description of this is confusing, encompassing many different aspects of repository functionality.)
  • B2.1 Repository has an identifiable, written definition for each AIP or class of information
    preserved by the repository.
  • B2.4. Repository has and uses a naming convention that can be shown to generate visible,
    unique identifiers for all AIPs.
  • B3.7 Repository actively monitors AIP integrity. (This includes maintaining checksums in a location separate from media files, and maintaining logs of the integrity checks.)
  • B4.2 Repository can demonstrate that referential integrity is created between all AIPs and
    associated descriptive information.
  • B5.2 Repository logs all access management failures, and staff review inappropriate
    "access denial" incidents. (This should probably apply both to Fedora and to HPSS.)
  • D1.1 Repository functions on well-supported operating systems and other core
    infrastructural software. (Not sure if HPSS will ever meet this...)
  • D1.2 Repository ensures that all platforms have a backup function sufficient for the
    repository's services and for the data held, e.g., metadata associated with access controls,
    repository main content, etc.
  • D1.4 Repository has mechanisms in place to insure any/multiple copies of digital objects
    are synchronized.
  • D1.5 Repository has effective mechanisms to detect data corruption or loss. For example, if the policy were the repository could not lose more than 0.001% of the collection per year, then the repository would need to confirm media integrity at least every 72 days to achieve an average time-to-recover of 36 days or about one tenth of a year.
  • D1.6 Repository reports to its administration all incidents of data corruption or loss, and
    steps taken to repair/replace corrupt or lost data.
  • D1.7 Repository has defined processes for storage media migration.
  • D1.8 Repository has a documented change management process that identifies changes to
    critical processes.
  • D1.9 Repository has a process for testing the effect of critical changes to the system.
  • D1.10 Repository has a process to stay current with the latest operating system security
    fixes
  • D3.4 Repository has written disaster preparedness and recovery plan(s), including at least
    one off-site copy of all deposited data. Multiple off-sites copies are expected of most repositories, but others may be able to justify not providing these.
  • D3.5 Repository tests disaster plans regularly.
  • D3.6 Repository has defined processes for service continuity and disaster recovery.

Possible threats to the preservation repository

  • Failure to access HPSS
  • Failure of HPSS tapes
  • Fire, tornado, or other catastrophic failure in WCC machine room
  • Disgruntled employee attempts to delete everything from HPSS
  • Disgruntled employee attempts to delelte random sets of items from HPSS

Other preservation-related items

OAIS is the primary model for building preservation systems.

RLG's older document, Trusted digital repositories: Attributes and Responsibilities.

The October 2005 RLG DigiNews describes requirements for "certification" of a digital repository. Many of these requirements are related to preservation.

There is a paper from Los Alamos about integrating OAI and OpenURL with an OAIS model.

There is a paper on bottom-up preservation issues that may also be useful.

The architecture of the LOCKSS system may be a useful guide, but LOCKSS currently relys on each participating institution "owning" a copy of the item. This is most useful for electronic journals.

Portico is a preservation archive for e-journals, based on Documentum. They perform extensive validation before accepting data into their repository.

The Florida Center for Library Automation is developing an archive system. This may provide some ideas, and they may eventually release code that we can use.

Vol. 54 (2005) no. 1 of 'Library Trends' is a theme issue, entitled: "Digital Preservation: Finding Balance", with many interesting articles.

Xena is a program that can take many file formats and convert them to a (supposedly open-format, long-term stable) XML format. It has been used with DSpace.

In the IU computer science department, Thomas Reichherzer and Geoffrey Brown are working on tools to aid "preservation via emulation".

Building an integrity checker

If all goes well, an integrity checker will never report a problem. But a non-existent or non-running integrity checker would exhibit the same behavior. So we must set up tests that ensure the integrity checker is regularly exercised. One simple way would be to put an object in the repository and purposefully corrupt it, then measure how long it takes for the integrity checker to notice.

Open questions

  1. Can we eventually store lots of small files (page images) directly in HPSS via the filesystem interface without aggregation, or will it simply take too long to retrieve these files?
  2. Should we store copies of our metadata in HPSS, or are the regular server backups good enough for this?
  • No labels