Child pages
  • Ingest Tool Workflow

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Corrected links that should have been relative instead of absolute.

Preferred workflow

All image and metadata files that have not been ingested to the repository should reside on digitize.dlib.indiana.edu. As of Summer 2007, digitize.dlib.indiana.edu is an alias for bleu, but the specific machine will change in the future, so it is best to use the 'digitize' name.

  1. All scanned images and metadata are placed in /digitize/data/(project name)
  2. Images that are processed by imageproc will have temporary files in /digitize/imageproc.
  3. After quality control and derivative generation, derivative files are placed in /digitize/toFedora/(project name)/derivatives. Master files are placed in both /digitize/toFedora/(project name)/masters and /digitize/hpss/(project name) and removed from /digitize/data.
  4. HPSSproc moves files from the hpss folder to MDSS as soon as possible, without worrying about Fedora's schedule.
  5. The Fedora ingest process on rhyme or thalia will ingest files from /digitize/toFedora as soon as possible, without worrying about the HPSSproc schedule. This may happen manually, or automatically, depending on the collection. When files have been ingested, they will be moved to /digitize/recycle.
  6. Files in /digitize/recycle will be removed when they are 30 days old.

Legacy workflow for older collections

Assuming that images are present before or at the same time as metadata:

  1. Objects are digitized manually. These usually end up on euterpe:/digitize.
  2. Quality control is performed and derivatives are created. This can happen through the imageProc system or manually, but derivative files typically reside on gavotte:/digitize/imageproc
  3. Derivative files (and metadata about masters) are ingested into Fedora on rhyme and/or thalia.
  4. When ingest is complete, master files are manually moved to erato for staging to HPSS. The HPSSproc process runs nightly and removes files that have been successfully transferred.
  5. Derivative files are manually deleted.

Updates to ingested objects:

  1. If metadata updates for MetsNav are needed, they are tested on a sample MetsNav instance on urania, then the files are re-ingested to Fedora.