Child pages
  • 2017-01-19 Developer meeting re: architecture and planning
Skip to end of metadata
Go to start of metadata

Date

2017-01-19

Attendees

Goals

  • Code architecture for ingestion and Preservation Plug-ins
  • Merging ingestion code with Preservation Plug-ins code
  • New github for PHYDO?

Discussion:

  1. Phydo an app or a gem?
    1. Acknowledging that we've gone back and forth on this:
      1. First HydraDAM was an app.
      2. Then we said it would be better as a gem.
      3. Then we said it would be better as a gem that was comprised of multiple, reusable component gems.
    2. The thinking now is that that Phydo will suffice as a working app, and that the app will be an integration of our reusable component gems. This approach has the following benefits:
      1. people can still pick and choose which parts of Phydo they want to implement
      2. Phydo (as a working app) can still be downloaded and run, if people choose to. Instead of "installing" it the way you do with Sufia and CurationConcerns, it would be come pre-built.
        1. It was noted that the end product that Phydo becomes, may not be the practical solution others need.
        2. In the same way that many in the Hydra community wanted to pick and choose specific features from Sufia without adopting the whole package, it's conceivable, if not probable, that they would feel the same about Phydo.
        3. Out of Sufia, came the demand for the stripped down version, CurationConcerns. And continued demand for modularity has lead to the Hydra Plugins WG.
        4. Instead of going through that same evolution with Phydo, we can start by targeting a more modular architecture.
      3. Apps are easier to develop than gems.
      4. Phydo will serve not only as the grant deliverable, but also as the integration point for all of the modular plugins we develop which include:
        1. search and discovery of preservation event models
        2. storing external content on Fedora objects
        3. support for asynchronous file operations (i.e. stage-for-download, run-fixity-check, etc)
        4. batch ingestion
        5. workflows around files
        6. (note that some of these may end up as contributions to the Hyrax core)
    3. We can start by renaming "hydradam2-app" to "phydo".
      1. They are essentially the same idea, and we can't think of any technical reason for why this wouldn't work.
      2. Avoids yet another repo.
  2. Ingest Normalization plugin (a.k.a. "Context YAML")
    1. This is the machinery that Adam and Amol have been working on re: ingest.
    2. The plugin is responsible for the following:
      1. reads metadata from a YAML (provided by the implementer) for file ingestion.
      2. reads a mapping from YAML to RDF properties (provided by the implementer).
      3. verifies existence of RDF properties on models (models are specified by the implementer).
      4. assigns metadata from YAML to RDF properties and saves the objects.
    3. The implementer (not the plugin) is responsible for the following:
      1. translating metadata formats (i.e. FITS, FFPROBE, FFMPEG) into expected YAML format.
      2. provide mapping from YAML to RDF properties, or use existing mapping, if one is available. The plugin _may_ provide mappings, but that is beyond the core scope of the plugin's functionality.
      3. configure ActiveFedora models to have the RDF properties.

Action items

  • rename github repo from "hydradam2-app" to "phydo".