Page tree
Skip to end of metadata
Go to start of metadata

FR Process

To use the Face Recognition (FR) MGM in a workflow, the following steps need to be performed:

  1. Face Recognition requires a training process before face matching can be done. The training process uses a collection supplement, which is ingested into AMPPD via batch ingest. The supplement is a zip file, once unzipped, it should contain one or multiple subdirectories, each subdirectory named after the person and contains a set of photos of that person. 
    1. Note: The zip file should be created by zipping all of the individual person's photo directories, not their parent directory. For ex, if you have a directory called photos, which contains 2 subdirectories called Wells and Nelms, you should zip the two subdirectories (NOT photos directory) into training.zip. 
  2. When creating a workflow that includes the FR MGM, make sure to populate the training_photos parameter in the FR step. The value of this parameter should be the name of the collection supplement which contains the photos zip file to be used for FR training.
    1. Note: The parameter value should be the exact string of the supplement name, with spaces and upper/lower cases as is. Collection name is not needed, as AMPPD can infer the collection from the primaryfile's item's collection.
  3. Before saving the workflow, also double check the other parameters
    1. reuse_trained: Leave it at the default true, if the training photos zip file hasn't been replaced since last training; otherwise set this parameter to false, to allow FR MGM to retrain.
    2. tolerance: the default is set at 0.60, which works best in most cases. Set it lower for stricter matching, higher vice versa.
  4. When submitting a bundle of primaryfiles to the FR workflow, AMPPD will translate the training_photos parameter from supplement name to it's actual full local pathname, which the FR MGM can use to grab the zip file.
    1. Note: If the primaryfiles submitted in the bundle are from multiple collections, each collection should contain a supplement with the same name as specified in the training_photos parameter, even though the zip files could be totally different.
  5. Once the workflow is submitted to Galaxy successfully, the FR MGM will run
    1. If reuse_trained is false, or if the previous training result can't be found, it will start the training process
      1. training_photos parameter now contains the full local path of the training photos zip file
      2. unzip the TrainingPhotos.zip into a temporary working directory with the same name under galaxy working directory for facial, for ex, in testing env, /srv/amp/galaxy_work/facial/ TrainingPhotos/, under this directory, we may have subdirectories, for ex, Person1 Name/, Person2 Name/, each containing photos of the person.
      3. run the training process
      4. generate training results, and save into a binary .frt file, for ex, TrainingPhotos.frt, under the same directory where the original TrainingPhotos.zip sits.
      5. With the above .frt file, next time FR is run and training is requested, the process will look for this same TrainingPhotos.frt for reuse.
    2. Now we have the training result, either read from previously saved TrainingPhotos.frt, or from the current training, we can start face matching on the input video and known faces.
      1. Matching results is generated in the standard AMP Face JSON file, for ex, dataset_3574.dat
      2. Detailed stdout and stderr logs can be found on the info page of the step in Galaxy UI, and in the log file corresponding to the output dataset on the server, for ex, in test env, /srv/amp/galaxy_work/logs/dataset_3574.dat_face_recognition.log.
  6. To view the FR results conveniently, one can output the AMP Face JSON to the Contact Sheet MGM in a workflow, in which case, the frames identified with known faces will show up in the contact sheet.

FR Source Code

  • galaxy/tools/amp_schema/facial_recognition.py: contains the hierarchy of classes needed by AMP Face Schema, with the top level class FaceRecognition.
  • galaxy/tools/amp_facial/dlib_face_recognition.xml: The tool config XML
  • galaxy/tools/amp_facial/dlib_face_training.py: containing methods related to FR training
  • galaxy/tools/amp_facial/dlib_face_recognition.py: the main python script containing logic for FR matching, and calling FR training as needed.

FR MGM Config

  • Tool ID: dlib_face_recognition
  • Dependencies
    • python 3
    • dlib
    • opencv
    • face_recognition
  •  Inputs
    • input_video: input video file for facial recognition
    • training_photos: absolute path of a zip file with subdirectories each named after a person containing his/her photos
    • reuse_trained: Flag indicating whether or not to reuse previous training results from the same training photos if exist
    • tolerance: tolerance level when matching faces, a lower value means stricter match
  • Outputs
    • amp_faces:AMP Faces Recognized
    • <supplement_file>.frt under the same directory as supplement_file>.zip: training results binary file with trained known faces name and encoding

Usage

  • Install dependencies:
    • pip3 install --user dlib
    • pip3 install --user face_recognition
    • pip3 install --user opencv-python
  • Run FR MGM on command line:
    • dlib_face_recognition.py root_dir input_video training_photos reuse_trained tolerance amp_faces
  • Run FR MGM in Galaxy as single tool or in a workflow


  • No labels