Digitizing and Importing a Recording
- Create a WAV File of the Recording
- Create Derivative Files of the Recording
- Copy Derivative Files to Variations Server
- Importing the Recording into Variations
- Using the Import Tracks Tool
Before digitizing a recording determine what Cataloging ID will be given to the recording. Each institution will should have a predetermined Cataloging ID scheme (filenaming) for naming records. If a Cataloging ID does not exist for the recording that is being ingested, one will need to be created.
For more information on Cataloging ID visit the Cataloging ID and System ID page.
Create a WAV File of the Recording
File Names are Important!
Several steps in the ingest process rely on the files being named following a strict convention. If you are using the Variations MP4 Encoder to create the derivative audio files, be sure that the initial WAV files follow the file naming scheme set up for your institution. By default, the file naming scheme is <cataloging id>-<sequence letter>.wav. For example, if b12345678 is the cataloging ID for a two-CD set, b12345678-a.wav would be the first CD and b12345678-b.wav would be the second CD. Variations will use the cataloging ID portion to look up a bib record via Z39.50. Variations uses the "-a" sequence letter to put the CDs or sides in the correct order.
Use CDex, Audacity, Wavlab, Soundforge or your favorite audio tool for the type of recording you are digitizing to produce a WAV file. Note the specifications for WAV files and that Variations expects all recordings like CDs to be one WAV file per CD or side instead of one per track.
If you would like to digitize a CD using Soundforge (on Windows), load the CD and select File->Extract Audio from CD.... Then select Read Entire Disc from the Action drop down box, select the Drive with the CD in it, and click OK.
Alternatively, select Tools->Scripting->Extract CD.
After the CD is done being extracted, select File->Save As, change the filename, set the Save as type to Wave (Microsoft), and click Save.
If you would like to digitize a CD using Foobar2000 (on Windows), load the CD and select File->Open audio CD.... Select the correct drive and click Rip. Optionally, edit any of the metadata. Make sure that Go to the Converter Setup dialog is selected and click Rip. At the Converter Setup dialog choose WAV as the Encoding preset, select Merge all tracks into one output file as the Output files, and click OK. Finally, choose the correct filename and place to save it and click Save.
If you would like to digitize a CD using CDex (on Windows), load the CD and select all of the tracks. Then run Convert->Extract a section of the CD. In the dialog, select WAV file. Check to make sure that the Start Position is the beginning of the first track and the Stop Position is the end of the last track. Then click OK. The digitized file defaults to C:\Documents and Settings\username.wav where username is the username of the currently logged in user. When CDex is done, copy and rename the file according to local convention.
Create Derivative Files of the Recording
Use an encoding program that can create MOV or MP4 files like the bundled MP4 Encoder or Quicktime Pro. Create as many derivative files at as many different bitrates as you would like, using the filenaming convention originalfilename.bitrate.extension. For example, a WAV file called b12345678-a.wav could have derivatives b12345678-a.28k.mp4 and b12345678-a.192k.mp4.
Currently, Variations only supports the following bitrates:
- 14 kbps
- 28 kbps
- 56 kbps
- 128 kbps
- 192 kbps
- 256 kbps
- 320 kbps
Note that using MP4 files requires setting the dml.digitizer.useMP4 property to true in dml.conf. This setting causes the file type in the media object to be set to video/mp4 instead of the default video/quicktime, which is for MOV files.
Using bundled MP4 Encoder
Directions on using the bundled MP4 Encoder can be found at this page.
Using Quicktime Pro
Using Quicktime Pro and mp4 files, open a wav file and select File->Export.... Choose the appropriate filename, set Export to Movie to MPEG-4, and click Options.... Select Audio from the drop down menu below the File Format drop down box. Select the bitrate needed using the Data Rate drop down box. Next change from the Audio settings to the Streaming settings. Check both the Enable streaming and Optimize for server boxes. Finally, click OK and then Save. (These settings will be saved and will be brought up the next time a file is exported.)
Alternatively for mov files, set Export to Movie to QuickTime Movie and click Options.... Click the Settings... button under Sound, change the Format to AAC, set Target Bit Rate to the desired bitrate, and click OK. Make sure that Prepare for Internet Streaming is checked and the drop down box is set to Hinted Streaming. Make sure that the Optimize Hints For Server option is checked and click OK. Finally, click OK and then Save. (These settings will be saved and will be brought up the next time a file is exported.)
Copy Derivative Files to Variations Server
This step can be skipped if the standalone encoder has been set to save the derivative files to the incoming directory.
- Connect either through Samba, sftp, or some other means to the content/loader/audio/incoming directory on the Variations server.
- Copy all derivative files, but NOT the WAV file.
- Double-check to make sure that the derivative filenames follow the local convention and the Variations derivative filename convention mentioned above.
- Make sure that derivative files are set with the correct permissions that the user dmlserv can read and write to them.
Importing the Recording into Variations
Have the physical item on hand when doing this step.
- If the item is a CD, put the CD (or the first CD) in the computer's CD drive.
- Obtain a unique Cataloging ID.
- Start the Variations digitizer client, ensuring first that your login has digitizer privileges. Then select Digitizer tab in the Search Window
- Enter the ID in the Create text field
- Select Recording
- Click Create. This will try to import the MARC record and then open the Variations Digitizing window.
- Importing the MARC record. Variations uses MARC records to import some bibliographic data. This happens automatically if Variations finds a match in the catalog. Variations will search on the local Z39.50 server for the ID entered and try to import the record if found. If no entry was found, a Z39.50 Search Window will pop up. A number of different fields are available for search. When the desired record is selected and displayed, click Import. If no record exists, click Cancel. No prior metadata will be imported in this case. To import the MARC record after the Variations record is created, click the Import button under Description to search in Z39.50 for the recording ID.
- Update the information in the Descriptionsection (left-hand side) of the window.
- Item Title. This should be filled in automatically by the MARC import process. If not, enter the title of the recording. The title is automatically copied to the top node of the container structure on the right-hand side of the window.
- Format. Choose the appropriate recording format from the list.
- Document Description and Publisher are imported from the MARC record and cannot be edited by the digitizer. These fields can be modified by the cataloger using the cataloging windows.
- Copyright Statement.This is displayed on the item's access page and should be copied verbatim from the back of the CD case or LP sleeve. Here are some examples:
- (C) 1997, 1967 Experience Hendrix, L.L.C.
- (P) & (C) 1993 HNH International
- (P) n.d. Vox
- Notes. Notes also import from the MARC record. More can be added as needed by local policy.
- Location. The location choices are set by your Variations systems administrator.
- Call Number. The call number is displayed on the item's access page, making it easier for users to locate the physical item should they desire to do so.
- Copy Number. This should indicate which copy was digitized.
- Viewable Comments. This field is displayed on the access page. It provides a way to tell users about problems with the item's condition or other necessary information.
- Availability. "Restricted access" will provide normal access to the recording according to institutional access policy. "Publicly available" (if implemented locally) provides access to this item for any Variations user.
- Import the Recording
- Click Add Sound File ..., locate the WAV file (not the derivative file) for the first CD (or side). This should add a CD or Side to the Container Structure. Repeat this process for all additional CDs or Sides in this container. Variations uses the WAV file to determine the length of the recording.
- Click Import Tracks ....
- This will populate the Structure with the track number and timings from the CD.
- With version 5.0.2 and later, you may have a choice of places from which to import track timings and titles. The Variations option will work if the CD is already digitized at IU, in which case this is the preferred choice for importing, as track hierarchy will also import if available. If importing the track times from a source other than the CD, be sure to check the timings later on to ensure correctness. Timings can be adjusted by selecting the track, changing the beginning offset, and clicking Next. For more information about Import Tracks options, see the Using the Import Tracks Tool, below.
- If tracks timings or titles were not automatically imported,
- enter the number of tracks to add to the default one and click Add
- for each track, fill in a Title and Beginning Offset (if needed), which determines where the track starts in the WAV file
- click Next or press Enter
- groupings can be made using the Add Grouping ... button
Instructions for creating container structure (track namings, hierarchy, etc.) are available in the Recording Container Structure Guide.
Diacritics and other special characters can be entered by hitting the F2 key.
When all metadata has been entered, click Save or Save & Exit which may take a while. If the derivative files were found on the server, they are processed, fully imported, and moved to their serving location in the audio-objects directory on the server. If the files were processed correctly, the Status should change to This record is valid. At this point, the recording should be available for listening by clicking the Listen button on in the Digitizing window.
Once a record is valid, the item should show up as "ready to Publish" in the progress listing on the digitizer tab in the Search window.
Before the 6.0 release, publishing was necessary to make content available to end users. As of 6.0, users can access content as soon as the recording is playable. Pressing the Publish button merely changes the record status to indicate that digitization is complete. This removes it from the progress listing on the digitizer tab.
In older, pre-6.0 versions of Variations, pressing the "Publish" button in the Digitizer window lets the system know that, when the nightly cron job is run, this recording should be moved from the cataloging server to the production server. When a file is ingested and saved in Variations, the record is created on the cataloging server and is only available to digitizers and catalogers. Once the file is published it is moved to the production server it is available to the end-users. This process prevents end-users from encountering partially digitized, incomplete, or incorrect records.
Using the Import Tracks Tool
After adding a sound file to the container, one may use the Import Tracks tool instead of manually entering the track titles and timings. To do this, select the CD/Side node in the Container Structure tree and click the "Import Tracks" button. A window will popup displaying items that match your sound file. The sources currently include the physical CD, Variations at IU, FreeDB, and MusicBrainz. Select the item which matches your sound file and click "Import". Note that the track timings returned from these services may be slightly off and should be checked and adjusted. For best results, open the "Import Tracks" window with the CD the wav file was made from in the CD drive. Note that this should be the only CD or DVD in any optical drive at the time.
- CD - This source pulls the track times from the CD in the drive. No track titles are returned.
- FreeDB - This source pulls track times and titles from FreeDB when the CD in is the drive. This source does not return any results when there is no CD in the drive.
- MusicBrainz - This source pulls track times and titles from MusicBrainz. This source works best when the CD is in the drive. If the CD is not in the drive, the Container title, and the number of tracks (if more than one) will be used to search MusicBrainz with at most 20 results returned. In this case, it will take about half a minute to return results.
- Variations - This source pulls track times, titles, and other Container Structure hierarchy from Variations at IU. If an OCLC number is present in the Container, then results will be returned based solely on this. If it is not present, then the search is based on the Container title, the length of the wav file, and track times if specified. Fuzzy matching is done on album length and track times. This should be the preferred source if a correct match exists.
- Wav Markers - This source pulls track times and titles from cue points embedded in the wav file. If there are more than one cue point at a given time (e.g. regions and markers), the behavior is undefined. This source does not work if the wav file was not added or replaced during the Variations session. Using wav markers may be of particular value for non-CD media, whether analog or "born digital." IU is using this option as part of our workflow for ingesting our born-digital Jacobs School of Music concert recordings.