Page tree
Skip to end of metadata
Go to start of metadata

Category description and use cases

Workflow example:

Output standard

Summary: 


Element

Datatype

Obligation

Definition

media

object

required

Wrapper for metadata about the source media file.

media.filename

string

required

Filename of the source file.

media.duration

string

required

The duration of the source file..

shots

array

required

The list of shots in the video. 

shots[*]

object

optional

A shot in a video. 

shots[*].type

string

required

The type of shot, “scene” or “shot”.

shots[*].start

string

required

The start time in seconds (s.fff).

shots[*].end

string

required

The end time in seconds (s.fff).

JSON Schema

Schema
{
    "$schema": "http://json-schema.org/schema#",
    "type": "object",
    "title": "Shot Detection Schema",
    "required": [
        "media",
        "shots"
    ],
    "properties": {
        "media": {
            "type": "object",
            "title": "Media",
            "description": "Wrapper for metadata about the source media file.",
            "required": [
                "filename",
                "duration"
            ],
            "properties": {
                "filename": {
                    "type": "string",
                    "title": "Filename",
                    "description": "Filename of the source file.",
                    "default": "",
                    "examples": [
                        "myfile.wav"
                    ]
                },
                "duration": {
                    "type": "string",
                    "title": "Duration",
                    "description": "Duration of the source file.",
                    "default": "",
                    "examples": [
                        "25.888"
                    ]
                }
            }
        },
        "shots": {
            "type": "array",
            "title": "Shots",
            "description": "The shots and/or scenes in a video.",
            "items": {
                "type": "object",
                "required": [
                    "type",
                    "start",
                    "end"
                ],
                "additionalProperties": false,
                "properties": {
                    "type": {
                        "type": "string",
                        "title": "Type",
                        "description": "The type of shot, 'scene' or 'shot'.",
                        "enum": [
                            "scene",
                            "shot"
                        ]
                    },
                    "start": {
                        "type": "string",
                        "title": "Start.",
                        "description": "Start time in seconds",
                        "default": 0.0,
                        "examples": [
                            "123.45"
                        ]
                    },
                    "end": {
                        "type": "string",
                        "title": "End",
                        "description": "End time in seconds",
                        "default": 0.0,
                        "examples": [
                            "123.45"
                        ]
                    }
                }
            }
        }
    }
}

Sample output

Sample Output
{
    "media": {
        "filename": "myvideo.mp4",
        "duration": "45.35"
     },
    "shots": [
        {
            "type": "scene",
            "start": "0.0",
            "end": "45.35"
        },
        {
            "type": "shot",
            "start": "0.0",
            "end": "10.89"
        },
        {
            "type": "shot",
            "start": "10.89",
            "end": "19.4"
        },
        {
            "type": "shot",
            "start": "19.4",
            "end": "45.35"
        }
    ]
}


Recommended tool(s)

Azure Video Indexer

Official documentation: <link>

Language: 

Description: 

Cost: <$ OR Free (open source)>

Social impact: 

Notes: 

Installation & requirements


Parameters

Input formats


Example Usage

<tool name> Example
 

Example Output

<tool name> Output
 

PySceneDetect

Official documentation: https://pyscenedetect.readthedocs.io/

Language: Python

Description: PySceneDetect is an open-source command-line application and Python library for detecting scene changes in videos. 

Cost: Free (open source)

Social impact: 

Notes: See also this Colab notebook.

Installation & requirements

Can be installed via pip: `pip install scenedetect`

Parameters

Input formats: mp4, likely others?

Example Usage

Can be used as a python library (which we don't have an example of) or run from the command line and the output reshaped to our JSON format:

pyscenedetect Command Line Example
scenedetect --input "video.mp4" --output output/path detect-content 

Example Output

Default output is a CSV, where the first row is a list of scene ending timecodes, and the second row begins a table with timecode information (see below). There is code in the pyscenedetect Colab notebook for reshaping this output to our standard JSON.

Timecode List:0:00:110:00:190:00:260:00:280:00:300:00:350:00:360:00:37
Scene NumberStart FrameStart TimecodeStart Time (seconds)End FrameEnd TimecodeEnd Time (seconds)Length (frames)Length (timecode)

Length (seconds)

100:00:0003320:00:1111.0673320:00:1111.067
23320:00:1111.0675770:00:1919.2332450:00:088.167
35770:00:1919.2337700:00:2625.6671930:00:066.433
47700:00:2625.6678440:00:2828.133740:00:022.467
58440:00:2828.1339140:00:3030.467700:00:022.333
69140:00:3030.46710450:00:3534.8331310:00:044.367
710450:00:3534.83310840:00:3636.133390:00:011.3
810840:00:3636.13311140:00:3737.133300:00:011

Other evaluated tools

Google Cloud Video Intelligence

Official documentation: Google Cloud Video Intelligence Documentation

Language: 

Description: 

Cost: <$ OR Free (open source)>

Social impact: 

Notes: 

Installation & requirements


Parameters


Input formats


Example Usage

<tool name> Example
 

Example Output

<tool name> Output
 

Evaluation summary

See Shot detection analysis

  • No labels