Skip to content

File format recommendations for image and video SOPs#

We recommend to store all relevant information on the image curation process in a human and machine-readable format. For structured information, we recommend the .yaml format which is compact (i.e. helps with fast transfer across networks), well-formed (i.e. helps with automating curation tasks), and natively supported by OSs and editors. For tabular data, we recommend to use .txt files.

*.txt files#

Encoding ASCII information in *.txt files still needs a clear format description. The following is being used throughout the MareHub AGV/I tools: - The file may contain optional comment lines anywhere. These have to begin with the # symbol. Content of those lines is ignored while reading the files. - The first non-comment line of a file must contain the column headers. - Columns are split by the TAB character (\t, chr(9)). - Rows are split by the line feed character (\n, chr(10)). - Columns that correspond to the MareHub AG V/I metadata vocabulary have to exactly match the vocabulary terms. - The file is ASCII/UTF8 encoded plain text.

*.yaml files#

  • Need to follow the general *.yaml specification.
  • Key or values that correspond to the MareHub AG V/I metadata vocabulary have to exactly match the vocabulary terms.
  • The file is ASCII/UTF8 encoded plain text.

Examples#

Example *.yaml file#

SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg:
    image-longitude: -123.854637
    image-latitude: 42.133426
    image-depth: 4230.3
    image-pixel-per-millimeter: 12.1
    image-meters-above-ground: 1.3
    image-coordinate-uncertainty: 4.2
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg:
    image-longitude: -123.854638
    image-latitude: 42.133427
    image-depth: 4230.4
    image-pixel-per-millimeter: 12.1
    image-meters-above-ground: 1.4
SO268-1_21-1_GMR_CAM-23_20190513_131417.jpg:
    image-longitude: -123.854639
    image-latitude: 42.133428
    image-depth: 4230.5
    image-pixel-per-millimeter: 12.0
    image-meters-above-ground: 1.5

Example *.txt file#

# This is a comment
image-filename  image-longitude image-latitude  image-depth image-pixel-per-millimeter  image-meters-above-ground
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg -123.854637 42.133426   4230.3  12.1    1.3
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg -123.854638 42.133427   4230.4  12.1    1.4
# There may be another comment here
SO268-1_21-1_GMR_CAM-23_20190513_131417.jpg -123.854639 42.133428   4230.5  12.0    1.5

Important files in the image curation process#

There are some information in the image curation process that need to be available to facilitate FAIRness of the imagery. In the end, it is the iFDO that provides this functionality but on the way to creating the iFDO the curation workflow may create intermediate information files. How those look is very much up to your individual data curation processes and tools. Anyhow we define some essential files here which are used in the software packages provided by the AG V/I.

Important *.yaml files#

Important *.txt files#

First, please note that its not required that you create all these files! In case you can directly create the iFDO that is also fine. We just found that its helpful to keep these intermediate files available as well.

Still image navigation file#

image-filename  image-longitude image-latitude  {image-depth,image-altitude}    image-coordinate-uncertainty
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg -123.854637 42.133426   4230.3  12.3
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg -123.854638 42.133427   4230.4  13.1
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-navigation.txt

Moving image navigation file#

image-filename  image-second  image-longitude image-latitude  {image-depth,image-altitude} image-coordinate-uncertainty
SO268-1_21-1_GMR_CAM-42_20190513_111213.mp4 0   -123.854637 42.133426   4230.3  12.3
SO268-1_21-1_GMR_CAM-42_20190513_111213.mp4 1   -123.854638 42.133427   4230.4  13.1
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-navigation.txt

Image UUID file#

image-filename  image-uuid
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 9999ba88-1a20-4efe-a0ac-6b4233490ad6
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 1c266c00-33e7-4e69-bc9a-f90fb1bce6d0
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-uuids.txt

Image acquisition start time file#

image-filename  image-datetime
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 2019-05-13 13:14:15.0000
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 2019-05-13 13:14:16.0000
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-start-times.txt

Image scaling file#

image-filename  image-pixel-per-millimeter
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 12.1
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 12.1
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-scaling.txt

Image SHA256 hash file#

image-filename  image-hash
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 83f30eb35d1325c44c85fba0cf478825c0a629d20177a945069934f6cd07e087
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 27c3585560f93a78995a038b5970a002315d2b525b999a24a1d13a5c5e5520f6
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-hashes.txt

Image content feature file#

image-filename  pfdo-proxy-entropy  pfdo-proxy-particle-count   pfdo-proxy-average-color mpeg7-colorlayoutdescriptor
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 0.475   1342    [27,245,64] [39,17,17,13,...]
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg: 0.657  4223    [29,233,61] [39,17,17,13,...]
SO268-1_21-1_GMR_CAM-23_20190513_131417.jpg: 0.632  2342    [31,231,57] [39,17,17,13,...]
...

When working with the AGV/I folder structure place this file at: /<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-features.txt