From MXF/JPEG2000 to MKV/FFV1: why we’re switching to a new file format

The choice of file format is crucial for preserving files in the long term because it determines how bits and bytes need to be ordered. Media players and editing tools need to ‘recognise’ and support this format so that they can play the content. Choosing the right file format therefore largely depends on how easily it can play (read) and edit the file.

Among other things, a digitally sustainable file format:

• saves the original content properties;

• is clearly and unambiguously specified (to limit the risk of different interpretations);

• allows the development of free software (without any licences or patents, so you always have access to tools for migrating content);

• has published specifications and is managed by an open standards organisation.

In 2013, we decided to use JPEG2000 in MXF together with our content partners. This file format was the best option at the time, even though it wasn’t an ideal solution. MXF saves the original file properties, but:

• also allows lots of different variants, which can lead to excessive complexity;

• wasn’t published under an open licence;

• is managed by a closed industrial consortium.

FFV1 in MKV was already under development at the time, but it wasn’t very mature yet and, therefore, we did not consider it a suitable format for the archive master files. But both MKV and FFV1 are now well on the way to becoming an industry standard and are already being used in various archives, such as Indiana University, the New York Public Library and BFI National Archive (British Film Institute). Furthermore:

• it’s an open format;

• FFV1 is lossless just like JPEG2000, meaning it preserves the original content properties;

• it’s easier to implement;

• it uses the preservation requirements as the starting point, and FFV1 can have separate checksums for each image in the video so that any preservation issues can be localised more precisely;

• MKV/FFV1 can capture the properties from any source materials correctly, which is less straightforward with MXF/JPEG2000;

• there are more playback and editing tools that can work with MKV/FFV1, often available as open source.

We therefore believe that MKV/FFV1 has the potential to become a more sustainable video file format than MXF/JPEG2000 in the short term. We also already use FFV1 for the long-term storage of some files that originate from DV, DVCAM and DVCPRO video cassettes. Tests have shown that we can use this file format to perform the conversion without losing any information, which is why we’re planning to convert files that are currently in the JPEG2000 in MXF format into FFV1 in MKV. In particular, this concerns video files that originate from digitised video carriers in the cultural heritage sector. We are currently leaving the archive master files (usually IMX D10 in MXF) untouched for the media sector.

This conversion will be a long-term task. All archive master files (several petabytes of data) will need to be read and rewritten for this, which is why we’re combining it with the migration from LTO-6 to LTO-8. You can read how we’re tackling the migration together with our partners here.