Migration and capacity expansion in the meemoo archive system
18 Jun 2020Archiving is a verb: our archive system isn’t just a place where files go to gather dust. In order to safeguard the readability – and so also the accessibility – of the content, and increase archive capacity, this summer we’re starting a large storage migration.
The meemoo archive system is constantly expanding with more and more content all the time. We digitise new audiovisual media and carrier types, and take on new digital collections. Our archive system grows by around 2 PB of content every year. At this rate, our archive infrastructure would reach its maximum capacity by the end of 2020.
LTO tape also has a finite lifetime as a storage medium, which is why we are now making changes in the existing archive infrastructure.
This intervention doesn’t mean that we’re going to buy new infrastructure, but rather that we’re going to build on previous investments. We’re continuing to use our existing tape infrastructure, but switching from LTO-6 tapes to LTO-8. LTO-6 tapes can store up to 2.5 TB each, but LTO-8 tapes can hold up to 12 TB. This will increase our storage capacity by a factor of 4 or 5, and it could grow to over 70 PB in the future.
How are we approaching this storage migration?
Because not all tapes are the same age and we only want to migrate when the need arises, we work in phases. Since the summer of 2020, all new content that meemoo ingests, is stored on the new LTO-8 tapes. Content that has already been archived is being migrated per organisation from November onwards. For example, this year we’re starting with regional broadcasters and in 2021 we’ll make a start with content from our other partners. Our account managers will contact any organisations whose turn is coming up. According to the current schedule, the total migration of content already stored with us will take six years.
For content partners with JPEG2000 masters in their collection, we’re also performing an archive format migration: converting files in JPEG2000 in MXF containers into FFV1 in MKV containers. You can read more about this here (only in Dutch). These organisations are also being contacted for this individually in 2021.