Skip to content

Backup and Preservation

Backups protect research data from accidental loss, corruption, or hardware failure. Preservation focuses on keeping important data understandable and usable after the active phase of a project has ended.

These are related, but they are not the same.

Backup and Preservation

Backup Preservation
Purpose Recover data after loss or damage Keep data usable and understandable over time
Time frame During active work After or beyond the project
Typical content Current working data and recent versions Selected final data, documentation, code, and metadata
Main concern Recovery Long-term access and reuse

A backup is not automatically a long-term archive. Similarly, preserving only the data files without documentation may make them difficult to understand or reuse later.

Warning

Perform regular backups of your data to a safe location. The ML Cloud does not perform data backups.

What Should Be Backed Up?

Prioritise files that would be difficult, expensive, or impossible to recreate, such as:

  • original research data
  • manually created annotations or labels
  • cleaned or curated datasets
  • important configuration files
  • analysis code that is not stored elsewhere
  • model checkpoints that are expensive to reproduce
  • documentation and metadata
  • final results and figures

Files that can easily be regenerated may not require the same level of protection.

Preparing Data for Preservation

Preservation usually involves selecting the important parts of a project rather than keeping every temporary or intermediate file.

Depending on the project, preserve:

  • the original or authoritative data
  • the final processed dataset
  • code required to reproduce the results
  • software and environment specifications
  • documentation of processing steps
  • metadata and data dictionaries
  • licence and access information
  • relevant publications or report references
  • checksums for integrity verification

The preserved files should be understandable to someone who was not directly involved in the project.

File formats and software

Where possible, preserve data in formats that are:

  • openly documented
  • widely used
  • not dependent on one specific application
  • suitable for future conversion
  • capable of retaining the required information

Proprietary formats may still be necessary, but consider also storing an open or widely readable version when this can be done without losing important information.

For software-dependent workflows, record:

  • software names and versions
  • package dependencies
  • environment files
  • container definitions
  • configuration files
  • instructions for running the analysis
Data integrity

Checksums can help confirm that preserved files have not changed or become corrupted.

For example:

sha256sum <filename>

Store the checksum separately or alongside the preserved data. Checksums confirm file integrity, but they do not replace a backup.

Personal, confidential, or licensed data

Data protection, consent, contracts, or licence conditions may affect:

  • where data may be backed up
  • who may access backup copies
  • how long the data may be retained
  • whether the data may be transferred
  • whether it must be encrypted
  • whether it must be deleted after the project

Do not place restricted data in an external backup or repository without confirming that the location and transfer are permitted.

At the End of a Project

Before closing, transferring, or deleting a project:

  • identify the data that must be retained
  • remove unnecessary temporary and duplicate files
  • confirm that important files have a reliable copy
  • update the README and other documentation
  • record dataset, code, and software versions
  • check licence and access restrictions
  • assign responsibility for the retained data
  • document the retention period
  • confirm that preserved files can be opened and understood
  • decide what may be deleted and when

Best Practices

When backing up and preserving research data:

  • Do not keep important data in only one location
  • Prioritise data that cannot easily be recreated
  • Preserve documentation together with the data
  • Use suitable and well-documented file formats
  • Record software and environment requirements
  • Check retention, licence, and data protection conditions

Need Help Planning a Backup or Preservation Strategy?

Contact us through the ticketing system. Include the cluster, storage location, approximate data size, type of data, required retention period, and any relevant licence or access restrictions.

Last update: July 27, 2026
Created: July 27, 2026