(v5.5.7 Migration) Binary files migration

Binary files migration

The upcoming major version 6.0 release of Server Pro and Community Edition will reduce the storage usage of binary files in half. An online migration is included in version 5.5.7 , allowing for minimal downtime as part of the upgrade.

Since Server Pro 4.x, binary files are stored twice: in the active files storage in "filestore" and in the full project history system. Moving forward, a single copy of each file will be stored in the full project history system.

The migration to the consolidated storage system is composed of two parts: A new flag for controlling the phase of the migration and a script that processes all active and soft-deleted projects.

Phases:

• OVERLEAF_FILESTORE_MIGRATION_LEVEL=0 (default), files are read and written to filestore. Files are written to history asynchronously.

• OVERLEAF_FILESTORE_MIGRATION_LEVEL=1 , files are read from history with fallback to filestore and written to both filestore and the history. Downgrade to OVERLEAF_FILESTORE_MIGRATION_LEVEL=0 is possible.

• OVERLEAF_FILESTORE_MIGRATION_LEVEL=2 files are read and written to history only. Downgrade to OVERLEAF_FILESTORE_MIGRATION_LEVEL=1 is not possible, unless it was performed "offline".

When storing data in S3 and using separate service accounts for filestore (OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID) and history (OVERLEAF_HISTORY_S3_ACCESS_KEY_ID): Please grant the filestore user read access to the history bucket for blobs OVERLEAF_HISTORY_PROJECT_BLOBS_BUCKET . The filestore service will serve reads from the compiler service moving forward.

It is highly recommended to perform the binary files migration in a non-production/sandbox environment first.

The standard Server Pro license allows you to run the application in a production environment as well as one in a non-production/sandbox environment; it is highly recommended that you provision a non-production environment for testing.

If you upgrade to Server Pro/CE version 6.0 and later decide that you want to downgrade to an earlier version, then you should restore from a full system backup.

Migration procedure

1

Create a backup

Create a full backup of your instance with a consistent snapshot of the mongo, redis and sharelatex directories.

2

Update

Toolkit: Use the $ bin/upgrade script to upgrade the toolkit to the latest version. When asked, do not confirm the prompt Upgrade image? — instead, manually edit config/version file and set the value to 5.5.7.

Legacy docker-compose.yml: Update the version of the sharelatex service to 5.5.7.

3

Estimate the number of affected projects

# Overleaf Toolkit users:
$ bin/docker-compose exec sharelatex /bin/bash -c "source /etc/overleaf/env.sh && source /etc/container_environment.sh && cd /overleaf/services/history-v1 && /sbin/setuser www-data node storage/scripts/back_fill_file_hash.mjs --report"

# Legacy docker-compose.yml users:
$ docker compose exec sharelatex /bin/bash -c "source /etc/overleaf/env.sh && source /etc/container_environment.sh && cd /overleaf/services/history-v1 && /sbin/setuser www-data node storage/scripts/back_fill_file_hash.mjs --report"

Example output:

Current status:
- Total number of projects: 10
- Total number of deleted projects: 5
Sampling 1000 projects to estimate progress...
Sampled stats for projects:
- Sampled projects: 9 (90% of all projects)
- Sampled projects with all hashes present: 5
- Percentage of projects that need back-filling hashes: 44% (estimated)
- Sampled projects have 11 files that need to be checked against the full project history system.
- Sampled projects have 3 files that need to be uploaded to the full project history system (estimating 27% of all files).
Sampled stats for deleted projects:
- Sampled deleted projects: 4 (80% of all deleted projects)
- Sampled deleted projects with all hashes present: 3
- Percentage of deleted projects that need back-filling hashes: 25% (estimated)
- Sampled deleted projects have 2 files that need to be checked against the full project history system.
- Sampled deleted projects have 1 files that need to be uploaded to the full project history system (estimating 50% of all files).

4

Flush project history queues

# Overleaf Toolkit users:
$ bin/docker-compose exec sharelatex /overleaf/bin/flush-history-queues

# Legacy docker-compose.yml users:
$ docker compose exec sharelatex /overleaf/bin/flush-history-queues

Repeat the flush until all projects have been flushed ("project_ids":0).

found projects {"project_ids":0,"limit":100000,"ts":"2025-09-01T10:35:33.353Z"}
total {"succeededProjects":0,"failedProjects":0}

In case "failedProjects" is not zero, please reach out to support and do not continue with the binary files migration.

5

Advance the migration phase to 1

Toolkit: Set OVERLEAF_FILESTORE_MIGRATION_LEVEL=1 in config/variables.env.

Legacy docker-compose.yml: Set OVERLEAF_FILESTORE_MIGRATION_LEVEL: '1' in the environment section of the sharelatex service.

6

Apply the configuration change and start the instance

Toolkit: bin/up -d

Legacy docker-compose.yml: docker compose up -d

7

Verify access to binary files

Open a project in the Overleaf editor in the browser and select a binary file, like an image.

8

Run the migration script

# Overleaf Toolkit users:
$ bin/docker-compose exec sharelatex /bin/bash -c "source /etc/overleaf/env.sh && source /etc/container_environment.sh && cd /overleaf/services/history-v1 && /sbin/setuser www-data node storage/scripts/back_fill_file_hash.mjs --all"

# Legacy docker-compose.yml users:
$ docker compose exec sharelatex /bin/bash -c "source /etc/overleaf/env.sh && source /etc/container_environment.sh && cd /overleaf/services/history-v1 && /sbin/setuser www-data node storage/scripts/back_fill_file_hash.mjs --all"

If you're persisting log files outside the sharelatex container, ensure that the logs directory owner is set to the www-data user (uid=33) so that the outputted log file can be written.

The output should look like this:

Set UV_THREADPOOL_SIZE=16
{"name":"default","hostname":"c25e9faaeb53","pid":971,"level":30,"backend":"fs","msg":"Loading backend","time":"2025-07-25T15:00:58.166Z","v":0}
Writing logs into /var/log/overleaf/file-migration-2025-07-25T15_00_58_199Z.log
Starting project file backup...
Loaded global blobs: 0
Processing non-deleted projects...
Processed 1 projects, elapsed time 0s
Done updating live projects
Processing deleted projects...
The collection deletedProjects appears to be empty.

Done updating deleted projects
Done.

If the migration is successful, you'll get an exit code of 0, and the last lines indicating no failures:

Done.

The log file will look like this (use the path as printed by the script):

$ docker cp sharelatex:/var/log/overleaf/file-migration-2025-07-25T15_00_58_199Z.log .
$ cat file-migration-2025-07-25T15_00_58_199Z.log
{"name":"file-migration","hostname":"c25e9faaeb53","pid":971,"level":30,"end":"68839a8f577b9f009d947b27 (2025-07-25T14:54:07.000Z)","msg":"actually completed batch","time":"2025-07-25T15:00:58.379Z","v":0}
{"name":"file-migration","hostname":"c25e9faaeb53","pid":971,"level":30,"time":"2025-07-25T15:00:58.383Z","LOGGING_IDENTIFIER":"4effa2000000000000000000","projects":1,"blobs":6,"filesWithHash":5,"filesWithoutHash":2,"filesDuplicated":0,"filesRetries":0,"filesFailed":0,"fileTreeUpdated":0,"badFileTrees":0,"globalBlobsCount":0,"globalBlobsEgress":0,"projectDeleted":0,"projectHardDeleted":0,"fileHardDeleted":0,"mongoUpdates":1,"readFromGCSCount":7,"readFromGCSIngress":28532,"writeToGCSCount":5,"writeToGCSEgress":300,"readFromGCSThroughputMiBPerSecond":0.14925639825786063,"eventLoop":{"idle":48.277844,"active":381.53244699971054,"utilization":0.8876763888372498},"diff":{"eventLoop":{"idle":48.223536,"active":134.04030200059555,"utilization":0.7354190687027976},"projects":1,"blobs":6,"filesWithHash":5,"filesWithoutHash":2,"filesDuplicated":0,"filesRetries":0,"filesFailed":0,"fileTreeUpdated":0,"badFileTrees":0,"globalBlobsCount":0,"globalBlobsEgress":0,"projectDeleted":0,"projectHardDeleted":0,"fileHardDeleted":0,"mongoUpdates":1,"readFromGCSCount":7,"readFromGCSIngress":28532,"writeToGCSCount":5,"writeToGCSEgress":300,"readFromGCSThroughputMiBPerSecond":0.14925639825786063},"deferredBatches":[],"msg":"file-migration stats","v":0}

9

Stop the instance

Toolkit: bin/stop sharelatex

Legacy docker-compose.yml: docker compose stop sharelatex

10

Make old files inaccessible to the application

You can now move the old files to secondary storage. We recommend keeping the files around for a while in case issues arise later.

# Toolkit users:
$ bin/docker-compose run --rm --entrypoint mv sharelatex --no-clobber --verbose /var/lib/overleaf/data/user_files /var/lib/overleaf/data/old_user_files

# Legacy docker-compose.yml users:
# We are assuming that you are using the default bind-mount in /var/lib/overleaf
$ docker compose run --rm --entrypoint mv sharelatex --no-clobber --verbose /var/lib/overleaf/data/user_files /var/lib/overleaf/data/old_user_files
# In case you are using selective bind-mounts, you can simply remove the bind-mount for /var/lib/overleaf/data/user_files inside the container.

11

Advance the migration phase to 2

Toolkit: Set OVERLEAF_FILESTORE_MIGRATION_LEVEL=2 in config/variables.env.

Legacy docker-compose.yml: Set OVERLEAF_FILESTORE_MIGRATION_LEVEL: '2' in the environment section of the sharelatex service.

12

Apply the configuration change and start the instance

Toolkit: bin/up -d

Legacy docker-compose.yml: docker compose up -d

13

Verify access to binary files

Open a project in the Overleaf editor in the browser and select a binary file, like an image.

Offline migration

If you want to prevent users from being able to log in while the binary file migration script is running, please follow these steps:

• Log into your Overleaf instance with an admin account

• Click on the Admin button and choose Manage Site

• Click the Open/Close Editor tab

• Click on the Close Editor button

• Click on the Disconnect all users button

Once this has been done, if any users are logged in they will get redirected to the maintenance page, and any new users visiting the log-in page will see the maintenance page and won't be able to log in.

You need to repeat these steps when restarting the instance. To re-open the site, simply restart the instance.

Online migration

It is possible to run the migration scripts while the application is still running. There are a few considerations to take into account:

• The migration process is IO intensive, you should monitor resource usage while the script is running.

• With a high processing concurrency, the event loop in filestore service might experience some blocking, which would lead to a degraded user experience. We recommend starting with the default values of --concurrency=10 and --concurrent-batches=1 .

• You can stop the script at any time. Starting it again will validate the previous projects and skip over files that have been processed already. This is useful in case you prefer to run the migration in less busy hours (e.g. at night).

Our recommendation is to close the site and run the migration offline in a maintenance window when your project count is less than 1000 projects (see output of the migration script when running with --report). If the number of projects is large you can run the script and monitor its progress, then decide whether to continue running it online or offline based on your particular case.

Clean up legacy binary file data

When you are done with the migration and verified that projects can still access all their files, you can remove the old file storage in /var/lib/overleaf/data/user_files. We highly recommend keeping these files around for a while - you can make them inaccessible to the application by renaming the folder first.

Troubleshooting

We will add troubleshooting advice here. Please note that while we normally offer support only to Server Pro customers, given the nature of this migration, we will also do our best to support CE customers who experience problems specific to the binary file migration.

If the binary file migration script fails (i.e. exits with an error or prints a non-zero number of failed projects), please send the following details to our support team by email [email protected], detailing:

Subject: Binary file migration problem

Body:

• Instance Type: CE or Server Pro (delete as appropriate)

• Installation Type: Overleaf toolkit or docker-compose.yml or other (delete as appropriate)

• Version: 5.5.x (toolkit: $ cat config/version)

• Migration script output (which should be located in the container under /var/log/overleaf)

• Report: (run migration script with --report)

• Processed projects: (as per the last run of the script)

• Duration of the migration:

• bin/doctor output (when using toolkit)

• Toolkit version: $ git rev-parse HEAD (when using Toolkit)

Consider attaching the log files for the filestore service to the email. You can find it at /var/log/overleaf/filestore.log inside the sharelatex container and export them like this:

$ docker cp sharelatex:/var/log/overleaf/filestore.log .
# replace <timestamp> with the timestamp as printed by the script
$ docker cp sharelatex:/var/log/overleaf/file-migration-<timestamp>.log .

Please redact any sensitive information from the log files before attaching them.

Missing files

Older versions of Server Pro/CE created file-tree entries before user uploads finished, which could cause files to appear as missing when an upload failed. You might find a few of these cases reported as errors when processing all the file-trees.

In case the number of missing files is low, consider manually reviewing these cases and delete them from the editor in the browser.

In case the number of missing files is high, consider reaching out to support, see email template above.

Finding broken file trees

The migration may fail for projects which have a malformed file tree (for example, where filenames are empty). You can find a list of these problems using the find_malformed_filetrees script which checks all projects in the database:

$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web && /sbin/setuser www-data node scripts/find_malformed_filetrees.mjs > /tmp/malformed-file-trees.json"

To fix the invalid paths, use the fix_malformed_filetree script, running the command once for each bad path:

$ bin/docker-compose exec sharelatex /bin/bash -c "cd /overleaf/services/web && /sbin/setuser www-data node scripts/fix_malformed_filetree.mjs --logs=/tmp/malformed-file-trees.json"