Skip to content

Backup and Restore

Backup and restore mechanism

Whitebox provides a way to create device backups that can be restored later. This is useful for preserving device configurations and moving between devices.

For creating and restoring backups, you can use the ./bin/export-whitebox-data.sh and ./bin/import-whitebox-data.sh scripts.

First, you'll need to connect to the device.

Connecting to Your Whitebox Device

Either connect a screen and keyboard to your Orange Pi, or find its IP to access it from another computer using the following command, which will list all the IPs open for connection on the SSH server port, which should include your Orange Pi:

sudo nmap -p 22 --open 192.168.1.0/24

replacing 192.168.1.0/24 with your local network IP range. You can SSH into your Orange Pi using:

ssh whitebox@192.168.1.120

replacing 192.168.1.120 with the IP identified via nmap.

The password for the whitebox user should be whitebox by default, or the one you set during installation.

Creating a Backup

After connecting to your Whitebox device, you can create a backup by running the following:

cd /whitebox
sudo ./bin/export-whitebox-data.sh

By default, this creates a file named whitebox-data-export.tar.gz.

The default export includes:

  • the Postgres database dump (database-dump.sql)
  • recorded media files (media/)
  • export metadata (metadata.json)
  • persistent Whitebox filesystem logs (whitebox-logs/) if /var/log/whitebox exists on the device
  • Docker app logs and inspect data (app-logs/) for all local containers still present, including stopped containers
  • host system logs (system-logs/) from dmesg/journalctl and, when available, a raw persisted journal snapshot

You should not perform any operations in Whitebox while the backup is being created.

There are several options to customize the backup process:

  • --archive-target FILE: Specify a different output file path (or directory) for the backup archive
  • -a / --no-app-logs: Exclude Docker app logs from app-logs/
  • -s / --no-system-logs: Exclude host system logs from system-logs/

whitebox-logs/ comes from /var/log/whitebox, which is created when persistent logging is enabled. Previous-boot journal entries in system-logs/ require persistent journald to have been enabled before the incident. Enable both going forward with:

cd /whitebox
sudo make setup-logging

When enabled, logs for services using Whitebox's wb-* syslog tags (for example backend/frontend/rq-worker/Stratux and other compose services using the default logging config) are written there.

This does not recover logs from before persistence was enabled, or logs already removed by cleanup, rotation, or deletion.

Additionally, you can exclude certain data from the backup. Note that backups using these might not be importable, so make sure you know what you're doing with them. You have the following options:

  • --no-database: Exclude the database from the backup
  • --no-media: Exclude recorded media files from the backup
  • --no-archive: Do not create an archive file; instead, just prepare the data for export in a temporary directory that'll be printed to the console

You can run ./bin/export-whitebox-data.sh --help to see all available options.

Once a backup is created, you can transfer the backup file to your computer using scp or any other file transfer method that you prefer.

Restoring a Backup

Restoring a backup will overwrite existing data on the device. In case it has any data you want to keep, make sure to create a backup before proceeding.

After connecting to your Whitebox device, you can restore a backup by running the following:

cd /whitebox
sudo ./bin/import-whitebox-data.sh PATH_TO_BACKUP_FILE

By default, this will import the database and recorded media from the specified backup file.

You should not perform any operations in Whitebox while the backup is being restored.

There are several options to customize the restore process:

  • --no-db: Do not restore the database from the backup
  • --no-media: Do not restore recorded media files from the backup

You probably don't need the above options, as loading only a part of the backup may cause Whitebox to be inoperational.

You can run ./bin/import-whitebox-data.sh --help to see all available options.

Anatomy of a Backup File

Backup files are .tar.gz archives that contain the following structure:

  • database-dump.sql: SQL dump of the Whitebox database (unless --no-database was used)
  • media/: Directory containing recorded media files (unless --no-media was used)
  • whitebox-logs/: Persistent filesystem logs from /var/log/whitebox (included when that directory exists)
  • app-logs/: Raw Docker container log files plus inspect.json for all local containers still present, including stopped containers (unless --no-app-logs was used)
  • system-logs/: Host system logs (unless --no-system-logs was used):
    • dmesg.log: current boot only
    • journalctl.log: full available journal output
    • journalctl-list-boots.log: available boots in the journal
    • journalctl-kernel.log: kernel messages from the available journal
    • raw-journal/: best-effort raw persisted journal snapshot when available
    • raw-journal-copy-errors.log: may exist when raw snapshot copying was partial, unsafe entries were removed, or capture was skipped
  • metadata.json: Metadata about the backup, such as version information, creation date, and included/excluded components

In short:

  • Default export: database + media + metadata + app/system logs (+ whitebox-logs/ if available)
  • --no-app-logs: omits app-logs/
  • --no-system-logs: omits system-logs/

If you're creating a backup file to diagnose an issue with Whitebox, the default logs can be very helpful. Note that dmesg.log covers only the current device boot; previous-boot journal logs are available only if persistent journald was enabled before the incident and the entries have not been removed.

Some of them can also be very verbose and contain sensitive information, so be cautious when sharing them with others, especially raw journal exports. For example, some countries may have specific regulations about sharing RF-acquired or GPS data.

Backup and restore the entire SD card

Sometimes, you might want to backup or clone the entire SD card used by your Whitebox device. You can do this using tools like dd.

Note that the instructions below may be dangerous if you specify the wrong device name, as you may overwrite important data. Always double-check the device names before running the commands.

If you are not comfortable with this, we advise you to use the backup and restore mechanism described above instead.

Backing up the SD card

To create a backup image of your SD card, first identify the device name of your SD card. You can use the lsblk command on Linux, or diskutil list on macOS.

You are probably looking for a device like /dev/sdX on Linux or /dev/diskN on macOS, where X is a letter and N is a number.

Once you've identified it, you can create a backup image using the following command:

# replace DEVICE_PATH with your SD card device (e.g., /dev/sdX)
# replace PATH_TO_WRITE_IMG_TO with the path where you want to save the image
sudo dd if=DEVICE_PATH of=PATH_TO_WRITE_IMG_TO bs

# For example:
sudo dd if=/dev/sda of=~/whitebox-sdcard-backup.img bs

This command will create a bit-for-bit copy of the SD card and save it as an .img file that you can later use to restore the SD card or clone it to another SD card. Note that this process will create a file as large as the SD card's TOTAL capacity, even if there's unused space, so make sure you have enough space.

Initializing a demo environment from an SD card image

Whitebox provides a utility to create a demo environment from an SD card image. This is useful for testing or development purposes without needing to write the image to an actual SD card.

This utility is intended to be used on a fresh Linux system, dedicated to creating the demo environment, and it should not be used on any other systems. It performs a destructive operation - it will stop all Docker containers, and remove all Docker images, volumes, and networks in the process, as a way to ensure a clean environment for the demo.

Once again, DO NOT USE THIS ON ANY SYSTEM THAT HAS DATA YOU WANT TO KEEP. We are not responsible for any data loss that may occur from using this utility.

To create a demo environment from an SD card image, you first need to have these installed:

  • Docker
  • jq
  • make

After installing these and downloading the SD card image, you'll first need to download the provisioning script. As any potentially existing Whitebox installation will likely be removed as part of the process, it is advisable not to run this from an existing Whitebox installation but download it separately.

To download it to /provision-demo-from-img.py and run it with the path to your SD card image, you can run the following command:

sudo curl https://gitlab.com/whitebox-aero/whitebox/-/raw/main/bin/provision-demo-from-img.py /provision-demo-from-img.py
sudo bash /provision-demo-from-img.py -i PATH_TO_SD_CARD_IMAGE.img

This will download the provisioning script and run it with the specified SD card image. The script will mount the image to a loop device, extract necessary data, remove any existing Docker containers and images to ensure the clean environment, and then set up the demo Whitebox environment using Docker. You will be prompted to adjust the .env file contents during the process.

Additionally, if you'd like to initialize the demo environment with a specific branch's code, you can provide a -b BRANCH_NAME argument, for example:

sudo bash /provision-demo-from-img.py -i PATH_TO_SD_CARD_IMAGE.img -b feature/special-branch-for-demo