Skip to content

Development Guide

This guide covers the development process and tools used in the Whitebox project and how you can develop new features or plugins for Whitebox and contribute to the project.

Prerequisites

  • Docker (for running the Whitebox server)
  • git-lfs (for downloading asset files)
  • git (for cloning the repository)

Understanding Docker Configuration

To facilitate easy deployment and testing, Whitebox includes Docker configurations. Dockerfiles define the necessary environment for running or developing the Whitebox, including all required dependencies and configuration. Compose files orchestrate the setup of the server, database, and other services.

All Whitebox containers build on a shared whitebox-base image that is published to the GitLab container registry. It bundles Python 3.14, Node.js 22, Poetry, and common system libraries so that backend, frontend, and CI jobs all start from the same consistent foundation. See Docker Registry Images for details.

The Dockerfile is usually configured in two stages: one for building the application and another for running it. This separation optimizes the build process and reduces the final image size.

Setting up Development Environment

The recommended way developers working on Whitebox is to use the dev containers. It is a pre-configured Docker container that includes all the necessary dependencies and tools for development. Once launched, the container will simply start an environment.

To access the environment, you will need to enter the shell of the container (think "SSH-ing" into the container). From there you can access the environment, install project dependencies, run the required services, etc, same as you would with an SSH session.

This allows you to develop code on your local machine and run it in the container without having to rebuild the container each time you make a change.

Starting Development Containers

For a primary checkout, create a .env file in the repository root. You can use the provided .env.example file as a template:

cp .env.example .env

Edit the .env file to set variables as needed. Linked worktrees may either have their own .env or reuse the primary checkout's ignored .env. The development wrapper selects configuration in this order:

  1. The file named by WHITEBOX_DEV_ENV_FILE (relative paths are resolved from the directory where the wrapper is invoked).
  2. .env in the active worktree.
  3. .env in the primary checkout, for a linked worktree.

It fails without reading Compose configuration if none of these is usable. To select a file explicitly, for example, run WHITEBOX_DEV_ENV_FILE=/path/to/dev.env make dev-up.

Start the development containers from the repository root:

make dev-up

Always use the root make dev-* targets for common operations or ./bin/dev-compose ... for other development Compose operations. The wrapper derives a stable project identity from the canonical worktree path and anchors Compose, source mounts, plugin mounts, and project resources to that worktree. Raw Compose invocations and global container-targeted commands bypass that boundary and can target the wrong checkout.

Published development ports are dynamic and bind to loopback by default so separate worktrees can run concurrently without exposing development services. Print the same-origin browser URL with make dev-url; discover a directly published service port with make dev-port service=<service> port=<container-port>. Other common commands are:

make dev-status
make dev-logs
make dev-down

On a trusted private network, VPN/overlay, or appropriately firewalled worker, bind published ports to every host interface by starting the project in host mode:

./bin/dev-compose --host up -d

This exposes development databases, debugger ports, and other unauthenticated services. To bind the configurable mappings to only one interface instead, provide its address directly, for example BIND_ADDR=10.42.0.1 ./bin/dev-compose up -d. The streaming profile's SRS mappings remain externally bound regardless of BIND_ADDR. Continue to use make dev-url and make dev-port to discover dynamic ports. If Docker reports 0.0.0.0, remote clients must substitute the worker's hostname or IP address; 0.0.0.0 is not a client destination.

Port bindings are selected when containers are created. docker compose restart retains existing bindings; running up -d with a different effective bind address may recreate affected containers.

Normal make dev-down preserves that worktree's project volumes. To enter the backend container shell, run:

make dev-backend-shell

To enter the frontend container shell, run:

make dev-frontend-shell

Both containers install the necessary development dependencies during build. They intentionally idle after startup; start backend and frontend processes as described below when you need them.

Hardware, host-monitoring, and streaming capabilities are opt-in because they control physical devices, host-global networking, or fixed streaming ports. Enable a profile through the wrapper, for example:

./bin/dev-compose --profile hardware up -d
./bin/dev-compose --profile host-monitoring up -d
./bin/dev-compose --profile streaming up -d

Each of these profiles may be owned by only one development project on a host at a time. The profile-free stack is the supported concurrent-worktree default.

Optional Opencode harness config

Contributors who use Opencode can find the shared Whitebox harness config in whitebox-llm-support. Follow that repository's README for setup; this guide intentionally does not duplicate those instructions. Project-specific agent behavior remains defined by the root AGENTS.md in each Whitebox repository.

To install new dependencies, make sure to use dev container shell. It is also recommended to rebuild the dev container after adding new dependencies to ensure they are cached at the build stage making future builds faster. To rebuild the containers:

./bin/dev-compose build --no-cache

Then start the containers:

make dev-up

Running the Django backend development server

  1. Ensure development environment is set up.

  2. Run the Whitebox server:

    ./bin/dev-compose exec backend-dev make run-dev

This starts the Django development server. Use make dev-url for the normal same-origin browser entrypoint, or make dev-port service=backend-dev port=8000 to discover its directly published address. Changes you make to the backend will be automatically reloaded.

Plugins' JSX files are transpiled only on backend startup. If you make changes to the plugins' JSX code, you will need to restart the backend server for them to get re-transpiled, and then refresh the frontend app.

Running the React frontend development server

  1. Ensure development environment is set up.

  2. Run the frontend development server:

    ./bin/dev-compose exec frontend-dev make run

Use make dev-url for the browser-facing nginx URL. If direct Vite access is needed, run make dev-port service=frontend-dev port=3000 to discover its published address. The development server has automatic reloading enabled, so you can see your changes in real-time.

This applies only the frontend project's code. In case you make changes to the plugins' JSX code, you will need to refresh the page for changes to take effect.

Running the RQ workers

  1. Ensure development environment is set up.

  2. Run the RQ worker:

    ./bin/dev-compose exec backend-dev make run-worker

This will start an RQ worker process that is going to process the tasks as they are added into the queue. If you need to run multiple workers for parallelized processing, you can run the run-worker command multiple times from multiple terminals.

Commands

Whitebox includes Makefiles with commands you can run inside the development containers for common development tasks.

Backend Commands

Setup Commands

These commands are for Whitebox setup, and need to be run inside the whitebox directory.

  • setup-network: Configure network interface naming for Orange Pi.

    Usage: 

    sudo make setup-network

    • Sets M.2 Wi-Fi interface's name to wlan-ap
    • Required for Wi-Fi access point setup

  • setup-sdr-udev: Set up udev rules for SDR devices.

    Usage:

    sudo make setup-sdr-udev
    • Downloads and installs Stratux/SDR device udev rules
    • Reloads udev rules automatically

Development Commands

First, enter the container's shell:

make dev-backend-shell

and then you can use the following Makefile commands (format: make <COMMAND>):

  • migrate: Apply database migrations
  • run: Start the backend server for production
  • run-dev: Start the backend server for development
  • run-for-test: Start the backend server in test mode

  • test: Run backend tests

  • build_federation_modules: Builds federation modules for all the plugins that are available to Whitebox. Useful for quicker iterations during development.

  • build_federation_modules_watch: Builds all plugin modules just as above, but also includes tests and watches for their file changes & rebuilds them

  • download_external_assets: For whitebox to function offline, it needs to download external assets like videos, external JS libraries, etc. for both plugins and whitebox itself. This command downloads all the external assets

  • clean: All the downloaded external assets will be removed. Whitebox will not function without these assets.

  • makemigrations: Create migrations for all the plugins

  • fmt: Format code using ruff

  • run-mkdocs: Run the MkDocs server for documentation

Frontend Commands

First, enter the container's shell:

make dev-frontend-shell

and then you can use the following Makefile commands (format: make <COMMAND>):

  • run: Start the frontend server for development
  • run-for-test: Start the frontend server in test mode
  • test: Run all frontend tests
    • unit_test: Run kernel's unit tests
    • integration_test: Run kernel's integration tests
    • federation_unit_test: Run plugins' unit tests (requires backend server running in test mode)
    • federation_integration_test: Run plugins' integration tests (requires backend server running in test mode)
    • test_e2e: Run end-to-end tests (requires frontend server running in test mode)
  • lint: Run lint checkers on the code

Plugin Commands

During development, plugins should be placed in the plugins/ directory, which is mounted into the backend service at /plugins for easy access.

Sometimes, work can span over multiple plugins. As there can be quite a bit of plugins in there, within /plugins directory, there's a Makefile that exposes some helper commands:

  • make [help] - Print a list of available commands
  • make check-dirty - List plugin repos that have uncommited changes
  • make diff-dirty - Show diff of all uncommited changes in all plugin repos
  • make sync-main - Check and pull main branch for all plugin repos
  • make checkout-if-available branch=BRANCH_NAME - Checkout branch in all repos where it exists and print poetry add command at the end to run in the backend container for convenience

Testing

Backend testing

Whitebox implements a custom test runner that allows for running tests for whitebox and plugins within a single test suite by discovering all plugin tests and loadin them dynamically.

To run tests, along with plugin tests, you need to make sure your plugin adheres to guidelines outlined in the Plugin Guide.

  1. Ensure the development environment is set up.

  2. Run the backend tests:

    ./bin/dev-compose exec backend-dev make test

Frontend testing

Whitebox uses Vitest (with Playwright in Browser Mode) for frontend unit testing, and Playwright for integration testing. To run frontend tests:

  1. Ensure the development environment is set up.

  2. Run the frontend tests:

    ./bin/dev-compose exec frontend-dev make test

This will run first the unit testing suite, and then the integration testing suite.

Federation testing

Federation modules require both frontend and backend to be running in order to test them - backend to build and serve the modules, and frontend to actually load and render the modules.

To run the federation tests, first run both servers in the testing mode (the two commands need to be run in separate terminals):

./bin/dev-compose exec backend-dev make run-for-test
./bin/dev-compose exec frontend-dev make run-for-test

Then, in a separate terminal, you can run the unit tests for the federation modules:

./bin/dev-compose exec frontend-dev make federation_test

or, run the integration tests for the federation modules:

./bin/dev-compose exec frontend-dev make federation_integration_test

End-to-end testing

Whitebox uses Playwright for end-to-end testing. The setup steps for end-to-end testing are slightly different from the other tests, as it requires running both the frontend and backend servers in conjunction.

As the frontend and backend are running in separate containers, and the tests will be run from the frontend container, the target hosts will be different—frontend will effectively live on localhost, while the backend will be on backend host (if using production containers), or backend-dev host (if using development containers).

For the tests to run successfully, you will need to ensure that the frontend application is configured to use the correct backend host from within the container.

To run end-to-end tests:

  1. Ensure development environment is set up.
  2. Run the backend environment that the frontend will use:
./bin/dev-compose exec backend-dev make run
  1. Run the frontend environment, ensuring that the backend host is set correctly for the frontend build:
./bin/dev-compose exec frontend-dev make run-for-test
  1. Run the test suite on the frontend container:
./bin/dev-compose exec frontend-dev make e2e_test

Running the tests on LambdaTest

Whitebox uses LambdaTest for running frontend integration, and end-to-end tests on multiple browsers and devices.

When ran during CI, the end-to-end tests will be run against the sandbox environment, which is setup as one of the steps before the LambdaTest testing step. You can also run the tests on LambdaTest from localhost, if you have an account.

To run the tests on LambdaTest, you will need the following:

  • LambdaTest username, later set as environment variable named LT_USERNAME
  • LambdaTest access key, later set as environment variable named LT_ACCESS_KEY

Similarly to the standard end-to-end test run, you'll need to setup frontend and backend environments, and then run the tests against them:

  1. Ensure development environment is set up.

  2. Run the backend environment that the frontend will use:

    ./bin/dev-compose exec backend-dev make run

  3. Run the frontend environment, ensuring that the backend host is set correctly for the frontend build:

    ./bin/dev-compose exec frontend-dev make run-for-test

  4. Run the LambdaTest test suites on the frontend container: 

    # Enter the frontend container shell
make dev-frontend-shell

# Set the LambdaTest credentials in the environment
export LT_USERNAME=<your_lambdatest_username>
export LT_ACCESS_KEY=<your_lambdatest_access_key>

# Run the integration tests on LambdaTest
yarn run test_integration:lambdatest-tunnel

# Run the end-to-end tests on LambdaTest
yarn run test_e2e:lambdatest-tunnel

Using *-tunnel commands, the tests will run using the LambdaTest tunnel which is preconfigured in the dev container. This allows for testing on local environments. This is useful when you want to run tests against the dev container running on your local machine.

You may also set environment variables like LT_USERNAME and LT_ACCESS_KEY in the .env file in the root of the project. Refer to the .env.example file for the required variables. This will allow you to run the tests from the host machine without having to set them each time like so:

./bin/dev-compose exec frontend-dev env $(./get-env) npm run test_integration:lambdatest-tunnel

Ensure you do poetry install at the root of the project to install all the required dependencies for the get-env script to work.

If you want to run the tests without the tunnel, against a sandbox, you can run the integration tests with:

FRONTEND_URL="TARGET_URL_OF_THE_SANDBOX" yarn run test_integration:lambdatest

or end-to-end tests with:

FRONTEND_URL="TARGET_URL_OF_THE_SANDBOX" yarn run test_e2e:lambdatest

Debugging

For comprehensive debugging procedures including Insta360 connection troubleshooting, check the Troubleshooting section.

VSCode

  1. Install VSCode and open the whitebox project in VSCode.
  2. Add Python language support by installing the extension.
  3. JS Language support and chrome debugger support is available by default.
  4. If using firefox browser, you would need to install this extension.

Backend Debugging

  1. On the host, run make dev-port service=backend-dev port=5678 and update Debug Backend's connect.port in .vscode/launch.json with the reported port.
  2. Enter the backend dev container's shell with make dev-backend-shell.
  3. Run make debug (the server waits for the debugger before starting).
  4. In Run & Debug, select Debug Backend and click the play button.

Frontend Debugging

  1. On the host, run make dev-port service=frontend-dev port=9229 and update Debug Vite Server's port in .vscode/launch.json with the reported port.
  2. Run make dev-url and update the Debug Chrome or Debug Firefox URL with the reported browser URL.
  3. Enter the frontend dev container's shell with make dev-frontend-shell, then run make debug.
  4. In Run & Debug, select Debug Frontend (Chrome) or Debug Frontend (Firefox) and click the play button.

The checked-in ports and URLs are defaults only; runtime-assigned values are specific to the active worktree and may change after container recreation.

PyCharm

Remote debugging on PyCharm works only on the Professional Edition and can take some work to fully set up. Community edition does not support remote debugging in any capacity.

You can find the detailed explanation and setup steps in PyCharm debugging.

Contributing

  1. Fork the repository
  2. Create a new branch for your feature
  3. Make your changes
  4. Run tests and ensure they pass
  5. Submit a pull request

Google Docstring Conventions should be followed for all code documentation.

Documentation

Documentation for Whitebox is generated using MkDocs. To run documentation locally, make sure you have Whitebox repository cloned and set up as per the instructions above.

To run the documentation server:

  1. Ensure development environment is set up.

  2. Enter the backend container shell:

    make dev-backend-shell
    3. Run the documentation server:

    make run-mkdocs

Versioning

Whitebox uses Semantic Versioning for versioning. In the CI configuration, the update_version stage is responsible for updating the version of the project. Backend and frontend versions are kept in sync, with the backend's version used as a reference point.

When a merge request is merged to main, first, patch version will be bumped in the pyproject.toml file for backend, and that same version number will then be applied to package.json for frontend. This is done by the script located in packaging/scripts/maintenance/whitebox_update_version.py.

Afterward, a commit will be made with the new version, which will then be tagged, and the CI will push these changes to the repository.

Adding temporary dependencies to CI

Sometimes you need to add a temporary dependency to the CI environment that you do not want to be included in the project's dependencies upon merge. You may want to do this when you are working on Whitebox core and a plugin in parallel. In these cases, the plugins' changes would only be available on its own branch, which are not published in PyPI, so you have to install them directly from Git.

To do this, you can add the dependency to the plugins-temporary Poetry group. Tests and sandbox will be run with these temporary dependencies installed (they take precedence over the "default" ones from the pyproject.toml file), but as it's an optional group, they won't be included in the final project dependencies. This change will be safe to merge, as the CI will perform the cleanup (for more info, take a look how the maintenance CI step works).

You can reference a Git branch directly in the poetry add command, by doing:

poetry add --group plugins-temporary git+GIT_BRANCH#BRANCH_NAME

For example, to add a temporary dependency with Git URL https://gitlab.com/whitebox-aero/whitebox-plugin-gps-display.git, with a branch feature/my-new-feature, you can run:

poetry add --group plugins-temporary git+https://gitlab.com/whitebox-aero/whitebox-plugin-gps-display.git#feature/my-new-feature

During development, dev containers automatically include the temporary dependencies group, so you can use them right away. However, if you want temporary dependencies in production, you need to set the TEMPORARY_DEPENDENCIES environment variable to 1 in .env file before building the production containers.

Next Steps