Development Guide

This guide covers the development process and tools used in the Whitebox project and how you can develop new features or plugins for Whitebox and contribute to the project.

Prerequisites¶

Docker (for running the Whitebox server)
git-lfs (for downloading asset files)
git (for cloning the repository)

Understanding Docker Configuration¶

To facilitate easy deployment and testing, Whitebox includes Docker configurations. Dockerfiles define the necessary environment for running or developing the Whitebox, including all required dependencies and configuration. Compose files orchestrate the setup of the server, database, and other services.

All Whitebox containers build on a shared whitebox-base image that is published to the GitLab container registry. It bundles Python 3.14, Node.js 22, Poetry, and common system libraries so that backend, frontend, and CI jobs all start from the same consistent foundation. See Docker Registry Images for details.

The Dockerfile is usually configured in two stages: one for building the application and another for running it. This separation optimizes the build process and reduces the final image size.

Setting up Development Environment¶

The recommended way developers working on Whitebox is to use the dev containers. It is a pre-configured Docker container that includes all the necessary dependencies and tools for development. Once launched, the container will simply start an environment.

To access the environment, you will need to enter the shell of the container (think "SSH-ing" into the container). From there you can access the environment, install project dependencies, run the required services, etc, same as you would with an SSH session.

This allows you to develop code on your local machine and run it in the container without having to rebuild the container each time you make a change.

Prepared offline environments¶

Offline operation starts with a prepared environment, not an empty machine. Before disconnecting, acquire the container images, locked Python/JavaScript dependencies, plugin archives, build tools and assets needed for your target platform. Run the intended build and checks while connected so missing resources are discovered before you need them. Whitebox does not provide an offline bootstrap or dependency export/import mechanism.

Production serves the built host and assembled plugin remotes without compiling JavaScript at startup. Editing with Vite still needs Node/Yarn, prepared core tools and each selected plugin's locked dependencies or matching installation caches. Prepare test browsers separately if running browser checks. See editable sessions and publication boundaries for coordinated release constraints.

Keep PostgreSQL, Redis and the required application services reachable locally. Provision managed assets with backend make download_external_assets before disconnecting. Map use also needs a local map bundle, including its style, tiles, glyphs and sprites; playback needs local recordings or a reachable local streaming service. Offline does not mean disabling the private LAN or replacing these services with Internet fallbacks.

Starting Development Containers¶

For a primary checkout, create a .env file in the repository root. You can use the provided .env.example file as a template:

cp .env.example .env

Edit the .env file to set variables as needed. Linked worktrees may either have their own .env or reuse the primary checkout's ignored .env. The development wrapper selects configuration in this order:

The file named by WHITEBOX_DEV_ENV_FILE (relative paths are resolved from the directory where the wrapper is invoked).
.env in the active worktree.
.env in the primary checkout, for a linked worktree.

It fails without reading Compose configuration if none of these is usable. To select a file explicitly, for example, run WHITEBOX_DEV_ENV_FILE=/path/to/dev.env make dev-up.

If the backend lock selects local plugin directories, first prepare the backend image with the local dependency build. The Docker build context does not include plugins/; runtime mounts alone cannot supply build-time plugin inputs.

Start the development containers from the repository root:

make dev-up

Always use the root make dev-* targets for common operations or ./bin/dev-compose ... for other development Compose operations. The wrapper derives a stable project identity from the canonical worktree path and anchors Compose, source mounts, plugin mounts, and project resources to that worktree. Raw Compose invocations and global container-targeted commands bypass that boundary and can target the wrong checkout.

Published development ports are dynamic and bind to loopback by default so separate worktrees can run concurrently without exposing development services. Print the same-origin browser URL with make dev-url; discover a directly published service port with make dev-port service=<service> port=<container-port>. Other common commands are:

make dev-status
make dev-logs
make dev-down

On a trusted private network, VPN/overlay, or appropriately firewalled worker, bind published ports to every host interface by starting the project in host mode:

./bin/dev-compose --host up -d

This exposes development databases, debugger ports, and other unauthenticated services. To bind the configurable mappings to only one interface instead, provide its address directly, for example BIND_ADDR=10.42.0.1 ./bin/dev-compose up -d. The streaming profile's SRS mappings remain externally bound regardless of BIND_ADDR. Continue to use make dev-url and make dev-port to discover dynamic ports. If Docker reports 0.0.0.0, remote clients must substitute the worker's hostname or IP address; 0.0.0.0 is not a client destination.

Port bindings are selected when containers are created. docker compose restart retains existing bindings; running up -d with a different effective bind address may recreate affected containers.

Normal make dev-down preserves that worktree's project volumes. To enter the backend container shell, run:

make dev-backend-shell

To enter the frontend container shell, run:

make dev-frontend-shell

Both containers install the necessary development dependencies during build. They intentionally idle after startup; start backend and frontend processes as described below when you need them. Before installing dependencies or starting Django, wait for backend-dev to print Development environment is ready in ./bin/dev-compose logs --follow backend-dev. up -d can return while its entrypoint is still applying migrations. Ctrl-C exits the log viewer without stopping the container.

Hardware, host-monitoring, and streaming capabilities are opt-in because they control physical devices, host-global networking, or fixed streaming ports. Enable a profile through the wrapper, for example:

./bin/dev-compose --profile hardware up -d
./bin/dev-compose --profile host-monitoring up -d
./bin/dev-compose --profile streaming up -d

Each of these profiles may be owned by only one development project on a host at a time. The profile-free stack is the supported concurrent-worktree default.

Optional Opencode harness config¶

Contributors who use Opencode can find the shared Whitebox harness config in whitebox-llm-support. Follow that repository's README for setup; this guide intentionally does not duplicate those instructions. Project-specific agent behavior remains defined by the root AGENTS.md in each Whitebox repository.

To install new dependencies, make sure to use dev container shell. It is also recommended to rebuild the dev container after adding new dependencies to ensure they are cached at the build stage making future builds faster. To rebuild the containers:

./bin/dev-compose build --no-cache

Then start the containers:

make dev-up

Running the Django backend development server¶

Ensure development environment is set up.

Run the Whitebox server:

./bin/dev-compose exec backend-dev make run-dev

This starts the Django development server. Use make dev-url for the normal same-origin browser entrypoint, or make dev-port service=backend-dev port=8000 to discover its directly published address. Changes you make to the backend will be automatically reloaded.

Plugin frontend code ships as source. Core explicitly assembles it after Python installation, before runtime; neither Python packaging nor backend startup runs JavaScript. See packaged frontend runtime below.

Running the React frontend development server¶

Ensure development environment is set up.

Run the frontend development server:

./bin/dev-compose exec frontend-dev make run

Use make dev-url for the browser-facing nginx URL. If direct Vite access is needed, run make dev-port service=frontend-dev port=3000 to discover its published address. The development server has automatic reloading enabled, so you can see your changes in real-time.

For editable plugin JSX/CSS, explicitly start an editable plugin session. Unselected plugins remain regular packaged resources, not watched source graphs.

Editable plugin sessions¶

Use this workflow when you want changes in a selected plugin checkout to appear in the running UI. A regular wheel stays unchanged when you edit that checkout. An editable Python installation imports Python from source; a frontend session watches and serves that source's JSX/CSS. You need both, and must opt Django into the session. For a first example, follow the Counter exercise.

Hot module replacement (HMR) replaces loaded modules without restarting the app. Compatible component edits can preserve React state, but store/service edits need a page reload to recreate their instances and subscriptions.

Prepare and select the plugins you will edit¶

Development images start with regular installed wheels. To edit plugin Python as well as JSX/CSS, explicitly install the intended checkout as editable; startup does not rebind packages. A genuine backend-only plugin needs no frontend session. Every frontend editable must be selected in a ready session, even if its source contains old generated assets. Unselected frontend plugins must remain regular wheel installs. One rejected frontend blocks the whole browser bootstrap.

Prerequisites: prepared development containers, matching source versions at host plugins/<distribution> (mounted at /plugins/<distribution>), and provisioned Python dependencies/build backend. Frontend plugins provide Python mappings, JSX and any genuine private descriptor/lock/Yarn inputs inside the Python package. Backend-dev needs Node 22, Yarn 4.9.1 and prepared core tools matching the mounted frontend/package.json, yarn.lock and .yarnrc.yml. From /app, run make prepare-frontend-tools after core tooling/config changes; copying receipts cannot repair stale tools. New plugins need not be in the image's normal/demo selection.

Stop any Django process before changing installations. From the core repository root, start prepared containers (use a disposable environment for recovery/testing):

./bin/dev-compose up -d --no-build --no-deps postgres-dev redis-dev backend-dev frontend-dev nginx-dev

Wait for Development environment is ready in ./bin/dev-compose logs --follow backend-dev, then Ctrl-C the log viewer (not the container). Only then install and select both plugins, for example:

./bin/dev-compose exec backend-dev poetry run python -m pip install --no-deps -e /plugins/whitebox-plugin-ui-components -e /plugins/whitebox-plugin-flight-management
python3 bin/dev-plugin-session.py whitebox-plugin-ui-components whitebox-plugin-flight-management

Leave this foreground session running in its terminal. The names are Python distribution names; select the plugins whose frontend sources you intend to edit, not every dependency of those plugins. Dependencies not being edited stay wheels.

--no-deps does not provision missing requirements. A group-wide poetry install --with plugins-temporary can rebind multiple directory-based plugins; it is not the normal HMR entry command. If you use it for dependency changes, select all resulting frontend editables or reinstall those not being edited as regular wheels. Inspect any Poetry input changes before publication.

The wrapper sets WHITEBOX_PLUGIN_DEV=1 for its session command only; leave the container default off. After READY, run ./bin/dev-compose exec -e WHITEBOX_PLUGIN_DEV=1 backend-dev make run-dev and ./bin/dev-compose exec frontend-dev make run in two other terminals, then open the printed Browser URL. Restart any already-running Django process: the session cannot change its environment or installed import bindings. The session mirrors only selected source and uses core tools plus immutable private dependency installs where needed, not installs in source binds. Plugins without dependencies need no Yarn install. First preparation can take longer. As at production startup, Django never runs Yarn or starts plugin Vite processes.

The source mirror is reusable, but each session recreates its generated dependency graph. Removing a descriptor or its last dependency cannot retain old private packages; shared download caches remain available outside that graph.

The browser should load normally. /plugins/federation/config/ returns HTTP 200 with gateway entry URLs for selected plugins and packaged asset URLs for the others. A frontend build validation failure returns HTTP 409 with a diagnostic and blocks bootstrap; inspect that diagnostic, the Django logs and the session terminal. An exposed component still needs an application consumer or slot to appear on screen.

The application and plugin gateway use two browser ports; see PC and tablet access if the browser runs on another machine. Use the session's printed Browser URL, not the gateway URL, to open App.

What happens when you save or stop?¶

Compatible React component edits preserve component state; CSS updates in place. Registered stores, service components, and other .js/.ts modules deliberately reload the page. React decides whether other JSX boundaries are refresh-safe. Configuration, lock, Python mapping or exposed-module changes stop the session. Restart the session and Django, then reload the page. Installation changes also need a new Django interpreter; editing an unselected checkout does not change its installed wheel. Editable HMR supports a host at the origin root, not an application subpath.

Use python3 bin/dev-plugin-session.py --status from the core root to inspect the generation, heartbeat and status. A ready lease expires when the session process stops refreshing its heartbeat. Normal Ctrl-C prints Editable session interrupted. and exits 130 after bounded cleanup, without a routine traceback. Actual startup/shutdown errors remain failures: inspect them instead of treating every nonzero result as cancellation. Ctrl-C stops only the session's processes, not Django or the core Vite server. A stopped, failed or expired session rejects bootstrap rather than serving stale source. Stop does not revert editable installs. Restart the selected session and Django to continue; private caches remain for reuse until container recreation.

To leave one plugin while continuing to edit others, reinstall its regular wheel, run WHITEBOX_PLUGIN_DEV=1 make build_federation_modules from /app while the other selected plugins still have a ready session, then restart the session selecting only the remaining frontend editables; restart Django and reload too.

Return to regular wheels¶

To return completely to wheels, stop your foreground session and Django server. Choose one installation recovery:

Recreate backend-dev from its prepared regular-wheel image: ./bin/dev-compose up -d --no-build --no-deps --force-recreate backend-dev. Wait for the readiness message again, then run ./bin/dev-compose restart nginx-dev to refresh its backend address. Recreation loses manual installs and private session caches: a scratch plugin not in the image must be reinstalled from its wheel.
Reinstall each changed plugin with ./bin/dev-compose exec backend-dev poetry run python -m pip install --no-deps --force-reinstall <built-wheel>.

Use the original compatible wheel or your built archive copied into the container; source-mode image builds retain wheels under /opt/whitebox-image-wheels. List available filenames with ./bin/dev-compose exec backend-dev ls /opt/whitebox-image-wheels. Other image inputs need not retain archives there. Your rebuilt wheel includes checkout edits; use the original archive if you want the earlier packaged state. For your own archive, follow plugin packaging, then copy it with ./bin/dev-compose cp <host-wheel> backend-dev:/opt/<wheel-filename> and use that container path for <built-wheel> above. Replace these placeholders with your actual wheel paths/filename. Core Vite can remain running.

After manual reinstalls, run ./bin/dev-compose exec backend-dev make prepare-frontend-tools build_federation_modules. Recreation from an unchanged matched image already restores its generation. Restart Django with development resolution off: ./bin/dev-compose exec -e WHITEBOX_PLUGIN_DEV=0 backend-dev make run-dev, then reload. The config endpoint should now return HTTP 200 with only packaged asset entry URLs. Reinstalling alone does not clear a stopped session while resolution is on.

For an old session schema, stop the identifiable old foreground session process normally, then start the current command to replace its record; rebuild older dev tooling if needed. Do not delete records or kill unknown lock/port owners automatically. With WHITEBOX_PLUGIN_DEV=0, leftover session records are ignored, but every frontend still needs a regular installation: turning the flag off does not admit source imports. This opt-in is independent of Django DEBUG and off in production; production images and Compose do not expose the extra gateway.

PC and tablet browser access¶

For local development, use make dev-url at the core root. The application port and plugin HMR gateway bind to loopback by default. A tablet cannot use the PC's 127.0.0.1: it needs a hostname/IP reachable on your trusted LAN or VPN.

To expose only browser ports, set the browser-specific binding before creating nginx-dev. For example, on a development host whose private interface is 10.42.0.1 (substitute your host's actual address):

export WHITEBOX_BROWSER_BIND_ADDR=10.42.0.1
./bin/dev-compose up -d --no-build --no-deps nginx-dev
make dev-url
make dev-port service=nginx-dev port=3100
python3 bin/dev-plugin-session.py --browser-host 10.42.0.1 whitebox-plugin-ui-components whitebox-plugin-flight-management

The last command requires those two plugins already installed editable as above; leave it running, wait for READY, then restart Django with WHITEBOX_PLUGIN_DEV=1 and reload. PC and tablet must open the same printed Browser URL, including scheme, hostname and port. The session accepts one fixed host origin; localhost, an IP and a DNS alias are different origins even when they reach the same machine. On Unexpected development browser origin, use the printed address or stop/restart the session with the desired --browser-host, restart Django and reload. Automatic multi-origin access is not supported.

Bindings are chosen at container creation: up may recreate nginx-dev when the binding changes; restart alone does not. Do this only in your owned environment. The application port falls back from WHITEBOX_BROWSER_BIND_ADDR to BIND_ADDR to loopback. The plugin gateway falls back directly to loopback, not BIND_ADDR: broad --host does not by itself expose the gateway. Leave BIND_ADDR unset/default to keep database, backend and debugger ports local. Avoid --host just to reach the UI; it exposes other unauthenticated services.

The helper prints HTTP URLs. HTTPS needs a developer-managed TLS gateway and explicit matching HTTPS origins when invoking the in-container session command; the host wrapper does not configure TLS. Never mix an HTTPS host and HTTP remotes. Editable HMR supports the origin root, not an application subpath. Production does not expose this development gateway.

Running the RQ workers¶

Ensure development environment is set up.

Run the RQ worker:

./bin/dev-compose exec backend-dev make run-worker

This will start an RQ worker process that is going to process the tasks as they are added into the queue. If you need to run multiple workers for parallelized processing, you can run the run-worker command multiple times from multiple terminals.

Commands¶

Whitebox includes Makefiles with commands you can run inside the development containers for common development tasks.

Backend Commands¶

Setup Commands¶

These commands are for Whitebox setup, and need to be run inside the whitebox directory.

setup-network: Configure network interface naming for Orange Pi.

Usage:
```
sudo make setup-network
```
- Sets M.2 Wi-Fi interface's name to wlan-ap
- Required for Wi-Fi access point setup
setup-sdr-udev: Set up udev rules for SDR devices.

Usage:
```
sudo make setup-sdr-udev
```
- Downloads and installs Stratux/SDR device udev rules
- Reloads udev rules automatically

Development Commands¶

First, enter the container's shell:

make dev-backend-shell

and then you can use the following Makefile commands (format: make <COMMAND>):

migrate: Apply database migrations
run: Start the backend server for production
run-dev: Start the backend server for development
test: Run backend tests
download_external_assets: For whitebox to function offline, it needs to download external assets like videos, external JS libraries, etc. for both plugins and whitebox itself. This command downloads all the external assets
clean: All the downloaded external assets will be removed. Whitebox will not function without these assets.
makemigrations: Create migrations for all the plugins
fmt: Format code using ruff
run-mkdocs: Run the MkDocs server for documentation

Frontend Commands¶

First, enter the container's shell:

make dev-frontend-shell

and then you can use the following Makefile commands (format: make <COMMAND>):

run: Start the frontend server for development
run-for-test: Start the frontend server in test mode
core_test: Run the ordinary core source suite, without servers
unit_test: Run core unit tests in watch mode
integration_test: Run core Playwright integration tests
federation_integration_test: Download and run selected installed plugins' Playwright specs
e2e_test: Run end-to-end tests
application_test: Run integration, federation integration and end-to-end tests
test: Run the ordinary core source suite; assembled application checks are explicit
lint: Run lint checkers on the code

Plugin Commands¶

During development, plugins should be placed in the plugins/ directory, which is mounted into the backend service at /plugins for easy access.

Sometimes, work can span over multiple plugins. As there can be quite a bit of plugins in there, within /plugins directory, there's a Makefile that exposes some helper commands:

make [help] - Print a list of available commands
make check-dirty - List plugin repos that have uncommited changes
make diff-dirty - Show diff of all uncommited changes in all plugin repos
make sync-main - Check and pull main branch for all plugin repos
make checkout-if-available branch=BRANCH_NAME - Checkout branch in all repos where it exists and print poetry add command at the end to run in the backend container for convenience

Testing¶

Backend testing¶

Whitebox implements a custom test runner that allows for running tests for whitebox and plugins within a single test suite by discovering all plugin tests and loadin them dynamically.

To run tests, along with plugin tests, you need to make sure your plugin adheres to guidelines outlined in the Plugin Guide.

Ensure the development environment is set up.

Run the backend tests:

./bin/dev-compose exec backend-dev make test

Frontend testing¶

Choose a check based on what changed:

Check	Use it for	Environment
Core source units	Core components and utilities	frontend-dev, no servers
Plugin source units	A plugin's components, stores and services with local mocks	backend-dev stages independent plugin installs
Owner application gate	Source tests, ordinary wheel install, core assembly and installed-owner admission/specs	Disposable prepared backend environment; see publication/check guide
Maintained built application	Installed plugin UI, native browser behavior and focused HMR	Explicit command; private owned containers
Application integration/E2E	A running application's routes and user flows	Backend and frontend test server
Editable HMR	Component refresh, reload and session lifecycle	Disposable source checkout and dev stack

From the repository root, with the development containers prepared, run core source units with Vitest/jsdom:

./bin/dev-compose exec frontend-dev make test

Source suites belong to their producers. Run a selected plugin's source tests when editing it, and use the explicit maintained application command for installed integration/native behavior. Demo and the noncore GPS simulator remain optional.

Packaged-application checks do not prove cold production startup without network egress; the maintained workflow does not provide automated network-isolation coverage. Production still uses prepared local assets and assembled frontends without startup JavaScript compilation, as described under prepared offline environments.

For build/session infrastructure changes, maintainers can run make test-frontend-harness from the core root. It checks assembly/admission, installed input inventories, session cleanup and source-runner failure behavior. It also verifies that a private Map build fails when its own Zustand dependency is omitted, even though core installs Zustand. These controls use disposable copies; both prepared dev containers, regular Map and matched core tools/browser are prerequisites. They are not steps in the first-plugin tutorial.

make test-source-runner runs just the source-runner contracts, including Linux process-group cleanup and positive/negative suite controls. It creates temporary dependency-free plugins rather than cloning or selecting other checkouts. Setup failure or timeout is a failure, not a successful negative control. For archive verification use the selected-plugin gate.

Maintained built-application checks¶

From the core worktree root, with Docker and an existing prepared backend-dev image containing regular locked plugin installations, run:

make test-application

The default scopes are normal contract hmr; demo is optional. This is the maintained assembled workflow; CI uses the same runtime owner. It builds the production host once and runs installed plugins' browser specifications plus core bootstrap/transport/shared contracts against a built preview. Feature/native assertions live in installed owners' federation/application/ suites, distinct from their live integration specs. Focused contract and real Vite development HMR checks use a disposable core test-owned plugin, absent from the normal installation. Its ordinary Python package carries JSX without private dependencies; the same production assembly implementation builds it along with installed owners. HMR subsequently uses its separate deliberate editable stage. No copied fixture toolchain or frontend packaging hook is needed. VideoJS's focused native MP4 check uses its own ordinary test-only source fixture, installed only for the aggregate contract or VideoJS owner scope. It still renders through the real App/slots. Owner gates stage dependency helpers, not dependency specs; core's normal aggregate runs all applicable installed scenarios. Demo stays in its explicit demo selection. The image needs Node/Yarn, ffmpeg and Chromium system libraries. Declared Python build requirements, immutable Yarn installs and the locked JavaScript Playwright browser may use configured indexes/caches. Missing managed media assets are fetched from installed manifests' URLs and integrity-checked; without the image cache each scope may fetch them again. This is not offline provisioning.

Use --scope normal, --scope contract, --scope hmr, or --scope demo to run one or several scopes (make test-application ARGS="--scope normal contract"); demo requires the optional locked demo distribution in the image. The default image is <worktree Compose project>-backend-dev:latest. Prepare images with local dependencies and regular wheel installations first; the selection uses the checkout's backend/poetry.lock. postgres:latest and redis:alpine must also already exist locally: the runner never pulls images. A missing image fails at docker image inspect; use --image IMAGE to select an existing prepared image. Missing/version-mismatched or editable plugin inputs fail rather than being resolved from a registry. Rebuild the prepared image with the required regular inputs.

The command owns temporary runtimes, a private plugin mount and synthetic PostgreSQL/Redis containers; it neither borrows running dev services nor changes your installed plugins. Exit 0 after the last Playwright result means success. The first failing scope aborts the remaining selections; the whole command is bounded to 30 minutes, including setup/build. Playwright results go to the terminal; attachments in <scope>/browser/ and <scope>/backend.log remain in .build/application-results, with <scope>/session.log for HMR. The common page setup records trace, screenshot, console, network and page state for each attempt; retries remain zero. Diagnose a failure using that scope's artifacts rather than treating a later unchanged green run as an explanation. Use --output DIR to separate runs; unselected scopes' older reports are not cleared. --browser-cache DIR reuses a read-only Chromium cache (default ~/.cache/ms-playwright); --javascript-cache DIR optionally reuses a prepared frontend install whose package descriptor, lock and .yarnrc.yml must match. Immutable preparation still runs before issuing the tooling receipt. Uncommitted/nonignored source files are included; symlinks in backend/frontend inputs and an existing plugins/whitebox-plugin-application-contract checkout are refused. Container-created report files may be root-owned. Failure returns nonzero and removes owned containers, volumes and temporary source copies. The explicit --scope failure deliberately fails after startup to check that cleanup path.

Only Chromium desktop/phone/responsive branches are covered here. Phone/responsive projects intentionally select named specs in playwright.application.config.js; update those lists when specs are renamed or removed, since an unmatched filename silently loses that branch's coverage. HTTP/WebSocket domain data is controlled; PMTiles, decoding, WebGL and the local WHEP peer are real. Physical hardware, live E2E/LambdaTest, cold network isolation and package publication are separate checks. The runner and scoped fixtures are in backend/tests/application_runtime.py and frontend/tests/application/. Those files compose owner-installed browser and Python support; device/transfer/session/location data, Map resources, native media recipes/peer/transition rules and annotations registration tests belong to their respective whitebox_test_plugin_<owner> packages. Generic manifest verification and network/queue prohibitions stay core. See where to add an owner scenario. The compact coverage map records intentional reductions and the deferred native flight-plan popup limitation.

Federation testing¶

Plugin source units run Vitest Browser Mode with Chromium, ordinary local imports and plugin-owned dependencies/mocks. They need no Django server or built remote. Prepare the browser prerequisite first. From the core root, use private container staging so installs do not write into the /plugins source binds:

make test-plugin-source plugins="whitebox-plugin-icons whitebox-plugin-ui-components"
make test-plugin-source plugins=whitebox-plugin-ui-components ARGS=--shuffle
make test-plugin-source plugins=whitebox-plugin-ui-components ARGS="--filter Button.test.jsx"

Missing selections and filters that run no tests fail. For owner checkout commands and writing mocks, see Testing JSX code.

For an owner source/application gate, follow the publication/check guide. Python packaging builds only that owner without JavaScript; core consumes installed locked inputs and explicitly assembles their separate remotes. Dependency source suites belong to their producers, not each consuming gate. Controlled application checks use real package code and native resources, not physical devices.

For live plugin integration tests, start the application as described under end-to-end testing, then select installed Python test packages that contain Playwright specs:

./bin/dev-compose exec -e FEDERATION_TEST_PACKAGES=whitebox_test_plugin_example frontend-dev make federation_integration_test

Replace whitebox_test_plugin_example with the required package names. Discovery reports spec counts; missing selections, selected packages without specs and failed downloads fail. Icons and UI components do not provide these specs.

Browser prerequisite for plugin source tests¶

The source runner uses core's JavaScript Playwright version, not the Python Playwright browser that may already be in backend-dev. A different Chromium revision in a cache is insufficient. From the core root on the Docker host, with backend-dev ready, run:

./bin/dev-compose exec -T backend-dev make prepare-frontend-tools
./bin/dev-compose exec -T backend-dev node /opt/whitebox/frontend-tools/node_modules/playwright/cli.js install --with-deps chromium

The second command downloads the matching browser and installs OS libraries inside backend-dev, not on the host. It requires network/package repository access and permission for the OS package installer. If your image already contains the matching browser and libraries, acquisition is unnecessary; keep the browser cache path consistent with the test process (PLAYWRIGHT_BROWSERS_PATH if set). Recreating the container may lose browsers installed manually.

If launch says the executable is missing, rerun acquisition with the prepared JavaScript tools and check the browser cache path. If it reports missing shared libraries, inspect the install --with-deps output. Unsupported OS releases or unavailable package repositories can block library installation: use a prepared supported image/browser environment rather than upgrading tools or inventing package names. For example, the locked install-deps path is not a supported fresh Ubuntu 26.04 acquisition. Running against preinstalled matching Chromium proves test execution only, not fresh browser/OS provisioning.

CI responsibilities¶

Core's run_test_frontend job runs core source units, the Make contract check and core Playwright integration tests against a built preview. run_frontend_aggregate runs command/runtime checks and the maintained installed normal/contract application gate plus focused optional demo. It assembles installed owners but does not reconstruct dependency Python packages or run their source suites. Plugin pipelines separately run Python, frontend-source and owner-application jobs. Sandbox E2E and LambdaTest jobs use the assembled application. These jobs do not imply an all-owner, all-browser matrix. See publication boundaries for dependency selection and publication.

Packaged frontend runtime¶

Frontend owners and application fixtures use the same existing pinned Poetry fork as the standard backend environment, including isolated Python builds. See the packaging prerequisite for the exact pin and selected-owner gate diagnostics. This keeps packaging selection and destination metadata consistent; runtime admission still checks installed content and its completed generation, not a builder-name allowlist.

The host and plugin builds use Vite 8 and Yarn 4.9.1. The host configures the node-modules linker; plugins with private dependencies must select it in their own packaged .yarnrc.yml. Core does not override that setting. See the minimal plugin configuration and dependency-resolution tradeoff. Frontend-dev keeps node_modules, install state and Vite's optimizer cache in its Compose-project volume. Its entrypoint runs yarn install --immutable on startup. Update the lockfile when dependencies change and rebuild the image. Production nginx serves the built host without installing packages.

Django validates the selected completed generation against installed inputs at discovery or refresh; it does not compile JavaScript. If bootstrap reports HTTP 409 with frontend_artifact_error, inspect the plugin-qualified diagnostics from /plugins/federation/config/. Install a compatible source-bearing package set, prepare changed core tools, then explicitly run make build_federation_modules. There is no compiler fallback. Backend registrations remain loaded even when a frontend artifact is invalid.

This is not per-plugin UI isolation: one invalid installed frontend artifact blocks the whole browser bootstrap. Start with the diagnosis/recovery steps, or source-only migration if the installed package has not adopted the build contract.

Keep installed packages/tools and physical generation assets immutable. After an upgrade, prepare and assemble a complete matched set before restarting all backend workers and reloading pages. A failed assembly leaves the old selector intact; new installed bytes cannot use that old output. Rollback restores the complete previous matched installation/tool/generation set. Hot upgrades and rolling mixed sets are unsupported. See the architecture overview and plugin package contract.

Served live 360 leveling test¶

This dedicated Playwright harness runs on the Docker host, against the active worktree's backend. It needs Node.js 22/Corepack, Docker/Compose, Chromium and its system libraries, plus prepared backend-dev, postgres-dev and redis-dev services with the backend's locked dependencies and regular plugin wheels already prepared. Device Insta360 and VideoJS ThreeJS must be regular installs, as must every other frontend owner; the Insta360 SDK must match the backend lock. If you previously used editables, first return to wheels. Do not run a group-wide Poetry install that rebinds directory owners before this test. No hardware or streaming Compose profile is needed.

From the repository root, using the default loopback port bindings:

# Keep the selected env file valid when the test changes working directory.
export WHITEBOX_DEV_ENV_FILE="$(./bin/dev-compose --print-env-file)"
./bin/dev-compose up -d --no-build --no-deps postgres-dev redis-dev backend-dev
# Wait for backend-dev's "Development environment is ready" before continuing.
./bin/dev-compose exec backend-dev make migrate
./bin/dev-compose exec -T -e WHITEBOX_PLUGIN_DEV=0 backend-dev poetry run python tests/live_leveling_server.py

Leave that backend server in its terminal. In a second terminal at the same root:

export WHITEBOX_DEV_ENV_FILE="$(./bin/dev-compose --print-env-file)"
export LIVE_LEVELING_BACKEND_URL="http://$(./bin/dev-compose port backend-dev 8000)"
(
  cd frontend
  # Host-local dependencies, without requiring writable container /opt/yarn paths.
  export YARN_CACHE_FOLDER="$HOME/.cache/yarn"
  export YARN_GLOBAL_FOLDER="$HOME/.cache/yarn-global"
  export YARN_INSTALL_STATE_PATH=node_modules/.yarn-state.gz
  export YARN_NODE_LINKER=node-modules
  corepack yarn install --immutable
  corepack yarn playwright install --with-deps chromium
  corepack yarn playwright test --config playwright.live-leveling.config.js --repeat-each=2
)

Browser system-library installation may require administrator access. Do not run host and container dependency installation concurrently in the same frontend directory. The test server requires a valid assembled generation; it does not compile federation modules. To test checkout changes, build and install regular wheels and run make prepare-frontend-tools build_federation_modules in /app, or rebuild the development image with python3 bin/build-backend.py --sources when the active lock selects those directory sources. Recreate backend-dev from that image, wait for readiness, then start the test server above; an image build alone does not replace an existing container's editable installations. This test uses packaged artifacts, not an editable HMR session. The test invokes bin/dev-compose exec -T backend-dev itself; do not supply a global container name. The producer is the Device plugin's test-only whitebox_test_plugin_device_insta360.live_leveling_producer helper, included in its wheel; it suppresses daemon startup so it never discovers camera hardware. Use only a disposable development database/event bus: the producer broadcasts synthetic device 532 observations to real subscribers.

LIVE_LEVELING_BACKEND_URL controls the fixture's HTTP/WebSocket proxy. Vite uses loopback port 3105 and fails if it is occupied. Playwright starts and stops Vite; LIVE_LEVELING_EXTERNAL_SERVER=1 instead requires you to own a fixture server on that port and stop it afterwards. Each scenario starts a test-only producer with WHITEBOX_INSTA360_LEVELING_MAX_HZ=0 (uncapped) or 10 (capped), using the daemon's real startup validation and publisher configuration. This does not change the running daemon's environment. Device unit tests cover precise cadence/backpressure; the browser checks observed sequences, not millisecond delivery promises.

The test downloads checksum-pinned Three.js, OrbitControls and Video.js assets; all hashes must match before it replaces any fixture assets. Internet access to the listed asset hosts in tests/live-leveling/globalSetup.js is required.

Strict SDK packets, callbacks, Device filtering/publishing, Redis/Channels, the production core subscriber/live component and served federated ThreeJS renderer are real. Camera discovery and WHEP/media input are simulated with an HTML5 Video.js tech and empty MediaStreams. This is not a hardware test, video/IMU synchronization check, or proof of camera-equivalent smoothness. Manually compare held inclinations, slow/quick rotations and motion recovery against camera preview, checking jitter, overshoot and lag before claiming smooth stability.

After testing, interrupt the backend server and stop only services you started: ./bin/dev-compose stop backend-dev redis-dev postgres-dev. Test producers exit on stdin EOF (with a bounded idle timeout if the driver disappears).

End-to-end testing¶

Whitebox uses Playwright for end-to-end testing. The setup steps for end-to-end testing are slightly different from the other tests, as it requires running both the frontend and backend servers in conjunction.

As the frontend and backend are running in separate containers, and the tests will be run from the frontend container, the target hosts will be different—frontend will effectively live on localhost, while the backend will be on backend host (if using production containers), or backend-dev host (if using development containers).

For the tests to run successfully, you will need to ensure that the frontend application is configured to use the correct backend host from within the container.

To run end-to-end tests:

Ensure development environment is set up.
Run the backend environment that the frontend will use:

./bin/dev-compose exec backend-dev make run

Run the frontend environment, ensuring that the backend host is set correctly for the frontend build:

./bin/dev-compose exec frontend-dev make run-for-test

Run the test suite on the frontend container:

bash ./bin/dev-compose exec frontend-dev make e2e_test

Use make application_test in frontend-dev to run core integration, selected plugin integration and E2E together. Set FEDERATION_TEST_PACKAGES for the plugin specs you intend to run, as described under federation testing.

Editable HMR checks¶

Run these from the root of a disposable checkout with backend-dev, frontend-dev and nginx-dev prepared. Install only each command's selected plugins as editables and use the private browser binding described under editable sessions. Leave the Django and Vite ports free; the browser command owns its test servers. Prepare the JavaScript Playwright Chromium revision by running a plugin source suite first. The Python Playwright browser in the image may use a different revision.

For --all, explicitly install every checkout owner with a Python-package jsx/ editable first. This command checks gateway readiness for that whole set:

make test-plugin-source plugins=whitebox-plugin-ui-components
python3 bin/test-plugin-hmr.py --all --browser-host <reachable-host>

It stops its session on exit but leaves those installations editable.

Before the browser check, return to regular wheels, then explicitly install only UI-components, Flight Management and Map editable:

./bin/dev-compose exec backend-dev poetry run python -m pip install --no-deps -e /plugins/whitebox-plugin-ui-components -e /plugins/whitebox-plugin-flight-management -e /plugins/whitebox-plugin-map
python3 bin/test-plugin-hmr.py --browser --disposable-source --browser-host <reachable-host>

Every other frontend must remain a regular wheel. The browser command selects those three owners itself and exercises component/CSS refresh, real private Map graph updates retaining native map identity, store/service reload and navigation using the real App and controlled domain fixtures. It temporarily edits source files and restores them on exit. Do not run it against an active contributor checkout. Afterwards, use the same return-to-wheels procedure before ordinary packaged use.

For session failure handling, the internal regression script also mutates its inputs. Run it only in that disposable environment:

./bin/dev-compose exec -T -e WHITEBOX_HMR_DISPOSABLE_SOURCE=1 backend-dev poetry run python tests/check_plugin_dev_failures.py

Images with local dependencies¶

If the active backend lock contains local directory dependencies, run from the repository root:

python3 bin/build-backend.py --sources
./bin/dev-compose up -d --no-build --no-deps backend-dev frontend-dev

Prepare the frontend-dev image separately before using --no-build. The wrapper selects local dependencies from the lock and supplies a narrow temporary Docker context. It excludes secrets, install trees, generated output and excluded camera media. It does not rewrite Poetry files or substitute registry releases for missing local sources.

The image supplies skipped directory dependencies as ordinary source wheels at their locked versions, prepares core tools and explicitly assembles before managed-asset downloads/image completion. Startup leaves those installs intact: it neither rebinds to /plugins nor compiles JavaScript. Checkout edits require an image rebuild or explicit editable installation and selection; changed versions need refreshed dependencies and a rebuilt image. No image/source artifact fallback is used.

For a production-like image from explicit ordinary source wheels:

python3 bin/build-backend.py --wheels /path/to/reviewed-wheels --production
./bin/dev-compose run --rm --no-deps backend-candidate-check

Production defaults to --selection normal. Supplied wheel-only inputs must cover every omitted locked directory dependency, including backend-only packages, at its locked version. Other supplied wheels must belong to the selected groups and match the lock; dependency sdists/reconstruction are not required. Missing or tampered supplied archives never fall back to registry or installed inputs. Use --selection demo to include the optional demo group. Source-dev builds default to demo; pass --selection normal to omit that optional group. The check service inspects the built image through command-mode discovery and HTTP asset views, without hardware daemons, source mounts or published ports. It is not a deployment or browser test.

Running the tests on LambdaTest¶

Whitebox uses LambdaTest for running frontend integration, and end-to-end tests on multiple browsers and devices.

When ran during CI, the end-to-end tests will be run against the sandbox environment, which is setup as one of the steps before the LambdaTest testing step. You can also run the tests on LambdaTest from localhost, if you have an account.

To run the tests on LambdaTest, you will need the following:

LambdaTest username, later set as environment variable named LT_USERNAME
LambdaTest access key, later set as environment variable named LT_ACCESS_KEY

Similarly to the standard end-to-end test run, you'll need to setup frontend and backend environments, and then run the tests against them:

Ensure development environment is set up.
Run the backend environment that the frontend will use:
```
./bin/dev-compose exec backend-dev make run
```
Run the frontend environment, ensuring that the backend host is set correctly for the frontend build:
```
./bin/dev-compose exec frontend-dev make run-for-test
```

Run the LambdaTest test suites on the frontend container:

# Enter the frontend container shell
make dev-frontend-shell

# Set the LambdaTest credentials in the environment
export LT_USERNAME=<your_lambdatest_username>
export LT_ACCESS_KEY=<your_lambdatest_access_key>

# Run the integration tests on LambdaTest
yarn run test_integration:lambdatest-tunnel

# Run the end-to-end tests on LambdaTest
yarn run test_e2e:lambdatest-tunnel

Using *-tunnel commands, the tests will run using the LambdaTest tunnel which is preconfigured in the dev container. This allows for testing on local environments. This is useful when you want to run tests against the dev container running on your local machine.

You may also set environment variables like LT_USERNAME and LT_ACCESS_KEY in the .env file in the root of the project. Refer to the .env.example file for the required variables. This will allow you to run the tests from the host machine without having to set them each time like so:

./bin/dev-compose exec frontend-dev env $(./get-env) npm run test_integration:lambdatest-tunnel

Ensure you do poetry install at the root of the project to install all the required dependencies for the get-env script to work.

If you want to run the tests without the tunnel, against a sandbox, you can run the integration tests with:

FRONTEND_URL="TARGET_URL_OF_THE_SANDBOX" yarn run test_integration:lambdatest

or end-to-end tests with:

FRONTEND_URL="TARGET_URL_OF_THE_SANDBOX" yarn run test_e2e:lambdatest

Debugging¶

For comprehensive debugging procedures including Insta360 connection troubleshooting, check the Troubleshooting section.

VSCode¶

Install VSCode and open the whitebox project in VSCode.
Add Python language support by installing the extension.
JS Language support and chrome debugger support is available by default.
If using firefox browser, you would need to install this extension.

Backend Debugging¶

On the host, run make dev-port service=backend-dev port=5678 and update Debug Backend's connect.port in .vscode/launch.json with the reported port.
Enter the backend dev container's shell with make dev-backend-shell.
Run make debug (the server waits for the debugger before starting).
In Run & Debug, select Debug Backend and click the play button.

Frontend Debugging¶

On the host, run make dev-port service=frontend-dev port=9229 and update Debug Vite Server's port in .vscode/launch.json with the reported port.
Run make dev-url and update the Debug Chrome or Debug Firefox URL with the reported browser URL.
Enter the frontend dev container's shell with make dev-frontend-shell, then run make debug.
In Run & Debug, select Debug Frontend (Chrome) or Debug Frontend (Firefox) and click the play button.

The checked-in ports and URLs are defaults only; runtime-assigned values are specific to the active worktree and may change after container recreation.

PyCharm¶

Remote debugging on PyCharm works only on the Professional Edition and can take some work to fully set up. Community edition does not support remote debugging in any capacity.

You can find the detailed explanation and setup steps in PyCharm debugging.

Contributing¶

Fork the repository
Create a new branch for your feature
Make your changes
Run tests and ensure they pass
Submit a pull request

Google Docstring Conventions should be followed for all code documentation.

Documentation¶

Documentation for Whitebox is generated using MkDocs. To run documentation locally, make sure you have Whitebox repository cloned and set up as per the instructions above.

To validate documentation, use the core repository root with Python 3.14 and the root Poetry documentation dependencies installed (poetry install):

make check-docs

This runs the installed-README selection regressions and mkdocs build. Generated official plugin pages also require the matching backend Poetry environment and normal provider packages to be installed and discoverable from backend/. A build without those providers checks structure, not their current README content; inspect the generated owner pages and retain any missing-metadata warnings.

To run the documentation server:

Ensure development environment is set up.
Enter the backend container shell:
```
make dev-backend-shell
```
3. Run the documentation server:
```
make run-mkdocs
```

Versioning¶

Whitebox uses Semantic Versioning for versioning. In the CI configuration, the update_version stage is responsible for updating the version of the project. Backend and frontend versions are kept in sync, with the backend's version used as a reference point.

When a merge request is merged to main, first, patch version will be bumped in the pyproject.toml file for backend, and that same version number will then be applied to package.json for frontend. This is done by the script located in packaging/scripts/maintenance/whitebox_update_version.py.

Afterward, a commit will be made with the new version, which will then be tagged, and the CI will push these changes to the repository.

Adding temporary dependencies to CI¶

Sometimes you need to add a temporary dependency to the CI environment that you do not want to be included in the project's dependencies upon merge. You may want to do this when you are working on Whitebox core and a plugin in parallel. In these cases, the plugins' changes would only be available on its own branch, which are not published in PyPI, so you have to install them directly from Git.

To do this, you can add the dependency to the plugins-temporary Poetry group. Tests and sandbox will be run with these temporary dependencies installed (they take precedence over the "default" ones from the pyproject.toml file), but as it's an optional group, they won't be included in the final project dependencies. This change will be safe to merge, as the CI will perform the cleanup (for more info, take a look how the maintenance CI step works).

You can reference a Git branch directly in the poetry add command, by doing:

poetry add --group plugins-temporary git+GIT_BRANCH#BRANCH_NAME

For example, to add a temporary dependency with Git URL https://gitlab.com/whitebox-aero/whitebox-plugin-gps-display.git, with a branch feature/my-new-feature, you can run:

poetry add --group plugins-temporary git+https://gitlab.com/whitebox-aero/whitebox-plugin-gps-display.git#feature/my-new-feature

During development, dev containers automatically include the temporary dependencies group, so you can use them right away. However, if you want temporary dependencies in production, you need to set the TEMPORARY_DEPENDENCIES environment variable to 1 in .env file before building the production containers.

Next Steps¶

How packaged frontends fit together¶

You do not need to trace implementation files to develop a plugin. Choose the reader path for your task:

Architecture: understand the host, separate remotes, sharing and failure/update model.
Counter tutorial: package and click a component, then optionally edit and test it.
Editable sessions: select plugins, save changes, diagnose a stopped session and return to wheels.
Assembly contract and publication boundaries: look up exact inputs, safety checks and coordinated release constraints.
Testing map: choose source, installed, live or HMR coverage.

Optional maintainer code tour¶

For changes to federation itself, start with backend/devtools/frontend_assembly.py and plugin-build.mjs (tool preparation and private builds). Then read backend/whitebox/plugin/frontend_artifacts.py, the manager's get_federation_config, and frontend/src/utils/federation.js (admission and browser registry). main.jsx and components/common/bridge/WhiteboxComponent.jsx show bootstrap and cached component identity; the store loader retains the provider's exported instance. backend/devtools/plugin_dev.py and plugin-vite.mjs own editable mirrors, lifecycle and HMR/reload rules. Follow their tests and the coverage map for intentional limits, including native popup coverage. This is an implementation tour, not a prerequisite or a substitute for the public contracts above.

Learn more about the Whitebox Architecture
Check out the Development Guide