agiledancers/docs/persistent-storage.md

PostgreSQL storage (S16)

The backend requires PostgreSQL 17. Flyway applies the versioned SQL under agiledancers-be/src/main/resources/db/migration before startup recovery and readiness. Never edit an already deployed migration; add the next version instead. The database owns domain identifiers and a monotonically increasing school revision. Application persistence uses JPA/Hibernate ORM: mapped entities and collections live in repository.entity, and repositories use EntityManager. The existing domain objects remain independent of JPA. Database constraints enforce school ownership of every relationship.

Production uses deploy/compose.yaml: PostgreSQL has no published port and keeps its data in the named volume agiledancers-production-database. Set a long random DATABASE_PASSWORD in the mode-600 production.env on the NAS before the first S16 deployment. Compose reads it directly from this file, including when Docker runs through sudo; exporting it in the SSH shell does not survive the default sudo environment filter. The password is never written to release metadata. Deployment preflight rejects a missing password before pulling images or replacing containers. The backend receives:

• DATABASE_URL=jdbc:postgresql://database:5432/agiledancers
• DATABASE_USER=agiledancers
• DATABASE_PASSWORD from production.env

Database health gates backend startup, and the backend readiness check includes the datasource. Acquisition failures reject requests with 503 and a message that the mutation was not stored. There is no volatile fallback. PostgreSQL's normal WAL, fsync and synchronous-commit defaults must remain enabled. Application writes are acknowledged only after commit. A network loss during acknowledgement can leave the client uncertain; reload current data before deciding to retry.

Deployment failure before the first S16 release

A missing DATABASE_PASSWORD can make compose config --quiet exit with status 15. Configure the password in the existing NAS production.env (trusted shell configuration, mode 600), for example DATABASE_PASSWORD='<long random value>'. Keep the same password after PostgreSQL initializes its volume: changing only this file does not change the database account password. Use Compose v2.17 or newer; the deployment reads production.env and then the retained release.env through two --env-file options, so sudo cannot discard the credential. Do not print expanded Compose configuration because it contains the password; use config --quiet.

If the attempt was rejected before container replacement, no pending journal or lock remains and the old release continues running. After merging the deployment fix and completing the export below, run a new deployment. Use retry only if the failed attempt already created a version tag. Do not remove a lock or pending journal if one does remain; follow the production recovery runbook.

Transition from the previous release

1. Export every school through JSON while the old backend is still running. Its data exists only in memory and cannot be recovered after it stops.
2. Keep those exports, configure the database password, then deploy S16. The first startup creates an empty relational database through Flyway.
3. Import the JSON files through the existing preview and confirmation workflow. School keys and file-local relationship semantics remain unchanged; runtime IDs are assigned by PostgreSQL. A replacement retains the target school ID.
4. Verify schools and saved assignments after a restart before removing exports. Import retains the existing behavior of importing schedules as drafts.

Deployments recreate only the application containers. Never run compose down -v or delete the named database volume. Rollback replaces application images, not schema migrations; only roll back to a database-compatible release. An image from before S16 cannot read the new database. Database/volume loss recovery and backup operations are deferred to S17.

Local development and verification

Quarkus Dev Services starts disposable PostgreSQL for development and tests when no datasource URL is configured; Docker must be accessible. For a durable local instance, create a PostgreSQL service with a persistent volume and pass QUARKUS_DATASOURCE_JDBC_URL, QUARKUS_DATASOURCE_USERNAME, and QUARKUS_DATASOURCE_PASSWORD to dev/test execution. Production uses the three DATABASE_* variables above. Never reuse a production database for tests.

JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 mvn -Dformat.mode=check clean verify
cd agiledancers-fe
npm test
npm run lint
npm run build

Standalone Docker builds run the complete backend tests against a reachable test database. Supply its configuration through BuildKit secrets (not image build arguments), for example with shell variables TEST_DATABASE_URL, TEST_DATABASE_USER and TEST_DATABASE_PASSWORD already set:

docker build --network=host \
  --secret id=database_url,env=TEST_DATABASE_URL \
  --secret id=database_user,env=TEST_DATABASE_USER \
  --secret id=database_password,env=TEST_DATABASE_PASSWORD \
  -t agiledancers .

CI runs verification against disposable PostgreSQL, packages the exact verified output using the prebuilt Docker target, and starts a separate database on its isolated browser-test network. Cleanup removes those test containers only.

The packaged-process regression test below checks graceful and abrupt backend termination, stable recovered run IDs through HTTP and SSE, committed results, interrupted calculations, database outages, and failed startup/migrations. It requires a built backend and an isolated disposable database container: the test deliberately stops and restarts the named container. It does not remove its volume. Pass matching connection options if they differ from the defaults.

python3 tooling/persistence/restart_test.py --database-container agiledancers-s16-test

S16 targets one active backend. The existing domain monitor serializes short snapshot/mutation/commit units of work; solver work runs outside that monitor and outside database transactions. Results use the school revision captured at start. Changes elsewhere in the same school may invalidate a run. Latest progress and outcome are durable; calculations left Running at startup become Interrupted and retain earlier saved assignments. There is no automatic resumption or run history.

JPA mapping and schema upgrades

Hibernate manages relationships, element collections, ordered assignments and orphan removal through mapped entities. Its @Version on the school entity implements optimistic concurrency. A child-only change marks the aggregate dirty, so the school version advances exactly once without manual version-update SQL. JTA owns the transaction boundary; run acceptance and completion are published only after commit. Detached domain snapshots do not expose Hibernate proxies through the API or to solver threads.

Flyway remains the only schema writer. Hibernate runs in validate mode and never creates or alters application tables. V1 is retained unchanged. V2 adds the aggregate change stamp, a JPA-managed identifier counter seeded above the existing sequence, and a deferred date/type uniqueness check for reordering dayparts. Existing school keys, entity IDs, assignments and scheduling status are preserved. The counter is locked while allocating IDs within a transaction. Rollback requires a release that understands V2's identifier allocation; the earlier JDBC implementation is not a compatible rollback target.

API projections

MapStruct 1.6.3 generates the REST-layer DomainViewMapper for domain objects to view records. REST resources inject the generated CDI bean. Relationship collections map to ID lists through one shared helper; validation remains a use-case result passed explicitly to the mapper. Preferred-daypart defaults call the existing application rule. View constructors preserve immutable snapshots.

The mapper sets unmappedTargetPolicy = ERROR: every target component must have a mapping, whether inferred from a matching property or declared explicitly. A newly added unmatched component therefore fails compilation. The backend compiler runs MapStruct together with Lombok and its pinned MapStruct binding. Generated classes stay under target/generated-sources/annotations; they are not maintained source.

Persistence entities and shared domain graph identity continue to use PeopleMapping, GroupMapping and ShowMapping. MapStruct does not construct or update that graph.