agiledancers/docs/persistent-storage.md

# PostgreSQL storage (S16)

The backend requires PostgreSQL 17. Flyway applies the versioned SQL under
`agiledancers-be/src/main/resources/db/migration` before startup recovery and
readiness. Never edit an already deployed migration; add the next version instead.
The database owns domain identifiers and a monotonically increasing school
revision. Application persistence uses JPA/Hibernate ORM: mapped entities and
collections live in `repository.entity`, and repositories use `EntityManager`.
The existing domain objects remain independent of JPA. Database constraints enforce
school ownership of every relationship.

Production uses `deploy/compose.yaml`: PostgreSQL has no published port and keeps
its data in the named volume `agiledancers-production-database`. Set a long random
`DATABASE_PASSWORD` in the mode-600 `production.env` on the NAS before the first
S16 deployment. Compose reads it directly from this file, including when Docker runs through
`sudo`; exporting it in the SSH shell does not survive the default sudo environment
filter. The password is never written to release metadata. Deployment preflight
rejects a missing password before pulling images or replacing containers. The backend receives:

- `DATABASE_URL=jdbc:postgresql://database:5432/agiledancers`
- `DATABASE_USER=agiledancers`
- `DATABASE_PASSWORD` from `production.env`

Database health gates backend startup, and the backend readiness check includes
the datasource. Acquisition failures reject requests with `503` and a message that
the mutation was not stored. There is no volatile fallback. PostgreSQL's normal
WAL, fsync and synchronous-commit defaults must remain enabled. Application writes
are acknowledged only after commit. A network loss during acknowledgement can
leave the client uncertain; reload current data before deciding to retry.

## Deployment failure before the first S16 release

A missing `DATABASE_PASSWORD` can make `compose config --quiet` exit with status
15. Configure the password in the existing NAS `production.env` (trusted shell
configuration, mode `600`), for example `DATABASE_PASSWORD='<long random value>'`.
Keep the same password after PostgreSQL initializes its volume: changing only this
file does not change the database account password. Use Compose v2.17 or newer;
the deployment reads `production.env` and then the retained `release.env` through
two `--env-file` options, so sudo cannot discard the credential. Do not print
expanded Compose configuration because it contains the password; use `config --quiet`.

If the attempt was rejected before container replacement, no pending journal or
lock remains and the old release continues running. After merging the deployment
fix and completing the export below, run a new deployment. Use retry only if the
failed attempt already created a version tag. Do not remove a lock or pending
journal if one does remain; follow the production recovery runbook.

## Transition from the previous release

1. Export **every school** through JSON while the old backend is still running.
   Its data exists only in memory and cannot be recovered after it stops.
2. Keep those exports, configure the database password, then deploy S16. The
   first startup creates an empty relational database through Flyway.
3. Import the JSON files through the existing preview and confirmation workflow.
   School keys and file-local relationship semantics remain unchanged; runtime
   IDs are assigned by PostgreSQL. A replacement retains the target school ID.
4. Verify schools and saved assignments after a restart before removing exports.
   Import retains the existing behavior of importing schedules as drafts.

Deployments recreate only the application containers. Never run `compose down -v`
or delete the named database volume. Rollback replaces application images, not
schema migrations; only roll back to a database-compatible release. An image from
before S16 cannot read the new database. Database/volume loss recovery and backup
operations are deferred to S17.

## Local development and verification

Quarkus Dev Services starts disposable PostgreSQL for development and tests when
no datasource URL is configured; Docker must be accessible. For a durable local
instance, create a PostgreSQL service with a persistent volume and pass
`QUARKUS_DATASOURCE_JDBC_URL`, `QUARKUS_DATASOURCE_USERNAME`, and
`QUARKUS_DATASOURCE_PASSWORD` to dev/test execution. Production uses the three
`DATABASE_*` variables above. Never reuse a production database for tests.

```sh
JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 mvn -Dformat.mode=check clean verify
cd agiledancers-fe
npm test
npm run lint
npm run build
```

Standalone Docker builds run the complete backend tests against a reachable test
database. Supply its configuration through BuildKit secrets (not image build
arguments), for example with shell variables `TEST_DATABASE_URL`,
`TEST_DATABASE_USER` and `TEST_DATABASE_PASSWORD` already set:

```sh
docker build --network=host \
  --secret id=database_url,env=TEST_DATABASE_URL \
  --secret id=database_user,env=TEST_DATABASE_USER \
  --secret id=database_password,env=TEST_DATABASE_PASSWORD \
  -t agiledancers .
```

CI runs verification against disposable PostgreSQL, packages the exact verified
output using the `prebuilt` Docker target, and starts a separate database on its
isolated browser-test network. Cleanup removes those test containers only.

The packaged-process regression test below checks graceful and abrupt backend
termination, stable recovered run IDs through HTTP and SSE, committed results,
interrupted calculations, database outages, and failed startup/migrations. It
requires a built backend and an **isolated disposable database container**: the
test deliberately stops and restarts the named container. It does not remove its
volume. Pass matching connection options if they differ from the defaults.

```sh
python3 tooling/persistence/restart_test.py --database-container agiledancers-s16-test
```

S16 targets one active backend. The existing domain monitor serializes short
snapshot/mutation/commit units of work; solver work runs outside that monitor and
outside database transactions. Results use the school revision captured at start.
Changes elsewhere in the same school may invalidate a run. Latest progress and
outcome are durable; calculations left Running at startup become Interrupted and
retain earlier saved assignments. There is no automatic resumption or run history.


## JPA mapping and schema upgrades

Hibernate manages relationships, element collections, ordered assignments and
orphan removal through mapped entities. Its `@Version` on the school entity
implements optimistic concurrency. A child-only change marks the aggregate dirty,
so the school version advances exactly once without manual version-update SQL.
JTA owns the transaction boundary; run acceptance and completion are published only
after commit. Detached domain snapshots do not expose Hibernate proxies through the
API or to solver threads.

Flyway remains the only schema writer. Hibernate runs in `validate` mode and never
creates or alters application tables. V1 is retained unchanged. V2 adds the aggregate
change stamp, a JPA-managed identifier counter seeded above the existing sequence,
and a deferred date/type uniqueness check for reordering dayparts. Existing school
keys, entity IDs, assignments and scheduling status are preserved. The counter is
locked while allocating IDs within a transaction. Rollback requires a release that
understands V2's identifier allocation; the earlier JDBC implementation is not a
compatible rollback target.


## API projections

MapStruct 1.6.3 generates the REST-layer `DomainViewMapper` for domain objects to
view records. REST resources inject the generated CDI bean. Relationship collections
map to ID lists through one shared helper; validation remains a use-case result
passed explicitly to the mapper. Preferred-daypart defaults call the existing
application rule. View constructors preserve immutable snapshots.

The mapper sets `unmappedTargetPolicy = ERROR`: every target component must have a
mapping, whether inferred from a matching property or declared explicitly. A newly
added unmatched component therefore fails compilation. The backend compiler runs
MapStruct together with Lombok and its pinned MapStruct binding. Generated classes
stay under `target/generated-sources/annotations`; they are not maintained source.

Persistence entities and shared domain graph identity continue to use PeopleMapping,
GroupMapping and ShowMapping. MapStruct does not construct or update that graph.