Wanja Krah
694bf3fa86
All checks were successful
Build, test and publish / verify (push) Successful in 7m31s
136 lines
7.1 KiB
Markdown
136 lines
7.1 KiB
Markdown
# PostgreSQL storage (S16)
|
|
|
|
The backend requires PostgreSQL 17. Flyway applies the versioned SQL under
|
|
`agiledancers-be/src/main/resources/db/migration` before startup recovery and
|
|
readiness. Never edit an already deployed migration; add the next version instead.
|
|
The database owns domain identifiers and a monotonically increasing school
|
|
revision. Application persistence uses JPA/Hibernate ORM: mapped entities and
|
|
collections live in `repository.entity`, and repositories use `EntityManager`.
|
|
The existing domain objects remain independent of JPA. Database constraints enforce
|
|
school ownership of every relationship.
|
|
|
|
Production uses `deploy/compose.yaml`: PostgreSQL has no published port and keeps
|
|
its data in the named volume `agiledancers-production-database`. Set a long random
|
|
`DATABASE_PASSWORD` in the mode-600 `production.env` on the NAS before the first
|
|
S16 deployment. The release helper exports it for Compose without writing it to
|
|
release metadata. The backend receives:
|
|
|
|
- `DATABASE_URL=jdbc:postgresql://database:5432/agiledancers`
|
|
- `DATABASE_USER=agiledancers`
|
|
- `DATABASE_PASSWORD` from `production.env`
|
|
|
|
Database health gates backend startup, and the backend readiness check includes
|
|
the datasource. Acquisition failures reject requests with `503` and a message that
|
|
the mutation was not stored. There is no volatile fallback. PostgreSQL's normal
|
|
WAL, fsync and synchronous-commit defaults must remain enabled. Application writes
|
|
are acknowledged only after commit. A network loss during acknowledgement can
|
|
leave the client uncertain; reload current data before deciding to retry.
|
|
|
|
## Transition from the previous release
|
|
|
|
1. Export **every school** through JSON while the old backend is still running.
|
|
Its data exists only in memory and cannot be recovered after it stops.
|
|
2. Keep those exports, configure the database password, then deploy S16. The
|
|
first startup creates an empty relational database through Flyway.
|
|
3. Import the JSON files through the existing preview and confirmation workflow.
|
|
School keys and file-local relationship semantics remain unchanged; runtime
|
|
IDs are assigned by PostgreSQL. A replacement retains the target school ID.
|
|
4. Verify schools and saved assignments after a restart before removing exports.
|
|
Import retains the existing behavior of importing schedules as drafts.
|
|
|
|
Deployments recreate only the application containers. Never run `compose down -v`
|
|
or delete the named database volume. Rollback replaces application images, not
|
|
schema migrations; only roll back to a database-compatible release. An image from
|
|
before S16 cannot read the new database. Database/volume loss recovery and backup
|
|
operations are deferred to S17.
|
|
|
|
## Local development and verification
|
|
|
|
Quarkus Dev Services starts disposable PostgreSQL for development and tests when
|
|
no datasource URL is configured; Docker must be accessible. For a durable local
|
|
instance, create a PostgreSQL service with a persistent volume and pass
|
|
`QUARKUS_DATASOURCE_JDBC_URL`, `QUARKUS_DATASOURCE_USERNAME`, and
|
|
`QUARKUS_DATASOURCE_PASSWORD` to dev/test execution. Production uses the three
|
|
`DATABASE_*` variables above. Never reuse a production database for tests.
|
|
|
|
```sh
|
|
JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 mvn -Dformat.mode=check clean verify
|
|
cd agiledancers-fe
|
|
npm test
|
|
npm run lint
|
|
npm run build
|
|
```
|
|
|
|
Standalone Docker builds run the complete backend tests against a reachable test
|
|
database. Supply its configuration through BuildKit secrets (not image build
|
|
arguments), for example with shell variables `TEST_DATABASE_URL`,
|
|
`TEST_DATABASE_USER` and `TEST_DATABASE_PASSWORD` already set:
|
|
|
|
```sh
|
|
docker build --network=host \
|
|
--secret id=database_url,env=TEST_DATABASE_URL \
|
|
--secret id=database_user,env=TEST_DATABASE_USER \
|
|
--secret id=database_password,env=TEST_DATABASE_PASSWORD \
|
|
-t agiledancers .
|
|
```
|
|
|
|
CI runs verification against disposable PostgreSQL, packages the exact verified
|
|
output using the `prebuilt` Docker target, and starts a separate database on its
|
|
isolated browser-test network. Cleanup removes those test containers only.
|
|
|
|
The packaged-process regression test below checks graceful and abrupt backend
|
|
termination, stable recovered run IDs through HTTP and SSE, committed results,
|
|
interrupted calculations, database outages, and failed startup/migrations. It
|
|
requires a built backend and an **isolated disposable database container**: the
|
|
test deliberately stops and restarts the named container. It does not remove its
|
|
volume. Pass matching connection options if they differ from the defaults.
|
|
|
|
```sh
|
|
python3 tooling/persistence/restart_test.py --database-container agiledancers-s16-test
|
|
```
|
|
|
|
S16 targets one active backend. The existing domain monitor serializes short
|
|
snapshot/mutation/commit units of work; solver work runs outside that monitor and
|
|
outside database transactions. Results use the school revision captured at start.
|
|
Changes elsewhere in the same school may invalidate a run. Latest progress and
|
|
outcome are durable; calculations left Running at startup become Interrupted and
|
|
retain earlier saved assignments. There is no automatic resumption or run history.
|
|
|
|
|
|
## JPA mapping and schema upgrades
|
|
|
|
Hibernate manages relationships, element collections, ordered assignments and
|
|
orphan removal through mapped entities. Its `@Version` on the school entity
|
|
implements optimistic concurrency. A child-only change marks the aggregate dirty,
|
|
so the school version advances exactly once without manual version-update SQL.
|
|
JTA owns the transaction boundary; run acceptance and completion are published only
|
|
after commit. Detached domain snapshots do not expose Hibernate proxies through the
|
|
API or to solver threads.
|
|
|
|
Flyway remains the only schema writer. Hibernate runs in `validate` mode and never
|
|
creates or alters application tables. V1 is retained unchanged. V2 adds the aggregate
|
|
change stamp, a JPA-managed identifier counter seeded above the existing sequence,
|
|
and a deferred date/type uniqueness check for reordering dayparts. Existing school
|
|
keys, entity IDs, assignments and scheduling status are preserved. The counter is
|
|
locked while allocating IDs within a transaction. Rollback requires a release that
|
|
understands V2's identifier allocation; the earlier JDBC implementation is not a
|
|
compatible rollback target.
|
|
|
|
|
|
## API projections
|
|
|
|
MapStruct 1.6.3 generates the REST-layer `DomainViewMapper` for domain objects to
|
|
view records. REST resources inject the generated CDI bean. Relationship collections
|
|
map to ID lists through one shared helper; validation remains a use-case result
|
|
passed explicitly to the mapper. Preferred-daypart defaults call the existing
|
|
application rule. View constructors preserve immutable snapshots.
|
|
|
|
The mapper sets `unmappedTargetPolicy = ERROR`: every target component must have a
|
|
mapping, whether inferred from a matching property or declared explicitly. A newly
|
|
added unmatched component therefore fails compilation. The backend compiler runs
|
|
MapStruct together with Lombok and its pinned MapStruct binding. Generated classes
|
|
stay under `target/generated-sources/annotations`; they are not maintained source.
|
|
|
|
Persistence entities and shared domain graph identity continue to use PeopleMapping,
|
|
GroupMapping and ShowMapping. MapStruct does not construct or update that graph.
|