S6 implementation and operational evidence

Status: IN_REVIEW. Production v1.0.0 is running on the Synology at http://192.168.2.28:8088. Main publication, the real Gitea deployment and the isolated NAS recovery/rollback acceptance checks passed. The evidence below distinguishes real production actions from deliberately isolated fault tests.

Confirmed environment

Work is on feature/S6_deployment.
Read-only authenticated inspection of the installed Gitea confirmed run fields path=build.yaml@refs/heads/main, status=completed, conclusion=success, event=push, head_branch=main and head_sha.
No remote Git tags existed at the time of that read-only inspection.
The maintainer created the dedicated DSM account agiledancers-deploy and confirmed SSH port 22. A separate Ed25519 deployment key was generated outside Git on the mini-pc and installed on the NAS.

Subsequent live checks confirmed key-based access after removing group/other write permissions from the account home. The maintainer installed the dedicated Docker sudo rule using deploy/enable-docker.sh; a subsequent noninteractive check reports linux/x86_64. NAS model is DS224+, Compose is v2.20.1-6047-g6817716, timeout -k and curl are available, and port 8088 was free. The daemon allows HTTP registry access only through 127.0.0.0/8. Deployment therefore uses the NAS-local 127.0.0.1:3000 alias of the same Gitea registry, without restarting Gitea/Kimai or changing the shared Docker daemon.

NAS setup and real image preparation

java ci/Ci.java configure-deployment succeeded using separate owner-only local pull/release tokens. It provisioned /volume1/docker/agiledancers-production, the private IP/port configuration and registry pull login, verified the real NAS preflight, and stored the three deployment secrets and five variables in Gitea. No secret values are recorded here.

A real preparation probe exercised the production coordinator/API/SSH adapter and NAS script, stopping explicitly at the tag-creation boundary. It retrieved both images for verified main commit b1544357bf6ea5e1607e1de1fa4872b9510690a8 (Gitea API build run ID 35), verified revision/platform labels, selected registry digests and passed Compose v2.20 configuration validation. It released the deployment lock without creating a Git tag, pending journal, successful release state or application containers. The displayed candidate v1.0.0 was not allocated as a release.

Component	Retrieved digest (NAS registry alias `127.0.0.1:3000/wanjakrah/`)
Backend	`sha256:d46af705c527a8eaa506407206cfce927edbc4b774951cf13da8702b5cb8c56d`
Frontend	`sha256:d6c09ad839b929591878195c7403cbf0bbf24ac6e8d335ef90d72b7982c4ba31`

Local probe evidence: ci/artifacts/deployment/setup.log, nas-image-probe.log and nas-image-probe.json (ignored by Git). Retained configuration on the NAS: releases/image-probe-bbb2a980-e624-4331-adb6-b8fffd51fa5d/. This is preparation evidence, not a successful production release or Gitea dispatch run.

Local checks

java ci/Ci.java test: 17 passed, including publication gates and process deadlines.
java ci/Ci.java release-test: 32 passed, including all version increments, input/main/manual gates, verified-build provenance, digest preparation failures, duplicate/stale release rejection, retry, version high-watermark after rollback, automatic recovery, failed recovery/first release, ambiguous state commits, lock contention, annotated-tag identity and registry aliases.
Seven of those release tests execute the actual POSIX NAS script against disposable command adapters, covering image/platform checks, published endpoint requests, retained-configuration mismatch, bounded readiness failure, overlapping replacement fencing and atomic state/journal handling. These are simulated failure-path tests, separate from the real NAS preparation probe above.
sh -n passed separately for deploy/remote.sh, deploy/provision.sh and deploy/enable-docker.sh.
mvn -N -Dformat.mode=check exec:exec@format: passed with no differences.
All workflow YAML parsed; assertions confirmed automatic builds, manual-only release/rollback triggers and the patch/minor/major form choices.
docker compose -f deploy/compose.yaml config --quiet passed with representative digest references; real NAS Compose validation passed during preparation too.
git diff --check: passed.

Backend/frontend application code is unchanged.

Full Gitea feature build

Feature build #36 (API run ID 37) completed successfully for implementation commit 525e9f9335e4343c28f91664c149d24da9284904 on feature/S6_deployment. This exercises the full existing verification path, including Maven verification, both helper test suites, frontend checks/build, both Docker image builds and container E2E. Image publication is skipped for the feature branch.

The subsequent evidence-only commit intentionally uses the accepted [skip ci] marker; implementation code is unchanged from this successful run.

Implementation PR #12 was reviewed and merged through the protected main branch. Production and post-merge acceptance evidence follow.

Production deployment and native workflow guards

The main SHA is b0d6592761cf3ceab6e1ffbf9ca50548584a0096.

Evidence	Gitea run	Result
Main build/test/publication	#37, API ID 38	Success
First manual `new` / `patch` release	#38, API ID 39	Success; production `v1.0.0`
Deliberate duplicate new release for the same SHA	#39, API ID 40	Expected rejection, production unchanged
Manual deployment from a temporary validation branch	#40, API ID 41	Expected skip, production unchanged

The original feature branch had been deleted after merge; its attempted dispatch returned 404 and is not counted as branch-guard evidence. The actual skipped run used validation/s6-deploy-guard, based on the existing documentation-only 60496de commit. That temporary branch was removed after verification.

The real annotated Git tag v1.0.0 has tag-object SHA 0b8df86a9de1729395baad35584e4accead29c44 and points directly to the main SHA above. Gitea tag protection rule ID 1, pattern v*, allows only wanjakrah (the account owning the dedicated release token) to create/delete release tags. The helper never moves or deletes them; this does not remove the trusted maintainer's administrative ability to manage tags.

Production image digests, using the NAS-local 127.0.0.1:3000/wanjakrah/ registry:

Component	Deployed digest
Backend	`sha256:1d6f5e35bb58341bfa7b6ed0b88e5bdebced60666ca289d5511613f35d9ee92e`
Frontend	`sha256:843a1ffcdbe708573ca55f51a04adee0d158f255f067225a20c6d41b382ce295`

Both running containers report v1.0.0 and the selected main SHA. Backend readiness, frontend and proxied API checks passed in the workflow. A subsequent read-only GET from the mini-pc to http://192.168.2.28:8088/api/dance-schools returned HTTP success and []. The successful state is recorded under releases/release-39-3b481acf-396d-41d6-96a6-6b7ef8710f18/, with no pending journal or deployment lock. As this is the first production release, its previous successful production release is correctly null.

Firewall and diagnostics

The initial readiness wait exposed an active DSM firewall blocking frontend-to- backend traffic. The maintainer added these All interfaces allow rules above deny rules: TCP 8088 from 192.168.2.0/24, and TCP 8080 from the actual production Docker subnet 172.18.0.0/16. The same first deployment then passed; there was no extra production version or forced bypass of readiness checks.

Downloaded and opened the actual workflow diagnostic ZIPs:

deployment-39-20261103135841995564629: two files; recorded outcome success.
deployment-40-20261103141439174434089: two files; recorded outcome rejected.

Both workflows request fourteen-day artifact retention, within the previously confirmed server retention limits. The ZIPs and run-view metadata are retained locally in ci/artifacts/deployment/live-workflows/. The older Gitea artifact list API limitation documented under S4 was avoided by downloading the actual run-page artifact archives.

Isolated real NAS failure and rollback acceptance

Twelve checks passed against real NAS Docker containers, using the production Release coordinator and remote shell script. They used only project agiledancers-s6-acceptance, loopback endpoint 127.0.0.1:18089 and network 172.20.0.0/16. State/configuration was isolated under /volume1/docker/agiledancers-s6-acceptance, with -first and -recovery sibling directories for first-release and failed-recovery cases.

The adapter changed only the project/network/configuration namespace and injected controlled faults. Release tags and main-head selections in this test adapter were virtual: no additional real Git tags were created. Build verification and image pulls used real Gitea/registry responses for verified main SHAs b1544357... and b0d65927.... The missing-image case also used verified main SHA c965f9b4.... Fault cases used a ten-second readiness window and healthy cases sixty seconds; the production workflow retains its normal five-minute replacement bound.

A healthy initial isolated release passed the real container/readiness checks.
A distinct candidate whose backend deliberately exited with status 23 failed; the real previous image pair was restored, the attempt remained failed, and its virtual version remained allocated.
Retrying the same version/SHA with a healthy configuration succeeded without allocating another version.
Manual rollback restored the distinct previous SHA without another version; a second rollback restored the newer successful release.
Retry of an older version after rollback was rejected by the high-watermark.
A real missing registry image was rejected before version allocation or replacement.
An unverified SHA was rejected using real Gitea build metadata.
Duplicate and stale releases were rejected while running containers stayed unchanged.
A competing NAS lock owner excluded a second release attempt.
An injected lost state-commit response retained the real pending journal and lock, blocked the next attempt and was resolved with explicit manual recovery.
A real failed first release retained its journal/lock; manual stop/archive recovery left no fabricated successful state.
Both a candidate and its recovery configuration were deliberately broken; automatic recovery failed visibly and retained its journal/lock. Repairing the retained previous configuration and following manual recovery restored the recorded previous SHA.

Raw program, logs and results.json are retained in ci/artifacts/deployment/nas-acceptance/. The test stack's containers and Docker network were removed. Test state/history remains for inspection. Final independent verification confirmed that production still ran the original v1.0.0 SHA, without a pending journal or lock, and that no acceptance containers/network remained. The maintainer was told the temporary firewall rule for 172.20.0.0/16 can be removed; the two production rules must remain.

These isolated tests prove the NAS replacement/recovery behavior, not a second successful production release. Native Gitea rollback against a populated production predecessor can be exercised after a normal subsequent release; the same coordinator's actual distinct-version rollback is demonstrated above.

Data boundary

The maintainer explicitly accepted redeployment data loss for the launching- partner phase until S16 and confirmed the operator-communication policy during refinement. The workflow and runbook display that limitation. No messages were sent to the partner on the maintainer's behalf. S16 is not implemented, so saved- data preservation and migration compatibility remain conditional S16 acceptance, not a durability claim for this release.

Required operational acceptance

Record NAS model/platform, Docker/Compose versions, private endpoint and production directory (no secret values).
Verify pinned SSH hostkey, key-only automation and separate registry pull credential. Release-token read access is verified; actual tag-writing acceptance awaits the first release.
Verify release-tag creation/protection with the dedicated token.
Merge workflows through the protected-branch process and identify the successful main build/publication run and SHA.
Demonstrate release form, first release tag and real Synology access; record workflow URL, version, SHA and both deployed digests.
Demonstrate rejected unverified/missing-image release and branch exclusion without modifying production.
Demonstrate duplicate/stale/concurrent release protection, retained versions after failure and exact-SHA retry without another version.
Demonstrate failed candidate recovery while the coordinator reports failure, with downloadable diagnostics.
Demonstrate manual rollback without a new version and retained image pairs.
Exercise first-release/recovery failure and interruption/manual recovery using isolated acceptance resources where appropriate.
Confirm fourteen-day artifact retention configuration and artifact downloads.
Confirm the maintainer's accepted pre-S16 data-loss/communication policy.
Once S16 applies, verify data preservation and migration compatibility during deployment and rollback (deferred until S16 implementation, not a blocker for the explicitly accepted pre-S16 release).

Identify isolated acceptance resources explicitly. Simulated tests are not real production evidence.

14 KiB Raw Blame History