S19: establish engineering repository
This commit is contained in:
4
.gitignore
vendored
Normal file
4
.gitignore
vendored
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
**/target/
|
||||||
|
**/__pycache__/
|
||||||
|
examples/work/
|
||||||
|
ci/artifacts/
|
||||||
10
AGENTS.md
Normal file
10
AGENTS.md
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
# Engineering guidance
|
||||||
|
|
||||||
|
- Edit resources/ as the source; generated copies in applications are consumers.
|
||||||
|
- Read resources/CODING_STANDARDS.md before changing Java; project architecture
|
||||||
|
and metrics apply to consumer backend modules as documented there.
|
||||||
|
- Keep upstream skills unchanged and retain resources/skills/upstream.json and licenses.
|
||||||
|
- Keep project-specific configuration, secrets and waivers out of shared defaults.
|
||||||
|
- Run the bootstrap, formatter, CI/release regression and generated-example checks.
|
||||||
|
- Never move a published release tag. Document consumer-file changes in UPGRADING.md.
|
||||||
|
- Architecture and specifications belong to the landscape repository.
|
||||||
144
README.md
Normal file
144
README.md
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
# Engineering
|
||||||
|
|
||||||
|
Versioned skills, Java standards, formatting, coverage checks, and the Gitea
|
||||||
|
Actions / Maven Java 25 / npm / Docker Compose recipe. Architecture and
|
||||||
|
specifications belong to [landscape](http://192.168.2.28:3000/wanjakrah/landscape).
|
||||||
|
Source attribution is in SOURCE.md; upstream skill licenses accompany each skill.
|
||||||
|
|
||||||
|
## Start a project
|
||||||
|
|
||||||
|
Clone this repository and check out a released tag. Use JDK 25, Maven, Python 3,
|
||||||
|
Git, Docker with Compose, and a supported Node version for a frontend.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 create-project.py my-project ../my-project --version v1.0.0 --components both --runner engineering
|
||||||
|
```
|
||||||
|
|
||||||
|
Choose `backend`, `frontend`, or `both`. The command refuses existing destinations
|
||||||
|
and creates local files only. The backend starter is Quarkus; the frontend starter
|
||||||
|
uses npm and a minimal static page. Replace these starters with application code
|
||||||
|
while preserving the configured build commands and container contracts.
|
||||||
|
|
||||||
|
Initialize the resulting Git repository and commit its generated resources.
|
||||||
|
Create the remote repository separately, configure its runner and secrets, then
|
||||||
|
push. The runner label must match an available **host** runner with the required
|
||||||
|
tools. `--runner` deliberately has no machine-specific default.
|
||||||
|
|
||||||
|
## Consumer layout and configuration
|
||||||
|
|
||||||
|
`engineering.lock.json` selects a repository, immutable semantic version tag and
|
||||||
|
full commit SHA. `scripts/engineering.py` is the independent bootstrap, and
|
||||||
|
`generated/` is the committed release snapshot. No sibling checkout or global
|
||||||
|
skill installation is needed for ordinary use. Regeneration uses Git credentials
|
||||||
|
or a Git credential helper; never place credentials in the lockfile URL.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mvn -N -Pregenerate-engineering validate
|
||||||
|
python3 scripts/engineering.py --check
|
||||||
|
mvn -Dformat.mode=check clean verify
|
||||||
|
java generated/ci/Ci.java verify
|
||||||
|
java generated/ci/Ci.java test
|
||||||
|
java generated/ci/Ci.java release-test
|
||||||
|
```
|
||||||
|
|
||||||
|
The Maven regeneration profile skips formatting and works without `generated/`.
|
||||||
|
CI checks regeneration in a temporary checkout before running the pipeline. Added,
|
||||||
|
removed, modified and executable-bit changes fail the check. Runtime `target/`
|
||||||
|
and Python cache output are not release resources. Updates remove obsolete
|
||||||
|
generated files and never overwrite project-owned files outside `generated/`.
|
||||||
|
|
||||||
|
`engineering.properties` supplies project defaults; environment variables take
|
||||||
|
precedence. Required settings are `PROJECT_NAME` and at least one of
|
||||||
|
`BACKEND_MODULE` / `FRONTEND_MODULE`. Modules must be relative paths within the
|
||||||
|
project. Image names are `<PROJECT_NAME>-backend` and `<PROJECT_NAME>-frontend`;
|
||||||
|
changing PROJECT_NAME configures that namespace without changing shared code.
|
||||||
|
|
||||||
|
`BACKEND_CONTEXT` defaults to `.`, and `FRONTEND_CONTEXT` to its module path.
|
||||||
|
Both Dockerfiles provide a `prebuilt` target packaging verified output and a
|
||||||
|
default standalone source build. `FORMAT_DIRECTORIES` lists comma-separated
|
||||||
|
handwritten Java directories. Backend verification prepares a standards classpath,
|
||||||
|
runs the shared standards checker and requires 80% line coverage. Application
|
||||||
|
waivers remain in `tooling/standards/waivers.tsv`; no waiver is supplied by default.
|
||||||
|
|
||||||
|
`FRONTEND_TEST_COMMAND` is a JSON argument array, defaulting to the established
|
||||||
|
Vitest JUnit command; the starter sets `["npm","test"]`. Frontend verification
|
||||||
|
runs npm ci, lint, tests, and build. `BROWSER_TESTS=true` enables the recipe's
|
||||||
|
Playwright/JaCoCo integration for a backend and frontend. Configure
|
||||||
|
`CI_READINESS_URL`, `PLAYWRIGHT_IMAGE`, `CI_DATABASE`, `DATABASE_NAME` and
|
||||||
|
`DATABASE_USER` as appropriate. The browser recipe expects backend port 8080,
|
||||||
|
frontend port 8080, a backend JaCoCo agent and CLI prepared by Maven, and npm's
|
||||||
|
`e2e` script. Projects without that integration leave BROWSER_TESTS=false.
|
||||||
|
|
||||||
|
## Gitea setup
|
||||||
|
|
||||||
|
Build workflows need a host runner, `TARGET_PLATFORM` (for example linux/amd64),
|
||||||
|
and `REGISTRY_USER` / `REGISTRY_TOKEN` secrets with the repository/package access
|
||||||
|
needed for registry publication. The runner must also be able to read the pinned
|
||||||
|
engineering repository during the regeneration check. For private engineering,
|
||||||
|
install a read-only credential helper on the runner. Application registry and
|
||||||
|
release tokens are not used as implicit bootstrap credentials.
|
||||||
|
|
||||||
|
Publication is restricted to main after verification. Immutable commit images
|
||||||
|
are pulled back and checked before moving main tags. A host lock serializes
|
||||||
|
publication; a newer main prevents an older run from moving those tags.
|
||||||
|
|
||||||
|
Manual deployment and rollback workflows require variables `DEPLOY_HOST`,
|
||||||
|
`DEPLOY_USER`, `DEPLOY_SSH_PORT`, `DEPLOY_DIRECTORY`, `DEPLOY_REGISTRY`, and secrets
|
||||||
|
`RELEASE_TOKEN`, `DEPLOY_SSH_KEY`, `DEPLOY_KNOWN_HOSTS`. Release eligibility requires
|
||||||
|
a successful main-push build.yaml run. Keep build.yaml's filename when adapting
|
||||||
|
the recipe. Configure Gitea access, registry access and SSH host trust explicitly.
|
||||||
|
|
||||||
|
Runner setup mechanisms are available as `configure-gitea`, `configure-docker`
|
||||||
|
and `install-runner`. Supply GITEA_SERVER_URL, CI_REPOSITORY, RUNNER_LABEL and
|
||||||
|
INSECURE_REGISTRY where applicable; the latter is only for a deliberately trusted
|
||||||
|
internal HTTP registry. Runner configuration and service templates are project
|
||||||
|
files under ci/runner/. Installation currently supports Linux x86_64 host runners.
|
||||||
|
|
||||||
|
## Deployment and recovery
|
||||||
|
|
||||||
|
Projects own deploy/compose.yaml, deploy/health.sh and production.env. Set
|
||||||
|
COMPOSE_PROJECT, DEPENDENCY_SERVICES and REQUIRED_DEPLOY_SECRETS in project
|
||||||
|
configuration. Backend/frontend services use BACKEND_IMAGE / FRONTEND_IMAGE,
|
||||||
|
RELEASE_SHA and RELEASE_VERSION. Missing roles are represented by `-` in release
|
||||||
|
state; at least one image is required. Dependency services are started without
|
||||||
|
recreating their storage. Application containers alone are recreated on releases.
|
||||||
|
|
||||||
|
The remote mechanism requires a dedicated directory with owner-only
|
||||||
|
production.env containing BIND_IP, FRONTEND_PORT, DOCKER_BIN, DOCKER_CONFIG and
|
||||||
|
DOCKER_SUDO. The bind IP must be private, and the port 1024..65535. Provision a
|
||||||
|
pull-only registry login. `generated/deploy/provision.sh` accepts directory,
|
||||||
|
registry user, registry, bind IP, port and Docker executable. Optional
|
||||||
|
`generated/deploy/enable-docker.sh` takes deployment account and Docker executable
|
||||||
|
and must be run by an administrator after reviewing the Docker permission grant.
|
||||||
|
|
||||||
|
deploy/health.sh defines `application_health`, returning nonzero until the app is
|
||||||
|
ready. It can use the shared `compose` and `docker_cmd` functions. Health scripts,
|
||||||
|
Compose configuration, digests and deployment settings are retained per release.
|
||||||
|
For pre-S19 retained releases lacking health.sh/settings, the current project
|
||||||
|
health function/settings provide compatibility; verify them before rolling back
|
||||||
|
across this migration. Existing release-state JSON remains compatible.
|
||||||
|
|
||||||
|
Locking, a pending journal, digest/revision checks, bounded readiness checks and
|
||||||
|
automatic recovery protect replacement. An ambiguous state write or failed
|
||||||
|
recovery retains the journal and lock for manual inspection. Inspect retained
|
||||||
|
state, pending.json, history and diagnostics before repairing a deployment; never
|
||||||
|
blindly delete a lock. Each application must document its own recovery procedure
|
||||||
|
and database-schema rollback limits. Production deployment remains manual.
|
||||||
|
|
||||||
|
## Releases and upgrades
|
||||||
|
|
||||||
|
Release the resources, mechanisms and templates together. Use semantic version
|
||||||
|
tags `vMAJOR.MINOR.PATCH`; never move or reuse a published tag. Breaking changes
|
||||||
|
increase MAJOR. Record required project-file changes in UPGRADING.md before tagging.
|
||||||
|
Consumers update both tag and commit, regenerate, review and commit the diff.
|
||||||
|
The bootstrap rejects a tag resolving to a different commit and never upgrades
|
||||||
|
automatically. Templates create project-owned files once; upgrading only replaces
|
||||||
|
generated/. Bootstrap/profile/workflow changes require deliberate consumer edits.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
Run `python3 -m unittest discover -s tests`. The example definition is
|
||||||
|
examples/recipe-demo.json. Generate it with create-project.py and use the same
|
||||||
|
pipeline as any consumer. `tests/acceptance.py` exercises generated projects and
|
||||||
|
isolated deployment/rollback without touching production. Validation evidence is
|
||||||
|
recorded in VALIDATION.md.
|
||||||
8
SOURCE.md
Normal file
8
SOURCE.md
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
# Source attribution
|
||||||
|
|
||||||
|
Initial snapshot: http://192.168.2.28:3000/wanjakrah/agiledancers
|
||||||
|
|
||||||
|
Source commit: `bd898e92d22f6c412e671908770f4d89c8ae12bc`. Original history remains in that repository.
|
||||||
|
|
||||||
|
Project-specific extraction and generalization are recorded in S19. Upstream
|
||||||
|
licenses and revision records accompany the resources they cover.
|
||||||
15
UPGRADING.md
Normal file
15
UPGRADING.md
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
# Engineering upgrades
|
||||||
|
|
||||||
|
## v1.0.0
|
||||||
|
|
||||||
|
Initial release. For Agile Dancers, paths to shared Java launchers, formatting,
|
||||||
|
standards, coverage and skills move to generated/. Add engineering.properties,
|
||||||
|
engineering.lock.json, scripts/engineering.py and the regenerate-engineering
|
||||||
|
Maven profile. Keep standards waivers and deployment configuration local. Rename
|
||||||
|
the runner's project-owned service template to ci/runner/runner.service.
|
||||||
|
|
||||||
|
Add the bootstrap check to each workflow and include generated/ and
|
||||||
|
engineering.properties in Docker source build inputs. Provide deploy/health.sh
|
||||||
|
for application readiness. Existing release-state JSON and existing Docker Compose
|
||||||
|
project/volume names are preserved. Review legacy-release readiness compatibility
|
||||||
|
before rollback; no production deployment is part of this migration.
|
||||||
30
VALIDATION.md
Normal file
30
VALIDATION.md
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
# S19 validation — 2026-11-08
|
||||||
|
|
||||||
|
- Bootstrap and project generator: **7 tests passed**. Covers deterministic
|
||||||
|
regeneration, modified/added/missing files, moved tags, obsolete-file removal,
|
||||||
|
project-file preservation, symlinks, existing destinations, all three component
|
||||||
|
selections and actual Maven regeneration with generated/ absent.
|
||||||
|
- Formatter integration: passed read-only checks, formatting, import cleanup,
|
||||||
|
UTF-8, idempotence, parser failures and discovery of newly added files.
|
||||||
|
- Shared CI behavior: **19 tests passed**, including optional components,
|
||||||
|
configuration precedence, path containment, publication gates, existing-tag
|
||||||
|
preservation, architecture checks and bounded process execution.
|
||||||
|
- Shared release behavior: **34 tests passed**, including recovery, ambiguous
|
||||||
|
outcomes, locking, actual remote shell, digest checks and readiness timeouts.
|
||||||
|
- Generated recipe-demo: backend verification, standards and coverage, frontend
|
||||||
|
lint/tests/build and both prebuilt Docker images passed using the common recipe.
|
||||||
|
- Real isolated deployment test: pushed uniquely named example images to the
|
||||||
|
internal Gitea registry, prepared digests, deployed two synthetic revisions,
|
||||||
|
rolled back, and verified both container revision labels. Temporary local
|
||||||
|
Compose containers were removed. Registry test images remain as evidence.
|
||||||
|
- Agile Dancers consumer: full pipeline passed with **73 backend tests**,
|
||||||
|
**32 frontend tests**, **4 browser tests**, **19 standards regression tests**,
|
||||||
|
both prebuilt images and combined backend coverage **1771/1841 (96.20%)**.
|
||||||
|
- Agile Dancers standalone Docker source builds: backend and frontend passed.
|
||||||
|
Backend verification used a disposable PostgreSQL instance and BuildKit secrets;
|
||||||
|
no production database or deployment was involved.
|
||||||
|
|
||||||
|
No production deployment was performed. Live NAS SSH connectivity, production
|
||||||
|
credentials and production rollback across existing retained releases remain
|
||||||
|
operational validation limits. Legacy release-state JSON remains compatible;
|
||||||
|
pre-S19 retained releases use current health/settings as documented in README.md.
|
||||||
99
bootstrap.py
Normal file
99
bootstrap.py
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Materialize a pinned engineering release without relying on generated files."""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
|
||||||
|
def git(directory, *arguments):
|
||||||
|
result = subprocess.run(['git', '-C', str(directory), *arguments],
|
||||||
|
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
|
||||||
|
env={**os.environ, 'GIT_TERMINAL_PROMPT': '0'}, timeout=180)
|
||||||
|
if result.returncode:
|
||||||
|
raise ValueError('Git operation failed; check repository access and credentials')
|
||||||
|
return result.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def files(directory):
|
||||||
|
result = {}
|
||||||
|
if not directory.exists():
|
||||||
|
return result
|
||||||
|
for path in directory.rglob('*'):
|
||||||
|
relative = path.relative_to(directory)
|
||||||
|
if any(part in {'target', '__pycache__'} for part in relative.parts):
|
||||||
|
continue
|
||||||
|
if path.is_symlink():
|
||||||
|
raise ValueError('Symlinks are not allowed in generated resources: ' + str(relative))
|
||||||
|
if path.is_file():
|
||||||
|
result[str(relative)] = (hashlib.sha256(path.read_bytes()).hexdigest(),
|
||||||
|
bool(path.stat().st_mode & 0o111))
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def materialize(root, check=False):
|
||||||
|
root = Path(root).resolve()
|
||||||
|
lock = json.loads((root / 'engineering.lock.json').read_text())
|
||||||
|
if not re.fullmatch(r'v\d+\.\d+\.\d+', lock['tag']):
|
||||||
|
raise ValueError('Engineering tag must be vMAJOR.MINOR.PATCH')
|
||||||
|
if not re.fullmatch(r'[0-9a-f]{40}', lock['commit']):
|
||||||
|
raise ValueError('Engineering commit must be a full SHA')
|
||||||
|
with tempfile.TemporaryDirectory(prefix='engineering-fetch-') as temporary:
|
||||||
|
checkout = Path(temporary)
|
||||||
|
git(checkout, 'init', '-q')
|
||||||
|
git(checkout, 'fetch', '--quiet', '--depth=1', '--no-tags', '--',
|
||||||
|
lock['repository'], 'refs/tags/' + lock['tag'])
|
||||||
|
actual = git(checkout, 'rev-parse', 'FETCH_HEAD^{commit}')
|
||||||
|
if actual != lock['commit']:
|
||||||
|
raise ValueError('Engineering tag moved: resolved commit differs from lock')
|
||||||
|
git(checkout, 'checkout', '--quiet', '--detach', actual)
|
||||||
|
source = checkout / 'resources'
|
||||||
|
if not source.is_dir():
|
||||||
|
raise ValueError('Engineering release has no resources directory')
|
||||||
|
files(source) # Validate before copying, including symlinks.
|
||||||
|
(source / 'SOURCE.json').write_text(json.dumps(lock, indent=2, sort_keys=True) + '\n')
|
||||||
|
destination = root / 'generated'
|
||||||
|
if destination.is_symlink():
|
||||||
|
raise ValueError('generated/ must not be a symlink')
|
||||||
|
expected, existing = files(source), files(destination)
|
||||||
|
if check:
|
||||||
|
changed = sorted(k for k in expected.keys() | existing.keys()
|
||||||
|
if expected.get(k) != existing.get(k))
|
||||||
|
if changed:
|
||||||
|
raise ValueError('Generated resources differ; regenerate and commit:\n' + '\n'.join(changed))
|
||||||
|
print('Engineering resources match ' + lock['tag'] + ' at ' + actual)
|
||||||
|
return
|
||||||
|
# Stage on the same filesystem. Preserve the previous tree until replacement succeeds.
|
||||||
|
with tempfile.TemporaryDirectory(prefix='.engineering-', dir=root) as staging:
|
||||||
|
staged = Path(staging) / 'next'
|
||||||
|
previous = Path(staging) / 'previous'
|
||||||
|
shutil.copytree(source, staged)
|
||||||
|
if destination.exists():
|
||||||
|
destination.rename(previous)
|
||||||
|
try:
|
||||||
|
staged.rename(destination)
|
||||||
|
except BaseException:
|
||||||
|
if previous.exists():
|
||||||
|
previous.rename(destination)
|
||||||
|
raise
|
||||||
|
print('Generated engineering resources from ' + lock['tag'] + ' at ' + actual)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('--check', action='store_true')
|
||||||
|
parser.add_argument('--root', type=Path, default=Path(__file__).resolve().parents[1])
|
||||||
|
args = parser.parse_args()
|
||||||
|
try:
|
||||||
|
materialize(args.root, args.check)
|
||||||
|
except (ValueError, KeyError, OSError, subprocess.TimeoutExpired) as error:
|
||||||
|
parser.exit(1, str(error) + '\n')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
94
create-project.py
Normal file
94
create-project.py
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Create a local project from the engineering recipe. Never creates remote repositories."""
|
||||||
|
import argparse
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
HOME = Path(__file__).resolve().parent
|
||||||
|
|
||||||
|
|
||||||
|
def create(destination, name, components, repository, tag, runner, landscape):
|
||||||
|
destination = Path(destination).resolve()
|
||||||
|
if not re.fullmatch('[a-z][a-z0-9-]*', name):
|
||||||
|
raise ValueError('Use a lowercase project name with optional digits and hyphens')
|
||||||
|
if destination.exists():
|
||||||
|
raise ValueError('Destination must not exist; existing project files are never overwritten')
|
||||||
|
commit = subprocess.check_output(['git', '-C', str(HOME), 'rev-parse', tag + '^{commit}'], text=True).strip()
|
||||||
|
replacements = {'@PROJECT@': name, '@RUNNER@': runner, '@LANDSCAPE@': landscape,
|
||||||
|
'@ENGINEERING@': repository.removesuffix('.git'),
|
||||||
|
'@MODULES@': '<module>backend</module>' if 'backend' in components else ''}
|
||||||
|
try:
|
||||||
|
destination.mkdir(parents=True)
|
||||||
|
for part in ['common', *components]:
|
||||||
|
for source in (HOME / 'templates' / part).rglob('*'):
|
||||||
|
if source.is_file():
|
||||||
|
target = destination / source.relative_to(HOME / 'templates' / part)
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
text = source.read_text()
|
||||||
|
for before, after in replacements.items():
|
||||||
|
text = text.replace(before, after)
|
||||||
|
target.write_text(text)
|
||||||
|
lock = {'repository': repository, 'tag': tag, 'commit': commit}
|
||||||
|
(destination / 'engineering.lock.json').write_text(json.dumps(lock, indent=2) + '\n')
|
||||||
|
(destination / 'scripts').mkdir()
|
||||||
|
shutil.copy2(HOME / 'bootstrap.py', destination / 'scripts/engineering.py')
|
||||||
|
settings = {'PROJECT_NAME': name, 'BACKEND_MODULE': 'backend' if 'backend' in components else '',
|
||||||
|
'FRONTEND_MODULE': 'frontend' if 'frontend' in components else '',
|
||||||
|
'FORMAT_DIRECTORIES': 'backend/src', 'COMPOSE_PROJECT': name + '-production',
|
||||||
|
'BROWSER_TESTS': 'false', 'CI_DATABASE': 'false',
|
||||||
|
'TARGET_PLATFORM': 'linux/amd64', 'RUNNER_LABEL': runner,
|
||||||
|
'FRONTEND_TEST_COMMAND': '["npm","test"]', 'DEPENDENCY_SERVICES': '',
|
||||||
|
'REQUIRED_DEPLOY_SECRETS': ''}
|
||||||
|
(destination / 'engineering.properties').write_text(''.join(k + '=' + v + '\n' for k, v in settings.items()))
|
||||||
|
services = []
|
||||||
|
published_role = 'frontend' if 'frontend' in components else 'backend'
|
||||||
|
for role in components:
|
||||||
|
services.append(f''' {role}:
|
||||||
|
image: ${{{role.upper()}_IMAGE:?Required verified digest}}
|
||||||
|
platform: linux/amd64
|
||||||
|
restart: unless-stopped
|
||||||
|
labels:
|
||||||
|
nl.agilecc.release: ${{RELEASE_VERSION:?Required version}}
|
||||||
|
nl.agilecc.revision: ${{RELEASE_SHA:?Required SHA}}
|
||||||
|
''')
|
||||||
|
if role == published_role:
|
||||||
|
services.append(" ports:\n - '${BIND_IP:?Required private IP}:${FRONTEND_PORT:?Required port}:8080'\n")
|
||||||
|
(destination / 'deploy/compose.yaml').write_text('services:\n' + ''.join(services))
|
||||||
|
if published_role == 'backend':
|
||||||
|
p = destination / 'deploy/health.sh'
|
||||||
|
p.write_text(p.read_text().replace('$FRONTEND_PORT/', '$FRONTEND_PORT/hello'))
|
||||||
|
spec = importlib.util.spec_from_file_location('bootstrap', HOME / 'bootstrap.py')
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
module.materialize(destination)
|
||||||
|
except BaseException:
|
||||||
|
shutil.rmtree(destination)
|
||||||
|
raise
|
||||||
|
return destination
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('name')
|
||||||
|
parser.add_argument('destination', type=Path)
|
||||||
|
parser.add_argument('--components', choices=['backend', 'frontend', 'both'], default='both')
|
||||||
|
parser.add_argument('--repository', default='http://192.168.2.28:3000/wanjakrah/engineering.git')
|
||||||
|
parser.add_argument('--version', required=True)
|
||||||
|
parser.add_argument('--runner', required=True, help='Existing Gitea host runner label')
|
||||||
|
parser.add_argument('--landscape', default='http://192.168.2.28:3000/wanjakrah/landscape')
|
||||||
|
args = parser.parse_args()
|
||||||
|
try:
|
||||||
|
path = create(args.destination, args.name,
|
||||||
|
['backend', 'frontend'] if args.components == 'both' else [args.components],
|
||||||
|
args.repository, args.version, args.runner, args.landscape)
|
||||||
|
print('Created ' + str(path))
|
||||||
|
except (ValueError, OSError, subprocess.CalledProcessError) as error:
|
||||||
|
parser.exit(1, str(error) + '\n')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
35
examples/README.md
Normal file
35
examples/README.md
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
# Recipe example
|
||||||
|
|
||||||
|
`recipe-demo.json` describes the small example project. Its source is produced by
|
||||||
|
the same templates as a new application; there is no application-specific pipeline.
|
||||||
|
|
||||||
|
From a released engineering checkout:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 create-project.py recipe-demo examples/work/recipe-demo --version v1.0.0 --components both --runner engineering
|
||||||
|
git -C examples/work/recipe-demo init -b main
|
||||||
|
git -C examples/work/recipe-demo add .
|
||||||
|
git -C examples/work/recipe-demo commit -m 'Generate recipe example'
|
||||||
|
cd examples/work/recipe-demo
|
||||||
|
java generated/ci/Ci.java verify
|
||||||
|
```
|
||||||
|
|
||||||
|
Use the actual host runner label for your environment. The backend starter's
|
||||||
|
`/hello` endpoint is a demonstration response used for recipe verification.
|
||||||
|
Before adding or exposing business interfaces, apply the landscape's approved
|
||||||
|
API decisions, including versioned JSON contracts, OpenAPI documentation, client
|
||||||
|
generation and compatibility checks. This example does not introduce a business
|
||||||
|
system or a cross-system data contract.
|
||||||
|
|
||||||
|
For a real isolated deployment test, run from engineering:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 tests/acceptance.py examples/work/recipe-demo --registry registry.example:3000 --namespace owner --user owner --token-file /secure/path/package-token
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates uniquely named example images in the selected registry, starts a
|
||||||
|
temporary local Compose project bound to loopback, deploys two synthetic
|
||||||
|
revisions, then rolls back and checks both container revision labels. It removes
|
||||||
|
the local test containers. Registry images remain as test evidence; they are not
|
||||||
|
production images or releases. The deployment token needs package write/read
|
||||||
|
access; it is supplied through stdin to Docker login, never committed.
|
||||||
5
examples/recipe-demo.json
Normal file
5
examples/recipe-demo.json
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "recipe-demo",
|
||||||
|
"components": ["backend", "frontend"],
|
||||||
|
"purpose": "Generate with create-project.py and validate the common pipeline without Agile Dancers assumptions."
|
||||||
|
}
|
||||||
16
maven-profile.xml
Normal file
16
maven-profile.xml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
|
||||||
|
<profiles>
|
||||||
|
<profile>
|
||||||
|
<id>regenerate-engineering</id>
|
||||||
|
<properties><format.skip>true</format.skip></properties>
|
||||||
|
<build><plugins><plugin>
|
||||||
|
<groupId>org.codehaus.mojo</groupId><artifactId>exec-maven-plugin</artifactId><version>3.6.3</version>
|
||||||
|
<inherited>false</inherited>
|
||||||
|
<executions><execution><id>regenerate-engineering</id><phase>validate</phase><goals><goal>exec</goal></goals>
|
||||||
|
<configuration combine.self="override"><executable>python3</executable>
|
||||||
|
<arguments><argument>${maven.multiModuleProjectDirectory}/scripts/engineering.py</argument></arguments>
|
||||||
|
</configuration>
|
||||||
|
</execution></executions>
|
||||||
|
</plugin></plugins></build>
|
||||||
|
</profile>
|
||||||
|
</profiles>
|
||||||
276
resources/CODING_STANDARDS.md
Normal file
276
resources/CODING_STANDARDS.md
Normal file
@@ -0,0 +1,276 @@
|
|||||||
|
# Backend coding standards
|
||||||
|
|
||||||
|
This document is the authoritative policy for handwritten production and test Java
|
||||||
|
in `<backend-module>`. Generated code is out of scope. The Java conventions below also apply to
|
||||||
|
handwritten Java helpers outside that module. Frontend conventions are owned by each consuming project.
|
||||||
|
|
||||||
|
## Scope and authority
|
||||||
|
|
||||||
|
- Cover handwritten production and test Java in `<backend-module>`. Exclude
|
||||||
|
generated code. Dependency injection and production architecture rules apply
|
||||||
|
where relevant. The Java conventions also cover Java helpers outside the backend; backend
|
||||||
|
architecture, metrics, coverage and automated checks remain backend-only.
|
||||||
|
Frontend code is out of scope.
|
||||||
|
- [Java coding guidelines](https://www.geeksforgeeks.org/java/coding-guidelines-in-java/)
|
||||||
|
supplied source material. Only the rules adopted in this local document apply;
|
||||||
|
changes to the article do not change project policy.
|
||||||
|
- The project's pinned Palantir formatter wins formatting conflicts, including
|
||||||
|
comment layout. S8 remains responsible for formatter integration.
|
||||||
|
- Do not adopt unrelated static-analysis rule bundles.
|
||||||
|
|
||||||
|
## General language guidance
|
||||||
|
|
||||||
|
- Always prefer newer stable official APIs, language features and officially
|
||||||
|
recommended coding practices over older alternatives when they suit the task
|
||||||
|
and are supported by the project's configured language/runtime version.
|
||||||
|
Preserve required behavior and compatibility; document concrete reasons for
|
||||||
|
retaining an older approach. This guidance applies to every language used in
|
||||||
|
the project.
|
||||||
|
|
||||||
|
## Java conventions
|
||||||
|
|
||||||
|
- Types use UpperCamelCase; methods and variables use lowerCamelCase; constants
|
||||||
|
use UPPER_SNAKE_CASE; package names are lowercase.
|
||||||
|
- One top-level type per file; explicit member visibility; `@Override` on
|
||||||
|
overriding methods. Do not invent visibility modifiers where Java forbids them.
|
||||||
|
- Opening braces stay on the declaration/control-statement line. All `if`, `for`
|
||||||
|
and `while` bodies require braces, including enhanced for and do/while bodies.
|
||||||
|
- No wildcard imports. Static imports are allowed only in test classes.
|
||||||
|
- Prefer `java.nio` APIs (including `java.nio.file` and `java.nio.charset`) over
|
||||||
|
equivalent `java.io` APIs. Use stream APIs where no suitable NIO equivalent
|
||||||
|
exists, such as process pipes or APIs requiring streams.
|
||||||
|
- Use varargs (`Type... values`) when an array is the last method or constructor
|
||||||
|
parameter, including `main(String... args)`. Preserve required override signatures
|
||||||
|
and framework/API compatibility.
|
||||||
|
- Prefer Java 25 `IO.print`/`IO.println` for direct standard output and `IO.readln`
|
||||||
|
for line-oriented standard input. `IO` has no standard-error variant; retain
|
||||||
|
`System.err` for error output. Use application logging where appropriate.
|
||||||
|
- Comments explain intent or constraints. Require Javadoc on public classes and
|
||||||
|
public methods that do more than basic POJO/data-carrier work. Trivial accessors
|
||||||
|
and plain data carriers do not need boilerplate documentation. Describe behavior,
|
||||||
|
constraints and failure conditions where relevant.
|
||||||
|
- Block comments use the following layout, subject to Palantir formatting:
|
||||||
|
|
||||||
|
```java
|
||||||
|
/*
|
||||||
|
This is an example of a block comment.
|
||||||
|
It may have more than one line.
|
||||||
|
*/
|
||||||
|
```
|
||||||
|
- Prefer records for immutable data carriers. Classes are allowed when mutation,
|
||||||
|
framework requirements or inheritance make records unsuitable. Preserve API and
|
||||||
|
serialization behavior. Record immutability is shallow: protect mutable
|
||||||
|
components appropriately.
|
||||||
|
- Require `final` for parameters, local variables and fields wherever they can
|
||||||
|
remain assigned once, including loop variables and catch parameters where Java
|
||||||
|
permits it. Field finality must remain compatible with framework injection and
|
||||||
|
lifecycle requirements. Permit reassignment when the algorithm needs it; do not
|
||||||
|
introduce artificial holder objects to avoid reassignment. This rule does not
|
||||||
|
require final classes or methods.
|
||||||
|
- Use explicit types unless the initializer visibly identifies the type. `var` is
|
||||||
|
allowed with `new School(...)` or an explicit cast, but not for method results,
|
||||||
|
literals or other inferred expressions. For example,
|
||||||
|
`final var school = new School(...)` is allowed, while a repository result uses
|
||||||
|
`final School school = repository.find(id)`. Borderline cases require review.
|
||||||
|
- No additional member-ordering or line-width policy beyond Palantir.
|
||||||
|
|
||||||
|
## Metrics
|
||||||
|
|
||||||
|
Targets guide development. Review signals require assessment, not automatic
|
||||||
|
failure. Blocking limits fail verification. Strict inequalities matter: a value
|
||||||
|
exactly at a blocking limit passes.
|
||||||
|
|
||||||
|
| Metric | Target | Review signal | Blocking limit |
|
||||||
|
| --- | ---: | ---: | ---: |
|
||||||
|
| Cyclomatic complexity per method/constructor | ≤ 5 | > 7 | > 10 |
|
||||||
|
| Cognitive complexity per method/constructor | ≤ 10 | > 10 | > 15 |
|
||||||
|
| Method/constructor length, code lines | ≤ 20 | > 30 | > 50 |
|
||||||
|
| Parameters per method | ≤ 3 | > 4 | > 5 |
|
||||||
|
| Parameters per explicitly written constructor | ≤ 4 | > 5 | > 7 |
|
||||||
|
| Injected dependencies per class | ≤ 4 | > 5 | > 7 |
|
||||||
|
| Fan-out per class, unique used types | ≤ 10 | > 15 | Review only |
|
||||||
|
| Own superclass depth, excluding Object | ≤ 1 | > 2 | > 3 |
|
||||||
|
| Control-flow nesting | ≤ 2 | > 3 | > 4 |
|
||||||
|
| Production duplicated lines density | ≤ 1% | > 1% | > 3% |
|
||||||
|
| Class length, code lines | ≤ 200 | > 300 | > 500 |
|
||||||
|
| Declared public methods per class, excluding trivial accessors | ≤ 10 | > 15 | Review only |
|
||||||
|
|
||||||
|
### Measurement definitions
|
||||||
|
|
||||||
|
- Use the pinned PMD algorithms as authoritative for cyclomatic and cognitive
|
||||||
|
complexity. Document their exact configuration, including switch and boolean
|
||||||
|
treatment. Do not accidentally introduce a class-level aggregate complexity gate.
|
||||||
|
- Measure each method, constructor and explicitly written record constructor
|
||||||
|
independently. Lambda bodies contribute to their enclosing executable;
|
||||||
|
methods in nested/local classes are separate executables.
|
||||||
|
- Parameter limits exclude implicit record constructors and generated members.
|
||||||
|
Record component counts remain a design review concern.
|
||||||
|
- Length means physical source lines after Palantir formatting, excluding blank
|
||||||
|
and comment-only lines. Declaration and brace-only lines count; a line containing
|
||||||
|
both code and comments counts. Do not substitute statement counts or NCSS.
|
||||||
|
Measure nested types separately and exclude them from enclosing class length.
|
||||||
|
- Count distinct injected collaborators by resolved type and qualifier. Include
|
||||||
|
constructor, field, setter, inherited and implicit constructor injection.
|
||||||
|
Repeated injection of the same type and qualifier counts once; differently
|
||||||
|
qualified instances of the same type count separately.
|
||||||
|
- Fan-out counts distinct explicitly referenced resolved types, including JDK and
|
||||||
|
framework types. Exclude primitives, the class itself and generated references.
|
||||||
|
- Inheritance depth counts project-owned superclass layers; exclude Object,
|
||||||
|
interfaces and external framework ancestry.
|
||||||
|
- Nesting combines `if`, loops, `switch` and `catch`, with the first structure at
|
||||||
|
depth one. An `else if` chain stays at one level. `else`, `try` and `finally`
|
||||||
|
add no level themselves. Lambdas retain surrounding depth; methods in nested
|
||||||
|
classes start fresh.
|
||||||
|
- Public-method counts include explicitly declared public instance and static
|
||||||
|
methods; exclude constructors, inherited methods and trivial accessors. A
|
||||||
|
trivial accessor only reads or assigns its corresponding field, with no
|
||||||
|
validation, branching or other work.
|
||||||
|
- Duplication limits apply only to production code. Duplication in tests is allowed
|
||||||
|
and has no target, review signal or blocking limit.
|
||||||
|
- Detect production clones with CPD, minimum 100 tokens, without ignoring
|
||||||
|
identifier or literal differences. Density is
|
||||||
|
`100 × distinct duplicated code-line locations / all production code-line locations`.
|
||||||
|
Exclude blank/comment-only lines from numerator and denominator. Count every
|
||||||
|
occurrence, including the first; merge overlapping ranges to count each source
|
||||||
|
line once. This project metric need not equal a future Sonar metric.
|
||||||
|
|
||||||
|
## Enforcement and developer workflow
|
||||||
|
|
||||||
|
- Use explicitly pinned Java-25-compatible Checkstyle and PMD/CPD versions.
|
||||||
|
Retain ArchUnit for architecture checks and add small targeted custom checks
|
||||||
|
where standard rules cannot accurately express the agreed semantics.
|
||||||
|
- Automate every blocking metric and explicit rule where reliable checking is
|
||||||
|
available. Identify rules requiring manual review explicitly; do not claim
|
||||||
|
they are automated. Review signals must appear in reports even when builds pass.
|
||||||
|
- Run standards checks during `mvn verify`; provide documented standalone commands.
|
||||||
|
Ordinary compile/test commands retain the current S8 formatting behavior.
|
||||||
|
- Standards checks report violations without rewriting code. CI passes these
|
||||||
|
checks before image publication. Parser failures or unsupported Java constructs
|
||||||
|
fail with the affected path and actionable diagnostics, rather than being skipped.
|
||||||
|
- Add a root AGENTS.md instruction requiring agents to read CODING_STANDARDS.md
|
||||||
|
before changing backend Java. Project review instructions must explicitly cover
|
||||||
|
manual rules, review signals in changed code and approved waivers, without
|
||||||
|
repeating checks already enforced by tooling.
|
||||||
|
- Keep upstream skill files unchanged. Put project-specific code-review guidance
|
||||||
|
in a companion file referenced by project instructions and used alongside
|
||||||
|
`skills/code-review/SKILL.md`.
|
||||||
|
- Waivers must be explicit, narrowly scoped and carry a recorded reason alongside
|
||||||
|
code or configuration. Agents need maintainer agreement before introducing one.
|
||||||
|
No blanket suppressions, exclusions of difficult files or weakened thresholds
|
||||||
|
merely to make verification pass.
|
||||||
|
|
||||||
|
## Coverage
|
||||||
|
|
||||||
|
- Require at least 80% backend line coverage across handwritten production code
|
||||||
|
for the module as a whole, not separately per class or package. Compare the
|
||||||
|
unrounded percentage against 80%.
|
||||||
|
- Local `mvn verify` must reach 80% using backend unit/integration tests alone.
|
||||||
|
This intentionally tightens the original combined-tests-only requirement.
|
||||||
|
- CI additionally instruments the backend exercised by Playwright, merges that
|
||||||
|
coverage with backend unit/integration coverage and gates the combined result.
|
||||||
|
The Playwright contribution need not separately reach 80%.
|
||||||
|
- Exclude generated code only; do not broadly exclude DTOs, configuration or
|
||||||
|
difficult code. Publish line and branch coverage reports; branch coverage has
|
||||||
|
no agreed blocking percentage.
|
||||||
|
- PIT measures mutation-test effectiveness, not line coverage, and is outside
|
||||||
|
S7's required implementation. Configuring Sonar in the pipeline remains out of
|
||||||
|
scope for S14. FindBugs/SpotBugs is not an additional required check suite.
|
||||||
|
|
||||||
|
|
||||||
|
## Implemented checks and manual review
|
||||||
|
|
||||||
|
Checkstyle **12.3.1**, PMD/CPD **7.28.0**, and JaCoCo **0.8.14** are pinned.
|
||||||
|
`generated/tooling/standards/checkstyle.xml` selects individual naming, structure, import and brace checks; no general analysis bundle is enabled.
|
||||||
|
`generated/tooling/standards/src/main/java/Standards.java` supplies the targeted measurements.
|
||||||
|
ArchUnit remains in the backend test suite for architectural boundaries.
|
||||||
|
|
||||||
|
PMD uses Java 25, the running JDK's `jrt-fs.jar`, compiled backend classes and the
|
||||||
|
resolved test dependency classpath. Cyclomatic complexity uses PMD's default
|
||||||
|
options: boolean paths count, assertions do not. Decisions include if/loops,
|
||||||
|
catch, ternary and throw; non-default switch alternatives count individually,
|
||||||
|
including comma-separated labels. There is no aggregate class-complexity gate.
|
||||||
|
Cognitive complexity uses PMD's boolean-operation sequences, nesting, switch,
|
||||||
|
recursion and flow rules. The scoped visitors retain those algorithms while
|
||||||
|
including lambdas in their enclosing executable and excluding nested/local type
|
||||||
|
bodies, whose methods are measured independently. Compact record constructors
|
||||||
|
use the same visitors and start with cyclomatic complexity one.
|
||||||
|
|
||||||
|
Custom checks measure physical code lines from regular parser tokens, including
|
||||||
|
brace/declaration lines and mixed code/comment lines. They count multiline literal
|
||||||
|
lines while excluding physically blank lines. Class length excludes nested-type
|
||||||
|
tokens, preserving lines shared with enclosing code. Other custom measurements
|
||||||
|
implement the definitions above: nesting, resolved type fan-out, project-owned
|
||||||
|
superclass depth, public non-accessor methods, and injected collaborators.
|
||||||
|
Injection includes inherited fields/setters, explicit `@Inject` constructors,
|
||||||
|
single constructors on bean classes, records' implicit canonical constructors,
|
||||||
|
and Lombok required/all-argument constructor injection. Qualifier keys use resolved
|
||||||
|
annotation names and values, including defaults; repeat sites are deduplicated.
|
||||||
|
The CPD configuration preserves identifiers/literals, uses 100 tokens and disables
|
||||||
|
skip blocks. Its line-location union counts overlapping occurrences only once.
|
||||||
|
|
||||||
|
Checkstyle and targeted checks enforce import/brace/naming/visibility rules,
|
||||||
|
`@Override`, visible constructor/cast/new-array `var` initializer forms, final parameters, locals (including
|
||||||
|
loops/catches and lambdas) and initialized fields. PMD's three selected finality
|
||||||
|
rules additionally cover control-flow assignment and private constructor-initialized
|
||||||
|
fields. Framework/lifecycle requirements, generated Lombok setters, and external
|
||||||
|
writes to public/protected fields need review before changing field finality.
|
||||||
|
Comment intent, Javadoc adequacy for nontrivial public contracts, record suitability
|
||||||
|
and protection of mutable components require manual review. These are **manual
|
||||||
|
rules**, as are ambiguous type inference, architecture/DI design decisions, NIO
|
||||||
|
API selection, trailing-array varargs, console I/O selection and the POJO
|
||||||
|
documentation boundary. Review all `DESIGN` record-component entries
|
||||||
|
and `REVIEW` signals, even when a build passes, using `docs/backend-review.md`.
|
||||||
|
|
||||||
|
Reports never rewrite Java. Formatter application remains S8's separate validate
|
||||||
|
execution. Generated output under `target` and generated source directories are
|
||||||
|
outside the handwritten `src/main/java` and `src/test/java` scope. No maintained
|
||||||
|
file, DTO, configuration class or package is excluded from standards or coverage.
|
||||||
|
JaCoCo filters compiler/Lombok-generated members; no handwritten-code exclusions
|
||||||
|
are configured. Line and branch results are published, with only lines gated.
|
||||||
|
|
||||||
|
Exact metric waivers live in `tooling/standards/waivers.tsv`; each records a metric,
|
||||||
|
fully resolved executable signature (or class), exact retained value and the
|
||||||
|
maintainer agreement/reason. No wildcard waivers are allowed, and a changed value
|
||||||
|
fails again. A waived blocking result remains visible as `WAIVER`. PMD suppression
|
||||||
|
annotations/comments cannot silently waive a rule. Adding a waiver always requires maintainer agreement.
|
||||||
|
|
||||||
|
## Commands and reports
|
||||||
|
|
||||||
|
Run from the root with JDK 25:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mvn -Dformat.mode=check clean verify
|
||||||
|
# Prepare standalone analysis without running tests:
|
||||||
|
mvn -pl <backend-module> test-compile dependency:build-classpath@standards-classpath
|
||||||
|
mvn -f generated/tooling/standards/pom.xml test exec:java
|
||||||
|
# Exact local coverage gate after a test/report build:
|
||||||
|
java generated/tooling/coverage/Coverage.java <backend-module>/target/site/jacoco/jacoco.xml
|
||||||
|
```
|
||||||
|
|
||||||
|
`verify` runs backend tests, builds resolved analysis inputs, generates JaCoCo
|
||||||
|
reports, runs checker regression tests and standards analysis, then gates exact
|
||||||
|
module-wide line coverage. `compile` and `test` retain existing formatting behavior
|
||||||
|
and do not run the verify-only standards checks. Standalone analysis requires its
|
||||||
|
classpath preparation; unresolved types fail with the path and preparation advice.
|
||||||
|
|
||||||
|
Full metrics and findings: `<backend-module>/target/standards/report.txt`.
|
||||||
|
Checker regression results: `generated/tooling/standards/target/surefire-reports`.
|
||||||
|
Coverage HTML/XML/CSV: `<backend-module>/target/site/jacoco` (local backend tests)
|
||||||
|
and `<backend-module>/target/site/jacoco-combined` (merged browser contribution).
|
||||||
|
The gate compares integer line counts (`5 * covered >= 4 * total`), rejecting an
|
||||||
|
empty report and percentages that would round up to 80%.
|
||||||
|
|
||||||
|
CI first passes local verify. It mounts the pinned JaCoCo agent into its disposable
|
||||||
|
backend test container, exposes the agent only on the isolated Docker network,
|
||||||
|
and uses the pinned CLI to dump `target/playwright.exec` after Playwright. A missing
|
||||||
|
or empty dump fails. The following commands merge both datasets and gate the result
|
||||||
|
before the verified-commit marker can authorize publication:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mvn -pl <backend-module> jacoco:merge@combined jacoco:report@combined-report exec:exec@combined-coverage
|
||||||
|
```
|
||||||
|
|
||||||
|
CI retains both coverage reports, execution data, standards findings and regression
|
||||||
|
results with its usual diagnostics artifacts. The agent is mounted only for tests;
|
||||||
|
published images contain no agent. PIT and Sonar configuration remain outside S7.
|
||||||
|
Application validation evidence and retained-signal assessments stay in the consuming project.
|
||||||
62
resources/ci/Ci.java
Normal file
62
resources/ci/Ci.java
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
import java.nio.file.*;
|
||||||
|
import java.time.*;
|
||||||
|
import java.time.format.DateTimeFormatter;
|
||||||
|
import java.util.*;
|
||||||
|
|
||||||
|
/** Run from the repository root with JDK 25: java ci/Ci.java <command>. */
|
||||||
|
public class Ci {
|
||||||
|
public static void main(String[] args) {
|
||||||
|
try {
|
||||||
|
if (Runtime.version().feature() < 25) throw new IllegalArgumentException("JDK 25 or newer is required");
|
||||||
|
if (args.length == 0) throw new IllegalArgumentException(usage());
|
||||||
|
Path root = Path.of("").toAbsolutePath();
|
||||||
|
var env = Recipe.environment(root, System.getenv());
|
||||||
|
var commands = new Commands(Duration.ofMinutes(args[0].equals("publish") ? 15 : 75));
|
||||||
|
switch (args[0]) {
|
||||||
|
case "verify", "publish", "cleanup" -> {
|
||||||
|
var pipeline = new Pipeline(root, env, commands);
|
||||||
|
switch (args[0]) {
|
||||||
|
case "verify" -> pipeline.verify();
|
||||||
|
case "publish" -> pipeline.publish();
|
||||||
|
default -> pipeline.cleanup();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "attempt" -> {
|
||||||
|
Path file = Path.of(Setup.required(env, "GITHUB_ENV"));
|
||||||
|
String id = DateTimeFormatter.ofPattern("yyyyMMddHHmmssSSSSSSSSS")
|
||||||
|
.withZone(ZoneOffset.UTC)
|
||||||
|
.format(Instant.now());
|
||||||
|
Files.writeString(
|
||||||
|
file, "CI_RUN_ATTEMPT=" + id + "\n", StandardOpenOption.CREATE, StandardOpenOption.APPEND);
|
||||||
|
}
|
||||||
|
case "configure-gitea" -> Setup.configureGitea(env);
|
||||||
|
case "configure-docker" -> {
|
||||||
|
if (args.length != 2) throw new IllegalArgumentException("Supply the daemon.json path");
|
||||||
|
Setup.configureDocker(Path.of(args[1]), Setup.required(env, "INSECURE_REGISTRY"));
|
||||||
|
}
|
||||||
|
case "install-runner" ->
|
||||||
|
Setup.installRunner(root, env, commands, args.length == 2 && args[1].equals("--prepare-only"));
|
||||||
|
case "test" -> CiTests.run();
|
||||||
|
case "deploy", "rollback" -> Release.execute(root, env, args[0]);
|
||||||
|
case "release-test" -> ReleaseTests.run();
|
||||||
|
case "configure-deployment" -> ReleaseSetup.configure(root, env);
|
||||||
|
default -> throw new IllegalArgumentException(usage());
|
||||||
|
}
|
||||||
|
} catch (Commands.Failure e) {
|
||||||
|
System.err.println(e.getMessage());
|
||||||
|
System.exit(e.exitCode);
|
||||||
|
} catch (Exception e) {
|
||||||
|
// Never dump HTTP bodies, process arguments, environment or tokens.
|
||||||
|
System.err.println(
|
||||||
|
e instanceof IllegalArgumentException || e instanceof Setup.ApiFailure
|
||||||
|
? e.getMessage()
|
||||||
|
: "CI failed (" + e.getClass().getSimpleName() + "). Check diagnostics and configuration.");
|
||||||
|
System.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static String usage() {
|
||||||
|
return "Usage: java ci/Ci.java verify|publish|cleanup|attempt|test|configure-gitea|configure-docker"
|
||||||
|
+ " <path>|install-runner [--prepare-only]|deploy|rollback|release-test|configure-deployment";
|
||||||
|
}
|
||||||
|
}
|
||||||
358
resources/ci/CiTests.java
Normal file
358
resources/ci/CiTests.java
Normal file
@@ -0,0 +1,358 @@
|
|||||||
|
import java.nio.file.*;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.*;
|
||||||
|
|
||||||
|
/** Dependency-free behavior tests: java ci/Ci.java test. */
|
||||||
|
class CiTests {
|
||||||
|
static final String SHA = "a".repeat(40);
|
||||||
|
static int count;
|
||||||
|
|
||||||
|
interface Test {
|
||||||
|
void run() throws Exception;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void test(String name, Test test) throws Exception {
|
||||||
|
test.run();
|
||||||
|
count++;
|
||||||
|
System.out.println("PASS " + name);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void equal(Object expected, Object actual) {
|
||||||
|
if (!Objects.equals(expected, actual)) throw new AssertionError("Expected " + expected + ", got " + actual);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void failure(Test action) throws Exception {
|
||||||
|
try {
|
||||||
|
action.run();
|
||||||
|
} catch (Exception expected) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
throw new AssertionError("Expected failure");
|
||||||
|
}
|
||||||
|
|
||||||
|
static void run() throws Exception {
|
||||||
|
count = 0;
|
||||||
|
test("recipe supports backend-only and frontend-only projects", () -> {
|
||||||
|
equal(List.of("backend"), Recipe.roles(Map.of("BACKEND_MODULE", "server")));
|
||||||
|
equal(List.of("frontend"), Recipe.roles(Map.of("FRONTEND_MODULE", "web")));
|
||||||
|
failure(() -> Recipe.roles(Map.of()));
|
||||||
|
failure(() -> Recipe.project(Map.of("PROJECT_NAME", "../unsafe")));
|
||||||
|
});
|
||||||
|
test("project configuration is overridden by environment and rejects escaping modules", () -> {
|
||||||
|
final Path root = Files.createTempDirectory("recipe-config-");
|
||||||
|
try {
|
||||||
|
Files.writeString(
|
||||||
|
root.resolve("engineering.properties"), "PROJECT_NAME=sample\nBACKEND_MODULE=server\n");
|
||||||
|
equal(
|
||||||
|
"override",
|
||||||
|
Recipe.environment(root, Map.of("PROJECT_NAME", "override"))
|
||||||
|
.get("PROJECT_NAME"));
|
||||||
|
failure(() -> Recipe.environment(root, Map.of("BACKEND_MODULE", "../outside")));
|
||||||
|
} finally {
|
||||||
|
Pipeline.deleteTree(root);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("JSON preserves configuration values and escaped text", () -> {
|
||||||
|
String json = "{\"text\":\"line\\nquote\\\"\\u20ac\",\"values\":[null,true,false,1,-2.5e2],\"nested\":{}}";
|
||||||
|
Object data = Json.parse(json);
|
||||||
|
equal(data, Json.parse(Json.write(data)));
|
||||||
|
equal("line\nquote\"€", Json.object(data).get("text"));
|
||||||
|
});
|
||||||
|
test("JSON rejects malformed/ambiguous input", () -> {
|
||||||
|
for (String value : List.of(
|
||||||
|
"{\"a\":null,\"a\":1}", "{} []", "[1,]", "01", "\"\\uZZZZ\"", "\"line\n\"", "truefalse", "+1"))
|
||||||
|
failure(() -> Json.parse(value));
|
||||||
|
});
|
||||||
|
test("Docker configuration merge is idempotent and preserves backup", () -> {
|
||||||
|
Path dir = Files.createTempDirectory("ci-docker-test-");
|
||||||
|
try {
|
||||||
|
Path file = dir.resolve("daemon.json");
|
||||||
|
String original = "{\"features\":{\"buildkit\":true},\"insecure-registries\":[\"existing:5000\"]}";
|
||||||
|
Files.writeString(file, original);
|
||||||
|
Setup.configureDocker(file, "registry.example:3000");
|
||||||
|
String first = Files.readString(file);
|
||||||
|
Setup.configureDocker(file, "registry.example:3000");
|
||||||
|
equal(first, Files.readString(file));
|
||||||
|
equal(
|
||||||
|
List.of("existing:5000", "registry.example:3000"),
|
||||||
|
Json.object(Json.parse(first)).get("insecure-registries"));
|
||||||
|
equal(original, Files.readString(dir.resolve("daemon.json.before-engineering")));
|
||||||
|
} finally {
|
||||||
|
Pipeline.deleteTree(dir);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Invalid Docker configuration is unchanged", () -> {
|
||||||
|
Path dir = Files.createTempDirectory("ci-docker-test-");
|
||||||
|
try {
|
||||||
|
Path file = dir.resolve("daemon.json");
|
||||||
|
String original = "{\"insecure-registries\":\"invalid\"}";
|
||||||
|
Files.writeString(file, original);
|
||||||
|
failure(() -> Setup.configureDocker(file, "registry.example:3000"));
|
||||||
|
equal(original, Files.readString(file));
|
||||||
|
} finally {
|
||||||
|
Pipeline.deleteTree(dir);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Feature branches never contact registry", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.pipeline.env.put("CI_REF", "refs/heads/feature/test");
|
||||||
|
f.pipeline.publish();
|
||||||
|
equal(0L, f.calls("login"));
|
||||||
|
equal(0L, f.calls("push"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Unverified commit cannot publish", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
Files.delete(f.root.resolve("ci/artifacts/verified-commit"));
|
||||||
|
failure(f.pipeline::publish);
|
||||||
|
equal(0L, f.calls("push"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Publication pullback and rerun preserve commit tags", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.pipeline.publish();
|
||||||
|
equal(4L, f.calls("push"));
|
||||||
|
f.pipeline.publish();
|
||||||
|
equal(6L, f.calls("push"));
|
||||||
|
equal(8L, f.calls("pull"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Partial publication is completed without overwriting backend", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.pushFailure = "frontend:" + SHA;
|
||||||
|
failure(f.pipeline::publish);
|
||||||
|
equal(
|
||||||
|
0L,
|
||||||
|
f.fake.calls.stream()
|
||||||
|
.filter(a -> a.contains("push") && a.getLast().endsWith(":main"))
|
||||||
|
.count());
|
||||||
|
f.fake.pushFailure = "";
|
||||||
|
f.pipeline.publish();
|
||||||
|
equal(
|
||||||
|
1L,
|
||||||
|
f.fake.calls.stream()
|
||||||
|
.filter(a -> a.contains("push") && a.getLast().endsWith("backend:" + SHA))
|
||||||
|
.count());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Authentication error is not a missing tag", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.manifestError = true;
|
||||||
|
failure(f.pipeline::publish);
|
||||||
|
equal(0L, f.calls("push"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Pull failure blocks moving tags", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.pullFailure = "backend:" + SHA;
|
||||||
|
failure(f.pipeline::publish);
|
||||||
|
equal(
|
||||||
|
0L,
|
||||||
|
f.fake.calls.stream()
|
||||||
|
.filter(a -> a.contains("push") && a.getLast().endsWith(":main"))
|
||||||
|
.count());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Older commit cannot move main", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.head = "c".repeat(40);
|
||||||
|
f.pipeline.publish();
|
||||||
|
equal(2L, f.calls("push"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Unavailable Docker fails before build with a concrete cause", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.dockerAvailable = false;
|
||||||
|
try {
|
||||||
|
f.pipeline.verify();
|
||||||
|
throw new AssertionError("Expected failure");
|
||||||
|
} catch (IllegalArgumentException error) {
|
||||||
|
equal(true, error.getMessage().contains("Docker is unavailable"));
|
||||||
|
}
|
||||||
|
equal(
|
||||||
|
0L,
|
||||||
|
f.fake.calls.stream()
|
||||||
|
.filter(a -> a.getFirst().equals("mvn"))
|
||||||
|
.count());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Architecture mismatch blocks publication", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.pipeline.env.put("TARGET_PLATFORM", "linux/arm64");
|
||||||
|
failure(f.pipeline::publish);
|
||||||
|
equal(0L, f.calls("push"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Failed verification removes success marker and preserves exit status", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.mavenFailure = true;
|
||||||
|
try {
|
||||||
|
f.pipeline.verify();
|
||||||
|
throw new AssertionError("Expected failure");
|
||||||
|
} catch (Commands.Failure error) {
|
||||||
|
equal(42, error.exitCode);
|
||||||
|
}
|
||||||
|
equal(
|
||||||
|
List.of("mvn", "-B", "-ntp", "-Dformat.mode=check", "clean", "verify"),
|
||||||
|
f.fake.calls.stream()
|
||||||
|
.filter(a -> a.getFirst().equals("mvn"))
|
||||||
|
.findFirst()
|
||||||
|
.orElseThrow());
|
||||||
|
equal(false, Files.exists(f.root.resolve("ci/artifacts/verified-commit")));
|
||||||
|
equal(0L, f.calls("build"));
|
||||||
|
equal(0L, f.calls("push"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Frontend lint runs after install and blocks tests and build on failure", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.fake.lintFailure = true;
|
||||||
|
try {
|
||||||
|
f.pipeline.verifyFrontend();
|
||||||
|
throw new AssertionError("Expected failure");
|
||||||
|
} catch (Commands.Failure error) {
|
||||||
|
equal(1, error.exitCode);
|
||||||
|
}
|
||||||
|
equal(List.of(List.of("npm", "ci"), List.of("npm", "run", "lint")), f.fake.calls);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Frontend verification checks before testing and building", () -> {
|
||||||
|
try (Fixture f = new Fixture()) {
|
||||||
|
f.pipeline.verifyFrontend();
|
||||||
|
equal(
|
||||||
|
List.of(
|
||||||
|
"ci",
|
||||||
|
"run lint",
|
||||||
|
"test -- --reporters=default --reporters=junit --output-file=unit-results/junit.xml",
|
||||||
|
"run build"),
|
||||||
|
f.fake.calls.stream()
|
||||||
|
.map(a -> String.join(" ", a.subList(1, a.size())))
|
||||||
|
.toList());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("Process execution enforces its deadline", () -> {
|
||||||
|
Path dir = Files.createTempDirectory("ci-process-test-");
|
||||||
|
try {
|
||||||
|
Path file = dir.resolve("Sleep.java");
|
||||||
|
Files.writeString(
|
||||||
|
file,
|
||||||
|
"class Sleep { public static void main(String[] args) throws Exception { Thread.sleep(10000);"
|
||||||
|
+ " } }");
|
||||||
|
Commands runner = new Commands(Duration.ofMillis(100));
|
||||||
|
try {
|
||||||
|
runner.execute(
|
||||||
|
List.of(System.getProperty("java.home") + "/bin/java", file.toString()),
|
||||||
|
dir,
|
||||||
|
System.getenv(),
|
||||||
|
null,
|
||||||
|
false);
|
||||||
|
throw new AssertionError("Expected timeout");
|
||||||
|
} catch (Commands.Failure error) {
|
||||||
|
equal(124, error.exitCode);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
Pipeline.deleteTree(dir);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
System.out.println(count + " CI tests passed");
|
||||||
|
}
|
||||||
|
|
||||||
|
static class Fixture implements AutoCloseable {
|
||||||
|
final Path root = Files.createTempDirectory("ci-publication-test-");
|
||||||
|
final FakeCommands fake = new FakeCommands();
|
||||||
|
final Pipeline pipeline;
|
||||||
|
|
||||||
|
Fixture() throws Exception {
|
||||||
|
Map<String, String> env = new HashMap<>(System.getenv());
|
||||||
|
env.putAll(Map.of(
|
||||||
|
"CI_COMMIT",
|
||||||
|
SHA,
|
||||||
|
"CI_RUN_ID",
|
||||||
|
"test",
|
||||||
|
"CI_RUN_ATTEMPT",
|
||||||
|
"1",
|
||||||
|
"CI_REF",
|
||||||
|
"refs/heads/main",
|
||||||
|
"CI_REPOSITORY",
|
||||||
|
"owner/recipe-demo",
|
||||||
|
"TARGET_PLATFORM",
|
||||||
|
"linux/amd64",
|
||||||
|
"GITEA_SERVER_URL",
|
||||||
|
"http://registry.example:3000",
|
||||||
|
"REGISTRY_USER",
|
||||||
|
"test",
|
||||||
|
"REGISTRY_TOKEN",
|
||||||
|
"dummy-token",
|
||||||
|
"CI_PUBLISH_LOCK_DIR",
|
||||||
|
root.resolve("lock").toString()));
|
||||||
|
env.put("PROJECT_NAME", "recipe-demo");
|
||||||
|
env.put("BACKEND_MODULE", "backend");
|
||||||
|
env.put("FRONTEND_MODULE", "frontend");
|
||||||
|
pipeline = new Pipeline(root, env, fake) {
|
||||||
|
@Override
|
||||||
|
String mainHead() {
|
||||||
|
return fake.head;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
Files.writeString(root.resolve("ci/artifacts/verified-commit"), SHA + "\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
long calls(String verb) {
|
||||||
|
return fake.calls.stream()
|
||||||
|
.filter(a -> a.size() > 1
|
||||||
|
&& a.getFirst().equals("docker")
|
||||||
|
&& a.get(1).equals(verb))
|
||||||
|
.count();
|
||||||
|
}
|
||||||
|
|
||||||
|
public void close() throws Exception {
|
||||||
|
pipeline.closeDockerConfig();
|
||||||
|
Pipeline.deleteTree(root);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static class FakeCommands extends Commands {
|
||||||
|
final List<List<String>> calls = new ArrayList<>();
|
||||||
|
final Map<String, String> images = new HashMap<>(), tags = new HashMap<>();
|
||||||
|
String head = SHA, pushFailure = "", pullFailure = "";
|
||||||
|
boolean manifestError, mavenFailure, lintFailure;
|
||||||
|
boolean dockerAvailable = true;
|
||||||
|
|
||||||
|
FakeCommands() {
|
||||||
|
super(Duration.ofMinutes(1));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
Result execute(List<String> args, Path cwd, Map<String, String> env, String input, boolean echo) {
|
||||||
|
calls.add(List.copyOf(args));
|
||||||
|
if (args.getFirst().equals("git")) return new Result(0, SHA);
|
||||||
|
if (args.getFirst().equals("mvn") && mavenFailure) return new Result(42, "deliberate failure");
|
||||||
|
if (args.equals(List.of("npm", "run", "lint")) && lintFailure)
|
||||||
|
return new Result(1, "deliberate lint failure");
|
||||||
|
if (!args.getFirst().equals("docker")) return new Result(0, "");
|
||||||
|
String command = args.get(1), ref = args.getLast();
|
||||||
|
if (command.equals("info"))
|
||||||
|
return new Result(dockerAvailable ? 0 : 1, args.contains("{{.Architecture}}") ? "amd64" : "linux");
|
||||||
|
if (command.equals("context")) return new Result(0, "unix:///tmp/test-docker.sock");
|
||||||
|
if (command.equals("manifest")) {
|
||||||
|
if (manifestError) return new Result(1, "unauthorized");
|
||||||
|
return images.containsKey(ref) ? new Result(0, "{}") : new Result(1, "manifest unknown");
|
||||||
|
}
|
||||||
|
if (command.equals("tag")) tags.put(ref, images.getOrDefault(args.get(2), SHA));
|
||||||
|
if (command.equals("push")) {
|
||||||
|
if (!pushFailure.isEmpty() && ref.contains(pushFailure)) return new Result(1, "push failed");
|
||||||
|
images.put(ref, tags.getOrDefault(ref, SHA));
|
||||||
|
}
|
||||||
|
if (command.equals("pull") && !pullFailure.isEmpty() && ref.contains(pullFailure))
|
||||||
|
return new Result(1, "pull failed");
|
||||||
|
if (command.equals("image"))
|
||||||
|
return new Result(
|
||||||
|
0,
|
||||||
|
args.stream().anyMatch(a -> a.contains("RepoDigests"))
|
||||||
|
? ref.substring(0, ref.lastIndexOf(':')) + "@sha256:" + "b".repeat(64)
|
||||||
|
: images.getOrDefault(ref, "missing"));
|
||||||
|
if (command.equals("container") || command.equals("network") && args.contains("inspect"))
|
||||||
|
return new Result(1, "");
|
||||||
|
return new Result(0, "");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
96
resources/ci/Commands.java
Normal file
96
resources/ci/Commands.java
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
import java.io.*;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.time.*;
|
||||||
|
import java.util.*;
|
||||||
|
import java.util.concurrent.*;
|
||||||
|
|
||||||
|
/** Executes argument lists directly, without shell interpolation or credential logging. */
|
||||||
|
class Commands {
|
||||||
|
record Result(int code, String output) {
|
||||||
|
String checked() {
|
||||||
|
if (code != 0) throw new Failure(code, "Command failed with exit status " + code);
|
||||||
|
return output.strip();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static class Failure extends RuntimeException {
|
||||||
|
final int exitCode;
|
||||||
|
|
||||||
|
Failure(int exitCode, String message) {
|
||||||
|
super(message);
|
||||||
|
this.exitCode = exitCode;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Instant deadline;
|
||||||
|
volatile Process active;
|
||||||
|
Path log;
|
||||||
|
|
||||||
|
void renew(Duration timeout) {
|
||||||
|
deadline = Instant.now().plus(timeout);
|
||||||
|
}
|
||||||
|
|
||||||
|
void cancel() {
|
||||||
|
Process process = active;
|
||||||
|
if (process != null) stop(process);
|
||||||
|
}
|
||||||
|
|
||||||
|
Commands(Duration timeout) {
|
||||||
|
deadline = Instant.now().plus(timeout);
|
||||||
|
}
|
||||||
|
|
||||||
|
Result execute(List<String> args, Path cwd, Map<String, String> env, String input, boolean echo) throws Exception {
|
||||||
|
if (!Instant.now().isBefore(deadline)) throw new Failure(124, "CI time limit exceeded");
|
||||||
|
ProcessBuilder builder =
|
||||||
|
new ProcessBuilder(args).directory(cwd.toFile()).redirectErrorStream(true);
|
||||||
|
builder.environment().clear();
|
||||||
|
builder.environment().putAll(env);
|
||||||
|
Process process = builder.start();
|
||||||
|
active = process;
|
||||||
|
ByteArrayOutputStream captured = new ByteArrayOutputStream();
|
||||||
|
Path destination = log;
|
||||||
|
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
|
||||||
|
Future<?> reader = executor.submit(() -> {
|
||||||
|
try (InputStream stream = process.getInputStream();
|
||||||
|
OutputStream file = destination == null
|
||||||
|
? OutputStream.nullOutputStream()
|
||||||
|
: Files.newOutputStream(
|
||||||
|
destination, StandardOpenOption.CREATE, StandardOpenOption.APPEND)) {
|
||||||
|
byte[] buffer = new byte[8192];
|
||||||
|
int count;
|
||||||
|
while ((count = stream.read(buffer)) != -1) {
|
||||||
|
captured.write(buffer, 0, count);
|
||||||
|
if (echo) {
|
||||||
|
System.out.write(buffer, 0, count);
|
||||||
|
System.out.flush();
|
||||||
|
}
|
||||||
|
file.write(buffer, 0, count);
|
||||||
|
}
|
||||||
|
} catch (IOException e) {
|
||||||
|
throw new UncheckedIOException(e);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
try (OutputStream stdin = process.getOutputStream()) {
|
||||||
|
if (input != null) stdin.write(input.getBytes(StandardCharsets.UTF_8));
|
||||||
|
}
|
||||||
|
if (!process.waitFor(
|
||||||
|
Math.max(1, Duration.between(Instant.now(), deadline).toMillis()), TimeUnit.MILLISECONDS)) {
|
||||||
|
stop(process);
|
||||||
|
throw new Failure(124, "CI time limit exceeded");
|
||||||
|
}
|
||||||
|
reader.get(10, TimeUnit.SECONDS);
|
||||||
|
return new Result(process.exitValue(), captured.toString(StandardCharsets.UTF_8));
|
||||||
|
} finally {
|
||||||
|
if (process.isAlive()) stop(process);
|
||||||
|
active = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void stop(Process process) {
|
||||||
|
process.descendants().forEach(ProcessHandle::destroyForcibly);
|
||||||
|
process.destroyForcibly();
|
||||||
|
}
|
||||||
|
}
|
||||||
154
resources/ci/Json.java
Normal file
154
resources/ci/Json.java
Normal file
@@ -0,0 +1,154 @@
|
|||||||
|
import java.math.BigDecimal;
|
||||||
|
import java.util.*;
|
||||||
|
|
||||||
|
/** Small dependency-free JSON codec for API/configuration objects, not application data. */
|
||||||
|
final class Json {
|
||||||
|
static Object parse(String source) {
|
||||||
|
Parser parser = new Parser(source);
|
||||||
|
Object result = parser.value();
|
||||||
|
parser.space();
|
||||||
|
if (parser.pos != source.length()) throw parser.invalid();
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
@SuppressWarnings("unchecked")
|
||||||
|
static Map<String, Object> object(Object value) {
|
||||||
|
if (!(value instanceof Map<?, ?>)) throw new IllegalArgumentException("Expected a JSON object");
|
||||||
|
return (Map<String, Object>) value;
|
||||||
|
}
|
||||||
|
|
||||||
|
static String write(Object value) {
|
||||||
|
if (value == null) return "null";
|
||||||
|
if (value instanceof String text) {
|
||||||
|
StringBuilder out = new StringBuilder("\"");
|
||||||
|
for (char c : text.toCharArray()) {
|
||||||
|
switch (c) {
|
||||||
|
case '"' -> out.append("\\\"");
|
||||||
|
case '\\' -> out.append("\\\\");
|
||||||
|
case '\n' -> out.append("\\n");
|
||||||
|
case '\r' -> out.append("\\r");
|
||||||
|
case '\t' -> out.append("\\t");
|
||||||
|
default -> {
|
||||||
|
if (c < 32) out.append(String.format("\\u%04x", (int) c));
|
||||||
|
else out.append(c);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out.append('"').toString();
|
||||||
|
}
|
||||||
|
if (value instanceof Boolean
|
||||||
|
|| value instanceof BigDecimal
|
||||||
|
|| value instanceof Integer
|
||||||
|
|| value instanceof Long) return value.toString();
|
||||||
|
if (value instanceof Map<?, ?> map) {
|
||||||
|
StringJoiner out = new StringJoiner(",", "{", "}");
|
||||||
|
map.forEach((k, v) -> out.add(write(Objects.toString(k)) + ":" + write(v)));
|
||||||
|
return out.toString();
|
||||||
|
}
|
||||||
|
if (value instanceof List<?> list) {
|
||||||
|
StringJoiner out = new StringJoiner(",", "[", "]");
|
||||||
|
list.forEach(v -> out.add(write(v)));
|
||||||
|
return out.toString();
|
||||||
|
}
|
||||||
|
throw new IllegalArgumentException("Unsupported JSON value");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static final class Parser {
|
||||||
|
final String source;
|
||||||
|
int pos;
|
||||||
|
|
||||||
|
Parser(String source) {
|
||||||
|
this.source = source;
|
||||||
|
}
|
||||||
|
|
||||||
|
IllegalArgumentException invalid() {
|
||||||
|
return new IllegalArgumentException("Invalid JSON at offset " + pos);
|
||||||
|
}
|
||||||
|
|
||||||
|
void space() {
|
||||||
|
while (pos < source.length() && " \r\n\t".indexOf(source.charAt(pos)) >= 0) pos++;
|
||||||
|
}
|
||||||
|
|
||||||
|
boolean take(char c) {
|
||||||
|
space();
|
||||||
|
if (pos < source.length() && source.charAt(pos) == c) {
|
||||||
|
pos++;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
Object value() {
|
||||||
|
space();
|
||||||
|
if (pos >= source.length()) throw invalid();
|
||||||
|
char c = source.charAt(pos);
|
||||||
|
if (c == '"') return string();
|
||||||
|
if (take('{')) {
|
||||||
|
Map<String, Object> out = new LinkedHashMap<>();
|
||||||
|
if (take('}')) return out;
|
||||||
|
do {
|
||||||
|
space();
|
||||||
|
if (pos >= source.length() || source.charAt(pos) != '"') throw invalid();
|
||||||
|
String key = string();
|
||||||
|
if (!take(':') || out.containsKey(key)) throw invalid();
|
||||||
|
out.put(key, value());
|
||||||
|
} while (take(','));
|
||||||
|
if (!take('}')) throw invalid();
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
if (take('[')) {
|
||||||
|
List<Object> out = new ArrayList<>();
|
||||||
|
if (take(']')) return out;
|
||||||
|
do {
|
||||||
|
out.add(value());
|
||||||
|
} while (take(','));
|
||||||
|
if (!take(']')) throw invalid();
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
for (String token : List.of("true", "false", "null")) {
|
||||||
|
if (source.startsWith(token, pos)) {
|
||||||
|
pos += token.length();
|
||||||
|
return token.equals("null") ? null : Boolean.valueOf(token);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
int start = pos;
|
||||||
|
while (pos < source.length() && "-+0123456789.eE".indexOf(source.charAt(pos)) >= 0) pos++;
|
||||||
|
String number = source.substring(start, pos);
|
||||||
|
if (!number.matches("-?(0|[1-9][0-9]*)(\\.[0-9]+)?([eE][+-]?[0-9]+)?")) throw invalid();
|
||||||
|
return new BigDecimal(number);
|
||||||
|
}
|
||||||
|
|
||||||
|
String string() {
|
||||||
|
pos++;
|
||||||
|
StringBuilder out = new StringBuilder();
|
||||||
|
while (pos < source.length()) {
|
||||||
|
char c = source.charAt(pos++);
|
||||||
|
if (c == '"') return out.toString();
|
||||||
|
if (c < 32) throw invalid();
|
||||||
|
if (c != '\\') {
|
||||||
|
out.append(c);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (pos >= source.length()) throw invalid();
|
||||||
|
char escaped = source.charAt(pos++);
|
||||||
|
switch (escaped) {
|
||||||
|
case '"', '\\', '/' -> out.append(escaped);
|
||||||
|
case 'b' -> out.append('\b');
|
||||||
|
case 'f' -> out.append('\f');
|
||||||
|
case 'n' -> out.append('\n');
|
||||||
|
case 'r' -> out.append('\r');
|
||||||
|
case 't' -> out.append('\t');
|
||||||
|
case 'u' -> {
|
||||||
|
if (pos + 4 > source.length()) throw invalid();
|
||||||
|
String digits = source.substring(pos, pos + 4);
|
||||||
|
if (!digits.matches("[0-9a-fA-F]{4}")) throw invalid();
|
||||||
|
out.append((char) Integer.parseInt(digits, 16));
|
||||||
|
pos += 4;
|
||||||
|
}
|
||||||
|
default -> throw invalid();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw invalid();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
513
resources/ci/Pipeline.java
Normal file
513
resources/ci/Pipeline.java
Normal file
@@ -0,0 +1,513 @@
|
|||||||
|
import java.io.*;
|
||||||
|
import java.nio.channels.*;
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.time.*;
|
||||||
|
import java.util.*;
|
||||||
|
|
||||||
|
/** Verification, publication and cleanup shared by local use and Gitea Actions. */
|
||||||
|
class Pipeline {
|
||||||
|
final Path root, artifacts;
|
||||||
|
final Map<String, String> env;
|
||||||
|
final Commands commands;
|
||||||
|
final String commit, runName, backendImage, frontendImage;
|
||||||
|
Path isolatedConfig;
|
||||||
|
|
||||||
|
Pipeline(Path root, Map<String, String> environment, Commands commands) throws Exception {
|
||||||
|
this.root = root;
|
||||||
|
this.env = Recipe.environment(root, environment);
|
||||||
|
this.commands = commands;
|
||||||
|
commit = value("CI_COMMIT", () -> capture("git", "rev-parse", "HEAD"));
|
||||||
|
String id = value("CI_RUN_ID", () -> "local"), attempt = value("CI_RUN_ATTEMPT", () -> "1");
|
||||||
|
check(commit.matches("[0-9a-f]{40}"), "Invalid commit SHA");
|
||||||
|
check((id + "-" + attempt).matches("[a-zA-Z0-9-]+"), "Invalid run identifier");
|
||||||
|
runName = Recipe.project(env) + "-" + id + "-" + attempt;
|
||||||
|
backendImage = Recipe.project(env) + "-ci-backend:" + commit + "-" + id + "-" + attempt;
|
||||||
|
frontendImage = Recipe.project(env) + "-ci-frontend:" + commit + "-" + id + "-" + attempt;
|
||||||
|
artifacts = root.resolve("ci/artifacts");
|
||||||
|
Files.createDirectories(artifacts);
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Value {
|
||||||
|
String get() throws Exception;
|
||||||
|
}
|
||||||
|
|
||||||
|
String value(String key, Value defaultValue) throws Exception {
|
||||||
|
if (env.getOrDefault(key, "").isBlank()) env.put(key, defaultValue.get());
|
||||||
|
return env.get(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void check(boolean condition, String message) {
|
||||||
|
if (!condition) throw new IllegalArgumentException(message);
|
||||||
|
}
|
||||||
|
|
||||||
|
Commands.Result result(boolean echo, String... args) throws Exception {
|
||||||
|
return commands.execute(List.of(args), root, env, null, echo);
|
||||||
|
}
|
||||||
|
|
||||||
|
String capture(String... args) throws Exception {
|
||||||
|
return result(false, args).checked();
|
||||||
|
}
|
||||||
|
|
||||||
|
void run(String... args) throws Exception {
|
||||||
|
result(true, args).checked();
|
||||||
|
}
|
||||||
|
|
||||||
|
void requirePlatform() throws Exception {
|
||||||
|
String target = Setup.required(env, "TARGET_PLATFORM");
|
||||||
|
var docker = result(false, "docker", "info", "--format", "{{.Architecture}}");
|
||||||
|
check(
|
||||||
|
docker.code() == 0,
|
||||||
|
"Docker is unavailable; start Docker Desktop before running CI (details in verify.log)");
|
||||||
|
String arch = docker.output().strip();
|
||||||
|
arch = switch (arch) {
|
||||||
|
case "x86_64" -> "amd64";
|
||||||
|
case "aarch64" -> "arm64";
|
||||||
|
default -> arch;
|
||||||
|
};
|
||||||
|
String nativePlatform = capture("docker", "info", "--format", "{{.OSType}}") + "/" + arch;
|
||||||
|
check(target.equals(nativePlatform), "Target " + target + " differs from Docker host " + nativePlatform);
|
||||||
|
}
|
||||||
|
|
||||||
|
void isolateDocker() throws Exception {
|
||||||
|
value("DOCKER_HOST", () -> capture("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}"));
|
||||||
|
env.remove("DOCKER_CONTEXT");
|
||||||
|
isolatedConfig = Files.createTempDirectory(Recipe.project(env) + "-docker-");
|
||||||
|
env.put("DOCKER_CONFIG", isolatedConfig.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
void verify() throws Exception {
|
||||||
|
Files.deleteIfExists(artifacts.resolve("verified-commit"));
|
||||||
|
env.put("CI", "true");
|
||||||
|
commands.log = artifacts.resolve("verify.log");
|
||||||
|
Files.writeString(commands.log, "");
|
||||||
|
Thread shutdown = new Thread(
|
||||||
|
() -> {
|
||||||
|
commands.cancel();
|
||||||
|
try {
|
||||||
|
cleanup();
|
||||||
|
closeDockerConfig();
|
||||||
|
} catch (Exception e) {
|
||||||
|
System.err.println("CI shutdown cleanup failed");
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"ci-cleanup");
|
||||||
|
Runtime.getRuntime().addShutdownHook(shutdown);
|
||||||
|
Exception failure = null;
|
||||||
|
try {
|
||||||
|
requirePlatform();
|
||||||
|
isolateDocker();
|
||||||
|
check(commit.equals(capture("git", "rev-parse", "HEAD")), "Checkout does not match pushed commit");
|
||||||
|
run("java", "-version");
|
||||||
|
if (Recipe.roles(env).contains("frontend")) run("node", "--version");
|
||||||
|
if (Recipe.roles(env).contains("backend"))
|
||||||
|
run("mvn", "-B", "-ntp", "-Dformat.mode=check", "clean", "verify");
|
||||||
|
CiTests.run();
|
||||||
|
ReleaseTests.run();
|
||||||
|
Path frontend = root.resolve(env.getOrDefault("FRONTEND_MODULE", "."));
|
||||||
|
if (Recipe.roles(env).contains("frontend")) verifyFrontend();
|
||||||
|
String platform = env.get("TARGET_PLATFORM");
|
||||||
|
for (String role : Recipe.roles(env)) {
|
||||||
|
run(
|
||||||
|
"docker",
|
||||||
|
"build",
|
||||||
|
"--platform",
|
||||||
|
platform,
|
||||||
|
"--target",
|
||||||
|
"prebuilt",
|
||||||
|
"--label",
|
||||||
|
"org.opencontainers.image.revision=" + commit,
|
||||||
|
"-t",
|
||||||
|
role.equals("backend") ? backendImage : frontendImage,
|
||||||
|
env.getOrDefault(
|
||||||
|
role.toUpperCase() + "_CONTEXT",
|
||||||
|
role.equals("backend") ? "." : env.get("FRONTEND_MODULE")));
|
||||||
|
}
|
||||||
|
if (Recipe.enabled(env, "BROWSER_TESTS")) {
|
||||||
|
Files.writeString(artifacts.resolve("containers-started"), "");
|
||||||
|
run("docker", "network", "create", "--label", Recipe.project(env) + ".ci.run=" + runName, runName);
|
||||||
|
if (Recipe.enabled(env, "CI_DATABASE")) {
|
||||||
|
run(
|
||||||
|
"docker",
|
||||||
|
"run",
|
||||||
|
"-d",
|
||||||
|
"--name",
|
||||||
|
runName + "-database",
|
||||||
|
"--network",
|
||||||
|
runName,
|
||||||
|
"--network-alias",
|
||||||
|
"database",
|
||||||
|
"--env",
|
||||||
|
"POSTGRES_USER=" + env.get("DATABASE_USER"),
|
||||||
|
"--env",
|
||||||
|
"POSTGRES_PASSWORD=ci-only",
|
||||||
|
"--env",
|
||||||
|
"POSTGRES_DB=" + env.get("DATABASE_NAME"),
|
||||||
|
"postgres:17.6");
|
||||||
|
for (int attempt = 0; attempt < 60; attempt++) {
|
||||||
|
if (result(
|
||||||
|
false,
|
||||||
|
"docker",
|
||||||
|
"exec",
|
||||||
|
runName + "-database",
|
||||||
|
"pg_isready",
|
||||||
|
"-U",
|
||||||
|
env.get("DATABASE_USER"))
|
||||||
|
.code()
|
||||||
|
== 0) break;
|
||||||
|
Thread.sleep(1000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
run(
|
||||||
|
"docker",
|
||||||
|
"run",
|
||||||
|
"-d",
|
||||||
|
"--name",
|
||||||
|
runName + "-backend",
|
||||||
|
"--label",
|
||||||
|
Recipe.project(env) + ".ci.run=" + runName,
|
||||||
|
"--platform",
|
||||||
|
platform,
|
||||||
|
"--network",
|
||||||
|
runName,
|
||||||
|
"--network-alias",
|
||||||
|
"backend",
|
||||||
|
"--memory",
|
||||||
|
"1g",
|
||||||
|
"--cpus",
|
||||||
|
"2",
|
||||||
|
"--volume",
|
||||||
|
root.resolve(env.get("BACKEND_MODULE") + "/target/coverage-tools/agent.jar")
|
||||||
|
+ ":/jacoco-agent.jar:ro",
|
||||||
|
"--env",
|
||||||
|
"JAVA_TOOL_OPTIONS=-javaagent:/jacoco-agent.jar=output=tcpserver,address=*,port=6300",
|
||||||
|
"--env",
|
||||||
|
"DATABASE_URL=jdbc:postgresql://database:5432/" + env.get("DATABASE_NAME"),
|
||||||
|
"--env",
|
||||||
|
"DATABASE_USER=" + env.get("DATABASE_USER"),
|
||||||
|
"--env",
|
||||||
|
"DATABASE_PASSWORD=ci-only",
|
||||||
|
backendImage);
|
||||||
|
run(
|
||||||
|
"docker",
|
||||||
|
"run",
|
||||||
|
"-d",
|
||||||
|
"--name",
|
||||||
|
runName + "-frontend",
|
||||||
|
"--label",
|
||||||
|
Recipe.project(env) + ".ci.run=" + runName,
|
||||||
|
"--platform",
|
||||||
|
platform,
|
||||||
|
"--network",
|
||||||
|
runName,
|
||||||
|
"--network-alias",
|
||||||
|
"frontend",
|
||||||
|
"--memory",
|
||||||
|
"256m",
|
||||||
|
"--cpus",
|
||||||
|
"1",
|
||||||
|
frontendImage);
|
||||||
|
waitForApplication();
|
||||||
|
String user = capture("id", "-u") + ":" + capture("id", "-g");
|
||||||
|
run(
|
||||||
|
"docker",
|
||||||
|
"run",
|
||||||
|
"--name",
|
||||||
|
runName + "-browser",
|
||||||
|
"--label",
|
||||||
|
Recipe.project(env) + ".ci.run=" + runName,
|
||||||
|
"--platform",
|
||||||
|
platform,
|
||||||
|
"--network",
|
||||||
|
runName,
|
||||||
|
"--memory",
|
||||||
|
"2g",
|
||||||
|
"--cpus",
|
||||||
|
"2",
|
||||||
|
"--shm-size=1g",
|
||||||
|
"--user",
|
||||||
|
user,
|
||||||
|
"--env",
|
||||||
|
"HOME=/tmp",
|
||||||
|
"--env",
|
||||||
|
"CI=true",
|
||||||
|
"--env",
|
||||||
|
"PLAYWRIGHT_BASE_URL=http://frontend:8080",
|
||||||
|
"--volume",
|
||||||
|
frontend + ":/workspace",
|
||||||
|
"--workdir",
|
||||||
|
"/workspace",
|
||||||
|
env.getOrDefault("PLAYWRIGHT_IMAGE", "mcr.microsoft.com/playwright:v1.64.1-noble"),
|
||||||
|
"npm",
|
||||||
|
"run",
|
||||||
|
"e2e");
|
||||||
|
run(
|
||||||
|
"docker",
|
||||||
|
"run",
|
||||||
|
"--rm",
|
||||||
|
"--network",
|
||||||
|
runName,
|
||||||
|
"--user",
|
||||||
|
user,
|
||||||
|
"--volume",
|
||||||
|
root.resolve(env.get("BACKEND_MODULE") + "/target") + ":/coverage",
|
||||||
|
"eclipse-temurin:25-jre",
|
||||||
|
"java",
|
||||||
|
"-jar",
|
||||||
|
"/coverage/coverage-tools/cli.jar",
|
||||||
|
"dump",
|
||||||
|
"--address",
|
||||||
|
"backend",
|
||||||
|
"--destfile",
|
||||||
|
"/coverage/playwright.exec");
|
||||||
|
check(
|
||||||
|
Files.size(root.resolve(env.get("BACKEND_MODULE") + "/target/playwright.exec")) > 0,
|
||||||
|
"Playwright backend coverage is missing");
|
||||||
|
run(
|
||||||
|
"mvn",
|
||||||
|
"-B",
|
||||||
|
"-ntp",
|
||||||
|
"-pl",
|
||||||
|
env.get("BACKEND_MODULE"),
|
||||||
|
"jacoco:merge@combined",
|
||||||
|
"jacoco:report@combined-report",
|
||||||
|
"exec:exec@combined-coverage");
|
||||||
|
}
|
||||||
|
Files.writeString(artifacts.resolve("verified-commit"), commit + "\n");
|
||||||
|
} catch (Exception e) {
|
||||||
|
failure = e;
|
||||||
|
throw e;
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
cleanup();
|
||||||
|
} catch (Exception e) {
|
||||||
|
if (failure != null) failure.addSuppressed(e);
|
||||||
|
else throw e;
|
||||||
|
} finally {
|
||||||
|
closeDockerConfig();
|
||||||
|
try {
|
||||||
|
Runtime.getRuntime().removeShutdownHook(shutdown);
|
||||||
|
} catch (IllegalStateException ignored) {
|
||||||
|
/* shutting down */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void verifyFrontend() throws Exception {
|
||||||
|
Path frontend = root.resolve(env.get("FRONTEND_MODULE"));
|
||||||
|
commands.execute(List.of("npm", "ci"), frontend, env, null, true).checked();
|
||||||
|
commands.execute(List.of("npm", "run", "lint"), frontend, env, null, true)
|
||||||
|
.checked();
|
||||||
|
final Object configured = Json.parse(
|
||||||
|
env.getOrDefault(
|
||||||
|
"FRONTEND_TEST_COMMAND",
|
||||||
|
"[\"npm\",\"test\",\"--\",\"--reporters=default\",\"--reporters=junit\",\"--output-file=unit-results/junit.xml\"]"));
|
||||||
|
final List<String> test =
|
||||||
|
((List<?>) configured).stream().map(Object::toString).toList();
|
||||||
|
commands.execute(test, frontend, env, null, true).checked();
|
||||||
|
commands.execute(List.of("npm", "run", "build"), frontend, env, null, true)
|
||||||
|
.checked();
|
||||||
|
}
|
||||||
|
|
||||||
|
void waitForApplication() throws Exception {
|
||||||
|
for (int i = 0; i < 60; i++) {
|
||||||
|
if (result(
|
||||||
|
false,
|
||||||
|
"docker",
|
||||||
|
"exec",
|
||||||
|
runName + "-frontend",
|
||||||
|
"wget",
|
||||||
|
"-q",
|
||||||
|
"-O",
|
||||||
|
"-",
|
||||||
|
Setup.required(env, "CI_READINESS_URL"))
|
||||||
|
.code()
|
||||||
|
== 0) return;
|
||||||
|
Thread.sleep(2000);
|
||||||
|
}
|
||||||
|
throw new IllegalArgumentException("Application readiness timed out");
|
||||||
|
}
|
||||||
|
|
||||||
|
synchronized void cleanup() throws Exception {
|
||||||
|
// A separate cleanup budget remains available after the build deadline.
|
||||||
|
commands.renew(Duration.ofMinutes(2));
|
||||||
|
List<String> failures = new ArrayList<>();
|
||||||
|
for (String path : List.of(
|
||||||
|
env.get("BACKEND_MODULE") + "/target/surefire-reports",
|
||||||
|
env.get("BACKEND_MODULE") + "/target/standards",
|
||||||
|
"generated/tooling/standards/target/surefire-reports",
|
||||||
|
env.get("BACKEND_MODULE") + "/target/site/jacoco",
|
||||||
|
env.get("BACKEND_MODULE") + "/target/site/jacoco-combined",
|
||||||
|
env.get("BACKEND_MODULE") + "/target/failsafe-reports",
|
||||||
|
env.get("FRONTEND_MODULE") + "/unit-results",
|
||||||
|
env.get("FRONTEND_MODULE") + "/test-results",
|
||||||
|
env.get("FRONTEND_MODULE") + "/playwright-report")) {
|
||||||
|
if (Files.isDirectory(root.resolve(path))) {
|
||||||
|
try {
|
||||||
|
copyTree(root.resolve(path), artifacts.resolve(path));
|
||||||
|
} catch (IOException e) {
|
||||||
|
failures.add(path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (String data : List.of("jacoco.exec", "playwright.exec", "jacoco-combined.exec")) {
|
||||||
|
final Path source =
|
||||||
|
root.resolve(env.get("BACKEND_MODULE") + "/target").resolve(data);
|
||||||
|
if (Files.isRegularFile(source))
|
||||||
|
Files.copy(source, artifacts.resolve(data), StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
}
|
||||||
|
if (result(false, "docker", "info").code() == 0) {
|
||||||
|
for (String role : List.of("backend", "frontend", "browser", "database")) {
|
||||||
|
String container = runName + "-" + role;
|
||||||
|
if (result(false, "docker", "container", "inspect", container).code() == 0) {
|
||||||
|
var logs = result(false, "docker", "logs", container);
|
||||||
|
Files.writeString(artifacts.resolve(role + ".log"), logs.output());
|
||||||
|
if (logs.code() != 0) failures.add(role + " logs");
|
||||||
|
if (result(false, "docker", "rm", "-f", "-v", container).code() != 0)
|
||||||
|
failures.add(role + " cleanup");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (result(false, "docker", "network", "inspect", runName).code() == 0
|
||||||
|
&& result(false, "docker", "network", "rm", runName).code() != 0) failures.add("network cleanup");
|
||||||
|
if (failures.isEmpty()) Files.deleteIfExists(artifacts.resolve("containers-started"));
|
||||||
|
} else if (Files.exists(artifacts.resolve("containers-started")))
|
||||||
|
failures.add("Docker unavailable during cleanup");
|
||||||
|
check(failures.isEmpty(), "Cleanup incomplete: " + String.join(", ", failures));
|
||||||
|
}
|
||||||
|
|
||||||
|
void publish() throws Exception {
|
||||||
|
if (!"refs/heads/main".equals(env.get("CI_REF"))) {
|
||||||
|
System.out.println("Branch is not main; no publication");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
check(
|
||||||
|
Files.exists(artifacts.resolve("verified-commit"))
|
||||||
|
&& Files.readString(artifacts.resolve("verified-commit"))
|
||||||
|
.strip()
|
||||||
|
.equals(commit),
|
||||||
|
"This commit has not passed verification");
|
||||||
|
String user = Setup.required(env, "REGISTRY_USER"), token = Setup.required(env, "REGISTRY_TOKEN");
|
||||||
|
String server = Setup.required(env, "GITEA_SERVER_URL");
|
||||||
|
String repository = Setup.required(env, "CI_REPOSITORY");
|
||||||
|
requirePlatform();
|
||||||
|
commands.log = artifacts.resolve("publish.log");
|
||||||
|
Files.writeString(commands.log, "");
|
||||||
|
var uri = java.net.URI.create(server);
|
||||||
|
check(
|
||||||
|
Set.of("http", "https").contains(uri.getScheme())
|
||||||
|
&& (uri.getPath().isEmpty() || uri.getPath().equals("/")),
|
||||||
|
"Registry must be hosted at the Gitea server root");
|
||||||
|
String registry = uri.getRawAuthority();
|
||||||
|
String owner = repository.split("/")[0].toLowerCase(Locale.ROOT);
|
||||||
|
check(owner.matches("[a-z0-9][a-z0-9_.-]*"), "Invalid registry namespace");
|
||||||
|
Path lockDir = Path.of(env.getOrDefault(
|
||||||
|
"CI_PUBLISH_LOCK_DIR",
|
||||||
|
env.getOrDefault(
|
||||||
|
"XDG_STATE_HOME",
|
||||||
|
Setup.home().resolve(".local/state").toString()) + "/" + Recipe.project(env) + "-ci"));
|
||||||
|
Files.createDirectories(lockDir);
|
||||||
|
// Shared host lock: Gitea 1.25 ignores workflow concurrency.
|
||||||
|
try (FileChannel channel = FileChannel.open(
|
||||||
|
lockDir.resolve("publish.lock"), StandardOpenOption.CREATE, StandardOpenOption.WRITE);
|
||||||
|
FileLock lock = awaitLock(channel)) {
|
||||||
|
isolateDocker();
|
||||||
|
try {
|
||||||
|
commands.execute(
|
||||||
|
List.of("docker", "login", registry, "--username", user, "--password-stdin"),
|
||||||
|
root,
|
||||||
|
env,
|
||||||
|
token,
|
||||||
|
true)
|
||||||
|
.checked();
|
||||||
|
String prefix = registry + "/" + owner + "/" + Recipe.project(env) + "-";
|
||||||
|
for (String role : Recipe.roles(env)) {
|
||||||
|
ensureImage(prefix + role, role.equals("backend") ? backendImage : frontendImage);
|
||||||
|
}
|
||||||
|
for (String role : Recipe.roles(env)) {
|
||||||
|
if (!commit.equals(mainHead())) {
|
||||||
|
System.out.println("Newer main exists; moving tags unchanged");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
String image = prefix + role;
|
||||||
|
run("docker", "tag", image + ":" + commit, image + ":main");
|
||||||
|
run("docker", "push", image + ":main");
|
||||||
|
run("docker", "pull", "--platform", env.get("TARGET_PLATFORM"), image + ":main");
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
closeDockerConfig();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
FileLock awaitLock(FileChannel channel) throws Exception {
|
||||||
|
Instant end = Instant.now().plusSeconds(600);
|
||||||
|
while (Instant.now().isBefore(end)) {
|
||||||
|
FileLock lock = channel.tryLock();
|
||||||
|
if (lock != null) return lock;
|
||||||
|
Thread.sleep(250);
|
||||||
|
}
|
||||||
|
throw new IllegalArgumentException("Publication lock timed out");
|
||||||
|
}
|
||||||
|
|
||||||
|
String mainHead() throws Exception {
|
||||||
|
Map<String, Object> branch = Json.object(Setup.api(
|
||||||
|
env.get("GITEA_SERVER_URL"),
|
||||||
|
env.get("REGISTRY_TOKEN"),
|
||||||
|
"GET",
|
||||||
|
"/api/v1/repos/" + env.get("CI_REPOSITORY") + "/branches/main",
|
||||||
|
null));
|
||||||
|
return Objects.toString(Json.object(branch.get("commit")).get("id"));
|
||||||
|
}
|
||||||
|
|
||||||
|
void ensureImage(String repository, String source) throws Exception {
|
||||||
|
String ref = repository + ":" + commit;
|
||||||
|
var manifest = result(false, "docker", "manifest", "inspect", "--insecure", ref);
|
||||||
|
if (manifest.code() == 0) System.out.println("Preserving existing " + ref);
|
||||||
|
else if (manifest.output().toLowerCase(Locale.ROOT).matches("(?s).*(manifest unknown|no such manifest).*")) {
|
||||||
|
run("docker", "tag", source, ref);
|
||||||
|
run("docker", "push", ref);
|
||||||
|
} else
|
||||||
|
throw new IllegalArgumentException(
|
||||||
|
"Registry manifest lookup failed; authentication/network errors are not missing tags");
|
||||||
|
run("docker", "pull", "--platform", env.get("TARGET_PLATFORM"), ref);
|
||||||
|
check(
|
||||||
|
commit.equals(capture(
|
||||||
|
"docker",
|
||||||
|
"image",
|
||||||
|
"inspect",
|
||||||
|
"--format",
|
||||||
|
"{{index .Config.Labels \"org.opencontainers.image.revision\"}}",
|
||||||
|
ref)),
|
||||||
|
"Existing image has an unexpected source revision");
|
||||||
|
String digests =
|
||||||
|
capture("docker", "image", "inspect", "--format", "{{range .RepoDigests}}{{println .}}{{end}}", ref);
|
||||||
|
check(digests.contains(repository + "@sha256:"), "Missing registry digest");
|
||||||
|
Files.writeString(
|
||||||
|
artifacts.resolve(repository.endsWith("backend") ? "backend-digests.txt" : "frontend-digests.txt"),
|
||||||
|
digests + "\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
synchronized void closeDockerConfig() throws IOException {
|
||||||
|
if (isolatedConfig != null) {
|
||||||
|
deleteTree(isolatedConfig);
|
||||||
|
isolatedConfig = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void copyTree(Path source, Path destination) throws IOException {
|
||||||
|
try (var paths = Files.walk(source)) {
|
||||||
|
for (Path path : paths.toList()) {
|
||||||
|
Path target = destination.resolve(source.relativize(path));
|
||||||
|
if (Files.isDirectory(path)) Files.createDirectories(target);
|
||||||
|
else Files.copy(path, target, StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void deleteTree(Path root) throws IOException {
|
||||||
|
if (!Files.exists(root)) return;
|
||||||
|
try (var paths = Files.walk(root)) {
|
||||||
|
for (Path path : paths.sorted(Comparator.reverseOrder()).toList()) Files.delete(path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
61
resources/ci/Recipe.java
Normal file
61
resources/ci/Recipe.java
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Properties;
|
||||||
|
|
||||||
|
/** Project-owned configuration for the shared engineering mechanisms. */
|
||||||
|
final class Recipe {
|
||||||
|
private Recipe() {}
|
||||||
|
|
||||||
|
static Map<String, String> environment(final Path root, final Map<String, String> input) throws Exception {
|
||||||
|
final var result = new HashMap<String, String>(input);
|
||||||
|
final Path file = root.resolve("engineering.properties");
|
||||||
|
if (Files.exists(file)) {
|
||||||
|
final var properties = new Properties();
|
||||||
|
try (final var stream = Files.newInputStream(file)) {
|
||||||
|
properties.load(stream);
|
||||||
|
}
|
||||||
|
for (final String key : properties.stringPropertyNames()) {
|
||||||
|
result.putIfAbsent(
|
||||||
|
key, properties.getProperty(key).replace("${user.home}", System.getProperty("user.home")));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (result.containsKey("PROJECT_NAME")) {
|
||||||
|
project(result);
|
||||||
|
for (final String role : roles(result)) {
|
||||||
|
final Path module = Path.of(result.get(role.toUpperCase() + "_MODULE"));
|
||||||
|
Pipeline.check(
|
||||||
|
!module.isAbsolute() && !module.normalize().startsWith(".."),
|
||||||
|
"Module must stay in the project");
|
||||||
|
}
|
||||||
|
Pipeline.check(
|
||||||
|
!enabled(result, "BROWSER_TESTS") || roles(result).size() == 2,
|
||||||
|
"Browser coverage requires both backend and frontend");
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
static String project(final Map<String, String> env) {
|
||||||
|
final String name = Setup.required(env, "PROJECT_NAME");
|
||||||
|
Pipeline.check(name.matches("[a-z][a-z0-9-]*"), "Invalid project name");
|
||||||
|
return name;
|
||||||
|
}
|
||||||
|
|
||||||
|
static List<String> roles(final Map<String, String> env) {
|
||||||
|
final var roles = new ArrayList<String>();
|
||||||
|
for (final String role : List.of("backend", "frontend")) {
|
||||||
|
if (!env.getOrDefault(role.toUpperCase() + "_MODULE", "").isBlank()) {
|
||||||
|
roles.add(role);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Pipeline.check(!roles.isEmpty(), "Configure BACKEND_MODULE, FRONTEND_MODULE, or both");
|
||||||
|
return roles;
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean enabled(final Map<String, String> env, final String key) {
|
||||||
|
return Boolean.parseBoolean(env.getOrDefault(key, "false"));
|
||||||
|
}
|
||||||
|
}
|
||||||
236
resources/ci/Release.java
Normal file
236
resources/ci/Release.java
Normal file
@@ -0,0 +1,236 @@
|
|||||||
|
import java.net.URI;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.LinkedHashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.UUID;
|
||||||
|
|
||||||
|
/** Manual release coordinator. All mutation decisions are made under the NAS lock. */
|
||||||
|
class Release {
|
||||||
|
final Path root, artifacts;
|
||||||
|
final Map<String, String> env;
|
||||||
|
final ReleaseApi api;
|
||||||
|
final ReleaseRemote remote;
|
||||||
|
final String mode, increment, retry, pinned, attempt;
|
||||||
|
ReleaseState.Deployment selected;
|
||||||
|
boolean locked, pending;
|
||||||
|
|
||||||
|
static void execute(Path root, Map<String, String> env, String operation) throws Exception {
|
||||||
|
validate(env, operation);
|
||||||
|
String attempt = "release-" + env.get("CI_RUN_ID") + "-" + UUID.randomUUID();
|
||||||
|
Path artifacts = root.resolve("ci/artifacts/deployment");
|
||||||
|
Files.createDirectories(artifacts);
|
||||||
|
var commands = new Commands(Duration.ofMinutes(15));
|
||||||
|
commands.log = artifacts.resolve(attempt + ".log");
|
||||||
|
Files.writeString(commands.log, "");
|
||||||
|
try (var remote = new ReleaseRemote(root, env, commands, attempt)) {
|
||||||
|
new Release(root, env, new ReleaseApi(env), remote, operation, attempt).run();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void validate(Map<String, String> env, String operation) {
|
||||||
|
Pipeline.check("refs/heads/main".equals(env.get("CI_REF")), "Production workflows must run from main");
|
||||||
|
Pipeline.check(
|
||||||
|
"workflow_dispatch".equals(env.get("CI_EVENT")), "Production requires a manual workflow dispatch");
|
||||||
|
Pipeline.check(Setup.required(env, "CI_RUN_ID").matches("[0-9]+"), "Invalid Gitea run ID");
|
||||||
|
ReleaseApi.sha(env.get("CI_COMMIT"));
|
||||||
|
Pipeline.check(List.of("deploy", "rollback").contains(operation), "Unknown release operation");
|
||||||
|
if (operation.equals("rollback")) return;
|
||||||
|
String mode = Setup.required(env, "RELEASE_MODE");
|
||||||
|
String increment = env.getOrDefault("RELEASE_INCREMENT", "");
|
||||||
|
String retry = env.getOrDefault("RELEASE_VERSION", "");
|
||||||
|
if (mode.equals("new")) {
|
||||||
|
Pipeline.check(List.of("patch", "minor", "major").contains(increment), "Choose patch, minor or major");
|
||||||
|
Pipeline.check(retry.isBlank(), "New release cannot specify an existing version");
|
||||||
|
} else {
|
||||||
|
Pipeline.check(mode.equals("retry"), "Choose new or retry release mode");
|
||||||
|
ReleaseVersion.parse(retry);
|
||||||
|
// Gitea shows all inputs; the new-release increment is ignored for retries.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Release(Path root, Map<String, String> env, ReleaseApi api, ReleaseRemote remote, String operation, String attempt)
|
||||||
|
throws Exception {
|
||||||
|
this.root = root;
|
||||||
|
this.env = env;
|
||||||
|
this.api = api;
|
||||||
|
this.remote = remote;
|
||||||
|
this.attempt = attempt;
|
||||||
|
artifacts = root.resolve("ci/artifacts/deployment");
|
||||||
|
Files.createDirectories(artifacts);
|
||||||
|
mode = operation.equals("rollback") ? "rollback" : env.get("RELEASE_MODE");
|
||||||
|
increment = env.getOrDefault("RELEASE_INCREMENT", "");
|
||||||
|
retry = env.getOrDefault("RELEASE_VERSION", "");
|
||||||
|
pinned = ReleaseApi.sha(env.get("CI_COMMIT"));
|
||||||
|
}
|
||||||
|
|
||||||
|
void run() throws Exception {
|
||||||
|
String outcome = "rejected";
|
||||||
|
try {
|
||||||
|
remote.acquire();
|
||||||
|
locked = true;
|
||||||
|
remote.call("preflight", null);
|
||||||
|
ReleaseState before = ReleaseState.parse(remote.call("state", null));
|
||||||
|
selected = select(before);
|
||||||
|
System.out.println("Selected " + selected.version() + " at " + selected.sha());
|
||||||
|
// Validate the recovery pair before any replacement too.
|
||||||
|
if (before.current() != null) ensure(before.current());
|
||||||
|
if (mode.equals("new")) {
|
||||||
|
api.create(selected.version(), selected.sha(), attempt);
|
||||||
|
Pipeline.check(
|
||||||
|
selected.sha().equals(api.head()),
|
||||||
|
"Main advanced; release tagged but production unchanged. Retry explicitly.");
|
||||||
|
}
|
||||||
|
var journal = new LinkedHashMap<String, Object>();
|
||||||
|
journal.put("attempt", attempt);
|
||||||
|
journal.put("mode", mode);
|
||||||
|
journal.put("candidate", selected.json());
|
||||||
|
journal.put("before", Json.parse(before.current() == null ? "{}" : before.json()));
|
||||||
|
remote.call("pending", Json.write(journal));
|
||||||
|
pending = true;
|
||||||
|
outcome = "replacement-started";
|
||||||
|
try {
|
||||||
|
remote.call("apply", null, selected.directory());
|
||||||
|
} catch (Exception failure) {
|
||||||
|
diagnose(selected);
|
||||||
|
outcome = recover(before);
|
||||||
|
throw new IllegalArgumentException(
|
||||||
|
"Deployment failed: " + outcome + ". See deployment diagnostics and recovery runbook.");
|
||||||
|
}
|
||||||
|
// A lost reply here is ambiguous: retain journal and lock, never guess
|
||||||
|
// whether a state commit happened and launch an overlapping recovery.
|
||||||
|
outcome = "healthy-state-commit-pending";
|
||||||
|
remote.call("commit", before.success(selected).json());
|
||||||
|
outcome = "success";
|
||||||
|
record(outcome);
|
||||||
|
remote.call("finish", null);
|
||||||
|
pending = false;
|
||||||
|
System.out.println("Production " + selected.version() + " is healthy (" + selected.sha() + ")");
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
record(outcome);
|
||||||
|
} finally {
|
||||||
|
if (locked && !pending) remote.release();
|
||||||
|
else if (locked)
|
||||||
|
System.err.println("NAS lock and pending journal retained. Follow docs/production-deployment.md for"
|
||||||
|
+ " recovery.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ReleaseState.Deployment select(ReleaseState before) throws Exception {
|
||||||
|
if (mode.equals("rollback")) {
|
||||||
|
Pipeline.check(before.previous() != null, "No previous successful production release is available");
|
||||||
|
ensure(before.previous());
|
||||||
|
return before.previous();
|
||||||
|
}
|
||||||
|
List<ReleaseApi.Tag> tags = api.tags();
|
||||||
|
ReleaseVersion version;
|
||||||
|
String commit;
|
||||||
|
if (mode.equals("new")) {
|
||||||
|
Pipeline.check(
|
||||||
|
pinned.equals(api.head()), "Main advanced since this dispatch; start a new release workflow");
|
||||||
|
Pipeline.check(
|
||||||
|
tags.stream().noneMatch(t -> t.sha().equals(pinned)),
|
||||||
|
"This commit already has a release tag; use retry or rollback");
|
||||||
|
version = ReleaseVersion.next(
|
||||||
|
tags.stream().map(ReleaseApi.Tag::version).toList(), increment);
|
||||||
|
commit = pinned;
|
||||||
|
} else {
|
||||||
|
version = ReleaseVersion.parse(retry);
|
||||||
|
var tag = tags.stream()
|
||||||
|
.filter(t -> t.version().equals(version))
|
||||||
|
.findFirst()
|
||||||
|
.orElseThrow(() -> new IllegalArgumentException("Retry release tag does not exist"));
|
||||||
|
api.annotated(tag);
|
||||||
|
commit = tag.sha();
|
||||||
|
}
|
||||||
|
Pipeline.check(
|
||||||
|
before.highest() == null || version.compareTo(before.highest()) >= (mode.equals("new") ? 1 : 0),
|
||||||
|
"A newer release has already succeeded; use explicit rollback for older versions");
|
||||||
|
String build = api.verified(commit);
|
||||||
|
remote.call("stage", Files.readString(root.resolve("deploy/compose.yaml")));
|
||||||
|
remote.call("stage-health", Files.readString(root.resolve("deploy/health.sh")));
|
||||||
|
String prepared = remote.call("prepare", null, version.toString(), commit, prefix());
|
||||||
|
return new ReleaseState.Deployment(
|
||||||
|
version, commit, digest(prepared, "BACKEND"), digest(prepared, "FRONTEND"), attempt, build);
|
||||||
|
}
|
||||||
|
|
||||||
|
String prefix() {
|
||||||
|
var uri = URI.create(api.server);
|
||||||
|
String authority = uri.getRawAuthority();
|
||||||
|
String registry = env.getOrDefault("DEPLOY_REGISTRY", "").strip();
|
||||||
|
if (registry.isEmpty()) registry = authority;
|
||||||
|
int port = uri.getPort() >= 0 ? uri.getPort() : uri.getScheme().equals("https") ? 443 : 80;
|
||||||
|
Pipeline.check(
|
||||||
|
registry.equals(authority) || registry.equals("127.0.0.1:" + port),
|
||||||
|
"Deployment registry must be the existing Gitea registry or its NAS loopback alias");
|
||||||
|
String owner = Setup.required(env, "CI_REPOSITORY").split("/")[0].toLowerCase(java.util.Locale.ROOT);
|
||||||
|
return registry + "/" + owner + "/" + Recipe.project(env) + "-";
|
||||||
|
}
|
||||||
|
|
||||||
|
void ensure(ReleaseState.Deployment release) throws Exception {
|
||||||
|
remote.call(
|
||||||
|
"ensure",
|
||||||
|
null,
|
||||||
|
release.directory(),
|
||||||
|
release.version().toString(),
|
||||||
|
release.sha(),
|
||||||
|
release.backend(),
|
||||||
|
release.frontend());
|
||||||
|
}
|
||||||
|
|
||||||
|
String digest(String response, String role) {
|
||||||
|
if (!Recipe.roles(env).contains(role.toLowerCase(java.util.Locale.ROOT))) return "-";
|
||||||
|
String value = response.lines()
|
||||||
|
.filter(line -> line.startsWith("S6_" + role + "="))
|
||||||
|
.map(line -> line.substring(("S6_" + role + "=").length()))
|
||||||
|
.reduce((a, b) -> {
|
||||||
|
throw new IllegalArgumentException("Ambiguous prepared image digest");
|
||||||
|
})
|
||||||
|
.orElseThrow(() -> new IllegalArgumentException("Missing prepared image digest"));
|
||||||
|
Pipeline.check(
|
||||||
|
value.startsWith(prefix() + role.toLowerCase(java.util.Locale.ROOT) + "@sha256:"),
|
||||||
|
"Unexpected image repository");
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
String recover(ReleaseState before) throws Exception {
|
||||||
|
if (before.current() == null) return "first-release-failed-manual-recovery-required";
|
||||||
|
try {
|
||||||
|
remote.call("apply", null, before.current().directory());
|
||||||
|
} catch (Exception failure) {
|
||||||
|
diagnose(before.current());
|
||||||
|
return "recovery-failed-manual-recovery-required";
|
||||||
|
}
|
||||||
|
record("failed-previous-release-restored");
|
||||||
|
remote.call("finish", null);
|
||||||
|
pending = false;
|
||||||
|
return "previous-release-restored";
|
||||||
|
}
|
||||||
|
|
||||||
|
void diagnose(ReleaseState.Deployment release) {
|
||||||
|
try {
|
||||||
|
Files.writeString(
|
||||||
|
artifacts.resolve(attempt + "-" + release.directory() + "-containers.log"),
|
||||||
|
remote.call("diagnose", null, release.directory()));
|
||||||
|
} catch (Exception ignored) {
|
||||||
|
System.err.println("Container diagnostics unavailable; use the NAS recovery runbook");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void record(String outcome) throws Exception {
|
||||||
|
var data = new LinkedHashMap<String, Object>();
|
||||||
|
data.put("attempt", attempt);
|
||||||
|
data.put("mode", mode);
|
||||||
|
data.put("outcome", outcome);
|
||||||
|
data.put("time", Instant.now().toString());
|
||||||
|
data.put("release", selected == null ? null : selected.json());
|
||||||
|
String json = Json.write(data) + "\n";
|
||||||
|
Files.writeString(artifacts.resolve(attempt + ".json"), json);
|
||||||
|
if (locked) remote.call("event", json);
|
||||||
|
}
|
||||||
|
}
|
||||||
124
resources/ci/ReleaseApi.java
Normal file
124
resources/ci/ReleaseApi.java
Normal file
@@ -0,0 +1,124 @@
|
|||||||
|
import java.net.URI;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Objects;
|
||||||
|
|
||||||
|
/** Gitea 1.25 API adapter. Never include response bodies or tokens in errors. */
|
||||||
|
class ReleaseApi {
|
||||||
|
record Tag(ReleaseVersion version, String sha, String object) {}
|
||||||
|
|
||||||
|
final String server, token, base;
|
||||||
|
|
||||||
|
ReleaseApi(Map<String, String> env) {
|
||||||
|
server = Setup.required(env, "GITEA_SERVER_URL");
|
||||||
|
var uri = URI.create(server);
|
||||||
|
Pipeline.check(
|
||||||
|
List.of("http", "https").contains(uri.getScheme())
|
||||||
|
&& uri.getUserInfo() == null
|
||||||
|
&& uri.getQuery() == null
|
||||||
|
&& uri.getFragment() == null
|
||||||
|
&& (uri.getPath().isEmpty() || uri.getPath().equals("/")),
|
||||||
|
"Gitea must use a server-root URL");
|
||||||
|
String repository = Setup.required(env, "CI_REPOSITORY");
|
||||||
|
Pipeline.check(repository.matches("[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+"), "Invalid repository name");
|
||||||
|
base = "/api/v1/repos/" + repository;
|
||||||
|
token = Setup.required(env, "RELEASE_TOKEN");
|
||||||
|
}
|
||||||
|
|
||||||
|
Object request(String method, String path, Object body) throws Exception {
|
||||||
|
return Setup.api(server, token, method, base + path, body);
|
||||||
|
}
|
||||||
|
|
||||||
|
String head() throws Exception {
|
||||||
|
return sha(
|
||||||
|
Json.object(Json.object(request("GET", "/branches/main", null)).get("commit"))
|
||||||
|
.get("id"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static String sha(Object value) {
|
||||||
|
String result = Objects.toString(value, "");
|
||||||
|
Pipeline.check(result.matches("[0-9a-f]{40}"), "Invalid release commit SHA");
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
List<Tag> tags() throws Exception {
|
||||||
|
List<Tag> tags = new ArrayList<>();
|
||||||
|
var seen = new java.util.HashSet<String>();
|
||||||
|
for (int page = 1; ; page++) {
|
||||||
|
var entries = (List<?>) request("GET", "/tags?limit=50&page=" + page, null);
|
||||||
|
if (entries.isEmpty()) return tags;
|
||||||
|
for (Object entry : entries) {
|
||||||
|
var tag = Json.object(entry);
|
||||||
|
String name = Objects.toString(tag.get("name"), "");
|
||||||
|
Pipeline.check(
|
||||||
|
seen.add(name),
|
||||||
|
"Remote tag pagination repeated a tag; cannot establish a safe version baseline");
|
||||||
|
// Unknown version-looking tags require explicit reconciliation, not a second baseline.
|
||||||
|
if (!name.matches("v?[0-9].*")) continue;
|
||||||
|
ReleaseVersion version = ReleaseVersion.parse(name);
|
||||||
|
String commit = sha(Json.object(tag.get("commit")).get("sha"));
|
||||||
|
String object = sha(tag.get("id"));
|
||||||
|
tags.add(new Tag(version, commit, object));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void annotated(Tag tag) throws Exception {
|
||||||
|
var data = Json.object(request("GET", "/git/tags/" + tag.object(), null));
|
||||||
|
Pipeline.check(
|
||||||
|
tag.version().toString().equals(data.get("tag"))
|
||||||
|
&& tag.sha().equals(Json.object(data.get("object")).get("sha")),
|
||||||
|
"Release tag must be annotated and point directly to its recorded commit");
|
||||||
|
}
|
||||||
|
|
||||||
|
void create(ReleaseVersion version, String commit, String run) throws Exception {
|
||||||
|
request(
|
||||||
|
"POST",
|
||||||
|
"/tags",
|
||||||
|
Map.of(
|
||||||
|
"tag_name",
|
||||||
|
version.toString(),
|
||||||
|
"target",
|
||||||
|
commit,
|
||||||
|
"message",
|
||||||
|
"Release " + version + "\nCommit: " + commit + "\nDeployment attempt: " + run));
|
||||||
|
Tag tag = tags().stream()
|
||||||
|
.filter(t -> t.version().equals(version))
|
||||||
|
.findFirst()
|
||||||
|
.orElseThrow();
|
||||||
|
Pipeline.check(tag.sha().equals(commit), "Created release tag does not match selected commit");
|
||||||
|
annotated(tag);
|
||||||
|
}
|
||||||
|
|
||||||
|
String verified(String commit) throws Exception {
|
||||||
|
// A successful build.yaml main-push run includes verification AND successful image publication.
|
||||||
|
// Do not accept another workflow, a branch build, or merely an existing image/tag.
|
||||||
|
var seen = new java.util.HashSet<String>();
|
||||||
|
for (int page = 1; ; page++) {
|
||||||
|
var response = Json.object(request(
|
||||||
|
"GET",
|
||||||
|
"/actions/runs?head_sha=" + commit + "&event=push&branch=main&status=success&limit=50&page=" + page,
|
||||||
|
null));
|
||||||
|
var runs = (List<?>) response.get("workflow_runs");
|
||||||
|
if (runs.isEmpty()) break;
|
||||||
|
boolean fresh = false;
|
||||||
|
for (Object entry : runs) {
|
||||||
|
var run = Json.object(entry);
|
||||||
|
fresh |= seen.add(Objects.toString(run.get("id")));
|
||||||
|
if (commit.equals(run.get("head_sha"))
|
||||||
|
&& "push".equals(run.get("event"))
|
||||||
|
&& List.of("main", "refs/heads/main").contains(Objects.toString(run.get("head_branch")))
|
||||||
|
&& "completed".equals(run.get("status"))
|
||||||
|
&& "success".equals(run.get("conclusion"))
|
||||||
|
&& "build.yaml@refs/heads/main".equals(run.get("path"))) {
|
||||||
|
return Objects.toString(run.get("id"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Older Gitea installations may ignore some list filters/pagination.
|
||||||
|
// Validate each entry locally and stop if a page makes no progress.
|
||||||
|
if (!fresh) break;
|
||||||
|
}
|
||||||
|
throw new IllegalArgumentException("No successful build.yaml main-push run verifies and publishes this SHA");
|
||||||
|
}
|
||||||
|
}
|
||||||
125
resources/ci/ReleaseRemote.java
Normal file
125
resources/ci/ReleaseRemote.java
Normal file
@@ -0,0 +1,125 @@
|
|||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.nio.file.attribute.PosixFilePermissions;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/** Argument-safe SSH adapter; server-side mkdir lock is shared by every runner. */
|
||||||
|
class ReleaseRemote implements AutoCloseable {
|
||||||
|
final Path root, credentials;
|
||||||
|
final Map<String, String> env;
|
||||||
|
final Commands commands;
|
||||||
|
final String owner, directory, host, user, port, script;
|
||||||
|
|
||||||
|
ReleaseRemote(Path root, Map<String, String> env, Commands commands, String owner) throws Exception {
|
||||||
|
this.root = root;
|
||||||
|
this.env = env;
|
||||||
|
this.commands = commands;
|
||||||
|
this.owner = owner;
|
||||||
|
directory = Setup.required(env, "DEPLOY_DIRECTORY");
|
||||||
|
host = Setup.required(env, "DEPLOY_HOST");
|
||||||
|
user = Setup.required(env, "DEPLOY_USER");
|
||||||
|
port = env.getOrDefault("DEPLOY_SSH_PORT", "22");
|
||||||
|
Pipeline.check(
|
||||||
|
directory.matches("/[a-zA-Z0-9_./-]+") && !directory.contains("..") && !directory.equals("/"),
|
||||||
|
"Use an absolute dedicated deployment directory without spaces or parent traversal");
|
||||||
|
Pipeline.check(
|
||||||
|
host.matches("[a-zA-Z0-9][a-zA-Z0-9.-]*")
|
||||||
|
&& user.matches("[a-zA-Z0-9_][a-zA-Z0-9_-]*")
|
||||||
|
&& port.matches("[0-9]{1,5}")
|
||||||
|
&& Integer.parseInt(port) > 0
|
||||||
|
&& Integer.parseInt(port) <= 65535,
|
||||||
|
"Invalid deployment SSH host, user or port");
|
||||||
|
script = "COMPOSE_PROJECT=" + quote(Setup.required(env, "COMPOSE_PROJECT")) + "\n"
|
||||||
|
+ "DEPLOY_ROLES=" + quote(String.join(" ", Recipe.roles(env))) + "\n"
|
||||||
|
+ "DEPENDENCY_SERVICES=" + quote(env.getOrDefault("DEPENDENCY_SERVICES", "")) + "\n"
|
||||||
|
+ "TARGET_PLATFORM=" + quote(env.getOrDefault("TARGET_PLATFORM", "linux/amd64")) + "\n"
|
||||||
|
+ "REQUIRED_SECRETS=" + quote(env.getOrDefault("REQUIRED_DEPLOY_SECRETS", "")) + "\n"
|
||||||
|
+ Files.readString(root.resolve("deploy/health.sh")) + "\n"
|
||||||
|
+ Files.readString(root.resolve("generated/deploy/remote.sh"));
|
||||||
|
credentials = Files.createTempDirectory("engineering-deploy-ssh-");
|
||||||
|
try {
|
||||||
|
writePrivate("identity", Setup.required(env, "DEPLOY_SSH_KEY") + "\n");
|
||||||
|
writePrivate("known_hosts", Setup.required(env, "DEPLOY_KNOWN_HOSTS") + "\n");
|
||||||
|
} catch (Exception e) {
|
||||||
|
close();
|
||||||
|
throw e;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void writePrivate(String name, String value) throws Exception {
|
||||||
|
Path file = credentials.resolve(name);
|
||||||
|
Files.createFile(file, PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString("rw-------")));
|
||||||
|
Files.writeString(file, value);
|
||||||
|
}
|
||||||
|
|
||||||
|
static String quote(String value) {
|
||||||
|
return "'" + value.replace("'", "'\"'\"'") + "'";
|
||||||
|
}
|
||||||
|
|
||||||
|
String call(String action, String input, String... arguments) throws Exception {
|
||||||
|
int seconds = switch (action) {
|
||||||
|
case "apply" -> 300;
|
||||||
|
case "prepare", "ensure" -> 600;
|
||||||
|
default -> 60;
|
||||||
|
};
|
||||||
|
var parameters = new ArrayList<>(List.of(action, directory, owner));
|
||||||
|
parameters.addAll(List.of(arguments));
|
||||||
|
return executeScript(script, input, seconds, parameters);
|
||||||
|
}
|
||||||
|
|
||||||
|
String executeScript(String source, String input, int seconds, List<String> parameters) throws Exception {
|
||||||
|
String remote = "timeout -k 5 " + seconds + " sh -c " + quote(source) + " --";
|
||||||
|
for (String argument : parameters) remote += " " + quote(argument);
|
||||||
|
var args = new ArrayList<>(List.of(
|
||||||
|
"ssh",
|
||||||
|
"-F",
|
||||||
|
"/dev/null",
|
||||||
|
"-T",
|
||||||
|
"-o",
|
||||||
|
"BatchMode=yes",
|
||||||
|
"-o",
|
||||||
|
"StrictHostKeyChecking=yes",
|
||||||
|
"-o",
|
||||||
|
"IdentitiesOnly=yes",
|
||||||
|
"-o",
|
||||||
|
"IdentityAgent=none",
|
||||||
|
"-o",
|
||||||
|
"ForwardAgent=no",
|
||||||
|
"-o",
|
||||||
|
"ConnectTimeout=10",
|
||||||
|
"-o",
|
||||||
|
"ServerAliveInterval=10",
|
||||||
|
"-o",
|
||||||
|
"ServerAliveCountMax=3",
|
||||||
|
"-o",
|
||||||
|
"LogLevel=ERROR",
|
||||||
|
"-o",
|
||||||
|
"UserKnownHostsFile=" + credentials.resolve("known_hosts"),
|
||||||
|
"-o",
|
||||||
|
"GlobalKnownHostsFile=/dev/null",
|
||||||
|
"-i",
|
||||||
|
credentials.resolve("identity").toString(),
|
||||||
|
"-p",
|
||||||
|
port,
|
||||||
|
user + "@" + host,
|
||||||
|
remote));
|
||||||
|
commands.renew(Duration.ofSeconds(seconds + 30));
|
||||||
|
return commands.execute(args, root, env, input, false).checked();
|
||||||
|
}
|
||||||
|
|
||||||
|
void acquire() throws Exception {
|
||||||
|
call("lock", null);
|
||||||
|
}
|
||||||
|
|
||||||
|
void release() throws Exception {
|
||||||
|
call("unlock", null);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void close() throws Exception {
|
||||||
|
Pipeline.deleteTree(credentials);
|
||||||
|
}
|
||||||
|
}
|
||||||
105
resources/ci/ReleaseSetup.java
Normal file
105
resources/ci/ReleaseSetup.java
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
import java.net.URI;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/** One-time provisioning; secrets stay in owner-only files or Gitea secret storage. */
|
||||||
|
class ReleaseSetup {
|
||||||
|
static void configure(Path root, Map<String, String> input) throws Exception {
|
||||||
|
var env = new HashMap<>(input);
|
||||||
|
env.putIfAbsent("DEPLOY_SSH_PORT", "22");
|
||||||
|
Path config = Setup.home().resolve(".config/" + Recipe.project(env) + "-runner");
|
||||||
|
env.putIfAbsent("GITEA_ADMIN_TOKEN_FILE", config.resolve("admin-token").toString());
|
||||||
|
env.putIfAbsent("RELEASE_TOKEN_FILE", config.resolve("release-token").toString());
|
||||||
|
env.putIfAbsent("PULL_TOKEN_FILE", config.resolve("pull-token").toString());
|
||||||
|
env.putIfAbsent("DEPLOY_SSH_KEY_FILE", config.resolve("deploy-key").toString());
|
||||||
|
String admin = Setup.secret(env, "GITEA_ADMIN_TOKEN_FILE");
|
||||||
|
String pull = Setup.secret(env, "PULL_TOKEN_FILE");
|
||||||
|
env.put("RELEASE_TOKEN", Setup.secret(env, "RELEASE_TOKEN_FILE"));
|
||||||
|
env.put("DEPLOY_SSH_KEY", Setup.secret(env, "DEPLOY_SSH_KEY_FILE"));
|
||||||
|
String bind = env.get("DEPLOY_BIND_IP"), port = env.get("DEPLOY_FRONTEND_PORT");
|
||||||
|
Pipeline.check(
|
||||||
|
bind.matches("[0-9.]+")
|
||||||
|
&& port.matches("[0-9]{4,5}")
|
||||||
|
&& Integer.parseInt(port) >= 1024
|
||||||
|
&& Integer.parseInt(port) <= 65535,
|
||||||
|
"Invalid private bind address or port");
|
||||||
|
var commands = new Commands(Duration.ofMinutes(15));
|
||||||
|
Path diagnostics = root.resolve("ci/artifacts/deployment");
|
||||||
|
Files.createDirectories(diagnostics);
|
||||||
|
commands.log = diagnostics.resolve("setup.log");
|
||||||
|
Files.writeString(commands.log, "");
|
||||||
|
String host = env.get("DEPLOY_HOST");
|
||||||
|
String lookup = env.get("DEPLOY_SSH_PORT").equals("22") ? host : "[" + host + "]:" + env.get("DEPLOY_SSH_PORT");
|
||||||
|
env.put(
|
||||||
|
"DEPLOY_KNOWN_HOSTS",
|
||||||
|
commands.execute(
|
||||||
|
List.of(
|
||||||
|
"ssh-keygen",
|
||||||
|
"-F",
|
||||||
|
lookup,
|
||||||
|
"-f",
|
||||||
|
Setup.home().resolve(".ssh/known_hosts").toString()),
|
||||||
|
root,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.checked());
|
||||||
|
Pipeline.check(!env.get("DEPLOY_KNOWN_HOSTS").isBlank(), "A previously trusted NAS hostkey is required");
|
||||||
|
var api = new ReleaseApi(env);
|
||||||
|
api.verified(api.head());
|
||||||
|
var uri = URI.create(api.server);
|
||||||
|
String registry = env.get("DEPLOY_REGISTRY");
|
||||||
|
int registryPort = uri.getPort() >= 0 ? uri.getPort() : uri.getScheme().equals("https") ? 443 : 80;
|
||||||
|
Pipeline.check(
|
||||||
|
registry.equals(uri.getRawAuthority()) || registry.equals("127.0.0.1:" + registryPort),
|
||||||
|
"Deployment registry must be the Gitea registry or its NAS loopback alias");
|
||||||
|
String registryUser =
|
||||||
|
env.getOrDefault("PULL_USER", env.get("CI_REPOSITORY").split("/")[0]);
|
||||||
|
Pipeline.check(registryUser.matches("[a-zA-Z0-9_.-]+"), "Invalid registry user");
|
||||||
|
try (var remote = new ReleaseRemote(root, env, commands, "setup")) {
|
||||||
|
System.out.println(remote.executeScript(
|
||||||
|
Files.readString(root.resolve("generated/deploy/provision.sh")),
|
||||||
|
pull + "\n",
|
||||||
|
300,
|
||||||
|
List.of(
|
||||||
|
remote.directory,
|
||||||
|
registryUser,
|
||||||
|
registry,
|
||||||
|
bind,
|
||||||
|
port,
|
||||||
|
Setup.required(env, "DEPLOY_DOCKER_BIN"))));
|
||||||
|
remote.acquire();
|
||||||
|
try {
|
||||||
|
remote.call("preflight", null);
|
||||||
|
} finally {
|
||||||
|
remote.release();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Provision credentials only after successful NAS login and release-read verification.
|
||||||
|
for (String secret : List.of("RELEASE_TOKEN", "DEPLOY_SSH_KEY", "DEPLOY_KNOWN_HOSTS"))
|
||||||
|
Setup.api(
|
||||||
|
api.server, admin, "PUT", api.base + "/actions/secrets/" + secret, Map.of("data", env.get(secret)));
|
||||||
|
for (String variable :
|
||||||
|
List.of("DEPLOY_HOST", "DEPLOY_USER", "DEPLOY_SSH_PORT", "DEPLOY_DIRECTORY", "DEPLOY_REGISTRY"))
|
||||||
|
variable(api, admin, variable, env.get(variable));
|
||||||
|
System.out.println(
|
||||||
|
"Production configuration and Gitea deployment secrets/variables installed; no release was created.");
|
||||||
|
}
|
||||||
|
|
||||||
|
static void variable(ReleaseApi api, String admin, String name, String value) throws Exception {
|
||||||
|
String path = api.base + "/actions/variables/" + name;
|
||||||
|
boolean exists;
|
||||||
|
try {
|
||||||
|
Setup.api(api.server, admin, "GET", path, null);
|
||||||
|
exists = true;
|
||||||
|
} catch (Setup.ApiFailure error) {
|
||||||
|
if (error.status != 404) throw error;
|
||||||
|
exists = false;
|
||||||
|
}
|
||||||
|
Setup.api(api.server, admin, exists ? "PUT" : "POST", path, Map.of("value", value));
|
||||||
|
}
|
||||||
|
}
|
||||||
244
resources/ci/ReleaseShellTests.java
Normal file
244
resources/ci/ReleaseShellTests.java
Normal file
@@ -0,0 +1,244 @@
|
|||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.nio.file.attribute.PosixFilePermissions;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/** Runs the actual NAS script with disposable command adapters, no Docker daemon. */
|
||||||
|
class ReleaseShellTests {
|
||||||
|
static void run() throws Exception {
|
||||||
|
ReleaseTests.test("actual NAS script prepares digest images and checks the published endpoint", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.prepare();
|
||||||
|
f.call("pending", "{}", "one").checked();
|
||||||
|
CiTests.equal("S6_HEALTHY", f.call("apply", null, "one", "one").checked());
|
||||||
|
String calls = Files.readString(f.directory.resolve("calls"));
|
||||||
|
CiTests.equal(true, calls.contains("http://backend:8080/q/health/ready"));
|
||||||
|
CiTests.equal(true, calls.contains("http://127.0.0.1:8088/api/items"));
|
||||||
|
CiTests.equal(false, calls.contains(" down"));
|
||||||
|
CiTests.equal(false, calls.contains("prune"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script rejects a missing database password before pulling images", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
Path config = f.directory.resolve("production.env");
|
||||||
|
Files.writeString(
|
||||||
|
config, Files.readString(config).replace("DATABASE_PASSWORD=fixture-only-password\n", ""));
|
||||||
|
f.call("lock", null, "one").checked();
|
||||||
|
Commands.Result result = f.call("preflight", null, "one");
|
||||||
|
CiTests.equal(1, result.code());
|
||||||
|
CiTests.equal(true, result.output().contains("Missing DATABASE_PASSWORD"));
|
||||||
|
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script passes database configuration through sudo", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.executable("sudo", "#!/bin/sh\n[ \"$1\" != -n ] || shift\nunset DATABASE_PASSWORD\nexec \"$@\"\n");
|
||||||
|
Path config = f.directory.resolve("production.env");
|
||||||
|
Files.writeString(config, Files.readString(config).replace("DOCKER_SUDO=0", "DOCKER_SUDO=1"));
|
||||||
|
f.prepare();
|
||||||
|
CiTests.equal(
|
||||||
|
false,
|
||||||
|
Files.readString(f.directory.resolve("releases/one/release.env"))
|
||||||
|
.contains("PASSWORD"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script rejects wrong image revision before replacement", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.env.put("WRONG_REVISION", "1");
|
||||||
|
CiTests.failure(f::prepare);
|
||||||
|
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script rejects unavailable image before replacement", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.env.put("FAIL_PULL", "1");
|
||||||
|
CiTests.failure(f::prepare);
|
||||||
|
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script rejects a changed retained release record", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.prepare();
|
||||||
|
CiTests.failure(() -> f.call(
|
||||||
|
"ensure", null, "one", "one", "v9.0.0", ReleaseTests.SHA, f.backend(), f.frontend())
|
||||||
|
.checked());
|
||||||
|
f.call("ensure", null, "one", "one", "v1.0.0", ReleaseTests.SHA, f.backend(), f.frontend())
|
||||||
|
.checked();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS replacement timeout is bounded and leaves recovery journal", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.prepare();
|
||||||
|
f.call("pending", "{}", "one").checked();
|
||||||
|
f.env.put("FAIL_HEALTH", "1");
|
||||||
|
CiTests.equal(1, f.call("apply", null, "one", "one", "1").code());
|
||||||
|
CiTests.equal(true, Files.exists(f.directory.resolve("pending.json")));
|
||||||
|
// Termination releases only the operation fence, not the deployment lock.
|
||||||
|
CiTests.equal(false, Files.exists(f.directory.resolve("deploy.lock/operation")));
|
||||||
|
CiTests.equal(true, Files.exists(f.directory.resolve("deploy.lock/owner")));
|
||||||
|
f.env.remove("FAIL_HEALTH");
|
||||||
|
f.call("apply", null, "one", "one").checked();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script fences overlapping remote replacements", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.prepare();
|
||||||
|
Files.createDirectory(f.directory.resolve("deploy.lock/operation"));
|
||||||
|
CiTests.failure(() -> f.call("apply", null, "one", "one").checked());
|
||||||
|
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ReleaseTests.test("actual NAS script stores state atomically and archives resolved journal", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.call("lock", null, "one").checked();
|
||||||
|
f.call("pending", "{\"attempt\":\"one\"}", "one").checked();
|
||||||
|
f.call("commit", "{\"current\":\"one\"}", "one").checked();
|
||||||
|
CiTests.equal(
|
||||||
|
"{\"current\":\"one\"}", f.call("state", null, "one").checked());
|
||||||
|
f.call("event", "{\"outcome\":\"success\"}", "one").checked();
|
||||||
|
f.call("finish", null, "one").checked();
|
||||||
|
f.call("unlock", null, "one").checked();
|
||||||
|
CiTests.equal(true, Files.exists(f.directory.resolve("history/one.json")));
|
||||||
|
CiTests.equal(false, Files.exists(f.directory.resolve("pending.json")));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
static class Fixture implements AutoCloseable {
|
||||||
|
final Path directory = Files.createTempDirectory("nas-script-test-");
|
||||||
|
final Map<String, String> env = new HashMap<>(System.getenv());
|
||||||
|
final String health = "application_health() {\n"
|
||||||
|
+ "frontend=$(compose ps -q frontend)\n"
|
||||||
|
+ "docker_cmd exec \"$frontend\" wget -q -T 5 -O /dev/null http://backend:8080/q/health/ready || return"
|
||||||
|
+ " 1\n"
|
||||||
|
+ "curl --fail http://127.0.0.1:8088/api/items -o /dev/null\n"
|
||||||
|
+ "}\n";
|
||||||
|
final String script = "COMPOSE_PROJECT=recipe-demo-test\n"
|
||||||
|
+ "DEPLOY_ROLES='backend frontend'\n"
|
||||||
|
+ "DEPENDENCY_SERVICES=database\n"
|
||||||
|
+ "TARGET_PLATFORM=linux/amd64\n"
|
||||||
|
+ "REQUIRED_SECRETS=DATABASE_PASSWORD\n"
|
||||||
|
+ health + "\n"
|
||||||
|
+ Files.readString(Path.of("generated/deploy/remote.sh"));
|
||||||
|
final Commands commands = new Commands(Duration.ofMinutes(1));
|
||||||
|
|
||||||
|
Fixture() throws Exception {
|
||||||
|
Files.createDirectory(directory.resolve("bin"));
|
||||||
|
executable("docker", DOCKER);
|
||||||
|
executable(
|
||||||
|
"curl",
|
||||||
|
"#!/bin/sh\nprintf 'curl %s\\n' \"$*\" >> \"$FIXTURE/calls\"\n[ \"${FAIL_HEALTH:-0}\" = 0 ]\n");
|
||||||
|
env.put("PATH", directory.resolve("bin") + ":" + env.get("PATH"));
|
||||||
|
env.put("FIXTURE", directory.toString());
|
||||||
|
Files.createDirectory(directory.resolve("docker-auth"));
|
||||||
|
Files.writeString(directory.resolve("docker-auth/config.json"), "{}");
|
||||||
|
Files.writeString(
|
||||||
|
directory.resolve("production.env"),
|
||||||
|
"DATABASE_PASSWORD=fixture-only-password\n"
|
||||||
|
+ "BIND_IP=127.0.0.1\n"
|
||||||
|
+ "FRONTEND_PORT=8088\n"
|
||||||
|
+ "DOCKER_SUDO=0\n"
|
||||||
|
+ "DOCKER_BIN="
|
||||||
|
+ directory.resolve("bin/docker") + "\nDOCKER_CONFIG=" + directory.resolve("docker-auth")
|
||||||
|
+ "\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
void executable(String name, String body) throws Exception {
|
||||||
|
Path path = directory.resolve("bin").resolve(name);
|
||||||
|
Files.writeString(path, body);
|
||||||
|
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rwx------"));
|
||||||
|
}
|
||||||
|
|
||||||
|
Commands.Result call(String action, String input, String owner, String... args) throws Exception {
|
||||||
|
var command = new ArrayList<>(
|
||||||
|
List.of("timeout", "-k", "1", "10", "sh", "-c", script, "--", action, directory.toString(), owner));
|
||||||
|
command.addAll(List.of(args));
|
||||||
|
commands.renew(Duration.ofSeconds(15));
|
||||||
|
return commands.execute(command, directory, env, input, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
String backend() {
|
||||||
|
return "nas:3000/owner/recipe-demo-backend@sha256:" + "1".repeat(64);
|
||||||
|
}
|
||||||
|
|
||||||
|
String frontend() {
|
||||||
|
return "nas:3000/owner/recipe-demo-frontend@sha256:" + "2".repeat(64);
|
||||||
|
}
|
||||||
|
|
||||||
|
void prepare() throws Exception {
|
||||||
|
call("lock", null, "one").checked();
|
||||||
|
call("preflight", null, "one").checked();
|
||||||
|
call("stage", "services: {}\n", "one").checked();
|
||||||
|
call("stage-health", health, "one").checked();
|
||||||
|
String result = call("prepare", null, "one", "v1.0.0", ReleaseTests.SHA, "nas:3000/owner/recipe-demo-")
|
||||||
|
.checked();
|
||||||
|
CiTests.equal(true, result.contains("S6_BACKEND=" + backend()));
|
||||||
|
CiTests.equal(true, result.contains("S6_FRONTEND=" + frontend()));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void close() throws Exception {
|
||||||
|
Pipeline.deleteTree(directory);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static final String DOCKER = """
|
||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
printf 'docker %s\n' "$*" >> "$FIXTURE/calls"
|
||||||
|
[ "$1" != --config ] || shift 2
|
||||||
|
case "$1" in
|
||||||
|
info) echo linux/x86_64;;
|
||||||
|
pull) [ "${FAIL_PULL:-0}" = 0 ];;
|
||||||
|
image)
|
||||||
|
format=$4; image=$5
|
||||||
|
case "$image" in *backend*) role=backend; digit=1;; *) role=frontend; digit=2;; esac
|
||||||
|
case "$format" in
|
||||||
|
*revision*)
|
||||||
|
if [ "${WRONG_REVISION:-0}" = 1 ]; then echo wrong; else printf '%040d\n' 0 | tr 0 a; fi;;
|
||||||
|
*Architecture*) echo linux/amd64;;
|
||||||
|
*RepoDigests*)
|
||||||
|
printf '%s@sha256:' "${image%:*}"; printf '%064d\n' 0 | tr 0 "$digit";;
|
||||||
|
*Id*) echo "sha256:$role";;
|
||||||
|
*) exit 1;;
|
||||||
|
esac;;
|
||||||
|
compose)
|
||||||
|
shift
|
||||||
|
[ "$1" != version ] || exit 0
|
||||||
|
shift 4
|
||||||
|
[ "$1" = --env-file ] || exit 1
|
||||||
|
production=$2
|
||||||
|
shift 2
|
||||||
|
[ "$1" = --env-file ] || exit 1
|
||||||
|
file=$2
|
||||||
|
shift 4
|
||||||
|
# Compose must read the secret from disk, even after sudo clears the environment.
|
||||||
|
test -f "$production"
|
||||||
|
. "$production"
|
||||||
|
[ "$DATABASE_PASSWORD" = fixture-only-password ] || exit 15
|
||||||
|
case "$1" in
|
||||||
|
config) test -f "$file";;
|
||||||
|
up) cp "$file" "$FIXTURE/live.env";;
|
||||||
|
ps) eval 'role=${'"$#"'}'; echo "$role";;
|
||||||
|
logs) echo 'sample container diagnostics';;
|
||||||
|
*) exit 1;;
|
||||||
|
esac;;
|
||||||
|
inspect)
|
||||||
|
. "$FIXTURE/live.env"
|
||||||
|
format=$3; id=$4
|
||||||
|
case "$format" in
|
||||||
|
*State.Running*) echo true;;
|
||||||
|
*Image*) echo "sha256:$id";;
|
||||||
|
*revision*) echo "$RELEASE_SHA";;
|
||||||
|
*release*) echo "$RELEASE_VERSION";;
|
||||||
|
*) exit 1;;
|
||||||
|
esac;;
|
||||||
|
exec) [ "${FAIL_HEALTH:-0}" = 0 ];;
|
||||||
|
*) exit 1;;
|
||||||
|
esac
|
||||||
|
""";
|
||||||
|
}
|
||||||
83
resources/ci/ReleaseState.java
Normal file
83
resources/ci/ReleaseState.java
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
import java.util.LinkedHashMap;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Objects;
|
||||||
|
|
||||||
|
/** Successful deployment state is separate from immutable allocated version tags. */
|
||||||
|
record ReleaseState(ReleaseState.Deployment current, ReleaseState.Deployment previous, ReleaseVersion highest) {
|
||||||
|
record Deployment(
|
||||||
|
ReleaseVersion version, String sha, String backend, String frontend, String directory, String build) {
|
||||||
|
Deployment {
|
||||||
|
ReleaseApi.sha(sha);
|
||||||
|
Pipeline.check(directory.matches("[a-zA-Z0-9-]+"), "Invalid retained release directory");
|
||||||
|
Pipeline.check(!backend.equals("-") || !frontend.equals("-"), "A release must contain an image");
|
||||||
|
for (String image : new String[] {backend, frontend})
|
||||||
|
Pipeline.check(
|
||||||
|
(image.equals("-") || image.matches("[a-zA-Z0-9.:/_-]+@sha256:[0-9a-f]{64}")),
|
||||||
|
"Invalid retained image digest");
|
||||||
|
}
|
||||||
|
|
||||||
|
Map<String, Object> json() {
|
||||||
|
return Map.of(
|
||||||
|
"version",
|
||||||
|
version.toString(),
|
||||||
|
"sha",
|
||||||
|
sha,
|
||||||
|
"backend",
|
||||||
|
backend,
|
||||||
|
"frontend",
|
||||||
|
frontend,
|
||||||
|
"directory",
|
||||||
|
directory,
|
||||||
|
"build",
|
||||||
|
build);
|
||||||
|
}
|
||||||
|
|
||||||
|
static Deployment parse(Object value) {
|
||||||
|
if (value == null) return null;
|
||||||
|
var data = Json.object(value);
|
||||||
|
return new Deployment(
|
||||||
|
ReleaseVersion.parse(text(data, "version")),
|
||||||
|
text(data, "sha"),
|
||||||
|
text(data, "backend"),
|
||||||
|
text(data, "frontend"),
|
||||||
|
text(data, "directory"),
|
||||||
|
text(data, "build"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static String text(Map<String, Object> data, String key) {
|
||||||
|
return Objects.toString(data.get(key), "");
|
||||||
|
}
|
||||||
|
|
||||||
|
static ReleaseState parse(String value) {
|
||||||
|
var data = Json.object(Json.parse(value));
|
||||||
|
if (data.isEmpty()) return new ReleaseState(null, null, null);
|
||||||
|
Pipeline.check("1".equals(text(data, "schema")), "Unsupported deployment state schema");
|
||||||
|
Deployment current = Deployment.parse(data.get("current"));
|
||||||
|
Deployment previous = Deployment.parse(data.get("previous"));
|
||||||
|
ReleaseVersion highest = ReleaseVersion.parse(text(data, "highest"));
|
||||||
|
Pipeline.check(
|
||||||
|
current != null
|
||||||
|
&& highest.compareTo(current.version()) >= 0
|
||||||
|
&& (previous == null || highest.compareTo(previous.version()) >= 0),
|
||||||
|
"Inconsistent successful release state");
|
||||||
|
return new ReleaseState(current, previous, highest);
|
||||||
|
}
|
||||||
|
|
||||||
|
String json() {
|
||||||
|
var data = new LinkedHashMap<String, Object>();
|
||||||
|
data.put("schema", "1");
|
||||||
|
data.put("current", current == null ? null : current.json());
|
||||||
|
data.put("previous", previous == null ? null : previous.json());
|
||||||
|
data.put("highest", highest == null ? null : highest.toString());
|
||||||
|
return Json.write(data) + "\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
ReleaseState success(Deployment selected) {
|
||||||
|
ReleaseVersion watermark =
|
||||||
|
highest == null || selected.version().compareTo(highest) > 0 ? selected.version() : highest;
|
||||||
|
// A redeploy of the same release must not discard the distinct rollback target.
|
||||||
|
Deployment fallback = current != null && current.version().equals(selected.version()) ? previous : current;
|
||||||
|
return new ReleaseState(selected, fallback, watermark);
|
||||||
|
}
|
||||||
|
}
|
||||||
555
resources/ci/ReleaseTests.java
Normal file
555
resources/ci/ReleaseTests.java
Normal file
@@ -0,0 +1,555 @@
|
|||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/** Release state-machine and real POSIX remote-script tests, without server writes. */
|
||||||
|
class ReleaseTests {
|
||||||
|
static final String SHA = "a".repeat(40), OLD = "b".repeat(40);
|
||||||
|
static int count;
|
||||||
|
|
||||||
|
static void test(String name, CiTests.Test test) throws Exception {
|
||||||
|
test.run();
|
||||||
|
count++;
|
||||||
|
System.out.println("PASS release: " + name);
|
||||||
|
}
|
||||||
|
|
||||||
|
static Map<String, String> environment() {
|
||||||
|
var env = new HashMap<>(System.getenv());
|
||||||
|
env.putAll(Map.of(
|
||||||
|
"CI_REF",
|
||||||
|
"refs/heads/main",
|
||||||
|
"CI_EVENT",
|
||||||
|
"workflow_dispatch",
|
||||||
|
"CI_RUN_ID",
|
||||||
|
"123",
|
||||||
|
"CI_COMMIT",
|
||||||
|
SHA,
|
||||||
|
"RELEASE_MODE",
|
||||||
|
"new",
|
||||||
|
"RELEASE_INCREMENT",
|
||||||
|
"patch",
|
||||||
|
"RELEASE_VERSION",
|
||||||
|
"",
|
||||||
|
"GITEA_SERVER_URL",
|
||||||
|
"http://nas:3000",
|
||||||
|
"CI_REPOSITORY",
|
||||||
|
"owner/recipe-demo",
|
||||||
|
"RELEASE_TOKEN",
|
||||||
|
"test"));
|
||||||
|
env.putAll(Map.of(
|
||||||
|
"DEPLOY_HOST",
|
||||||
|
"nas",
|
||||||
|
"DEPLOY_USER",
|
||||||
|
"deploy",
|
||||||
|
"DEPLOY_DIRECTORY",
|
||||||
|
"/tmp/production",
|
||||||
|
"DEPLOY_SSH_KEY",
|
||||||
|
"test-private-key",
|
||||||
|
"DEPLOY_KNOWN_HOSTS",
|
||||||
|
"test-pinned-host"));
|
||||||
|
return env;
|
||||||
|
}
|
||||||
|
|
||||||
|
static ReleaseState.Deployment deployment(String version, String sha, String directory) {
|
||||||
|
return new ReleaseState.Deployment(
|
||||||
|
ReleaseVersion.parse(version),
|
||||||
|
sha,
|
||||||
|
"nas:3000/owner/recipe-demo-backend@sha256:" + "1".repeat(64),
|
||||||
|
"nas:3000/owner/recipe-demo-frontend@sha256:" + "2".repeat(64),
|
||||||
|
directory,
|
||||||
|
"10");
|
||||||
|
}
|
||||||
|
|
||||||
|
static ReleaseState prior() {
|
||||||
|
return new ReleaseState(
|
||||||
|
deployment("v1.0.1", OLD, "previous-current"),
|
||||||
|
deployment("v1.0.0", OLD, "previous-old"),
|
||||||
|
ReleaseVersion.parse("v1.0.1"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static void run() throws Exception {
|
||||||
|
count = 0;
|
||||||
|
test("first release and all increments reset the correct components", () -> {
|
||||||
|
CiTests.equal("v1.0.0", ReleaseVersion.next(List.of(), "major").toString());
|
||||||
|
var versions = List.of(ReleaseVersion.parse("v1.2.3"), ReleaseVersion.parse("v1.1.9"));
|
||||||
|
CiTests.equal("v1.2.4", ReleaseVersion.next(versions, "patch").toString());
|
||||||
|
CiTests.equal("v1.3.0", ReleaseVersion.next(versions, "minor").toString());
|
||||||
|
CiTests.equal("v2.0.0", ReleaseVersion.next(versions, "major").toString());
|
||||||
|
CiTests.equal(
|
||||||
|
"v1.0.100000000000000000000",
|
||||||
|
ReleaseVersion.next(List.of(ReleaseVersion.parse("v1.0.99999999999999999999")), "patch")
|
||||||
|
.toString());
|
||||||
|
for (String invalid : List.of("1.0.0", "v01.0.0", "v1.0.0-rc1", "v1.0.0+build", "v1.0", "v-1.0.0"))
|
||||||
|
CiTests.failure(() -> ReleaseVersion.parse(invalid));
|
||||||
|
});
|
||||||
|
test("branch, nonmanual, malformed input and conflicting new-release inputs are rejected", () -> {
|
||||||
|
var env = environment();
|
||||||
|
Release.validate(env, "deploy");
|
||||||
|
env.put("CI_REF", "refs/heads/feature/S6");
|
||||||
|
CiTests.failure(() -> Release.validate(env, "deploy"));
|
||||||
|
env.put("CI_REF", "refs/heads/main");
|
||||||
|
env.put("CI_EVENT", "push");
|
||||||
|
CiTests.failure(() -> Release.validate(env, "rollback"));
|
||||||
|
env.put("CI_EVENT", "workflow_dispatch");
|
||||||
|
env.put("RELEASE_VERSION", "v1.0.0");
|
||||||
|
CiTests.failure(() -> Release.validate(env, "deploy"));
|
||||||
|
env.put("RELEASE_MODE", "retry");
|
||||||
|
Release.validate(env, "deploy");
|
||||||
|
env.put("RELEASE_VERSION", "bad");
|
||||||
|
CiTests.failure(() -> Release.validate(env, "deploy"));
|
||||||
|
});
|
||||||
|
test("successful release stages images before tag, checks main again and commits state", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.release.run();
|
||||||
|
CiTests.equal("v1.0.0", f.remote.state.current().version().toString());
|
||||||
|
CiTests.equal(false, f.remote.locked);
|
||||||
|
CiTests.equal(false, f.remote.pending);
|
||||||
|
CiTests.equal(true, f.events.indexOf("prepare") < f.events.indexOf("tag"));
|
||||||
|
CiTests.equal(true, f.events.indexOf("tag") < f.events.lastIndexOf("head"));
|
||||||
|
CiTests.equal(true, f.events.lastIndexOf("head") < f.events.indexOf("apply:attempt"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("unverified commit never creates a tag or replaces production", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.api.verified = false;
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("tag"));
|
||||||
|
CiTests.equal(false, f.events.contains("apply:attempt"));
|
||||||
|
CiTests.equal(false, f.remote.locked);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("missing images prevent tag allocation and replacement", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.remote.failAction = "prepare";
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("tag"));
|
||||||
|
CiTests.equal(false, f.remote.pending);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("a tagged commit cannot receive another new version", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.api.tags.add(new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.0"), SHA, OLD));
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("prepare"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("stale dispatch is rejected before pulling or tagging", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.api.head = OLD;
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("prepare"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("main advancing after tag creation preserves tag without changing production", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.api.advanceAfterTag = true;
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(1, f.api.tags.size());
|
||||||
|
CiTests.equal(false, f.remote.pending);
|
||||||
|
CiTests.equal(false, f.events.contains("apply:attempt"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("unavailable recovery images block replacement", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.withPrior();
|
||||||
|
f.remote.failAction = "ensure";
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("tag"));
|
||||||
|
CiTests.equal(false, f.remote.pending);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("failed candidate restores prior version and still fails the workflow", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.withPrior();
|
||||||
|
f.remote.failedDirectories.add("attempt");
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(
|
||||||
|
List.of("apply:attempt", "apply:previous-current"),
|
||||||
|
f.events.stream().filter(e -> e.startsWith("apply:")).toList());
|
||||||
|
CiTests.equal(prior(), f.remote.state);
|
||||||
|
CiTests.equal(false, f.remote.pending);
|
||||||
|
CiTests.equal(false, f.remote.locked);
|
||||||
|
CiTests.equal(2, f.api.tags.size());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("failed first release retains journal and lock for manual recovery", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.remote.failedDirectories.add("attempt");
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(true, f.remote.pending);
|
||||||
|
CiTests.equal(true, f.remote.locked);
|
||||||
|
CiTests.equal(null, f.remote.state.current());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("failed automatic recovery retains journal and lock without retry loop", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.withPrior();
|
||||||
|
f.remote.failedDirectories.addAll(List.of("attempt", "previous-current"));
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(
|
||||||
|
2L,
|
||||||
|
f.events.stream().filter(e -> e.startsWith("apply:")).count());
|
||||||
|
CiTests.equal(true, f.remote.pending);
|
||||||
|
CiTests.equal(true, f.remote.locked);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("ambiguous state commit never launches speculative rollback", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.withPrior();
|
||||||
|
f.remote.failAction = "commit";
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(
|
||||||
|
1L,
|
||||||
|
f.events.stream().filter(e -> e.startsWith("apply:")).count());
|
||||||
|
CiTests.equal(true, f.remote.pending);
|
||||||
|
CiTests.equal(true, f.remote.locked);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("retry keeps existing version and SHA after main advances", () -> {
|
||||||
|
try (var f = new Fixture("retry")) {
|
||||||
|
f.api.head = OLD;
|
||||||
|
f.release.run();
|
||||||
|
CiTests.equal(SHA, f.remote.state.current().sha());
|
||||||
|
CiTests.equal("v1.0.0", f.remote.state.current().version().toString());
|
||||||
|
CiTests.equal(false, f.events.contains("tag"));
|
||||||
|
CiTests.equal(true, f.events.contains("annotated"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("retry cannot cross the successful-version high watermark after rollback", () -> {
|
||||||
|
try (var f = new Fixture("retry")) {
|
||||||
|
f.remote.state = prior().success(prior().previous());
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("apply:attempt"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("manual rollback preserves version baseline and creates no tag", () -> {
|
||||||
|
try (var f = new Fixture("rollback")) {
|
||||||
|
f.remote.state = prior();
|
||||||
|
f.release.run();
|
||||||
|
CiTests.equal("v1.0.0", f.remote.state.current().version().toString());
|
||||||
|
CiTests.equal("v1.0.1", f.remote.state.previous().version().toString());
|
||||||
|
CiTests.equal("v1.0.1", f.remote.state.highest().toString());
|
||||||
|
CiTests.equal(false, f.events.contains("tag"));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("redeploying current version preserves a distinct rollback target", () -> {
|
||||||
|
var state = prior();
|
||||||
|
CiTests.equal(state.previous(), state.success(state.current()).previous());
|
||||||
|
CiTests.equal(state, ReleaseState.parse(state.json()));
|
||||||
|
});
|
||||||
|
test("NAS lock contention cannot allocate tags or replace production", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.remote.locked = true;
|
||||||
|
CiTests.failure(f.release::run);
|
||||||
|
CiTests.equal(false, f.events.contains("tag"));
|
||||||
|
CiTests.equal(true, f.remote.locked);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
apiTests();
|
||||||
|
shellTests();
|
||||||
|
ReleaseShellTests.run();
|
||||||
|
System.out.println(count + " release behavior tests passed");
|
||||||
|
}
|
||||||
|
|
||||||
|
static class FakeApi extends ReleaseApi {
|
||||||
|
final List<String> events;
|
||||||
|
final List<Tag> tags = new ArrayList<>();
|
||||||
|
String head = SHA;
|
||||||
|
boolean verified = true, advanceAfterTag;
|
||||||
|
|
||||||
|
FakeApi(Map<String, String> env, List<String> events) {
|
||||||
|
super(env);
|
||||||
|
this.events = events;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
String head() {
|
||||||
|
events.add("head");
|
||||||
|
return head;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
List<Tag> tags() {
|
||||||
|
return List.copyOf(tags);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
String verified(String sha) {
|
||||||
|
Pipeline.check(verified, "Unverified test commit");
|
||||||
|
events.add("verified");
|
||||||
|
return "10";
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
void annotated(Tag tag) {
|
||||||
|
events.add("annotated");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
void create(ReleaseVersion version, String sha, String attempt) {
|
||||||
|
events.add("tag");
|
||||||
|
tags.add(new Tag(version, sha, OLD));
|
||||||
|
if (advanceAfterTag) head = OLD;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static class FakeRemote extends ReleaseRemote {
|
||||||
|
final List<String> events;
|
||||||
|
final List<String> failedDirectories = new ArrayList<>();
|
||||||
|
ReleaseState state = new ReleaseState(null, null, null);
|
||||||
|
boolean locked, pending;
|
||||||
|
String failAction = "";
|
||||||
|
|
||||||
|
FakeRemote(Path root, Map<String, String> env, List<String> events) throws Exception {
|
||||||
|
super(root, env, new Commands(Duration.ofMinutes(1)), "attempt");
|
||||||
|
this.events = events;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
String call(String action, String input, String... args) {
|
||||||
|
events.add(action.equals("apply") ? action + ":" + args[0] : action);
|
||||||
|
Pipeline.check(!action.equals(failAction), "Injected remote failure");
|
||||||
|
switch (action) {
|
||||||
|
case "lock" -> {
|
||||||
|
Pipeline.check(!locked && !pending, "Locked");
|
||||||
|
locked = true;
|
||||||
|
}
|
||||||
|
case "unlock" -> {
|
||||||
|
Pipeline.check(!pending, "Pending");
|
||||||
|
locked = false;
|
||||||
|
}
|
||||||
|
case "state" -> {
|
||||||
|
return state.current() == null ? "{}" : state.json();
|
||||||
|
}
|
||||||
|
case "pending" -> pending = true;
|
||||||
|
case "finish" -> pending = false;
|
||||||
|
case "commit" -> state = ReleaseState.parse(input);
|
||||||
|
case "prepare" -> {
|
||||||
|
return "S6_BACKEND=" + deployment("v1.0.0", SHA, "attempt").backend() + "\nS6_FRONTEND="
|
||||||
|
+ deployment("v1.0.0", SHA, "attempt").frontend();
|
||||||
|
}
|
||||||
|
case "apply" -> Pipeline.check(!failedDirectories.contains(args[0]), "Injected unhealthy release");
|
||||||
|
default -> {}
|
||||||
|
}
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static class Fixture implements AutoCloseable {
|
||||||
|
final Path root = Files.createTempDirectory("release-test-");
|
||||||
|
final Map<String, String> env = environment();
|
||||||
|
final List<String> events = new ArrayList<>();
|
||||||
|
final FakeApi api;
|
||||||
|
final FakeRemote remote;
|
||||||
|
final Release release;
|
||||||
|
|
||||||
|
Fixture() throws Exception {
|
||||||
|
this("new");
|
||||||
|
}
|
||||||
|
|
||||||
|
Fixture(String mode) throws Exception {
|
||||||
|
Files.createDirectories(root.resolve("deploy"));
|
||||||
|
Files.createDirectories(root.resolve("generated/deploy"));
|
||||||
|
env.put("PROJECT_NAME", "recipe-demo");
|
||||||
|
env.put("BACKEND_MODULE", "backend");
|
||||||
|
env.put("FRONTEND_MODULE", "frontend");
|
||||||
|
env.put("COMPOSE_PROJECT", "recipe-demo-test");
|
||||||
|
Files.writeString(root.resolve("generated/deploy/remote.sh"), "");
|
||||||
|
Files.writeString(root.resolve("deploy/compose.yaml"), "services: {}\n");
|
||||||
|
Files.writeString(root.resolve("deploy/health.sh"), "application_health() { true; }\n");
|
||||||
|
env.put("RELEASE_MODE", mode);
|
||||||
|
if (mode.equals("retry")) env.put("RELEASE_VERSION", "v1.0.0");
|
||||||
|
api = new FakeApi(env, events);
|
||||||
|
if (mode.equals("retry")) api.tags.add(new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.0"), SHA, OLD));
|
||||||
|
remote = new FakeRemote(root, env, events);
|
||||||
|
release = new Release(root, env, api, remote, mode.equals("rollback") ? "rollback" : "deploy", "attempt");
|
||||||
|
}
|
||||||
|
|
||||||
|
void withPrior() {
|
||||||
|
remote.state = prior();
|
||||||
|
api.tags.add(new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.1"), OLD, OLD));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void close() throws Exception {
|
||||||
|
remote.close();
|
||||||
|
Pipeline.deleteTree(root);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void apiTests() throws Exception {
|
||||||
|
test("NAS registry alias keeps the same repository and rejects unrelated registries", () -> {
|
||||||
|
try (var f = new Fixture()) {
|
||||||
|
f.env.put("DEPLOY_REGISTRY", "127.0.0.1:3000");
|
||||||
|
CiTests.equal("127.0.0.1:3000/owner/recipe-demo-", f.release.prefix());
|
||||||
|
f.env.put("DEPLOY_REGISTRY", "other-registry:3000");
|
||||||
|
CiTests.failure(f.release::prefix);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("repeating remote tag pages fail closed instead of allocating from incomplete history", () -> {
|
||||||
|
var api = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return List.of(Map.of("name", "v1.0.0", "id", OLD, "commit", Map.of("sha", SHA)));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
CiTests.failure(api::tags);
|
||||||
|
});
|
||||||
|
test("annotated release tag must identify the recorded version and exact SHA", () -> {
|
||||||
|
var tag = new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.0"), SHA, OLD);
|
||||||
|
var api = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return Map.of("tag", "v1.0.0", "object", Map.of("sha", SHA));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
api.annotated(tag);
|
||||||
|
var wrong = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return Map.of("tag", "v1.0.0", "object", Map.of("sha", OLD));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
CiTests.failure(() -> wrong.annotated(tag));
|
||||||
|
});
|
||||||
|
test("Gitea API eligibility validates SHA, event, branch, workflow path and successful conclusion", () -> {
|
||||||
|
var valid = Map.<String, Object>of(
|
||||||
|
"id",
|
||||||
|
1,
|
||||||
|
"head_sha",
|
||||||
|
SHA,
|
||||||
|
"event",
|
||||||
|
"push",
|
||||||
|
"head_branch",
|
||||||
|
"main",
|
||||||
|
"path",
|
||||||
|
"build.yaml@refs/heads/main",
|
||||||
|
"status",
|
||||||
|
"completed",
|
||||||
|
"conclusion",
|
||||||
|
"success");
|
||||||
|
for (String field : List.of("head_sha", "event", "head_branch", "path", "status", "conclusion")) {
|
||||||
|
var invalid = new HashMap<>(valid);
|
||||||
|
invalid.put(field, "wrong");
|
||||||
|
var api = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return Map.of("workflow_runs", List.of(invalid));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
CiTests.failure(() -> api.verified(SHA));
|
||||||
|
}
|
||||||
|
var api = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return Map.of("workflow_runs", List.of(valid));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
CiTests.equal("1", api.verified(SHA));
|
||||||
|
});
|
||||||
|
test("remote tag pagination includes failed-release baseline and rejects ambiguous tag formats", () -> {
|
||||||
|
var api = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return path.endsWith("page=1")
|
||||||
|
? List.of(Map.of("name", "v1.2.3", "id", OLD, "commit", Map.of("sha", SHA)))
|
||||||
|
: List.of();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
CiTests.equal(
|
||||||
|
"v1.2.4",
|
||||||
|
ReleaseVersion.next(
|
||||||
|
api.tags().stream()
|
||||||
|
.map(ReleaseApi.Tag::version)
|
||||||
|
.toList(),
|
||||||
|
"patch")
|
||||||
|
.toString());
|
||||||
|
var invalid = new ReleaseApi(environment()) {
|
||||||
|
@Override
|
||||||
|
Object request(String method, String path, Object body) {
|
||||||
|
return List.of(Map.of("name", "1.0", "id", OLD, "commit", Map.of("sha", SHA)));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
CiTests.failure(invalid::tags);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
static void shellTests() throws Exception {
|
||||||
|
test("real remote shell atomically excludes a second owner and retains unresolved journal", () -> {
|
||||||
|
Path directory = Files.createTempDirectory("release-shell-test-");
|
||||||
|
try {
|
||||||
|
String script = Files.readString(Path.of("generated/deploy/remote.sh"));
|
||||||
|
var commands = new Commands(Duration.ofSeconds(30));
|
||||||
|
var env = environment();
|
||||||
|
var first = commands.execute(
|
||||||
|
List.of("sh", "-c", script, "--", "lock", directory.toString(), "one"),
|
||||||
|
directory,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false);
|
||||||
|
first.checked();
|
||||||
|
CiTests.equal(
|
||||||
|
1,
|
||||||
|
commands.execute(
|
||||||
|
List.of("sh", "-c", script, "--", "lock", directory.toString(), "two"),
|
||||||
|
directory,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.code());
|
||||||
|
commands.execute(
|
||||||
|
List.of("sh", "-c", script, "--", "pending", directory.toString(), "one"),
|
||||||
|
directory,
|
||||||
|
env,
|
||||||
|
"{\"candidate\":\"test\"}",
|
||||||
|
false)
|
||||||
|
.checked();
|
||||||
|
CiTests.equal(
|
||||||
|
1,
|
||||||
|
commands.execute(
|
||||||
|
List.of("sh", "-c", script, "--", "unlock", directory.toString(), "one"),
|
||||||
|
directory,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.code());
|
||||||
|
CiTests.equal(true, Files.exists(directory.resolve("pending.json")));
|
||||||
|
commands.execute(
|
||||||
|
List.of("sh", "-c", script, "--", "finish", directory.toString(), "one"),
|
||||||
|
directory,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.checked();
|
||||||
|
commands.execute(
|
||||||
|
List.of("sh", "-c", script, "--", "unlock", directory.toString(), "one"),
|
||||||
|
directory,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.checked();
|
||||||
|
CiTests.equal(false, Files.exists(directory.resolve("deploy.lock")));
|
||||||
|
CiTests.equal(true, Files.exists(directory.resolve("history/one.json")));
|
||||||
|
} finally {
|
||||||
|
Pipeline.deleteTree(directory);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test("SSH shell quoting preserves metacharacters as data", () -> {
|
||||||
|
String value = "quote' dollar$ backtick` newline\n";
|
||||||
|
var commands = new Commands(Duration.ofSeconds(5));
|
||||||
|
CiTests.equal(
|
||||||
|
value,
|
||||||
|
commands.execute(
|
||||||
|
List.of("sh", "-c", "printf '%s' " + ReleaseRemote.quote(value)),
|
||||||
|
Path.of("."),
|
||||||
|
environment(),
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.output());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
39
resources/ci/ReleaseVersion.java
Normal file
39
resources/ci/ReleaseVersion.java
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
import java.math.BigInteger;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
/** Strict stable release versions; arbitrary-size integers avoid overflow/reuse. */
|
||||||
|
record ReleaseVersion(BigInteger major, BigInteger minor, BigInteger patch) implements Comparable<ReleaseVersion> {
|
||||||
|
private static final Pattern FORMAT = Pattern.compile("v(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)");
|
||||||
|
|
||||||
|
static ReleaseVersion parse(String value) {
|
||||||
|
var match = FORMAT.matcher(value);
|
||||||
|
Pipeline.check(match.matches(), "Expected release version vMAJOR.MINOR.PATCH");
|
||||||
|
return new ReleaseVersion(
|
||||||
|
new BigInteger(match.group(1)), new BigInteger(match.group(2)), new BigInteger(match.group(3)));
|
||||||
|
}
|
||||||
|
|
||||||
|
static ReleaseVersion next(List<ReleaseVersion> versions, String increment) {
|
||||||
|
Pipeline.check(List.of("patch", "minor", "major").contains(increment), "Choose patch, minor or major");
|
||||||
|
var highest = versions.stream().max(ReleaseVersion::compareTo);
|
||||||
|
if (highest.isEmpty()) return parse("v1.0.0");
|
||||||
|
var current = highest.get();
|
||||||
|
return switch (increment) {
|
||||||
|
case "major" -> new ReleaseVersion(current.major.add(BigInteger.ONE), BigInteger.ZERO, BigInteger.ZERO);
|
||||||
|
case "minor" -> new ReleaseVersion(current.major, current.minor.add(BigInteger.ONE), BigInteger.ZERO);
|
||||||
|
default -> new ReleaseVersion(current.major, current.minor, current.patch.add(BigInteger.ONE));
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public int compareTo(ReleaseVersion other) {
|
||||||
|
int value = major.compareTo(other.major);
|
||||||
|
if (value == 0) value = minor.compareTo(other.minor);
|
||||||
|
return value == 0 ? patch.compareTo(other.patch) : value;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String toString() {
|
||||||
|
return "v" + major + "." + minor + "." + patch;
|
||||||
|
}
|
||||||
|
}
|
||||||
287
resources/ci/Setup.java
Normal file
287
resources/ci/Setup.java
Normal file
@@ -0,0 +1,287 @@
|
|||||||
|
import java.io.*;
|
||||||
|
import java.net.*;
|
||||||
|
import java.net.http.*;
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.nio.file.attribute.*;
|
||||||
|
import java.security.MessageDigest;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.*;
|
||||||
|
|
||||||
|
class Setup {
|
||||||
|
static Path home() {
|
||||||
|
return Path.of(System.getProperty("user.home"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static String required(Map<String, String> env, String key) {
|
||||||
|
String value = env.get(key);
|
||||||
|
if (value == null || value.isBlank()) throw new IllegalArgumentException("Missing " + key);
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
static String secret(Map<String, String> env, String key) throws IOException {
|
||||||
|
Path path = Path.of(required(env, key));
|
||||||
|
Set<PosixFilePermission> permissions = Files.getPosixFilePermissions(path);
|
||||||
|
if (permissions.stream()
|
||||||
|
.anyMatch(p -> p.name().startsWith("GROUP_") || p.name().startsWith("OTHERS_")))
|
||||||
|
throw new IllegalArgumentException(key + " must refer to an owner-only file (mode 600)");
|
||||||
|
String token = Files.readString(path).strip();
|
||||||
|
if (token.isEmpty()) throw new IllegalArgumentException(key + " is empty");
|
||||||
|
return token;
|
||||||
|
}
|
||||||
|
|
||||||
|
static class ApiFailure extends IOException {
|
||||||
|
final int status;
|
||||||
|
|
||||||
|
ApiFailure(int status) {
|
||||||
|
super("Gitea HTTP " + status);
|
||||||
|
this.status = status;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static Object api(String server, String token, String method, String path, Object body) throws Exception {
|
||||||
|
Pipeline.check(token.matches("[a-zA-Z0-9_-]+"), "Invalid token format");
|
||||||
|
HttpRequest request = HttpRequest.newBuilder(URI.create(server.replaceAll("/+$", "") + path))
|
||||||
|
.timeout(Duration.ofSeconds(30))
|
||||||
|
.header("Authorization", "token " + token)
|
||||||
|
.header("Content-Type", "application/json")
|
||||||
|
.method(
|
||||||
|
method,
|
||||||
|
body == null
|
||||||
|
? HttpRequest.BodyPublishers.noBody()
|
||||||
|
: HttpRequest.BodyPublishers.ofString(Json.write(body)))
|
||||||
|
.build();
|
||||||
|
try (HttpClient client =
|
||||||
|
HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(30)).build()) {
|
||||||
|
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
|
||||||
|
if (response.statusCode() < 200 || response.statusCode() >= 300)
|
||||||
|
throw new ApiFailure(response.statusCode());
|
||||||
|
return response.body().isBlank() ? null : Json.parse(response.body());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void configureGitea(Map<String, String> env) throws Exception {
|
||||||
|
String admin = secret(env, "GITEA_ADMIN_TOKEN_FILE"), registry = secret(env, "REGISTRY_TOKEN_FILE");
|
||||||
|
String user = required(env, "REGISTRY_USER"), platform = required(env, "TARGET_PLATFORM");
|
||||||
|
Pipeline.check(
|
||||||
|
Set.of("linux/amd64", "linux/arm64").contains(platform),
|
||||||
|
"Set TARGET_PLATFORM from the verified Synology architecture");
|
||||||
|
String server = required(env, "GITEA_SERVER_URL"), base = "/api/v1/repos/" + required(env, "CI_REPOSITORY");
|
||||||
|
api(server, admin, "PATCH", base, Map.of("has_actions", true));
|
||||||
|
api(server, admin, "PUT", base + "/actions/secrets/REGISTRY_USER", Map.of("data", user));
|
||||||
|
api(server, admin, "PUT", base + "/actions/secrets/REGISTRY_TOKEN", Map.of("data", registry));
|
||||||
|
String variable = base + "/actions/variables/TARGET_PLATFORM";
|
||||||
|
boolean exists;
|
||||||
|
try {
|
||||||
|
api(server, admin, "GET", variable, null);
|
||||||
|
exists = true;
|
||||||
|
} catch (ApiFailure e) {
|
||||||
|
if (e.status != 404) throw e;
|
||||||
|
exists = false;
|
||||||
|
}
|
||||||
|
api(server, admin, exists ? "PUT" : "POST", variable, Map.of("value", platform));
|
||||||
|
Map<String, Object> response =
|
||||||
|
Json.object(api(server, admin, "POST", base + "/actions/runners/registration-token", null));
|
||||||
|
Path directory = home().resolve(".config/" + Recipe.project(env) + "-runner");
|
||||||
|
privateDirectory(directory);
|
||||||
|
privateWrite(directory.resolve("registration-token"), Objects.toString(response.get("token")));
|
||||||
|
System.out.println("Actions, registry secrets and platform configured; runner token saved outside Git.");
|
||||||
|
}
|
||||||
|
|
||||||
|
static void configureDocker(Path file, String registry) throws IOException {
|
||||||
|
Map<String, Object> data =
|
||||||
|
Files.exists(file) ? Json.object(Json.parse(Files.readString(file))) : new LinkedHashMap<>();
|
||||||
|
Object entry = data.getOrDefault("insecure-registries", new ArrayList<>());
|
||||||
|
if (!(entry instanceof List<?>))
|
||||||
|
throw new IllegalArgumentException("Existing insecure-registries must be an array; unchanged");
|
||||||
|
List<Object> registries = new ArrayList<>((List<?>) entry);
|
||||||
|
if (registries.contains(registry)) {
|
||||||
|
System.out.println("Internal registry already configured; no change");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
registries.add(registry);
|
||||||
|
data.put("insecure-registries", registries);
|
||||||
|
Path absolute = file.toAbsolutePath(), parent = absolute.getParent();
|
||||||
|
Files.createDirectories(parent);
|
||||||
|
if (Files.exists(file)) {
|
||||||
|
Path backup = file.resolveSibling(file.getFileName() + ".before-engineering");
|
||||||
|
if (!Files.exists(backup)) Files.copy(file, backup, StandardCopyOption.COPY_ATTRIBUTES);
|
||||||
|
}
|
||||||
|
Path temp = Files.createTempFile(parent, "daemon-", ".json");
|
||||||
|
try {
|
||||||
|
Files.writeString(temp, Json.write(data) + "\n");
|
||||||
|
if (Files.exists(file)) Files.setPosixFilePermissions(temp, Files.getPosixFilePermissions(file));
|
||||||
|
Files.move(temp, absolute, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
} finally {
|
||||||
|
Files.deleteIfExists(temp);
|
||||||
|
}
|
||||||
|
System.out.println("Internal registry added; restart/apply Docker Engine configuration to activate");
|
||||||
|
}
|
||||||
|
|
||||||
|
static void installRunner(Path root, Map<String, String> env, Commands commands, boolean prepareOnly)
|
||||||
|
throws Exception {
|
||||||
|
Pipeline.check(
|
||||||
|
System.getProperty("os.name").equals("Linux")
|
||||||
|
&& Set.of("amd64", "x86_64").contains(System.getProperty("os.arch")),
|
||||||
|
"Installer targets the existing Linux x86_64 mini-PC");
|
||||||
|
Path runnerDir = home().resolve(".local/share/" + Recipe.project(env) + "-runner"),
|
||||||
|
configDir = home().resolve(".config/" + Recipe.project(env) + "-runner");
|
||||||
|
Path bin = home().resolve(".local/bin"), units = home().resolve(".config/systemd/user");
|
||||||
|
for (Path path : List.of(runnerDir, configDir, bin, units)) privateDirectory(path);
|
||||||
|
for (String tool : List.of("node", "java", "mvn", "docker", "git", "systemctl")) executable(env, tool);
|
||||||
|
String javaHome = System.getProperty("java.home");
|
||||||
|
String version = commands.execute(List.of("node", "--version"), root, env, null, false)
|
||||||
|
.checked();
|
||||||
|
String[] parts = version.replaceFirst("^v", "").split("\\.");
|
||||||
|
int major = Integer.parseInt(parts[0]), minor = Integer.parseInt(parts[1]), patch = Integer.parseInt(parts[2]);
|
||||||
|
Pipeline.check(
|
||||||
|
(major == 22 && (minor > 22 || minor == 22 && patch >= 3))
|
||||||
|
|| (major == 24 && minor >= 15)
|
||||||
|
|| major >= 26,
|
||||||
|
"Supported Node version required");
|
||||||
|
Path runner = bin.resolve("act_runner");
|
||||||
|
if (!Files.isExecutable(runner)) {
|
||||||
|
URI uri = URI.create("https://dl.gitea.com/act_runner/0.2.13/act_runner-0.2.13-linux-amd64");
|
||||||
|
byte[] bytes;
|
||||||
|
try (HttpClient client = HttpClient.newBuilder()
|
||||||
|
.followRedirects(HttpClient.Redirect.NORMAL)
|
||||||
|
.build()) {
|
||||||
|
var response = client.send(
|
||||||
|
HttpRequest.newBuilder(uri)
|
||||||
|
.timeout(Duration.ofMinutes(2))
|
||||||
|
.build(),
|
||||||
|
HttpResponse.BodyHandlers.ofByteArray());
|
||||||
|
Pipeline.check(response.statusCode() == 200, "Runner download failed");
|
||||||
|
bytes = response.body();
|
||||||
|
}
|
||||||
|
String hash = HexFormat.of()
|
||||||
|
.formatHex(MessageDigest.getInstance("SHA-256").digest(bytes));
|
||||||
|
Pipeline.check(
|
||||||
|
hash.equals("3acac8b506ac8cadc88a55155b5d6378f0fab0b8f62d1e0c0450f4ccd69733e2"),
|
||||||
|
"Runner checksum mismatch");
|
||||||
|
Files.write(runner, bytes);
|
||||||
|
Files.setPosixFilePermissions(runner, PosixFilePermissions.fromString("rwxr-xr-x"));
|
||||||
|
}
|
||||||
|
String actual = commands.execute(List.of(runner.toString(), "--version"), root, env, null, false)
|
||||||
|
.checked();
|
||||||
|
Pipeline.check(actual.contains("v0.2.13"), "Expected runner v0.2.13");
|
||||||
|
privateWrite(runnerDir.resolve("config.yaml"), Files.readString(root.resolve("ci/runner/config.yaml")));
|
||||||
|
Files.copy(
|
||||||
|
root.resolve("ci/runner/runner.service"),
|
||||||
|
units.resolve(Recipe.project(env) + "-runner.service"),
|
||||||
|
StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
Files.copy(
|
||||||
|
root.resolve("generated/ci/runner/WaitForDocker.java"),
|
||||||
|
runnerDir.resolve("WaitForDocker.java"),
|
||||||
|
StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
String dockerHost = env.get("DOCKER_HOST");
|
||||||
|
if (dockerHost == null || dockerHost.isBlank())
|
||||||
|
dockerHost = commands.execute(
|
||||||
|
List.of("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}"),
|
||||||
|
root,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
false)
|
||||||
|
.checked();
|
||||||
|
String path = javaHome + "/bin:" + executable(env, "node").getParent() + ":"
|
||||||
|
+ executable(env, "mvn").getParent() + ":/usr/local/bin:/usr/bin:/bin";
|
||||||
|
privateWrite(
|
||||||
|
configDir.resolve("environment"),
|
||||||
|
"JAVA_HOME=" + quoted(javaHome) + "\nPATH=" + quoted(path) + "\nDOCKER_HOST=" + quoted(dockerHost)
|
||||||
|
+ "\n");
|
||||||
|
boolean desktop = dockerHost.endsWith("/.docker/desktop/docker.sock");
|
||||||
|
Path overrides = units.resolve(Recipe.project(env) + "-runner.service.d");
|
||||||
|
Path desktopOverride = overrides.resolve("10-docker-desktop.conf");
|
||||||
|
if (desktop) {
|
||||||
|
Files.createDirectories(overrides);
|
||||||
|
Files.copy(
|
||||||
|
root.resolve("ci/runner/docker-desktop.conf"),
|
||||||
|
desktopOverride,
|
||||||
|
StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
} else {
|
||||||
|
Files.deleteIfExists(desktopOverride);
|
||||||
|
}
|
||||||
|
commands.execute(List.of("systemctl", "--user", "daemon-reload"), root, env, null, true)
|
||||||
|
.checked();
|
||||||
|
if (prepareOnly) {
|
||||||
|
System.out.println("Runner installed; registration and startup still required.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
commands.execute(List.of("docker", "info"), root, env, null, false).checked();
|
||||||
|
if (!Files.exists(runnerDir.resolve(".runner"))) {
|
||||||
|
Path tokenPath = Path.of(env.getOrDefault(
|
||||||
|
"RUNNER_REGISTRATION_TOKEN_FILE",
|
||||||
|
configDir.resolve("registration-token").toString()));
|
||||||
|
Map<String, String> tokenEnv = Map.of("RUNNER_TOKEN_FILE", tokenPath.toString());
|
||||||
|
String token = secret(tokenEnv, "RUNNER_TOKEN_FILE");
|
||||||
|
commands.execute(
|
||||||
|
List.of(
|
||||||
|
runner.toString(),
|
||||||
|
"register",
|
||||||
|
"--no-interactive",
|
||||||
|
"--instance",
|
||||||
|
required(env, "GITEA_SERVER_URL"),
|
||||||
|
"--name",
|
||||||
|
required(env, "RUNNER_LABEL"),
|
||||||
|
"--labels",
|
||||||
|
required(env, "RUNNER_LABEL") + ":host",
|
||||||
|
"--token",
|
||||||
|
token),
|
||||||
|
runnerDir,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
true)
|
||||||
|
.checked();
|
||||||
|
Files.setPosixFilePermissions(runnerDir.resolve(".runner"), PosixFilePermissions.fromString("rw-------"));
|
||||||
|
}
|
||||||
|
if (desktop) {
|
||||||
|
commands.execute(
|
||||||
|
List.of("systemctl", "--user", "enable", "--now", "docker-desktop.service"),
|
||||||
|
root,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
true)
|
||||||
|
.checked();
|
||||||
|
// Replace any previous default.target link with the desktop-login target.
|
||||||
|
commands.execute(
|
||||||
|
List.of("systemctl", "--user", "disable", Recipe.project(env) + "-runner.service"),
|
||||||
|
root,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
true)
|
||||||
|
.checked();
|
||||||
|
}
|
||||||
|
commands.execute(
|
||||||
|
List.of("systemctl", "--user", "enable", "--now", Recipe.project(env) + "-runner.service"),
|
||||||
|
root,
|
||||||
|
env,
|
||||||
|
null,
|
||||||
|
true)
|
||||||
|
.checked();
|
||||||
|
System.out.println("Runner registered and started; keep the mini-PC and Docker available.");
|
||||||
|
}
|
||||||
|
|
||||||
|
static Path executable(Map<String, String> env, String name) {
|
||||||
|
for (String dir : env.getOrDefault("PATH", "").split(File.pathSeparator)) {
|
||||||
|
Path path = Path.of(dir, name);
|
||||||
|
if (Files.isRegularFile(path) && Files.isExecutable(path)) return path.toAbsolutePath();
|
||||||
|
}
|
||||||
|
throw new IllegalArgumentException("Missing executable: " + name);
|
||||||
|
}
|
||||||
|
|
||||||
|
static String quoted(String value) {
|
||||||
|
Pipeline.check(!value.contains("\n") && !value.contains("\r"), "Invalid service environment value");
|
||||||
|
return "\"" + value.replace("\\", "\\\\").replace("\"", "\\\"") + "\"";
|
||||||
|
}
|
||||||
|
|
||||||
|
static void privateDirectory(Path path) throws IOException {
|
||||||
|
Files.createDirectories(path);
|
||||||
|
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rwx------"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static void privateWrite(Path path, String text) throws IOException {
|
||||||
|
if (!Files.exists(path))
|
||||||
|
Files.createFile(path, PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString("rw-------")));
|
||||||
|
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rw-------"));
|
||||||
|
Files.writeString(path, text);
|
||||||
|
}
|
||||||
|
}
|
||||||
35
resources/ci/runner/WaitForDocker.java
Normal file
35
resources/ci/runner/WaitForDocker.java
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
import java.time.Duration;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
|
||||||
|
/** Source-launcher startup probe; uses the service's configured DOCKER_HOST. */
|
||||||
|
class WaitForDocker {
|
||||||
|
public static void main(String[] args) throws Exception {
|
||||||
|
long seconds = args.length == 0 ? 120 : Long.parseLong(args[0]);
|
||||||
|
if (seconds < 1 || seconds > 300) throw new IllegalArgumentException("Timeout must be 1–300 seconds");
|
||||||
|
long deadline = System.nanoTime() + Duration.ofSeconds(seconds).toNanos();
|
||||||
|
System.out.println("Waiting for the configured Docker engine...");
|
||||||
|
while (System.nanoTime() < deadline) {
|
||||||
|
Process process = new ProcessBuilder("docker", "info", "--format", "{{.ServerVersion}}")
|
||||||
|
.redirectOutput(ProcessBuilder.Redirect.DISCARD)
|
||||||
|
.redirectError(ProcessBuilder.Redirect.DISCARD)
|
||||||
|
.start();
|
||||||
|
try {
|
||||||
|
long remaining = Math.max(1, deadline - System.nanoTime());
|
||||||
|
if (process.waitFor(Math.min(remaining, TimeUnit.SECONDS.toNanos(5)), TimeUnit.NANOSECONDS)
|
||||||
|
&& process.exitValue() == 0) {
|
||||||
|
System.out.println("Docker is available; starting the Gitea runner.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
if (process.isAlive()) {
|
||||||
|
process.destroyForcibly();
|
||||||
|
process.waitFor();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
long remaining = deadline - System.nanoTime();
|
||||||
|
if (remaining > 0) TimeUnit.NANOSECONDS.sleep(Math.min(remaining, TimeUnit.SECONDS.toNanos(2)));
|
||||||
|
}
|
||||||
|
System.err.println("Docker did not become available within " + seconds + " seconds; runner not started.");
|
||||||
|
System.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
30
resources/deploy/enable-docker.sh
Normal file
30
resources/deploy/enable-docker.sh
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# One-time DSM administrator action, run with sudo; does not restart any service.
|
||||||
|
set -eu
|
||||||
|
test "$(id -u)" = 0 || { echo 'Run this setup with sudo' >&2; exit 1; }
|
||||||
|
account=${1:?Supply deployment account}
|
||||||
|
docker=${2:?Supply absolute Docker executable}
|
||||||
|
case "$account" in ''|*[!a-zA-Z0-9_-]*) exit 1;; esac
|
||||||
|
case "$docker" in /*) ;; *) exit 1;; esac
|
||||||
|
target=/etc/sudoers.d/"$account"
|
||||||
|
rule="$account ALL=(root) NOPASSWD: $docker"
|
||||||
|
test -x "$docker" && id "$account" >/dev/null
|
||||||
|
test -d /etc/sudoers.d || { echo 'Missing sudoers.d; no changes made' >&2; exit 1; }
|
||||||
|
if [ -e "$target" ]; then
|
||||||
|
test "$(cat "$target")" = "$rule" || { echo 'Existing sudoers entry differs; no changes made' >&2; exit 1; }
|
||||||
|
else
|
||||||
|
umask 077
|
||||||
|
temporary=$(mktemp /etc/sudoers.d/"$account".XXXXXX)
|
||||||
|
trap 'rm -f "$temporary"' EXIT
|
||||||
|
printf '%s\n' "$rule" > "$temporary"
|
||||||
|
chmod 440 "$temporary"
|
||||||
|
if command -v visudo >/dev/null 2>&1; then visudo -cf "$temporary"; fi
|
||||||
|
# A dot in the temporary name keeps sudo from loading the uncommitted file.
|
||||||
|
mv "$temporary" "$target"
|
||||||
|
if ! sudo -n -u "$account" sudo -n "$docker" info --format '{{.OSType}}/{{.Architecture}}'; then
|
||||||
|
rm "$target"
|
||||||
|
echo 'Docker permission check failed; new sudoers entry removed' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
echo 'Dedicated Docker sudo permission is configured; no services were restarted.'
|
||||||
38
resources/deploy/provision.sh
Normal file
38
resources/deploy/provision.sh
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Account-level provisioning. Docker sudo must already be enabled separately.
|
||||||
|
set -eu
|
||||||
|
umask 077
|
||||||
|
root=$1
|
||||||
|
registry_user=$2
|
||||||
|
registry=$3
|
||||||
|
bind=$4
|
||||||
|
port=$5
|
||||||
|
docker=${6:?Supply the Docker executable path}
|
||||||
|
test -x "$docker"
|
||||||
|
sudo -n "$docker" compose version
|
||||||
|
sudo -n "$docker" info --format '{{.OSType}}/{{.Architecture}}'
|
||||||
|
test ! -L "$root"
|
||||||
|
mkdir -p "$root" "$root/docker-auth"
|
||||||
|
chmod 700 "$root" "$root/docker-auth"
|
||||||
|
if [ ! -f "$root/production.env" ]; then
|
||||||
|
# First provisioning must not take a port already occupied by another service.
|
||||||
|
command -v netstat >/dev/null
|
||||||
|
if netstat -lnt | awk -v port=":$port" '$4 ~ (port "$") { found=1 } END { exit !found }'; then
|
||||||
|
echo 'Frontend port is already occupied; production configuration unchanged' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
printf 'BIND_IP=%s\nFRONTEND_PORT=%s\nDOCKER_BIN=%s\nDOCKER_SUDO=1\nDOCKER_CONFIG=%s/docker-auth\n' \
|
||||||
|
"$bind" "$port" "$docker" "$root" > "$root/production.env"
|
||||||
|
chmod 600 "$root/production.env"
|
||||||
|
else
|
||||||
|
# Preserve existing config; refuse to silently rebind a configured deployment.
|
||||||
|
. "$root/production.env"
|
||||||
|
test "$BIND_IP" = "$bind" && test "$FRONTEND_PORT" = "$port" \
|
||||||
|
&& test "$DOCKER_CONFIG" = "$root/docker-auth" || {
|
||||||
|
echo 'Existing production.env differs; review it before provisioning' >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
fi
|
||||||
|
# Credential enters only via stdin. No password, environment or auth config is logged.
|
||||||
|
sudo -n "$docker" --config "$root/docker-auth" login "$registry" --username "$registry_user" --password-stdin
|
||||||
|
echo 'Production directory and dedicated registry login prepared; no containers were changed.'
|
||||||
204
resources/deploy/remote.sh
Normal file
204
resources/deploy/remote.sh
Normal file
@@ -0,0 +1,204 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Invoked over a pinned SSH connection by ReleaseRemote. No secrets in arguments.
|
||||||
|
# mkdir is the NAS-wide lock; a disconnected runner leaves it for explicit recovery.
|
||||||
|
set -eu
|
||||||
|
umask 077
|
||||||
|
action=$1
|
||||||
|
root=$2
|
||||||
|
owner=$3
|
||||||
|
shift 3
|
||||||
|
die() { echo "Deployment: $*" >&2; exit 1; }
|
||||||
|
case "$root" in /*) ;; *) die 'Deployment directory must be absolute';; esac
|
||||||
|
case "$owner" in ''|*[!a-zA-Z0-9-]*) die 'Invalid lock owner';; esac
|
||||||
|
test -d "$root" || die 'Provision the deployment directory first'
|
||||||
|
cd "$root"
|
||||||
|
lock="$root/deploy.lock"
|
||||||
|
release="$root/releases/$owner"
|
||||||
|
|
||||||
|
if [ "$action" = lock ]; then
|
||||||
|
test ! -e pending.json || die 'Unresolved deployment: follow manual recovery runbook'
|
||||||
|
mkdir "$lock" 2>/dev/null || die 'Deployment locked: another run or manual recovery owns the NAS'
|
||||||
|
printf '%s\n' "$owner" > "$lock/owner"
|
||||||
|
# Check again after acquiring, covering races with another lock holder.
|
||||||
|
if [ -e pending.json ]; then
|
||||||
|
rm "$lock/owner"
|
||||||
|
rmdir "$lock"
|
||||||
|
die 'Unresolved deployment: follow manual recovery runbook'
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
test -f "$lock/owner" && [ "$(cat "$lock/owner")" = "$owner" ] || die 'Deployment lock ownership mismatch'
|
||||||
|
|
||||||
|
case "$action" in
|
||||||
|
unlock)
|
||||||
|
test ! -e pending.json || die 'Pending deployment retained; manual recovery required'
|
||||||
|
rm "$lock/owner"
|
||||||
|
rmdir "$lock"
|
||||||
|
exit 0;;
|
||||||
|
state)
|
||||||
|
if [ -f state.json ]; then cat state.json; else printf '{}\n'; fi
|
||||||
|
exit 0;;
|
||||||
|
pending)
|
||||||
|
test ! -e pending.json || die 'Deployment already pending'
|
||||||
|
cat > pending.json.tmp
|
||||||
|
mv pending.json.tmp pending.json
|
||||||
|
exit 0;;
|
||||||
|
commit)
|
||||||
|
test -f pending.json || die 'No pending deployment'
|
||||||
|
cat > state.json.tmp
|
||||||
|
mv state.json.tmp state.json
|
||||||
|
exit 0;;
|
||||||
|
finish)
|
||||||
|
test -f pending.json || die 'No pending deployment'
|
||||||
|
mkdir -p history
|
||||||
|
mv pending.json "history/$owner.json"
|
||||||
|
exit 0;;
|
||||||
|
event)
|
||||||
|
mkdir -p history
|
||||||
|
cat > "history/$owner-outcome.json.tmp"
|
||||||
|
mv "history/$owner-outcome.json.tmp" "history/$owner-outcome.json"
|
||||||
|
exit 0;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
test -f production.env || die 'Missing production.env; follow provisioning runbook'
|
||||||
|
# This account-owned file is trusted shell configuration, never a workflow input.
|
||||||
|
. "$root/production.env"
|
||||||
|
: "${BIND_IP:?}" "${FRONTEND_PORT:?}" "${DOCKER_BIN:?}" "${DOCKER_CONFIG:?}"
|
||||||
|
export DOCKER_CONFIG
|
||||||
|
docker_cmd() {
|
||||||
|
if [ "${DOCKER_SUDO:-1}" = 1 ]; then
|
||||||
|
sudo -n "$DOCKER_BIN" --config "$DOCKER_CONFIG" "$@"
|
||||||
|
else
|
||||||
|
"$DOCKER_BIN" --config "$DOCKER_CONFIG" "$@"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
: "${COMPOSE_PROJECT:?}" "${DEPLOY_ROLES:?}" "${TARGET_PLATFORM:?}"
|
||||||
|
case "$COMPOSE_PROJECT" in ''|*[!a-z0-9_-]*) die 'Invalid Compose project';; esac
|
||||||
|
case "$DEPENDENCY_SERVICES" in *[!a-zA-Z0-9_\ -]*) die 'Invalid dependency services';; esac
|
||||||
|
case "$DEPLOY_ROLES" in backend|frontend|'backend frontend') ;; *) die 'Invalid deployment roles';; esac
|
||||||
|
case "$TARGET_PLATFORM" in linux/amd64|linux/arm64) ;; *) die 'Invalid target platform';; esac
|
||||||
|
compose() {
|
||||||
|
docker_cmd compose --project-name "$COMPOSE_PROJECT" --project-directory "$root" \
|
||||||
|
--env-file "$root/production.env" --env-file "$release/release.env" -f "$release/compose.yaml" "$@"
|
||||||
|
}
|
||||||
|
identity() {
|
||||||
|
image=$1
|
||||||
|
sha=$2
|
||||||
|
test "$(docker_cmd image inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$image")" = "$sha" \
|
||||||
|
|| die 'Image revision does not match selected SHA'
|
||||||
|
test "$(docker_cmd image inspect --format '{{.Os}}/{{.Architecture}}' "$image")" = "$TARGET_PLATFORM" \
|
||||||
|
|| die 'Image platform differs from target'
|
||||||
|
}
|
||||||
|
use_release() {
|
||||||
|
case "$1" in ''|*[!a-zA-Z0-9-]*) die 'Invalid release directory';; esac
|
||||||
|
release="$root/releases/$1"
|
||||||
|
test -f "$release/compose.yaml" && test -f "$release/release.env" || die 'Missing retained release configuration'
|
||||||
|
}
|
||||||
|
health() {
|
||||||
|
# Compare image IDs, revision labels and the container release labels.
|
||||||
|
for role in $DEPLOY_ROLES; do
|
||||||
|
id=$(compose ps -q "$role")
|
||||||
|
if [ "$role" = backend ]; then expected=$BACKEND_IMAGE; else expected=$FRONTEND_IMAGE; fi
|
||||||
|
test "$(docker_cmd inspect --format '{{.State.Running}}' "$id")" = true || return 1
|
||||||
|
test "$(docker_cmd inspect --format '{{.Image}}' "$id")" = "$(docker_cmd image inspect --format '{{.Id}}' "$expected")" || return 1
|
||||||
|
test "$(docker_cmd inspect --format '{{index .Config.Labels "nl.agilecc.revision"}}' "$id")" = "$RELEASE_SHA" || return 1
|
||||||
|
test "$(docker_cmd inspect --format '{{index .Config.Labels "nl.agilecc.release"}}' "$id")" = "$RELEASE_VERSION" || return 1
|
||||||
|
done
|
||||||
|
if [ -f "$release/health.sh" ]; then . "$release/health.sh"; fi
|
||||||
|
application_health
|
||||||
|
}
|
||||||
|
|
||||||
|
case "$action" in
|
||||||
|
preflight)
|
||||||
|
for secret in ${REQUIRED_SECRETS:-}; do
|
||||||
|
case "$secret" in ''|*[!A-Z0-9_]*) die 'Invalid required secret name';; esac
|
||||||
|
eval 'value=${'"$secret"':-}'
|
||||||
|
test -n "$value" || die "Missing $secret in production.env"
|
||||||
|
done
|
||||||
|
command -v timeout >/dev/null || die 'timeout is required for bounded remote operations'
|
||||||
|
command -v curl >/dev/null || die 'curl is required'
|
||||||
|
test -x "$DOCKER_BIN" || die 'Docker executable missing'
|
||||||
|
docker_cmd compose version >/dev/null
|
||||||
|
actual=$(docker_cmd info --format '{{.OSType}}/{{.Architecture}}')
|
||||||
|
case "$actual" in linux/x86_64) actual=linux/amd64;; linux/aarch64) actual=linux/arm64;; esac
|
||||||
|
test "$actual" = "$TARGET_PLATFORM" || die 'Docker platform differs from target'
|
||||||
|
test -d "$DOCKER_CONFIG" && test -f "$DOCKER_CONFIG/config.json" || die 'Provision registry pull login first'
|
||||||
|
# Explicit private IPv4 binding; prevent accidental 0.0.0.0 publication.
|
||||||
|
case "$BIND_IP" in
|
||||||
|
10.*|192.168.*|172.16.*|172.17.*|172.18.*|172.19.*|172.2[0-9].*|172.30.*|172.31.*|127.*) ;;
|
||||||
|
*) die 'Bind frontend to a private IPv4 address';;
|
||||||
|
esac
|
||||||
|
case "$FRONTEND_PORT" in ''|*[!0-9]*) die 'Invalid frontend port';; esac
|
||||||
|
test "$FRONTEND_PORT" -ge 1024 && test "$FRONTEND_PORT" -le 65535 || die 'Use port 1024..65535'
|
||||||
|
;;
|
||||||
|
stage)
|
||||||
|
mkdir -p "$release"
|
||||||
|
test ! -e "$release/compose.yaml" || die 'Attempt configuration already exists'
|
||||||
|
cat > "$release/compose.yaml"
|
||||||
|
;;
|
||||||
|
stage-health)
|
||||||
|
test -f "$release/compose.yaml" || die 'Stage Compose first'
|
||||||
|
test ! -e "$release/health.sh" || die 'Attempt health configuration already exists'
|
||||||
|
cat > "$release/health.sh"
|
||||||
|
;;
|
||||||
|
prepare)
|
||||||
|
version=$1; sha=$2; prefix=$3
|
||||||
|
backend='-'; frontend='-'
|
||||||
|
for role in $DEPLOY_ROLES; do
|
||||||
|
repository="$prefix$role"
|
||||||
|
image="$repository:$sha"
|
||||||
|
docker_cmd pull --platform "$TARGET_PLATFORM" "$image" >&2
|
||||||
|
identity "$image" "$sha"
|
||||||
|
digest=$(docker_cmd image inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$image" \
|
||||||
|
| awk -v prefix="$repository@sha256:" 'index($0,prefix)==1 { print; exit }')
|
||||||
|
test -n "$digest" || die 'Missing registry digest'
|
||||||
|
if [ "$role" = backend ]; then backend=$digest; else frontend=$digest; fi
|
||||||
|
done
|
||||||
|
printf 'RELEASE_VERSION=%s\nRELEASE_SHA=%s\nBACKEND_IMAGE=%s\nFRONTEND_IMAGE=%s\nBIND_IP=%s\nFRONTEND_PORT=%s\n' \
|
||||||
|
"$version" "$sha" "$backend" "$frontend" "$BIND_IP" "$FRONTEND_PORT" > "$release/release.env"
|
||||||
|
printf 'COMPOSE_PROJECT=%s\nDEPLOY_ROLES="%s"\nDEPENDENCY_SERVICES="%s"\nTARGET_PLATFORM=%s\n' \
|
||||||
|
"$COMPOSE_PROJECT" "$DEPLOY_ROLES" "${DEPENDENCY_SERVICES:-}" "$TARGET_PLATFORM" >> "$release/release.env"
|
||||||
|
compose config --quiet
|
||||||
|
printf 'S6_BACKEND=%s\nS6_FRONTEND=%s\n' "$backend" "$frontend"
|
||||||
|
;;
|
||||||
|
ensure)
|
||||||
|
use_release "$1"
|
||||||
|
. "$release/release.env"
|
||||||
|
test "$RELEASE_VERSION" = "$2" && test "$RELEASE_SHA" = "$3" \
|
||||||
|
&& test "$BACKEND_IMAGE" = "$4" && test "$FRONTEND_IMAGE" = "$5" \
|
||||||
|
|| die 'Retained configuration differs from recorded successful release'
|
||||||
|
for image in "$BACKEND_IMAGE" "$FRONTEND_IMAGE"; do
|
||||||
|
[ "$image" != - ] || continue
|
||||||
|
docker_cmd pull --platform "$TARGET_PLATFORM" "$image" >&2
|
||||||
|
identity "$image" "$RELEASE_SHA"
|
||||||
|
done
|
||||||
|
compose config --quiet
|
||||||
|
;;
|
||||||
|
apply)
|
||||||
|
mkdir "$lock/operation" 2>/dev/null || die 'A remote replacement is still active; manual recovery required'
|
||||||
|
trap 'rmdir "$lock/operation"' EXIT
|
||||||
|
use_release "$1"
|
||||||
|
. "$release/release.env"
|
||||||
|
health_seconds=${2:-280}
|
||||||
|
case "$health_seconds" in ''|*[!0-9]*) die 'Invalid readiness window';; esac
|
||||||
|
test "$health_seconds" -ge 1 && test "$health_seconds" -le 280 || die 'Invalid readiness window'
|
||||||
|
end=$(( $(date +%s) + health_seconds ))
|
||||||
|
# Always recreate the frontend too: nginx must resolve the new backend IP.
|
||||||
|
# Recreate application containers only; the database and volume survive releases.
|
||||||
|
if [ -n "${DEPENDENCY_SERVICES:-}" ]; then
|
||||||
|
compose up -d --no-build --pull missing --timeout 20 $DEPENDENCY_SERVICES
|
||||||
|
fi
|
||||||
|
compose up -d --no-deps --no-build --pull never --force-recreate --timeout 20 $DEPLOY_ROLES
|
||||||
|
while ! health; do
|
||||||
|
test "$(date +%s)" -lt "$end" || die 'Readiness checks timed out'
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
echo 'S6_HEALTHY'
|
||||||
|
;;
|
||||||
|
diagnose)
|
||||||
|
use_release "$1"
|
||||||
|
compose ps
|
||||||
|
compose logs --no-color --tail 200
|
||||||
|
;;
|
||||||
|
*) die 'Unknown remote deployment operation';;
|
||||||
|
esac
|
||||||
19
resources/docs/backend-review.md
Normal file
19
resources/docs/backend-review.md
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
# Backend review companion
|
||||||
|
|
||||||
|
Use alongside `generated/skills/code-review/SKILL.md` and `generated/CODING_STANDARDS.md`.
|
||||||
|
|
||||||
|
Review changed code for intent-revealing comments, missing public-contract Javadoc,
|
||||||
|
appropriate record use, defensive copying of mutable record components, and
|
||||||
|
compatibility with APIs, JSON and persistence. Assess ambiguous `var` initializers
|
||||||
|
and framework/lifecycle exceptions to field finality. Confirm architecture and
|
||||||
|
injection decisions make sense for production and tests. Check NIO API preference,
|
||||||
|
varargs for trailing array parameters, and Java 25 IO use for direct console
|
||||||
|
input/output; retain System.err for diagnostics. Require Javadoc for public
|
||||||
|
classes and methods beyond basic POJO/data-carrier behavior.
|
||||||
|
|
||||||
|
Read standards reports even when verification passes. Assess every review signal
|
||||||
|
in changed code, including fan-out, public methods and record component counts.
|
||||||
|
Record a rationale for retained signals in application validation evidence or the change description.
|
||||||
|
Check narrowly scoped waivers against recorded maintainer agreement and reason.
|
||||||
|
Never introduce blanket exclusions or weaken thresholds to pass a build.
|
||||||
|
Do not repeat mechanical checks already enforced by tooling; verify their result.
|
||||||
23
resources/skills/README.md
Normal file
23
resources/skills/README.md
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
# Engineering skills
|
||||||
|
|
||||||
|
Reusable agent instructions for consuming repositories. Invoke them explicitly in the
|
||||||
|
conversation, for example:
|
||||||
|
|
||||||
|
> Gebruik grill-me om de JSON-import en -export met mij uit te werken.
|
||||||
|
|
||||||
|
## Installed skills
|
||||||
|
|
||||||
|
| Skill | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| [code-review](code-review/SKILL.md) | Review changes against documented coding standards and the originating spec. |
|
||||||
|
| [grill-me](grill-me/SKILL.md) | Entry point for reviewing a plan or design through questions. |
|
||||||
|
| [grilling](grilling/SKILL.md) | Interview instructions required by `grill-me`. |
|
||||||
|
|
||||||
|
The skills are copied from [Matt Pocock's skills.sh listing](https://skills.sh/mattpocock/skills/grill-me)
|
||||||
|
and its [upstream repository](https://github.com/mattpocock/skills).
|
||||||
|
The exact upstream revisions and paths are recorded in [upstream.json](upstream.json).
|
||||||
|
The upstream [MIT license](LICENSE.mattpocock) is included.
|
||||||
|
|
||||||
|
These project-local files are available to the agent from the next conversation
|
||||||
|
turn through the instructions in the consuming project's root `AGENTS.md`. They are not installed
|
||||||
|
globally. Update upstream sources in engineering, retaining licenses and revision records, then release a new engineering version. Consumers regenerate their pinned copies.
|
||||||
24
resources/skills/adr-brainstorm/ATTRIBUTION.md
Normal file
24
resources/skills/adr-brainstorm/ATTRIBUTION.md
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
# License and attribution
|
||||||
|
|
||||||
|
ADR brainstorm is maintained by Wanja Krah and is distributed under the MIT
|
||||||
|
License. See [LICENSE.wanjakrah](LICENSE.wanjakrah) for the license covering
|
||||||
|
Wanja Krah's original contributions.
|
||||||
|
|
||||||
|
The design-tree approach and related interviewing guidance in `SKILL.md` were
|
||||||
|
adapted from Matt Pocock's `grilling` skill, invoked by his `grill-me` skill.
|
||||||
|
This includes tracking dependent decisions, settling prerequisites before
|
||||||
|
asking dependent questions, investigating accessible facts, and leaving
|
||||||
|
decisions with the user. The wording and workflow have been adapted for ADR
|
||||||
|
brainstorming; this is not an unmodified copy of the upstream skill.
|
||||||
|
|
||||||
|
Upstream source:
|
||||||
|
- Repository: https://github.com/mattpocock/skills
|
||||||
|
- Revision: `59db609eb1e18aaf59b63284d0e52255aff0c63c`
|
||||||
|
- Paths: `skills/productivity/grilling` and `skills/productivity/grill-me`
|
||||||
|
- License: MIT, copyright (c) 2026 Matt Pocock
|
||||||
|
|
||||||
|
Matt Pocock's full copyright and license notice is preserved in
|
||||||
|
[LICENSE.mattpocock](LICENSE.mattpocock) for the adapted material. Keep both
|
||||||
|
license files with redistributed copies or substantial portions of this skill.
|
||||||
|
The two notices preserve attribution under the same MIT terms; they do not
|
||||||
|
introduce an additional license restriction or imply upstream endorsement.
|
||||||
21
resources/skills/adr-brainstorm/LICENSE.mattpocock
Normal file
21
resources/skills/adr-brainstorm/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Matt Pocock
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
21
resources/skills/adr-brainstorm/LICENSE.wanjakrah
Normal file
21
resources/skills/adr-brainstorm/LICENSE.wanjakrah
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Wanja Krah
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
103
resources/skills/adr-brainstorm/SKILL.md
Normal file
103
resources/skills/adr-brainstorm/SKILL.md
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
---
|
||||||
|
name: adr-brainstorm
|
||||||
|
description: "Brainstorm with the user about a new ADR. Use when the user wants to stress-test their thinking, or uses any combination of 'adr' and 'brainstorm' in their trigger phrase."
|
||||||
|
---
|
||||||
|
|
||||||
|
An ADR is an Architecture Decision Record. Such a record contains a decision on the architecture or design for the application landscape the user is maintaining.
|
||||||
|
If an ADR-folder exists, read the README.md of that folder to get a full understanding of any specific rules and properties of how the user works with ADR's that might not be listed in this Skill.
|
||||||
|
|
||||||
|
The goal of the ADR is to provide an architectural solution or a design solution for a given context. Therefor, two to four different approaches must be worked out with their pro's and con's, from which the user should select one as their preferred solution.
|
||||||
|
|
||||||
|
Below, the brainstorm session is mapped out in multiple steps. Each step must be completed succesfully, before continuing to the next step. Success means reaching a shared understanding and whatever the step entitles as its success criteria.
|
||||||
|
|
||||||
|
If the user, or the relevant ADR, indicates that an ADR is to supersede another ADR, read that ADR first.
|
||||||
|
|
||||||
|
## Explore decisions with the user
|
||||||
|
|
||||||
|
### Follow a design tree
|
||||||
|
|
||||||
|
Map the architectural question as a design tree: each decision may
|
||||||
|
introduce dependent decisions. Track which decisions are agreed,
|
||||||
|
open, awaiting factual investigation, or explicitly deferred.
|
||||||
|
|
||||||
|
Ask dependent questions only after their prerequisites are settled.
|
||||||
|
Revisit earlier decisions when new evidence changes their assumptions,
|
||||||
|
and explain which dependent decisions need reconsideration.
|
||||||
|
|
||||||
|
Keep the tree within the ADR's agreed scope. Suggest follow-up work
|
||||||
|
for independent topics rather than continually expanding the ADR.
|
||||||
|
A deferred topic must have a clear boundary and must not leave the
|
||||||
|
main decision ambiguous.
|
||||||
|
|
||||||
|
Finish when the user has agreed on the architectural decision and its material tradeoffs. Identify remaining implementation questions as follow-up work. Do not leave questions open that could materially change the chosen approach.
|
||||||
|
|
||||||
|
### Establish facts before asking for decisions
|
||||||
|
|
||||||
|
Finding accessible facts is the agent's responsibility. Inspect relevant
|
||||||
|
code, configuration, specifications, ADRs, and authoritative documentation
|
||||||
|
before asking the user questions those sources can answer.
|
||||||
|
|
||||||
|
Distinguish observed facts, assumptions, recommendations, and decisions.
|
||||||
|
Cite sources for material factual claims and explain uncertainty where
|
||||||
|
evidence is unavailable or conflicting. Ask the user for requirements,
|
||||||
|
priorities, constraints, and knowledge unavailable from accessible sources.
|
||||||
|
|
||||||
|
Evaluate alternatives against the agreed requirements. Present credible
|
||||||
|
alternatives fairly, including drawbacks of the recommended approach.
|
||||||
|
Do not infer that an option is unsuitable merely because the current
|
||||||
|
implementation uses something else.
|
||||||
|
|
||||||
|
### Keep the conversation collaborative
|
||||||
|
|
||||||
|
Use short rounds, usually one main question or a few closely related
|
||||||
|
questions. Number questions when asking more than one. Do not ask every
|
||||||
|
available question at once if that would hinder discussion.
|
||||||
|
|
||||||
|
For decision questions, normally provide a recommended answer and its
|
||||||
|
reason. Where natural, phrase the question so "yes" clearly accepts that
|
||||||
|
recommendation. Make its scope explicit; avoid combining independent
|
||||||
|
decisions into one yes/no question.
|
||||||
|
|
||||||
|
Use open questions when discovering requirements or when a recommendation
|
||||||
|
would prematurely narrow the discussion. Allow disagreement, alternatives,
|
||||||
|
and back-and-forth exploration before seeking a decision.
|
||||||
|
|
||||||
|
### Leave architectural decisions with the user
|
||||||
|
|
||||||
|
The agent investigates, explains tradeoffs, challenges assumptions, and
|
||||||
|
recommends. The user selects the architectural approach and accepts its
|
||||||
|
material tradeoffs.
|
||||||
|
|
||||||
|
Do not treat silence, a factual answer, or agreement on one decision as
|
||||||
|
approval of other decisions. Preserve earlier agreements without repeatedly
|
||||||
|
requesting confirmation unless their meaning or supporting facts change.
|
||||||
|
|
||||||
|
At meaningful milestones, summarize the agreed decisions and remaining
|
||||||
|
questions. Before finalizing the ADR, confirm shared understanding of the
|
||||||
|
complete proposal. Keep recommendations and unresolved assumptions out of
|
||||||
|
the Decisions section until the user accepts them.
|
||||||
|
|
||||||
|
### Guide the process through steps
|
||||||
|
|
||||||
|
Coöperate with the user via the following steps. Communicate each step in which step you are, and what the step entails. Use the template adr-template.md as a template for the final ADR.
|
||||||
|
|
||||||
|
The _first step_ is to get to a shared understanding of the probived context and the problem area the user is in need for a decision. Success for step one means you can describe the context in one to fifty sentences.
|
||||||
|
|
||||||
|
The _second step_ is to get to two to four different architectural solutions or design solutions for the described context. Every solution needs a short description of one to fifty sentences and a list of one to ten pro's and one to ten con's. allow discussion of consequences when they affect the choice. Record them formally in _step four_, but don’t postpone discovering migration costs or operational burdens until after selecting an option.
|
||||||
|
Success for step two means there are more than one credible solutions with a set of relevant pro's and con's by which a user is able to choose, or not choose that solution for the next step.
|
||||||
|
|
||||||
|
The _third step_ is to pick the best solution. Ths user must decide what solution he prefers. The choosen solution will be presented as the ADR decision. A decision may branch into a small tree of related decisions, small meaning one to five levels deep. Make the architectural decision, its scope, and its material tradeoffs unambiguous; record remaining implementation questions as follow-up work. Success of step three means there is a shared understanding of the choosen solution and what it entails.
|
||||||
|
|
||||||
|
The _fourth step_ is to make clear what the consequences will be for the application landscape. Success of step four means there is a clear understanding of the changes that should be made to software landscape and the way futurue work should be implemented.
|
||||||
|
|
||||||
|
The _fifth step_ is fill out the other properties of the ADR that is required by the template. Succes of step five means the ID of the ADR is unique, the title is short and comprehandible, and gives the user an overall idea of what the ADR is about. If the ADR supersedes another ADR, that is listed as well.
|
||||||
|
Set the Expiration Date of when the ADR should be reviewd. This should be a relative date, to be made absolute by the user as soon as the user moves the ADR to APPROVED.
|
||||||
|
Finally, set the status to REFINED. It's up to the user to set the Status to either APPROVED - meaning the ADR is accepted and to be applied for all future work - or REJECTED - meaning the ADR is not accepted and should not be taken in consideration.
|
||||||
|
|
||||||
|
When updating the ADR with the result of the brainstorm, make it both easy to glanse through, as to read in full, meaning:
|
||||||
|
- stick to the template
|
||||||
|
- keep to the formatting as is custom for markdown-files
|
||||||
|
- high use of headers and sub-headers
|
||||||
|
- high use of bullit points, lists, and tables where applicable
|
||||||
|
- keep sentences short and precise where possible
|
||||||
|
- create images with ASCII-art where possible
|
||||||
20
resources/skills/adr-brainstorm/adr-template.md
Normal file
20
resources/skills/adr-brainstorm/adr-template.md
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
**ID:** <insert ADR followed by a unique number> \
|
||||||
|
**Title:** <insert short and descriptive title> \
|
||||||
|
**Supersedes:** - \
|
||||||
|
**Status:** NEW \
|
||||||
|
**Expiration Date:** to be determined
|
||||||
|
|
||||||
|
## Context
|
||||||
|
To be determined
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
To be determined
|
||||||
|
|
||||||
|
## Options considered
|
||||||
|
To be determined
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
To be determined
|
||||||
|
|
||||||
|
## References
|
||||||
|
<insert a list of references consulted, along with a timestamp of when it was consulted>
|
||||||
3
resources/skills/adr-brainstorm/agents/openai.yaml
Normal file
3
resources/skills/adr-brainstorm/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
interface:
|
||||||
|
display_name: "ADR Brainstorm"
|
||||||
|
short_description: "Brainstorm with the user on an ADR to get it from NEW to REFINED"
|
||||||
21
resources/skills/code-review/LICENSE.mattpocock
Normal file
21
resources/skills/code-review/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Matt Pocock
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
89
resources/skills/code-review/SKILL.md
Normal file
89
resources/skills/code-review/SKILL.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
---
|
||||||
|
name: code-review
|
||||||
|
description: "Review the changes since a fixed point (commit, branch, tag, or merge-base) along two axes: Standards (does the code follow this repo's documented coding standards?) and Spec (does the code match what the originating issue/spec asked for?). Runs both reviews in parallel sub-agents and reports them side by side. Use when the user wants to review a branch, a PR, work-in-progress changes, or asks to \"review since X\"."
|
||||||
|
---
|
||||||
|
|
||||||
|
Two-axis review of the diff between `HEAD` and a fixed point the user supplies:
|
||||||
|
|
||||||
|
- **Standards**: does the code conform to this repo's documented coding standards?
|
||||||
|
- **Spec**: does the code faithfully implement the originating issue / spec?
|
||||||
|
|
||||||
|
Both axes run as **parallel sub-agents** so they don't pollute each other's context, then this skill aggregates their findings. Start every sub-agent with a fresh context window.
|
||||||
|
|
||||||
|
The issue tracker should have been provided to you. If not, tell the user to run `/setup-matt-pocock-skills`.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
### 1. Pin the fixed point
|
||||||
|
|
||||||
|
Whatever the user said is the fixed point (a commit SHA, branch name, tag, `main`, `HEAD~5`, etc.). If they didn't specify one, ask for it.
|
||||||
|
|
||||||
|
Capture the diff command once: `git diff <fixed-point>...HEAD` (three-dot, so the comparison is against the merge-base). Also note the list of commits via `git log <fixed-point>..HEAD --oneline`.
|
||||||
|
|
||||||
|
Before going further, confirm the fixed point resolves (`git rev-parse <fixed-point>`) and the diff is non-empty. A bad ref or empty diff should fail here, not inside two parallel sub-agents.
|
||||||
|
|
||||||
|
### 2. Identify the spec source
|
||||||
|
|
||||||
|
Look for the originating spec, in this order:
|
||||||
|
|
||||||
|
1. Issue references in the commit messages (`#123`, `Closes #45`, GitLab `!67`, etc.), fetched via the workflow in the tracker doc.
|
||||||
|
2. A path the user passed as an argument.
|
||||||
|
3. A spec file under `docs/`, `specs/`, or `.scratch/` matching the branch name or feature.
|
||||||
|
4. If nothing is found, ask the user where the spec is. If they say there isn't one, the **Spec** sub-agent will skip and report "no spec available".
|
||||||
|
|
||||||
|
### 3. Identify the standards sources
|
||||||
|
|
||||||
|
Search the repo for every file that documents how code should be written. When `CODING_STANDARDS.md` or `CONTRIBUTING.md` exists, it must be on the list.
|
||||||
|
|
||||||
|
On top of whatever the repo documents, the Standards axis always carries the **smell baseline** below: a fixed set of Fowler code smells (_Refactoring_, ch.3) that applies even when a repo documents nothing. Two rules bind it:
|
||||||
|
|
||||||
|
- **The repo overrides.** A documented repo standard always wins; where it endorses something the baseline would flag, suppress the smell.
|
||||||
|
- **Always a judgement call.** Each smell is a labelled heuristic ("possible Feature Envy"), never a hard violation. Like any standard here, skip anything tooling already enforces.
|
||||||
|
|
||||||
|
Each smell reads *what it is* → *how to fix*; match it against the diff:
|
||||||
|
|
||||||
|
- **Mysterious Name**: a function, variable, or type whose name doesn't reveal what it does or holds. → rename it; if no honest name comes, the design's murky.
|
||||||
|
- **Duplicated Code**: the same logic shape appears in more than one hunk or file in the change. → extract the shared shape, call it from both.
|
||||||
|
- **Feature Envy**: a method that reaches into another object's data more than its own. → move the method onto the data it envies.
|
||||||
|
- **Data Clumps**: the same few fields or params keep travelling together (a type wanting to be born). → bundle them into one type, pass that.
|
||||||
|
- **Primitive Obsession**: a primitive or string standing in for a domain concept that deserves its own type. → give the concept its own small type.
|
||||||
|
- **Repeated Switches**: the same `switch`/`if`-cascade on the same type recurs across the change. → replace with polymorphism, or one map both sites share.
|
||||||
|
- **Shotgun Surgery**: one logical change forces scattered edits across many files in the diff. → gather what changes together into one module.
|
||||||
|
- **Divergent Change**: one file or module is edited for several unrelated reasons. → split so each module changes for one reason.
|
||||||
|
- **Speculative Generality**: abstraction, parameters, or hooks added for needs the spec doesn't have. → delete it; inline back until a real need shows.
|
||||||
|
- **Message Chains**: long `a.b().c().d()` navigation the caller shouldn't depend on. → hide the walk behind one method on the first object.
|
||||||
|
- **Middle Man**: a class or function that mostly just delegates onward. → cut it, call the real target direct.
|
||||||
|
- **Refused Bequest**: a subclass or implementer that ignores or overrides most of what it inherits. → drop the inheritance, use composition.
|
||||||
|
|
||||||
|
### 4. Spawn both sub-agents in parallel
|
||||||
|
|
||||||
|
Issue both sub-agent calls together, in the foreground, and aggregate the reports they return.
|
||||||
|
|
||||||
|
**Standards sub-agent prompt** should include:
|
||||||
|
|
||||||
|
- The full diff command and commit list.
|
||||||
|
- The list of standards-source files you found in step 3, **plus the smell baseline from step 3** pasted in full (the sub-agent has no other access to it).
|
||||||
|
- The brief: "Report, per file/hunk where relevant, (a) every place the diff violates a documented standard: cite the standard (file + the rule); and (b) any baseline smell you spot: name it and quote the hunk. Distinguish hard violations from judgement calls: documented-standard breaches can be hard, but baseline smells are always judgement calls, and a documented repo standard overrides the baseline. Skip anything tooling enforces. Under 400 words."
|
||||||
|
|
||||||
|
**Spec sub-agent prompt** should include:
|
||||||
|
|
||||||
|
- The diff command and commit list.
|
||||||
|
- The path or fetched contents of the spec.
|
||||||
|
- The brief: "Report: (a) requirements the spec asked for that are missing or partial; (b) behaviour in the diff that wasn't asked for (scope creep); (c) requirements that look implemented but where the implementation looks wrong. Quote the spec line for each finding. Under 400 words."
|
||||||
|
|
||||||
|
If the spec is missing, skip the Spec sub-agent and note this in the final report.
|
||||||
|
|
||||||
|
### 5. Aggregate
|
||||||
|
|
||||||
|
Present the two reports under `## Standards` and `## Spec` headings, verbatim or lightly cleaned. Do **not** merge or rerank findings, because the two axes are deliberately separate (see _Why two axes_).
|
||||||
|
|
||||||
|
End with a one-line summary: total findings per axis, and the worst issue _within each axis_ (if any). Don't pick a single winner across axes: that's the reranking the separation exists to prevent.
|
||||||
|
|
||||||
|
## Why two axes
|
||||||
|
|
||||||
|
A change can pass one axis and fail the other:
|
||||||
|
|
||||||
|
- Code that follows every standard but implements the wrong thing → **Standards pass, Spec fail.**
|
||||||
|
- Code that does exactly what the issue asked but breaks the project's conventions → **Spec pass, Standards fail.**
|
||||||
|
|
||||||
|
Reporting them separately stops one axis from masking the other.
|
||||||
3
resources/skills/code-review/agents/openai.yaml
Normal file
3
resources/skills/code-review/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
interface:
|
||||||
|
display_name: "Code Review"
|
||||||
|
short_description: "Review a diff on standards and spec"
|
||||||
21
resources/skills/grill-me/LICENSE.mattpocock
Normal file
21
resources/skills/grill-me/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Matt Pocock
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
7
resources/skills/grill-me/SKILL.md
Normal file
7
resources/skills/grill-me/SKILL.md
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
---
|
||||||
|
name: grill-me
|
||||||
|
description: A relentless interview to sharpen a plan or design.
|
||||||
|
disable-model-invocation: true
|
||||||
|
---
|
||||||
|
|
||||||
|
Call the Skill tool with "grilling".
|
||||||
5
resources/skills/grill-me/agents/openai.yaml
Normal file
5
resources/skills/grill-me/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
interface:
|
||||||
|
display_name: "Grill Me"
|
||||||
|
short_description: "Sharpen a plan through interview"
|
||||||
|
policy:
|
||||||
|
allow_implicit_invocation: false
|
||||||
21
resources/skills/grilling/LICENSE.mattpocock
Normal file
21
resources/skills/grilling/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Matt Pocock
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
30
resources/skills/grilling/SKILL.md
Normal file
30
resources/skills/grilling/SKILL.md
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
---
|
||||||
|
name: grilling
|
||||||
|
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
|
||||||
|
---
|
||||||
|
|
||||||
|
Interview the user relentlessly until you reach a shared understanding. Map this as a **design tree**: every decision branches into the decisions that hang off it.
|
||||||
|
|
||||||
|
Work the tree in **rounds**. The **frontier** is every decision whose prerequisites are already settled: the questions you can ask _now_ without guessing at answers you haven't heard yet. Ask the whole frontier in one round: number each question and give your recommended answer. Then wait for the user's answers before the next round.
|
||||||
|
|
||||||
|
Format a round like so:
|
||||||
|
|
||||||
|
```
|
||||||
|
❓ **Q1** - **<question title>**: <question body, might be multiple paragraphs, including multiple choices>
|
||||||
|
|
||||||
|
➡️ <your recommended answer>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
❓ **Q2** - **<question title>**: <question body, might be multiple paragraphs, including multiple choices>
|
||||||
|
|
||||||
|
➡️ <your recommended answer>
|
||||||
|
```
|
||||||
|
|
||||||
|
Word each question so "yes" accepts your recommended answer.
|
||||||
|
|
||||||
|
Each round the user answers reshapes the tree: settled decisions push the frontier outward and unblock questions that depended on them. Recompute the frontier and ask the next round. A question whose answer depends on another question still open in this round belongs to a _later_ round, not this one.
|
||||||
|
|
||||||
|
Finding _facts_ is your job, never the user's. When a frontier question needs a fact from the environment (filesystem, tools, etc.), dispatch a sub-agent to find it; don't ask the user for anything you could look up yourself. Don't block on it: a running exploration is an unsettled prerequisite, so only the questions downstream of it wait for the sub-agent to report; ask the rest of the frontier now. The _decisions_ are the user's: put each to them and wait.
|
||||||
|
|
||||||
|
The session is done when the frontier is empty: every branch of the design tree visited, nothing left silently assumed. Do not act on it until the user confirms you have reached a shared understanding.
|
||||||
3
resources/skills/grilling/agents/openai.yaml
Normal file
3
resources/skills/grilling/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
interface:
|
||||||
|
display_name: "Grilling"
|
||||||
|
short_description: "Stress-test thinking a round of questions at a time"
|
||||||
14
resources/skills/upstream.json
Normal file
14
resources/skills/upstream.json
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"repository": "https://github.com/mattpocock/skills",
|
||||||
|
"commit": "59db609eb1e18aaf59b63284d0e52255aff0c63c",
|
||||||
|
"source": "https://skills.sh/mattpocock/skills/grill-me",
|
||||||
|
"skills": {
|
||||||
|
"grill-me": "skills/productivity/grill-me",
|
||||||
|
"grilling": "skills/productivity/grilling",
|
||||||
|
"code-review": {
|
||||||
|
"path": "skills/engineering/code-review",
|
||||||
|
"commit": "a496f5513d4a4cf889bc48e189e579f4b0bdf6aa",
|
||||||
|
"source": "https://www.skills.sh/mattpocock/skills/code-review"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
30
resources/tooling/coverage/Coverage.java
Normal file
30
resources/tooling/coverage/Coverage.java
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
import java.nio.file.Path;
|
||||||
|
import javax.xml.parsers.DocumentBuilderFactory;
|
||||||
|
import org.w3c.dom.Element;
|
||||||
|
|
||||||
|
/** Exact module-wide line gate; consumes JaCoCo XML without rounding percentages. */
|
||||||
|
public class Coverage {
|
||||||
|
public static void main(String[] args) throws Exception {
|
||||||
|
final var factory = DocumentBuilderFactory.newInstance();
|
||||||
|
factory.setFeature("http://xml.org/sax/features/external-general-entities", false);
|
||||||
|
factory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
|
||||||
|
factory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
|
||||||
|
final Element report =
|
||||||
|
factory.newDocumentBuilder().parse(Path.of(args[0]).toFile()).getDocumentElement();
|
||||||
|
for (var child = report.getFirstChild(); child != null; child = child.getNextSibling()) {
|
||||||
|
if (child instanceof Element counter
|
||||||
|
&& counter.getTagName().equals("counter")
|
||||||
|
&& counter.getAttribute("type").equals("LINE")) {
|
||||||
|
final long covered = Long.parseLong(counter.getAttribute("covered"));
|
||||||
|
final long missed = Long.parseLong(counter.getAttribute("missed"));
|
||||||
|
final long total = covered + missed;
|
||||||
|
System.out.println(
|
||||||
|
"Backend line coverage: " + covered + "/" + total + " (" + 100.0 * covered / total + "%)");
|
||||||
|
if (total == 0 || 5 * covered < 4 * total)
|
||||||
|
throw new IllegalStateException("Backend line coverage must reach 80%: " + args[0]);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new IllegalStateException("Missing module LINE counter: " + args[0]);
|
||||||
|
}
|
||||||
|
}
|
||||||
136
resources/tooling/format/Format.java
Normal file
136
resources/tooling/format/Format.java
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
import com.palantir.javaformat.java.Formatter;
|
||||||
|
import com.palantir.javaformat.java.JavaFormatterOptions;
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.security.MessageDigest;
|
||||||
|
import java.util.*;
|
||||||
|
import org.openrewrite.*;
|
||||||
|
import org.openrewrite.internal.InMemoryLargeSourceSet;
|
||||||
|
import org.openrewrite.text.*;
|
||||||
|
|
||||||
|
/** Source-launcher bootstrap: no project-specific artifact is resolved or installed. */
|
||||||
|
public class Format {
|
||||||
|
public static void main(String[] args) throws Exception {
|
||||||
|
Path root = Path.of(args[0]).toAbsolutePath().normalize();
|
||||||
|
boolean check = switch (args[1]) {
|
||||||
|
case "check" -> true;
|
||||||
|
case "apply" -> false;
|
||||||
|
default -> throw new IllegalArgumentException("format.mode must be apply or check");
|
||||||
|
};
|
||||||
|
Path cacheFile = root.resolve("target/palantir-cache.properties");
|
||||||
|
var cache = new Properties();
|
||||||
|
if (Files.exists(cacheFile)) {
|
||||||
|
try (var input = Files.newInputStream(cacheFile)) {
|
||||||
|
cache.load(input);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
String configuration = digest(Files.readString(root.resolve("pom.xml"))
|
||||||
|
+ Files.readString(root.resolve("generated/tooling/format/Format.java"))
|
||||||
|
+ System.getProperty("java.version")
|
||||||
|
+ System.getProperty("java.class.path"));
|
||||||
|
if (!configuration.equals(cache.getProperty("configuration"))) cache.clear();
|
||||||
|
cache.setProperty("configuration", configuration);
|
||||||
|
List<SourceFile> sources = new ArrayList<>();
|
||||||
|
var settings = new Properties();
|
||||||
|
try (var input = Files.newInputStream(root.resolve("engineering.properties"))) {
|
||||||
|
settings.load(input);
|
||||||
|
}
|
||||||
|
for (String directory :
|
||||||
|
settings.getProperty("FORMAT_DIRECTORIES", "backend/src").split(",")) {
|
||||||
|
if (!Files.isDirectory(root.resolve(directory))) continue;
|
||||||
|
try (var paths = Files.walk(root.resolve(directory))) {
|
||||||
|
for (Path path : paths.filter(p -> p.toString().endsWith(".java"))
|
||||||
|
.sorted()
|
||||||
|
.toList()) {
|
||||||
|
try {
|
||||||
|
String content = Files.readString(path);
|
||||||
|
String key = root.relativize(path).toString();
|
||||||
|
if (!digest(content).equals(cache.getProperty(key))) {
|
||||||
|
sources.add(PlainText.builder()
|
||||||
|
.sourcePath(root.relativize(path))
|
||||||
|
.text(content)
|
||||||
|
.build());
|
||||||
|
}
|
||||||
|
} catch (Exception error) {
|
||||||
|
throw new IllegalStateException(path + ": " + error.getMessage(), error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var context = new InMemoryExecutionContext(error -> {
|
||||||
|
throw new IllegalStateException("Formatting failed: " + error.getMessage(), error);
|
||||||
|
});
|
||||||
|
var results = new PalantirRecipe()
|
||||||
|
.run(new InMemoryLargeSourceSet(sources), context)
|
||||||
|
.getChangeset()
|
||||||
|
.getAllResults();
|
||||||
|
for (Result result : results) {
|
||||||
|
System.out.println((check ? "Needs formatting: " : "Formatted: ")
|
||||||
|
+ result.getAfter().getSourcePath());
|
||||||
|
}
|
||||||
|
if (check && !results.isEmpty()) {
|
||||||
|
throw new IllegalStateException("Run mvn -N exec:exec@format to apply Palantir formatting.");
|
||||||
|
}
|
||||||
|
// Complete parsing/formatting before writing any maintained file. Each replacement is atomic.
|
||||||
|
if (!check) {
|
||||||
|
for (Result result : results) {
|
||||||
|
Path destination = root.resolve(result.getAfter().getSourcePath());
|
||||||
|
Path temporary = Files.createTempFile(destination.getParent(), ".palantir-", ".tmp");
|
||||||
|
try {
|
||||||
|
Files.writeString(temporary, result.getAfter().printAll());
|
||||||
|
Files.setPosixFilePermissions(temporary, Files.getPosixFilePermissions(destination));
|
||||||
|
Files.move(
|
||||||
|
temporary,
|
||||||
|
destination,
|
||||||
|
StandardCopyOption.ATOMIC_MOVE,
|
||||||
|
StandardCopyOption.REPLACE_EXISTING);
|
||||||
|
} finally {
|
||||||
|
Files.deleteIfExists(temporary);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (SourceFile source : sources) {
|
||||||
|
cache.setProperty(
|
||||||
|
source.getSourcePath().toString(), digest(Files.readString(root.resolve(source.getSourcePath()))));
|
||||||
|
}
|
||||||
|
Files.createDirectories(cacheFile.getParent());
|
||||||
|
try (var output = Files.newOutputStream(cacheFile)) {
|
||||||
|
cache.store(output, "Successfully formatted content only");
|
||||||
|
}
|
||||||
|
System.out.println(
|
||||||
|
"Palantir " + args[1] + ": " + sources.size() + " Java files, " + results.size() + " differences");
|
||||||
|
}
|
||||||
|
|
||||||
|
static String digest(String value) throws Exception {
|
||||||
|
return HexFormat.of()
|
||||||
|
.formatHex(MessageDigest.getInstance("SHA-256")
|
||||||
|
.digest(value.getBytes(java.nio.charset.StandardCharsets.UTF_8)));
|
||||||
|
}
|
||||||
|
|
||||||
|
public static final class PalantirRecipe extends Recipe {
|
||||||
|
public String getDisplayName() {
|
||||||
|
return "Palantir Java Format";
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getDescription() {
|
||||||
|
return "Apply the pinned Palantir engine and its native import handling.";
|
||||||
|
}
|
||||||
|
|
||||||
|
public TreeVisitor<?, ExecutionContext> getVisitor() {
|
||||||
|
return new PlainTextVisitor<ExecutionContext>() {
|
||||||
|
final Formatter formatter = Formatter.createFormatter(JavaFormatterOptions.builder()
|
||||||
|
.style(JavaFormatterOptions.Style.PALANTIR)
|
||||||
|
.formatJavadoc(false)
|
||||||
|
.build());
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public PlainText visitText(PlainText text, ExecutionContext context) {
|
||||||
|
try {
|
||||||
|
return text.withText(formatter.formatSourceAndFixImports(text.getText()));
|
||||||
|
} catch (Exception | LinkageError error) {
|
||||||
|
throw new IllegalStateException(text.getSourcePath() + ": " + error, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
21
resources/tooling/standards/checkstyle.xml
Normal file
21
resources/tooling/standards/checkstyle.xml
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
<!DOCTYPE module PUBLIC "-//Checkstyle//DTD Checkstyle Configuration 1.3//EN" "https://checkstyle.org/dtds/configuration_1_3.dtd">
|
||||||
|
<module name="Checker">
|
||||||
|
<module name="TreeWalker">
|
||||||
|
<module name="TypeName"/>
|
||||||
|
<module name="MethodName"/>
|
||||||
|
<module name="MemberName"/>
|
||||||
|
<module name="ParameterName"/>
|
||||||
|
<module name="LocalVariableName"/>
|
||||||
|
<module name="LocalFinalVariableName"/>
|
||||||
|
<module name="StaticVariableName"/>
|
||||||
|
<module name="LambdaParameterName"/>
|
||||||
|
<module name="RecordComponentName"/>
|
||||||
|
<module name="LeftCurly"/>
|
||||||
|
<module name="CatchParameterName"/>
|
||||||
|
<module name="ConstantName"/>
|
||||||
|
<module name="PackageName"><property name="format" value="^[a-z]+(\.[a-z][a-z0-9]*)*$"/></module>
|
||||||
|
<module name="OneTopLevelClass"/>
|
||||||
|
<module name="AvoidStarImport"/>
|
||||||
|
<module name="NeedBraces"/>
|
||||||
|
</module>
|
||||||
|
</module>
|
||||||
16
resources/tooling/standards/pom.xml
Normal file
16
resources/tooling/standards/pom.xml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
<groupId>nl.agilecc</groupId><artifactId>backend-standards</artifactId><version>1</version>
|
||||||
|
<properties><maven.compiler.release>25</maven.compiler.release><project.build.sourceEncoding>UTF-8</project.build.sourceEncoding></properties>
|
||||||
|
<dependencies>
|
||||||
|
<dependency><groupId>jakarta.inject</groupId><artifactId>jakarta.inject-api</artifactId><version>2.0.1</version><scope>test</scope></dependency>
|
||||||
|
<dependency><groupId>net.sourceforge.pmd</groupId><artifactId>pmd-java</artifactId><version>7.28.0</version></dependency>
|
||||||
|
<dependency><groupId>com.puppycrawl.tools</groupId><artifactId>checkstyle</artifactId><version>12.3.1</version></dependency>
|
||||||
|
<dependency><groupId>org.junit.jupiter</groupId><artifactId>junit-jupiter</artifactId><version>5.14.1</version><scope>test</scope></dependency>
|
||||||
|
</dependencies>
|
||||||
|
<build><plugins>
|
||||||
|
<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.16.0</version></plugin>
|
||||||
|
<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-surefire-plugin</artifactId><version>3.6.1</version></plugin>
|
||||||
|
<plugin><groupId>org.codehaus.mojo</groupId><artifactId>exec-maven-plugin</artifactId><version>3.6.3</version><configuration><mainClass>Standards</mainClass><arguments><argument>${project.basedir}/../../..</argument></arguments></configuration></plugin>
|
||||||
|
</plugins></build>
|
||||||
|
</project>
|
||||||
590
resources/tooling/standards/src/main/java/Standards.java
Normal file
590
resources/tooling/standards/src/main/java/Standards.java
Normal file
@@ -0,0 +1,590 @@
|
|||||||
|
import com.puppycrawl.tools.checkstyle.*;
|
||||||
|
import com.puppycrawl.tools.checkstyle.api.*;
|
||||||
|
import java.io.*;
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.util.*;
|
||||||
|
import net.sourceforge.pmd.*;
|
||||||
|
import net.sourceforge.pmd.cpd.*;
|
||||||
|
import net.sourceforge.pmd.lang.*;
|
||||||
|
import net.sourceforge.pmd.lang.ast.*;
|
||||||
|
import net.sourceforge.pmd.lang.java.ast.*;
|
||||||
|
import net.sourceforge.pmd.lang.java.rule.AbstractJavaRule;
|
||||||
|
import net.sourceforge.pmd.lang.java.symbols.*;
|
||||||
|
import net.sourceforge.pmd.lang.java.types.*;
|
||||||
|
import net.sourceforge.pmd.lang.metrics.MetricOptions;
|
||||||
|
import net.sourceforge.pmd.lang.rule.RuleSet;
|
||||||
|
|
||||||
|
/** Deliberately independent of the application build and its annotation processors. */
|
||||||
|
public class Standards {
|
||||||
|
static final List<String> findings = new ArrayList<>();
|
||||||
|
static final Map<Path, Set<Integer>> codeLines = new HashMap<>();
|
||||||
|
static final Set<String> ownedTypes = new HashSet<>();
|
||||||
|
static final Map<String, ASTTypeDeclaration> types = new HashMap<>();
|
||||||
|
static final List<ASTCompilationUnit> units = new ArrayList<>();
|
||||||
|
static Path root;
|
||||||
|
static String module;
|
||||||
|
|
||||||
|
record Waiver(String metric, String executable, double value, String reason) {}
|
||||||
|
|
||||||
|
static final List<Waiver> waivers = new ArrayList<>();
|
||||||
|
|
||||||
|
public static void main(String[] args) throws Exception {
|
||||||
|
findings.clear();
|
||||||
|
codeLines.clear();
|
||||||
|
types.clear();
|
||||||
|
units.clear();
|
||||||
|
ownedTypes.clear();
|
||||||
|
root = Path.of(args[0]).toAbsolutePath().normalize();
|
||||||
|
final var settings = new Properties();
|
||||||
|
try (var input = Files.newInputStream(root.resolve("engineering.properties"))) {
|
||||||
|
settings.load(input);
|
||||||
|
}
|
||||||
|
module = settings.getProperty("BACKEND_MODULE");
|
||||||
|
if (module == null || module.isBlank()) throw new IllegalArgumentException("BACKEND_MODULE is required");
|
||||||
|
waivers.clear();
|
||||||
|
final Path waiverFile = root.resolve("tooling/standards/waivers.tsv");
|
||||||
|
if (Files.exists(waiverFile))
|
||||||
|
for (String line : Files.readAllLines(waiverFile)) {
|
||||||
|
if (line.isBlank() || line.startsWith("#")) continue;
|
||||||
|
final String[] fields = line.split("\t", -1);
|
||||||
|
if (fields.length != 4 || fields[0].contains("*") || fields[1].contains("*") || fields[3].isBlank())
|
||||||
|
throw new IllegalArgumentException("Invalid exact waiver in " + waiverFile + ": " + line);
|
||||||
|
waivers.add(new Waiver(fields[0], fields[1], Double.parseDouble(fields[2]), fields[3]));
|
||||||
|
}
|
||||||
|
final Path output = root.resolve(module + "/target/standards");
|
||||||
|
Files.createDirectories(output);
|
||||||
|
final List<Path> files = new ArrayList<>();
|
||||||
|
for (String scope : List.of("main", "test")) {
|
||||||
|
try (var paths = Files.walk(root.resolve(module + "/src/" + scope + "/java"))) {
|
||||||
|
paths.filter(p -> p.toString().endsWith(".java")).sorted().forEach(files::add);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (Path file : files) {
|
||||||
|
final var matcher =
|
||||||
|
java.util.regex.Pattern.compile("package\\s+([\\w.]+)\\s*;").matcher(Files.readString(file));
|
||||||
|
if (matcher.find())
|
||||||
|
ownedTypes.add(
|
||||||
|
matcher.group(1) + "." + file.getFileName().toString().replace(".java", ""));
|
||||||
|
}
|
||||||
|
checkstyle(files);
|
||||||
|
final PMDConfiguration config = new PMDConfiguration();
|
||||||
|
config.setDefaultLanguageVersion(
|
||||||
|
LanguageRegistry.PMD.getLanguageById("java").getVersion("25"));
|
||||||
|
config.setThreads(1);
|
||||||
|
config.prependAuxClasspath(
|
||||||
|
Path.of(System.getProperty("java.home"), "lib/jrt-fs.jar").toString());
|
||||||
|
config.setIgnoreIncrementalAnalysis(true);
|
||||||
|
// Compiled application/test dependencies permit PMD to resolve external types.
|
||||||
|
final Path cp = root.resolve(module + "/target/standards-classpath.txt");
|
||||||
|
if (Files.exists(cp))
|
||||||
|
config.prependAuxClasspath(
|
||||||
|
Files.readString(cp) + File.pathSeparator + root.resolve(module + "/target/classes"));
|
||||||
|
try (var analysis = PmdAnalysis.create(config)) {
|
||||||
|
analysis.addRuleSet(RuleSet.forSingleRule(new Measurements()));
|
||||||
|
for (String rule : List.of(
|
||||||
|
"category/java/codestyle.xml/LocalVariableCouldBeFinal",
|
||||||
|
"category/java/codestyle.xml/MethodArgumentCouldBeFinal",
|
||||||
|
"category/java/design.xml/ImmutableField"))
|
||||||
|
analysis.addRuleSet(analysis.newRuleSetLoader().loadFromResource(rule));
|
||||||
|
for (Path file : files) analysis.files().addFile(file);
|
||||||
|
final var report = analysis.performAnalysisAndCollectReport();
|
||||||
|
for (var error : report.getProcessingErrors())
|
||||||
|
findings.add("BLOCK parser " + error.getFileId() + " " + error.getMsg());
|
||||||
|
for (var violation : report.getViolations())
|
||||||
|
findings.add("BLOCK " + violation.getFileId() + ":" + violation.getBeginLine() + " "
|
||||||
|
+ violation.getRule().getName() + " " + violation.getDescription());
|
||||||
|
for (var violation : report.getSuppressedViolations())
|
||||||
|
findings.add("BLOCK Unapproved suppression: " + violation);
|
||||||
|
for (var error : report.getConfigurationErrors()) findings.add("BLOCK configuration " + error);
|
||||||
|
}
|
||||||
|
duplication(files);
|
||||||
|
Collections.sort(findings);
|
||||||
|
Files.write(output.resolve("report.txt"), findings);
|
||||||
|
findings.stream().filter(f -> !f.startsWith("METRIC ")).forEach(System.out::println);
|
||||||
|
System.out.println(
|
||||||
|
"Backend standards: " + files.size() + " source files; full metrics: " + output.resolve("report.txt"));
|
||||||
|
if (findings.stream().anyMatch(s -> s.startsWith("BLOCK")))
|
||||||
|
throw new IllegalStateException("Backend standards failed; see " + output.resolve("report.txt"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static void checkstyle(List<Path> files) throws Exception {
|
||||||
|
final Checker checker = new Checker();
|
||||||
|
checker.setModuleClassLoader(Standards.class.getClassLoader());
|
||||||
|
checker.configure(ConfigurationLoader.loadConfiguration(
|
||||||
|
root.resolve("generated/tooling/standards/checkstyle.xml").toString(),
|
||||||
|
new PropertiesExpander(new Properties())));
|
||||||
|
checker.addListener(new AuditListener() {
|
||||||
|
public void auditStarted(AuditEvent e) {}
|
||||||
|
|
||||||
|
public void auditFinished(AuditEvent e) {}
|
||||||
|
|
||||||
|
public void fileStarted(AuditEvent e) {}
|
||||||
|
|
||||||
|
public void fileFinished(AuditEvent e) {}
|
||||||
|
|
||||||
|
public void addError(AuditEvent e) {
|
||||||
|
findings.add("BLOCK " + e.getFileName() + ":" + e.getLine() + " " + e.getMessage());
|
||||||
|
}
|
||||||
|
|
||||||
|
public void addException(AuditEvent e, Throwable t) {
|
||||||
|
findings.add("BLOCK " + e.getFileName() + " parser: " + t);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
checker.process(files.stream().map(Path::toFile).toList());
|
||||||
|
} finally {
|
||||||
|
checker.destroy();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static class Measurements extends AbstractJavaRule {
|
||||||
|
public Measurements() {
|
||||||
|
setName("BackendMeasurements");
|
||||||
|
setMessage("Backend standards");
|
||||||
|
setLanguage(LanguageRegistry.PMD.getLanguageById("java"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Object visit(ASTCompilationUnit unit, Object data) {
|
||||||
|
units.add(unit);
|
||||||
|
for (var type : all(unit, ASTTypeDeclaration.class))
|
||||||
|
types.put(type.getSymbol().getBinaryName(), type);
|
||||||
|
final Set<Integer> lines = lines(unit);
|
||||||
|
codeLines.put(path(unit), lines);
|
||||||
|
for (var node : all(unit, JavaNode.class)) {
|
||||||
|
if (node instanceof ASTExecutableDeclaration executable) executable(executable);
|
||||||
|
if (node instanceof ASTCompactConstructorDeclaration compact) {
|
||||||
|
final var cyclo = new org.apache.commons.lang3.mutable.MutableInt(1);
|
||||||
|
compact.acceptVisitor(new ScopedCyclo(compact), cyclo);
|
||||||
|
final var cognitive =
|
||||||
|
new net.sourceforge.pmd.lang.java.metrics.internal.CognitiveComplexityVisitor.State(
|
||||||
|
compact);
|
||||||
|
compact.acceptVisitor(new ScopedCognitive(), cognitive);
|
||||||
|
metric(compact, "cyclomatic", cyclo.intValue(), 7, 10);
|
||||||
|
metric(compact, "cognitive", cognitive.getComplexity(), 10, 15);
|
||||||
|
metric(compact, "executableLines", lines(compact).size(), 30, 50);
|
||||||
|
metric(compact, "nesting", nesting(compact.getBody(), 0), 3, 4);
|
||||||
|
metric(
|
||||||
|
compact,
|
||||||
|
"parameters",
|
||||||
|
compact.getEnclosingType().getRecordComponents().size(),
|
||||||
|
5,
|
||||||
|
7);
|
||||||
|
visibility(compact);
|
||||||
|
}
|
||||||
|
if (node instanceof ASTVariableId variable) variable(variable);
|
||||||
|
if (node instanceof ASTTypeDeclaration nested
|
||||||
|
&& nested.isNested()
|
||||||
|
&& !nested.isLocal()
|
||||||
|
&& !nested.isAnonymous()) visibility(nested);
|
||||||
|
if (node instanceof ASTImportDeclaration imp && imp.isStatic() && production(unit))
|
||||||
|
block(imp, "Static imports are test-only");
|
||||||
|
if (node instanceof ASTExecutableDeclaration || node instanceof ASTFieldDeclaration)
|
||||||
|
visibility((ModifierOwner) node);
|
||||||
|
if (node instanceof ASTMethodDeclaration method
|
||||||
|
&& method.isOverride()
|
||||||
|
&& !method.isAnnotationPresent("java.lang.Override"))
|
||||||
|
block(method, "Overriding methods require @Override");
|
||||||
|
}
|
||||||
|
for (var type : all(unit, ASTTypeDeclaration.class)) typeMetrics(type);
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static Path path(Node node) {
|
||||||
|
return Path.of(node.getTextDocument().getFileId().getAbsolutePath())
|
||||||
|
.toAbsolutePath()
|
||||||
|
.normalize();
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean production(Node node) {
|
||||||
|
return path(node).startsWith(root.resolve(module + "/src/main/java"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static <T> List<T> all(Node node, Class<T> clazz) {
|
||||||
|
final List<T> result = new ArrayList<>();
|
||||||
|
if (clazz.isInstance(node)) result.add(clazz.cast(node));
|
||||||
|
for (int i = 0; i < node.getNumChildren(); i++) result.addAll(all(node.getChild(i), clazz));
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void block(Node node, String message) {
|
||||||
|
findings.add("BLOCK " + location(node) + " " + message);
|
||||||
|
}
|
||||||
|
|
||||||
|
static String location(Node node) {
|
||||||
|
return root.relativize(path(node)) + ":" + node.getBeginLine();
|
||||||
|
}
|
||||||
|
|
||||||
|
static void metric(Node node, String name, double value, double review, double limit) {
|
||||||
|
if (value > limit)
|
||||||
|
for (Waiver waiver : waivers) {
|
||||||
|
if (waiver.metric().equals(name)
|
||||||
|
&& waiver.executable().equals(identity(node))
|
||||||
|
&& waiver.value() == value) {
|
||||||
|
findings.add("WAIVER " + location(node) + " " + name + "=" + value + " " + waiver.reason());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
findings.add((value > limit ? "BLOCK " : value > review ? "REVIEW " : "METRIC ") + location(node) + " " + name
|
||||||
|
+ "=" + value);
|
||||||
|
}
|
||||||
|
|
||||||
|
static String identity(Node node) {
|
||||||
|
if (node instanceof ASTCompactConstructorDeclaration compact) {
|
||||||
|
final List<String> parameters = new ArrayList<>();
|
||||||
|
for (var component : compact.getEnclosingType().getRecordComponents())
|
||||||
|
parameters.add(component.getTypeMirror().toString());
|
||||||
|
return compact.getEnclosingType().getBinaryName() + "#<init>(" + String.join(",", parameters) + ")";
|
||||||
|
}
|
||||||
|
if (node instanceof ASTExecutableDeclaration executable) {
|
||||||
|
final List<String> parameters = new ArrayList<>();
|
||||||
|
for (var parameter : executable.getFormalParameters())
|
||||||
|
parameters.add(parameter.getTypeMirror().toString());
|
||||||
|
return executable.getEnclosingType().getBinaryName() + "#"
|
||||||
|
+ (executable instanceof ASTConstructorDeclaration ? "<init>" : executable.getName())
|
||||||
|
+ "(" + String.join(",", parameters) + ")";
|
||||||
|
}
|
||||||
|
return node instanceof ASTTypeDeclaration type ? type.getBinaryName() : location(node);
|
||||||
|
}
|
||||||
|
|
||||||
|
static Set<Integer> lines(JavaNode node) {
|
||||||
|
final Set<Integer> result = new HashSet<>();
|
||||||
|
var token = node.getFirstToken();
|
||||||
|
while (token != null && !token.isEof()) {
|
||||||
|
final var loc = token.getReportLocation();
|
||||||
|
for (int line = loc.getStartLine(); line <= loc.getEndLine(); line++) result.add(line);
|
||||||
|
if (token == node.getLastToken()) break;
|
||||||
|
token = token.getNext();
|
||||||
|
}
|
||||||
|
final String[] sourceLines = node.getTextDocument().getText().toString().split("\\R", -1);
|
||||||
|
result.removeIf(line -> line > sourceLines.length || sourceLines[line - 1].isBlank());
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void executable(ASTExecutableDeclaration method) {
|
||||||
|
{
|
||||||
|
metric(
|
||||||
|
method,
|
||||||
|
"parameters",
|
||||||
|
method.getArity(),
|
||||||
|
method instanceof ASTMethodDeclaration ? 4 : 5,
|
||||||
|
method instanceof ASTMethodDeclaration ? 5 : 7);
|
||||||
|
}
|
||||||
|
if (method.getBody() == null) return;
|
||||||
|
metric(method, "cyclomatic", cyclomatic(method), 7, 10);
|
||||||
|
metric(method, "cognitive", cognitive(method), 10, 15);
|
||||||
|
metric(method, "executableLines", lines(method).size(), 30, 50);
|
||||||
|
metric(method, "nesting", nesting(method.getBody(), 0), 3, 4);
|
||||||
|
}
|
||||||
|
|
||||||
|
static final class ScopedCyclo extends net.sourceforge.pmd.lang.java.metrics.internal.CycloVisitor {
|
||||||
|
ScopedCyclo(JavaNode node) {
|
||||||
|
super(MetricOptions.emptyOptions(), node);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Void visit(ASTLambdaExpression lambda, org.apache.commons.lang3.mutable.MutableInt value) {
|
||||||
|
for (int i = 0; i < lambda.getNumChildren(); i++) lambda.getChild(i).acceptVisitor(this, value);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static final class ScopedCognitive
|
||||||
|
extends net.sourceforge.pmd.lang.java.metrics.internal.CognitiveComplexityVisitor {
|
||||||
|
@Override
|
||||||
|
public Void visit(ASTClassBody body, State state) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Void visitJavaNode(JavaNode node, State state) {
|
||||||
|
return node instanceof ASTTypeDeclaration || node instanceof ASTTypeBody
|
||||||
|
? null
|
||||||
|
: super.visitJavaNode(node, state);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static int cyclomatic(ASTExecutableDeclaration method) {
|
||||||
|
final var value = new org.apache.commons.lang3.mutable.MutableInt();
|
||||||
|
method.acceptVisitor(new ScopedCyclo(method), value);
|
||||||
|
return value.intValue();
|
||||||
|
}
|
||||||
|
|
||||||
|
static int cognitive(ASTExecutableDeclaration method) {
|
||||||
|
final var state = new net.sourceforge.pmd.lang.java.metrics.internal.CognitiveComplexityVisitor.State(method);
|
||||||
|
method.acceptVisitor(new ScopedCognitive(), state);
|
||||||
|
return state.getComplexity();
|
||||||
|
}
|
||||||
|
|
||||||
|
static int nesting(Node node, int depth) {
|
||||||
|
if (node instanceof ASTTypeDeclaration) return depth;
|
||||||
|
final String kind = node.getXPathNodeName();
|
||||||
|
final boolean control = Set.of(
|
||||||
|
"IfStatement",
|
||||||
|
"ForStatement",
|
||||||
|
"ForeachStatement",
|
||||||
|
"WhileStatement",
|
||||||
|
"DoStatement",
|
||||||
|
"SwitchStatement",
|
||||||
|
"SwitchExpression",
|
||||||
|
"CatchClause")
|
||||||
|
.contains(kind);
|
||||||
|
final boolean elseIf = node instanceof ASTIfStatement
|
||||||
|
&& node.getParent() instanceof ASTIfStatement p
|
||||||
|
&& p.getElseBranch() == node;
|
||||||
|
final int current = depth + (control && !elseIf ? 1 : 0);
|
||||||
|
int maximum = current;
|
||||||
|
for (int i = 0; i < node.getNumChildren(); i++) maximum = Math.max(maximum, nesting(node.getChild(i), current));
|
||||||
|
return maximum;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void visibility(ModifierOwner member) {
|
||||||
|
final ASTTypeDeclaration owner = ((JavaNode) member).getEnclosingType();
|
||||||
|
if (owner == null || owner.isAnnotation()) return;
|
||||||
|
if (!member.getModifiers().hasAnyExplicitly(JModifier.PUBLIC, JModifier.PROTECTED, JModifier.PRIVATE))
|
||||||
|
block(member, "Declare explicit member visibility");
|
||||||
|
}
|
||||||
|
|
||||||
|
static void variable(ASTVariableId id) {
|
||||||
|
if (id.isEnumConstant() && !id.getName().matches("[A-Z][A-Z0-9]*(_[A-Z0-9]+)*"))
|
||||||
|
block(id, "Enum constants require UPPER_SNAKE_CASE");
|
||||||
|
if (id.isRecordComponent() || id.isEnumConstant() || id.isUnnamed()) return;
|
||||||
|
if (id.isTypeInferred() && !id.isLambdaParameter() && !id.isPatternBinding()) {
|
||||||
|
final ASTExpression initializer = id.getInitializer();
|
||||||
|
if (!(initializer instanceof ASTConstructorCall)
|
||||||
|
&& !(initializer instanceof ASTCastExpression)
|
||||||
|
&& !(initializer instanceof ASTArrayAllocation)) {
|
||||||
|
block(id, "var requires a visible constructor or cast type");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (id.isLambdaParameter()
|
||||||
|
&& id.getParent() instanceof ASTLambdaParameter parameter
|
||||||
|
&& parameter.hasVarKeyword()) block(id, "Use an explicit lambda parameter type with final");
|
||||||
|
if (id.isFinal()) return;
|
||||||
|
final boolean written =
|
||||||
|
id.getLocalUsages().stream().anyMatch(u -> u.getAccessType() != ASTAssignableExpr.AccessType.READ);
|
||||||
|
if (written) return;
|
||||||
|
if (id.isLambdaParameter()) {
|
||||||
|
block(id, "Assigned-once lambda parameter requires final: " + id.getName());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (id.isField()) {
|
||||||
|
final ASTFieldDeclaration field =
|
||||||
|
id.ancestors(ASTFieldDeclaration.class).first();
|
||||||
|
if (field == null
|
||||||
|
|| id.getInitializer() == null
|
||||||
|
|| field.isAnnotationPresent("jakarta.inject.Inject")
|
||||||
|
|| field.isAnnotationPresent("lombok.Setter")
|
||||||
|
|| id.getEnclosingType().isAnnotationPresent("lombok.Setter")
|
||||||
|
|| id.getEnclosingType().isAnnotationPresent("lombok.Data")) return;
|
||||||
|
}
|
||||||
|
block(id, "Assigned-once variable requires final: " + id.getName());
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean owned(String name) {
|
||||||
|
return types.containsKey(name) || ownedTypes.stream().anyMatch(n -> name.equals(n) || name.startsWith(n + "$"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static void typeMetrics(ASTTypeDeclaration type) {
|
||||||
|
final Set<Integer> ownLines = new HashSet<>();
|
||||||
|
final Set<Integer> scopeLines = codeLines.getOrDefault(path(type), lines(type));
|
||||||
|
final var nestedTypes = all(type, ASTTypeDeclaration.class).stream()
|
||||||
|
.filter(t -> t != type)
|
||||||
|
.toList();
|
||||||
|
var token = type.getFirstToken();
|
||||||
|
while (token != null && !token.isEof()) {
|
||||||
|
final var region = token.getRegion();
|
||||||
|
if (nestedTypes.stream().noneMatch(t -> t.getTextRegion().contains(region))) {
|
||||||
|
final var loc = token.getReportLocation();
|
||||||
|
for (int line = loc.getStartLine(); line <= loc.getEndLine(); line++)
|
||||||
|
if (scopeLines.contains(line)) ownLines.add(line);
|
||||||
|
}
|
||||||
|
if (token == type.getLastToken()) break;
|
||||||
|
token = token.getNext();
|
||||||
|
}
|
||||||
|
metric(type, "classLines", ownLines.size(), 300, 500);
|
||||||
|
if (type.isRecord())
|
||||||
|
findings.add("DESIGN " + location(type) + " recordComponents="
|
||||||
|
+ type.getRecordComponents().size() + " (manual design review)");
|
||||||
|
int depth = 0;
|
||||||
|
var superclass = type.getSymbol().getSuperclass();
|
||||||
|
while (superclass != null && owned(superclass.getBinaryName())) {
|
||||||
|
depth++;
|
||||||
|
superclass = superclass.getSuperclass();
|
||||||
|
}
|
||||||
|
metric(type, "ownSuperclassDepth", depth, 2, 3);
|
||||||
|
final Set<String> fanout = new HashSet<>();
|
||||||
|
for (var ref : all(type, ASTClassType.class)) {
|
||||||
|
if (ref.getEnclosingType() != type) continue;
|
||||||
|
final var symbol = ref.getTypeMirror().getSymbol();
|
||||||
|
if (symbol != null && symbol.isUnresolved())
|
||||||
|
block(
|
||||||
|
ref,
|
||||||
|
"Cannot resolve type " + ref.getSimpleName()
|
||||||
|
+ "; compile the backend and regenerate target/standards-classpath.txt before"
|
||||||
|
+ " checking");
|
||||||
|
if (symbol instanceof JClassSymbol resolved && !resolved.equals(type.getSymbol()))
|
||||||
|
fanout.add(resolved.getBinaryName());
|
||||||
|
}
|
||||||
|
metric(type, "fanOut", fanout.size(), 15, Double.POSITIVE_INFINITY);
|
||||||
|
int publicMethods = 0;
|
||||||
|
for (var method : type.getOperations())
|
||||||
|
if (method instanceof ASTMethodDeclaration
|
||||||
|
&& method.hasExplicitModifiers(JModifier.PUBLIC)
|
||||||
|
&& !trivial(method)) publicMethods++;
|
||||||
|
metric(type, "publicMethods", publicMethods, 15, Double.POSITIVE_INFINITY);
|
||||||
|
final Set<Dependency> dependencies = new HashSet<>();
|
||||||
|
injected(type, dependencies, new HashSet<>());
|
||||||
|
metric(type, "injectedDependencies", dependencies.size(), 5, 7);
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean trivial(ASTExecutableDeclaration method) {
|
||||||
|
if (method.getBody() == null || method.getBody().getNumChildren() != 1) return false;
|
||||||
|
final JavaNode statement = method.getBody().getChild(0);
|
||||||
|
final String name = method.getName();
|
||||||
|
String property = name;
|
||||||
|
if (name.startsWith("get") || name.startsWith("set")) property = name.substring(3);
|
||||||
|
else if (name.startsWith("is")) property = name.substring(2);
|
||||||
|
if (property.isEmpty()) return false;
|
||||||
|
property = Character.toLowerCase(property.charAt(0)) + property.substring(1);
|
||||||
|
final String fieldName = property;
|
||||||
|
if (method.getEnclosingType().getSymbol().getDeclaredFields().stream()
|
||||||
|
.noneMatch(f -> f.getSimpleName().equals(fieldName))) return false;
|
||||||
|
if (method.getArity() == 0)
|
||||||
|
return statement instanceof ASTReturnStatement returned
|
||||||
|
&& accessorField(returned.getExpr(), method, property);
|
||||||
|
if (method.getArity() != 1 || !name.startsWith("set")) return false;
|
||||||
|
return statement instanceof ASTExpressionStatement expression
|
||||||
|
&& expression.getExpr() instanceof ASTAssignmentExpression assignment
|
||||||
|
&& !assignment.isCompound()
|
||||||
|
&& accessorField(assignment.getLeftOperand(), method, property)
|
||||||
|
&& assignment.getRightOperand() instanceof ASTVariableAccess value
|
||||||
|
&& method.getFormalParameters().get(0).getVarId().getSymbol().equals(value.getReferencedSym());
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean accessorField(ASTExpression expression, ASTExecutableDeclaration method, String property) {
|
||||||
|
if (!(expression instanceof ASTAssignableExpr.ASTNamedReferenceExpr reference)) return false;
|
||||||
|
if (expression instanceof ASTFieldAccess field && !(field.getQualifier() instanceof ASTThisExpression))
|
||||||
|
return false;
|
||||||
|
return method.getEnclosingType().getSymbol().getDeclaredFields().stream()
|
||||||
|
.anyMatch(
|
||||||
|
field -> field.getSimpleName().equals(property) && field.equals(reference.getReferencedSym()));
|
||||||
|
}
|
||||||
|
|
||||||
|
record Dependency(JTypeMirror type, Set<String> qualifiers) {}
|
||||||
|
|
||||||
|
static boolean annotated(AnnotableSymbol symbol, String annotation) {
|
||||||
|
return symbol.getDeclaredAnnotations().stream()
|
||||||
|
.anyMatch(a -> a.getBinaryName().equals(annotation));
|
||||||
|
}
|
||||||
|
|
||||||
|
static Dependency dependency(JVariableSymbol variable) {
|
||||||
|
return dependency(variable, Substitution.EMPTY);
|
||||||
|
}
|
||||||
|
|
||||||
|
static Dependency dependency(JVariableSymbol variable, Substitution substitution) {
|
||||||
|
final Set<String> qualifiers = new HashSet<>();
|
||||||
|
for (var annotation : variable.getDeclaredAnnotations()) {
|
||||||
|
if (annotation.getBinaryName().equals("jakarta.inject.Named")
|
||||||
|
|| annotated(annotation.getAnnotationSymbol(), "jakarta.inject.Qualifier")) {
|
||||||
|
final Map<String, String> attributes = new TreeMap<>();
|
||||||
|
for (String name : annotation.getAttributeNames())
|
||||||
|
attributes.put(name, String.valueOf(annotation.getAttribute(name)));
|
||||||
|
qualifiers.add(annotation.getBinaryName() + attributes);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return new Dependency(variable.getTypeMirror(substitution), qualifiers);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void injected(ASTTypeDeclaration type, Set<Dependency> dependencies, Set<String> visited) {
|
||||||
|
if (!visited.add(type.getBinaryName())) return;
|
||||||
|
final var constructors = type.getSymbol().getConstructors();
|
||||||
|
final boolean bean = type.getDeclaredAnnotations().any(a -> {
|
||||||
|
final var symbol = a.getTypeMirror().getSymbol();
|
||||||
|
return Set.of(
|
||||||
|
"jakarta.inject.Singleton",
|
||||||
|
"jakarta.enterprise.context.ApplicationScoped",
|
||||||
|
"jakarta.enterprise.context.RequestScoped",
|
||||||
|
"jakarta.enterprise.context.Dependent",
|
||||||
|
"jakarta.enterprise.context.SessionScoped")
|
||||||
|
.contains(symbol.getCanonicalName())
|
||||||
|
|| annotated(symbol, "jakarta.inject.Scope")
|
||||||
|
|| annotated(symbol, "jakarta.enterprise.context.NormalScope")
|
||||||
|
|| annotated(symbol, "jakarta.enterprise.inject.Stereotype");
|
||||||
|
});
|
||||||
|
for (var constructor : constructors) {
|
||||||
|
if (annotated(constructor, "jakarta.inject.Inject") || bean && constructors.size() == 1) {
|
||||||
|
for (var parameter : constructor.getFormalParameters()) dependencies.add(dependency(parameter));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// PMD deliberately sees handwritten source, so account for Lombok's generated constructor.
|
||||||
|
if (bean
|
||||||
|
&& type.getOperations()
|
||||||
|
.filterIs(ASTConstructorDeclaration.class)
|
||||||
|
.isEmpty()) {
|
||||||
|
final boolean allArgs = type.isAnnotationPresent("lombok.AllArgsConstructor");
|
||||||
|
final boolean requiredArgs = type.isAnnotationPresent("lombok.RequiredArgsConstructor");
|
||||||
|
if (allArgs || requiredArgs)
|
||||||
|
for (var field : type.getDeclarations(ASTFieldDeclaration.class)) {
|
||||||
|
if (!field.isStatic())
|
||||||
|
for (var id : field.getVarIds())
|
||||||
|
if (allArgs
|
||||||
|
|| id.getInitializer() == null
|
||||||
|
&& (id.isFinal() || field.isAnnotationPresent("lombok.NonNull")))
|
||||||
|
dependencies.add(dependency(id.getSymbol()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
inheritedInjection(type.getTypeMirror(), dependencies, new HashSet<>());
|
||||||
|
}
|
||||||
|
|
||||||
|
static void inheritedInjection(JClassType resolvedType, Set<Dependency> dependencies, Set<String> visited) {
|
||||||
|
if (resolvedType == null) return;
|
||||||
|
final JClassSymbol symbol = resolvedType.getSymbol();
|
||||||
|
final Substitution substitution = resolvedType.getTypeParamSubst();
|
||||||
|
if (!visited.add(symbol.getBinaryName())) return;
|
||||||
|
for (var field : symbol.getDeclaredFields())
|
||||||
|
if (annotated(field, "jakarta.inject.Inject")) dependencies.add(dependency(field, substitution));
|
||||||
|
for (var method : symbol.getDeclaredMethods())
|
||||||
|
if (annotated(method, "jakarta.inject.Inject"))
|
||||||
|
for (var parameter : method.getFormalParameters())
|
||||||
|
dependencies.add(dependency(parameter, substitution));
|
||||||
|
inheritedInjection(resolvedType.getSuperClass(), dependencies, visited);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void duplication(List<Path> files) throws Exception {
|
||||||
|
final var config = new CPDConfiguration();
|
||||||
|
config.setMinimumTileSize(100);
|
||||||
|
config.setIgnoreIdentifiers(false);
|
||||||
|
config.setIgnoreLiterals(false);
|
||||||
|
config.setSkipLexicalErrors(false);
|
||||||
|
config.setNoSkipBlocks(true);
|
||||||
|
final Set<String> duplicated = new HashSet<>();
|
||||||
|
try (var cpd = CpdAnalysis.create(config)) {
|
||||||
|
for (var file : files)
|
||||||
|
if (file.startsWith(root.resolve(module + "/src/main/java")))
|
||||||
|
cpd.files().addFile(file);
|
||||||
|
cpd.performAnalysis(report -> {
|
||||||
|
for (var error : report.getProcessingErrors()) findings.add("BLOCK CPD parser " + error);
|
||||||
|
for (var match : report.getMatches())
|
||||||
|
for (var mark : match) {
|
||||||
|
final var loc = mark.getLocation();
|
||||||
|
final Path path = Path.of(loc.getFileId().getAbsolutePath())
|
||||||
|
.toAbsolutePath()
|
||||||
|
.normalize();
|
||||||
|
for (int line = loc.getStartLine(); line <= loc.getEndLine(); line++)
|
||||||
|
if (codeLines.getOrDefault(path, Set.of()).contains(line))
|
||||||
|
duplicated.add(path + ":" + line);
|
||||||
|
findings.add("CLONE " + path + ":" + loc.getStartLine() + "-" + loc.getEndLine());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
final long total = codeLines.entrySet().stream()
|
||||||
|
.filter(e -> e.getKey().startsWith(root.resolve(module + "/src/main/java")))
|
||||||
|
.mapToLong(e -> e.getValue().size())
|
||||||
|
.sum();
|
||||||
|
final double density = total == 0 ? 0 : 100.0 * duplicated.size() / total;
|
||||||
|
findings.add((density > 3 ? "BLOCK " : density > 1 ? "REVIEW " : "METRIC ") + "productionDuplication=" + density
|
||||||
|
+ "% (" + duplicated.size() + "/" + total + ")");
|
||||||
|
}
|
||||||
|
}
|
||||||
372
resources/tooling/standards/src/test/java/StandardsTest.java
Normal file
372
resources/tooling/standards/src/test/java/StandardsTest.java
Normal file
@@ -0,0 +1,372 @@
|
|||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.util.*;
|
||||||
|
import net.sourceforge.pmd.*;
|
||||||
|
import net.sourceforge.pmd.lang.*;
|
||||||
|
import net.sourceforge.pmd.lang.document.FileId;
|
||||||
|
import net.sourceforge.pmd.lang.java.ast.*;
|
||||||
|
import net.sourceforge.pmd.lang.rule.RuleSet;
|
||||||
|
import org.junit.jupiter.api.*;
|
||||||
|
|
||||||
|
class StandardsTest {
|
||||||
|
private List<String> analyze(String text) throws Exception {
|
||||||
|
Standards.root = Path.of("").toAbsolutePath();
|
||||||
|
Standards.module = "backend";
|
||||||
|
Standards.ownedTypes.clear();
|
||||||
|
Standards.findings.clear();
|
||||||
|
Standards.units.clear();
|
||||||
|
Standards.types.clear();
|
||||||
|
Standards.codeLines.clear();
|
||||||
|
final var config = new PMDConfiguration();
|
||||||
|
config.setThreads(1);
|
||||||
|
config.prependAuxClasspath(
|
||||||
|
Path.of(System.getProperty("java.home"), "lib/jrt-fs.jar").toString());
|
||||||
|
config.prependAuxClasspath(
|
||||||
|
System.getProperty("surefire.test.class.path", System.getProperty("java.class.path")));
|
||||||
|
config.setIgnoreIncrementalAnalysis(true);
|
||||||
|
config.setDefaultLanguageVersion(
|
||||||
|
LanguageRegistry.PMD.getLanguageById("java").getVersion("25"));
|
||||||
|
try (var analysis = PmdAnalysis.create(config)) {
|
||||||
|
analysis.addRuleSet(RuleSet.forSingleRule(new Standards.Measurements()));
|
||||||
|
analysis.files().addSourceFile(FileId.fromPath(Standards.root.resolve("Fixture.java")), text);
|
||||||
|
var report = analysis.performAnalysisAndCollectReport();
|
||||||
|
assertTrue(
|
||||||
|
report.getProcessingErrors().isEmpty(),
|
||||||
|
report.getProcessingErrors().toString());
|
||||||
|
}
|
||||||
|
return List.copyOf(Standards.findings);
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean contains(List<String> findings, String prefix, String metric) {
|
||||||
|
return findings.stream().anyMatch(s -> s.startsWith(prefix) && s.endsWith(metric));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void cyclomaticAndCognitiveBoundaries() throws Exception {
|
||||||
|
var at = analyze("class Fixture { public void run(final boolean b) { " + "if (b) {}".repeat(9) + "} }");
|
||||||
|
assertTrue(contains(at, "REVIEW", "cyclomatic=10.0"));
|
||||||
|
var over = analyze("class Fixture { public void run(final boolean b) { " + "if (b) {}".repeat(10) + "} }");
|
||||||
|
assertTrue(contains(over, "BLOCK", "cyclomatic=11.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture { public void run(final boolean b) {" + "if (b) {}".repeat(15) + "} }"),
|
||||||
|
"REVIEW",
|
||||||
|
"cognitive=15.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture { public void run(final boolean b) {" + "if (b) {}".repeat(16) + "} }"),
|
||||||
|
"BLOCK",
|
||||||
|
"cognitive=16.0"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void methodAndConstructorParameterBoundariesAndImplicitRecords() throws Exception {
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||||
|
+ " final int e) {} }"),
|
||||||
|
"REVIEW",
|
||||||
|
"parameters=5.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||||
|
+ " final int e, final int f) {} }"),
|
||||||
|
"BLOCK",
|
||||||
|
"parameters=6.0"));
|
||||||
|
var implicit = analyze("record Fixture(int a, int b, int c, int d, int e, int f, int g, int h) {}");
|
||||||
|
assertFalse(implicit.stream().anyMatch(s -> s.contains("parameters=")));
|
||||||
|
var compact = analyze(
|
||||||
|
"record Fixture(int a, int b, int c, int d, int e, int f, int g, int h) { public Fixture" + " {} }");
|
||||||
|
assertTrue(contains(compact, "BLOCK", "parameters=8.0"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void physicalLinesExcludeCommentsAndCountBraces() throws Exception {
|
||||||
|
var findings = analyze("class Fixture {\n"
|
||||||
|
+ "public void run() {\n"
|
||||||
|
+ "// comment\n"
|
||||||
|
+ "/* multi\n"
|
||||||
|
+ " comment */\n\n"
|
||||||
|
+ "System.out.println(1); // code\n"
|
||||||
|
+ "}\n"
|
||||||
|
+ "}");
|
||||||
|
assertTrue(contains(findings, "METRIC", "executableLines=3.0"));
|
||||||
|
assertTrue(contains(findings, "METRIC", "classLines=5.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture {\npublic void run() {\n" + "System.out.println(1);\n".repeat(48) + "}\n}"),
|
||||||
|
"REVIEW",
|
||||||
|
"executableLines=50.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture {\npublic void run() {\n" + "System.out.println(1);\n".repeat(49) + "}\n}"),
|
||||||
|
"BLOCK",
|
||||||
|
"executableLines=51.0"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void nestingElseIfCatchSwitchLambdaAndNestedMethods() throws Exception {
|
||||||
|
var chain = analyze(
|
||||||
|
"class Fixture { public void run(final boolean b) { if (b) {} else if (b) {} else if (b)" + " {} } }");
|
||||||
|
assertTrue(contains(chain, "METRIC", "nesting=1.0"));
|
||||||
|
var nested = analyze("class Fixture { public void run(final boolean b) { if (b) { while (b) { switch (1) {"
|
||||||
|
+ " default -> { try {} catch (Exception e) { Runnable r = () -> { if (b) {} }; } } }"
|
||||||
|
+ " } } } }");
|
||||||
|
assertTrue(contains(nested, "BLOCK", "nesting=5.0"));
|
||||||
|
var separate = analyze("class Fixture { public void run(final boolean b) { if (b) { class Inner { public void"
|
||||||
|
+ " go() { if (b) {} } } } } }");
|
||||||
|
assertFalse(separate.stream().anyMatch(s -> s.endsWith("nesting=2.0")));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void lambdaComplexityContributesAndNestedClassDoesNot() throws Exception {
|
||||||
|
var findings = analyze("class Fixture { public void run(final boolean b) { Runnable r = () -> { if (b) {} };"
|
||||||
|
+ " class Inner { public void nested() { if (b) {} } } } }");
|
||||||
|
assertEquals(
|
||||||
|
2, findings.stream().filter(s -> s.endsWith("cyclomatic=2.0")).count(), findings.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void varAndFinalRules() throws Exception {
|
||||||
|
var findings =
|
||||||
|
analyze("class Fixture { public void run(int p) { var a = new Object(); var b = (Object) a; var c"
|
||||||
|
+ " = name(); int count = 0; count++; } private String name() { return \"x\"; } }");
|
||||||
|
assertEquals(
|
||||||
|
1, findings.stream().filter(s -> s.contains("var requires")).count());
|
||||||
|
assertTrue(findings.stream().anyMatch(s -> s.contains("requires final: p")));
|
||||||
|
assertFalse(findings.stream().anyMatch(s -> s.contains("requires final: count")));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void patternBindingsRequireFinalOnlyWhenAssignedOnce() throws Exception {
|
||||||
|
var findings = analyze("""
|
||||||
|
record Pair(Object key, Object value) {}
|
||||||
|
class Fixture {
|
||||||
|
public void run(final Object input) {
|
||||||
|
if (input instanceof String unchanged) { System.out.println(unchanged); }
|
||||||
|
if (input instanceof final String protectedValue) { System.out.println(protectedValue); }
|
||||||
|
if (input instanceof String reassigned) { reassigned = reassigned.trim(); System.out.println(reassigned); }
|
||||||
|
switch (input) {
|
||||||
|
case String switched -> System.out.println(switched);
|
||||||
|
default -> {}
|
||||||
|
}
|
||||||
|
if (input instanceof Pair(var key, var value)) {
|
||||||
|
System.out.println(key);
|
||||||
|
System.out.println(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
""");
|
||||||
|
for (String binding : List.of("unchanged", "switched", "key", "value"))
|
||||||
|
assertTrue(findings.stream().anyMatch(s -> s.contains("requires final: " + binding)), findings.toString());
|
||||||
|
for (String binding : List.of("protectedValue", "reassigned"))
|
||||||
|
assertFalse(findings.stream().anyMatch(s -> s.contains("requires final: " + binding)), findings.toString());
|
||||||
|
assertFalse(findings.stream().anyMatch(s -> s.contains("var requires")), findings.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void inheritanceCountsOnlyProjectSuperclasses() throws Exception {
|
||||||
|
var findings = analyze("class A {} class B extends A {} class C extends B {} class D extends C {} class Fixture"
|
||||||
|
+ " extends D {} class External extends java.util.ArrayList<String> {}");
|
||||||
|
assertTrue(contains(findings, "REVIEW", "ownSuperclassDepth=3.0"));
|
||||||
|
assertTrue(contains(findings, "BLOCK", "ownSuperclassDepth=4.0"));
|
||||||
|
assertTrue(contains(findings, "METRIC", "ownSuperclassDepth=0.0"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void injectionUsesResolvedTypesQualifiersInheritedMembersAndImplicitConstructors() throws Exception {
|
||||||
|
var findings = analyze("""
|
||||||
|
class Parent {
|
||||||
|
@jakarta.inject.Inject private String inherited;
|
||||||
|
@jakarta.inject.Inject public void setValue(final Integer value) {}
|
||||||
|
}
|
||||||
|
@jakarta.inject.Singleton class Fixture extends Parent {
|
||||||
|
private final String value;
|
||||||
|
public Fixture(@jakarta.inject.Named("a") final String first,
|
||||||
|
@jakarta.inject.Named("a") final String repeated,
|
||||||
|
@jakarta.inject.Named("b") final String second) { value = first; }
|
||||||
|
@jakarta.inject.Inject private String repeatedInherited;
|
||||||
|
}
|
||||||
|
""");
|
||||||
|
assertTrue(contains(findings, "METRIC", "injectedDependencies=4.0"), findings.toString());
|
||||||
|
String fields = "";
|
||||||
|
for (int i = 0; i < 7; i++)
|
||||||
|
fields += "@jakarta.inject.Inject @jakarta.inject.Named(\"" + i + "\") private String d" + i + ";";
|
||||||
|
assertTrue(contains(analyze("class Fixture {" + fields + "}"), "REVIEW", "injectedDependencies=7.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture {" + fields + "@jakarta.inject.Inject private Integer eighth;}"),
|
||||||
|
"BLOCK",
|
||||||
|
"injectedDependencies=8.0"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void fanoutCountsDistinctJdkTypesAndPublicMethodsExcludeOnlyTrueAccessors() throws Exception {
|
||||||
|
var findings = analyze("""
|
||||||
|
class Fixture {
|
||||||
|
private String name;
|
||||||
|
private java.util.List<String> values;
|
||||||
|
private java.util.List<String> repeated;
|
||||||
|
public String getName() { return name; }
|
||||||
|
public void setName(final String value) { this.name = value; }
|
||||||
|
public String getOther() { return name; }
|
||||||
|
public String getValidatedName() { if (name == null) { throw new IllegalStateException(); } return name; }
|
||||||
|
public static int calculate() { return 1; }
|
||||||
|
}
|
||||||
|
""");
|
||||||
|
assertTrue(contains(findings, "METRIC", "fanOut=3.0"), findings.toString());
|
||||||
|
assertTrue(contains(findings, "METRIC", "publicMethods=3.0"), findings.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void commentedAccessorsRemainTrivialWhileAdditionalWorkStillCounts() throws Exception {
|
||||||
|
var findings = analyze("""
|
||||||
|
class Fixture {
|
||||||
|
private String name;
|
||||||
|
private String trimmed;
|
||||||
|
private String logged;
|
||||||
|
private String calculated;
|
||||||
|
private String appended;
|
||||||
|
public String getName() { /* Intent comment. */ return name; }
|
||||||
|
public void setName(final String value) { this./* Field comment. */name = value; }
|
||||||
|
public String name() { // Read the stored value.
|
||||||
|
return this.name;
|
||||||
|
}
|
||||||
|
public void setName(final String value, final boolean validate) { this.name = value; }
|
||||||
|
public void setTrimmed(final String value) { this.trimmed = value.trim(); }
|
||||||
|
public String getLogged() { System.out.println(logged); return logged; }
|
||||||
|
public String getCalculated() { return calculated.trim(); }
|
||||||
|
public void setAppended(final String value) { this.appended += value; }
|
||||||
|
}
|
||||||
|
""");
|
||||||
|
assertTrue(contains(findings, "METRIC", "publicMethods=5.0"), findings.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void classLengthBoundaryAndNestedTypeLines() throws Exception {
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture {\n" + "private int field;\n".repeat(498) + "}"), "REVIEW", "classLines=500.0"));
|
||||||
|
assertTrue(contains(
|
||||||
|
analyze("class Fixture {\n" + "private int field;\n".repeat(499) + "}"), "BLOCK", "classLines=501.0"));
|
||||||
|
var separate = analyze("class Fixture {\nprivate class Inner {\nprivate int value;\n}\nprivate int other;\n}");
|
||||||
|
assertEquals(
|
||||||
|
2, separate.stream().filter(s -> s.endsWith("classLines=3.0")).count());
|
||||||
|
assertTrue(contains(analyze("class Fixture { private class Inner {} }"), "METRIC", "classLines=1.0"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void switchBooleanAndThrowTreatmentIsPinned() throws Exception {
|
||||||
|
var findings = analyze("class Fixture { public int run(final boolean a, final boolean b) { if (a && b) { throw"
|
||||||
|
+ " new IllegalStateException(); } return switch (1) { case 0, 1 -> 1; default -> 2;"
|
||||||
|
+ " }; } }");
|
||||||
|
assertTrue(contains(findings, "METRIC", "cyclomatic=6.0"), findings.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
private Path project(String production, String tests) throws Exception {
|
||||||
|
Path directory = Files.createTempDirectory("s7-fixture-");
|
||||||
|
Files.writeString(directory.resolve("engineering.properties"), "BACKEND_MODULE=backend\n");
|
||||||
|
Path prod = directory.resolve("backend/src/main/java");
|
||||||
|
Path test = directory.resolve("backend/src/test/java");
|
||||||
|
Files.createDirectories(prod);
|
||||||
|
Files.createDirectories(test);
|
||||||
|
Path config = directory.resolve("generated/tooling/standards/checkstyle.xml");
|
||||||
|
Files.createDirectories(config.getParent());
|
||||||
|
Files.copy(Path.of(System.getProperty("basedir"), "checkstyle.xml"), config);
|
||||||
|
if (production != null) Files.writeString(prod.resolve("Fixture.java"), production);
|
||||||
|
if (tests != null) Files.writeString(test.resolve("ExampleTest.java"), tests);
|
||||||
|
return directory;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void checksFailWithoutRewritingSourcesAndExposeReviewSignals() throws Exception {
|
||||||
|
String source = "package example;\n"
|
||||||
|
+ "import java.util.*;\n"
|
||||||
|
+ "class Fixture { public void run(int x) { if (x == 0) System.out.println(x); } }";
|
||||||
|
Path directory = project(source, null);
|
||||||
|
assertThrows(IllegalStateException.class, () -> Standards.main(new String[] {directory.toString()}));
|
||||||
|
assertEquals(source, Files.readString(directory.resolve("backend/src/main/java/Fixture.java")));
|
||||||
|
String report = Files.readString(directory.resolve("backend/target/standards/report.txt"));
|
||||||
|
assertTrue(report.contains("form of import"), report);
|
||||||
|
assertTrue(report.contains("must use '{}'s"), report);
|
||||||
|
assertTrue(report.contains("requires final: x"), report);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void productionCpdCountsEveryOccurrenceAndTestClonesDoNotGate() throws Exception {
|
||||||
|
String method = "public void first() {\n" + "System.out.println(123);\n".repeat(30) + "}\n";
|
||||||
|
String source = "package example;\nclass Fixture {\n" + method + method.replace("first", "second") + "}";
|
||||||
|
Path directory = project(source, null);
|
||||||
|
assertThrows(IllegalStateException.class, () -> Standards.main(new String[] {directory.toString()}));
|
||||||
|
String report = Files.readString(directory.resolve("backend/target/standards/report.txt"));
|
||||||
|
assertTrue(report.contains("BLOCK productionDuplication=") && report.contains("(64/67)"), report);
|
||||||
|
assertTrue(report.lines().filter(l -> l.startsWith("CLONE")).count() >= 2, report);
|
||||||
|
Path testProject =
|
||||||
|
project("package example; class Fixture {}", source.replace("class Fixture", "class ExampleTest"));
|
||||||
|
Standards.main(new String[] {testProject.toString()});
|
||||||
|
String testReport = Files.readString(testProject.resolve("backend/target/standards/report.txt"));
|
||||||
|
assertTrue(testReport.contains("productionDuplication=0.0%"), testReport);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void parserFailureNamesTheAffectedFileAndFails() throws Exception {
|
||||||
|
Path directory = project("package example; class Fixture { public void broken( }", null);
|
||||||
|
Exception failure = assertThrows(Exception.class, () -> Standards.main(new String[] {directory.toString()}));
|
||||||
|
assertTrue(
|
||||||
|
failure.toString().contains("Fixture.java")
|
||||||
|
|| Standards.findings.stream().anyMatch(f -> f.contains("Fixture.java")),
|
||||||
|
failure.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void exactCoverageGateAcceptsEightyPercentAndRejectsRoundedNearMiss() throws Exception {
|
||||||
|
Path directory = Files.createTempDirectory("s7-coverage-");
|
||||||
|
Path xml = directory.resolve("report.xml");
|
||||||
|
Path helper = Path.of(System.getProperty("basedir"), "../coverage/Coverage.java")
|
||||||
|
.normalize();
|
||||||
|
for (long[] counts : List.of(new long[] {4, 1, 0}, new long[] {799999, 200001, 1}, new long[] {0, 0, 1})) {
|
||||||
|
Files.writeString(
|
||||||
|
xml,
|
||||||
|
"<report><counter type=\"LINE\" covered=\"" + counts[0] + "\" missed=\"" + counts[1]
|
||||||
|
+ "\"/></report>");
|
||||||
|
Process process = new ProcessBuilder(
|
||||||
|
Path.of(System.getProperty("java.home"), "bin/java").toString(),
|
||||||
|
helper.toString(),
|
||||||
|
xml.toString())
|
||||||
|
.redirectErrorStream(true)
|
||||||
|
.redirectOutput(directory.resolve("output.log").toFile())
|
||||||
|
.start();
|
||||||
|
assertEquals(counts[2], process.waitFor(), Files.readString(directory.resolve("output.log")));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void overridesAndImplicitRecordInjectionAreMeasured() throws Exception {
|
||||||
|
var override = analyze("class Fixture extends java.util.ArrayList<String> { public void clear() {} }");
|
||||||
|
assertTrue(override.stream().anyMatch(f -> f.contains("require @Override")), override.toString());
|
||||||
|
var record = analyze("@jakarta.inject.Singleton record Fixture(String one, String repeated, Integer two) {}");
|
||||||
|
assertTrue(contains(record, "METRIC", "injectedDependencies=2.0"), record.toString());
|
||||||
|
var inherited = analyze("class Parent<T> { @jakarta.inject.Inject private T dependency; } class Fixture extends"
|
||||||
|
+ " Parent<String> { @jakarta.inject.Inject private String repeated; }");
|
||||||
|
assertEquals(
|
||||||
|
2,
|
||||||
|
inherited.stream()
|
||||||
|
.filter(f -> f.endsWith("injectedDependencies=1.0"))
|
||||||
|
.count(),
|
||||||
|
inherited.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void waiverMatchesOnlyTheExactMetricSignatureAndRetainedValue() throws Exception {
|
||||||
|
Standards.waivers.add(new Standards.Waiver(
|
||||||
|
"parameters",
|
||||||
|
"Fixture#run(int,int,int,int,int,int)",
|
||||||
|
6,
|
||||||
|
"Test fixture agreement: preserve an example signature"));
|
||||||
|
try {
|
||||||
|
var waived = analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||||
|
+ " final int e, final int f) {} }");
|
||||||
|
assertTrue(
|
||||||
|
waived.stream().anyMatch(f -> f.startsWith("WAIVER") && f.contains("parameters=6.0")),
|
||||||
|
waived.toString());
|
||||||
|
var changed = analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||||
|
+ " final int e, final int f, final int g) {} }");
|
||||||
|
assertTrue(contains(changed, "BLOCK", "parameters=7.0"));
|
||||||
|
} finally {
|
||||||
|
Standards.waivers.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
7
templates/backend/.dockerignore
Normal file
7
templates/backend/.dockerignore
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
.git
|
||||||
|
**/node_modules
|
||||||
|
ci/artifacts
|
||||||
|
**/target
|
||||||
|
!backend/target
|
||||||
|
backend/target/*
|
||||||
|
!backend/target/quarkus-app
|
||||||
13
templates/backend/Dockerfile
Normal file
13
templates/backend/Dockerfile
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
FROM maven:3.9-eclipse-temurin-25 AS build
|
||||||
|
WORKDIR /workspace
|
||||||
|
COPY . .
|
||||||
|
RUN mvn -B -ntp -Dformat.mode=check clean verify
|
||||||
|
FROM eclipse-temurin:25-jre AS runtime
|
||||||
|
WORKDIR /app
|
||||||
|
USER 10001:10001
|
||||||
|
EXPOSE 8080
|
||||||
|
ENTRYPOINT ["java", "-jar", "/app/quarkus-run.jar"]
|
||||||
|
FROM runtime AS prebuilt
|
||||||
|
COPY --chown=10001:10001 backend/target/quarkus-app/ ./
|
||||||
|
FROM runtime AS standalone
|
||||||
|
COPY --from=build --chown=10001:10001 /workspace/backend/target/quarkus-app/ ./
|
||||||
99
templates/backend/backend/pom.xml
Normal file
99
templates/backend/backend/pom.xml
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||||
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||||
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
<parent>
|
||||||
|
<groupId>org.example.@PROJECT@</groupId>
|
||||||
|
<artifactId>@PROJECT@</artifactId>
|
||||||
|
<version>1.0-SNAPSHOT</version>
|
||||||
|
</parent>
|
||||||
|
|
||||||
|
<artifactId>backend</artifactId>
|
||||||
|
<packaging>jar</packaging>
|
||||||
|
|
||||||
|
<dependencies>
|
||||||
|
<dependency><groupId>io.quarkus</groupId><artifactId>quarkus-jacoco</artifactId><scope>test</scope></dependency>
|
||||||
|
<dependency><groupId>io.quarkus</groupId><artifactId>quarkus-rest-jackson</artifactId></dependency>
|
||||||
|
<dependency><groupId>io.quarkus</groupId><artifactId>quarkus-smallrye-health</artifactId></dependency>
|
||||||
|
<dependency><groupId>io.quarkus</groupId><artifactId>quarkus-junit</artifactId><scope>test</scope></dependency>
|
||||||
|
<dependency><groupId>io.rest-assured</groupId><artifactId>rest-assured</artifactId><scope>test</scope></dependency>
|
||||||
|
</dependencies>
|
||||||
|
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId><artifactId>maven-dependency-plugin</artifactId><version>3.8.1</version>
|
||||||
|
<executions><execution><id>standards-classpath</id><phase>prepare-package</phase><goals><goal>build-classpath</goal></goals>
|
||||||
|
<configuration><outputFile>${project.build.directory}/standards-classpath.txt</outputFile></configuration>
|
||||||
|
</execution>
|
||||||
|
<execution><id>coverage-tools</id><phase>prepare-package</phase><goals><goal>copy</goal></goals>
|
||||||
|
<configuration><artifactItems>
|
||||||
|
<artifactItem><groupId>org.jacoco</groupId><artifactId>org.jacoco.agent</artifactId><version>0.8.14</version><classifier>runtime</classifier><destFileName>agent.jar</destFileName></artifactItem>
|
||||||
|
<artifactItem><groupId>org.jacoco</groupId><artifactId>org.jacoco.cli</artifactId><version>0.8.14</version><classifier>nodeps</classifier><destFileName>cli.jar</destFileName></artifactItem>
|
||||||
|
</artifactItems><outputDirectory>${project.build.directory}/coverage-tools</outputDirectory></configuration>
|
||||||
|
</execution></executions>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.codehaus.mojo</groupId><artifactId>exec-maven-plugin</artifactId><version>3.6.3</version>
|
||||||
|
<executions><execution><id>standards</id><phase>verify</phase><goals><goal>exec</goal></goals>
|
||||||
|
<configuration>
|
||||||
|
<executable>${maven.home}/bin/mvn</executable>
|
||||||
|
<workingDirectory>${maven.multiModuleProjectDirectory}</workingDirectory>
|
||||||
|
<arguments><argument>-B</argument><argument>-ntp</argument><argument>-f</argument>
|
||||||
|
<argument>${maven.multiModuleProjectDirectory}/generated/tooling/standards/pom.xml</argument>
|
||||||
|
<argument>test</argument><argument>exec:java</argument>
|
||||||
|
</arguments>
|
||||||
|
</configuration>
|
||||||
|
</execution>
|
||||||
|
<execution><id>coverage</id><phase>verify</phase><goals><goal>exec</goal></goals>
|
||||||
|
<configuration><executable>${java.home}/bin/java</executable><arguments>
|
||||||
|
<argument>${maven.multiModuleProjectDirectory}/generated/tooling/coverage/Coverage.java</argument>
|
||||||
|
<argument>${project.build.directory}/site/jacoco/jacoco.xml</argument>
|
||||||
|
</arguments></configuration>
|
||||||
|
</execution>
|
||||||
|
<execution><id>combined-coverage</id><phase>none</phase><goals><goal>exec</goal></goals>
|
||||||
|
<configuration><executable>${java.home}/bin/java</executable><arguments>
|
||||||
|
<argument>${maven.multiModuleProjectDirectory}/generated/tooling/coverage/Coverage.java</argument>
|
||||||
|
<argument>${project.build.directory}/site/jacoco-combined/jacoco.xml</argument>
|
||||||
|
</arguments></configuration>
|
||||||
|
</execution></executions>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.jacoco</groupId><artifactId>jacoco-maven-plugin</artifactId><version>0.8.14</version>
|
||||||
|
<executions>
|
||||||
|
<execution><id>agent</id><goals><goal>prepare-agent</goal></goals><configuration>
|
||||||
|
<exclClassLoaders>*QuarkusClassLoader</exclClassLoaders><destFile>${project.build.directory}/jacoco.exec</destFile><append>true</append>
|
||||||
|
</configuration></execution>
|
||||||
|
<execution><id>report</id><phase>post-integration-test</phase><goals><goal>report</goal></goals></execution>
|
||||||
|
<execution><id>combined</id><phase>none</phase><goals><goal>merge</goal></goals><configuration>
|
||||||
|
<fileSets><fileSet><directory>${project.build.directory}</directory><includes><include>jacoco.exec</include><include>playwright.exec</include></includes></fileSet></fileSets>
|
||||||
|
<destFile>${project.build.directory}/jacoco-combined.exec</destFile>
|
||||||
|
</configuration></execution>
|
||||||
|
<execution><id>combined-report</id><phase>none</phase><goals><goal>report</goal></goals><configuration>
|
||||||
|
<dataFile>${project.build.directory}/jacoco-combined.exec</dataFile>
|
||||||
|
<outputDirectory>${project.build.directory}/site/jacoco-combined</outputDirectory>
|
||||||
|
</configuration></execution>
|
||||||
|
</executions>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>io.quarkus</groupId>
|
||||||
|
<artifactId>quarkus-maven-plugin</artifactId>
|
||||||
|
<version>${quarkus.platform.version}</version>
|
||||||
|
<extensions>true</extensions>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<goals>
|
||||||
|
<goal>build</goal>
|
||||||
|
<goal>generate-code</goal>
|
||||||
|
<goal>generate-code-tests</goal>
|
||||||
|
</goals>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
|
||||||
|
</project>
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
package org.example;
|
||||||
|
|
||||||
|
import jakarta.ws.rs.GET;
|
||||||
|
import jakarta.ws.rs.Path;
|
||||||
|
|
||||||
|
/** Minimal endpoint for the generated recipe. */
|
||||||
|
@Path("/hello")
|
||||||
|
public class GreetingResource {
|
||||||
|
/** Return the starter greeting. */
|
||||||
|
@GET
|
||||||
|
public String hello() {
|
||||||
|
return "Hello";
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
quarkus.http.host=0.0.0.0
|
||||||
|
%test.quarkus.jacoco.data-file=target/jacoco.exec
|
||||||
|
%test.quarkus.jacoco.reuse-data-file=true
|
||||||
|
%test.quarkus.jacoco.report=false
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
package org.example;
|
||||||
|
|
||||||
|
import static io.restassured.RestAssured.given;
|
||||||
|
import static org.hamcrest.Matchers.is;
|
||||||
|
|
||||||
|
import io.quarkus.test.junit.QuarkusTest;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
|
/** Exercise the starter over HTTP. */
|
||||||
|
@QuarkusTest
|
||||||
|
public class GreetingResourceTest {
|
||||||
|
@Test
|
||||||
|
public void returnsGreeting() {
|
||||||
|
given().when().get("/hello").then().statusCode(200).body(is("Hello"));
|
||||||
|
}
|
||||||
|
}
|
||||||
46
templates/common/.gitea/workflows/build.yaml
Normal file
46
templates/common/.gitea/workflows/build.yaml
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
name: Build, test and publish
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: ['**']
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
verify:
|
||||||
|
runs-on: @RUNNER@
|
||||||
|
env:
|
||||||
|
CI: 'true'
|
||||||
|
TARGET_PLATFORM: ${{ vars.TARGET_PLATFORM }}
|
||||||
|
GITEA_SERVER_URL: ${{ gitea.server_url }}
|
||||||
|
CI_REPOSITORY: ${{ gitea.repository }}
|
||||||
|
CI_COMMIT: ${{ gitea.sha }}
|
||||||
|
CI_REF: ${{ gitea.ref }}
|
||||||
|
CI_RUN_ID: ${{ gitea.run_id }}
|
||||||
|
steps:
|
||||||
|
- name: Check out pushed commit
|
||||||
|
uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- name: Verify pinned engineering resources
|
||||||
|
run: python3 scripts/engineering.py --check
|
||||||
|
- name: Identify workflow attempt
|
||||||
|
# Gitea 1.25 does not populate gitea.run_attempt. Use a unique value for
|
||||||
|
# container names and diagnostic artifacts, including reruns of one SHA.
|
||||||
|
run: java generated/ci/Ci.java attempt
|
||||||
|
- name: Build and test both images
|
||||||
|
run: java generated/ci/Ci.java verify
|
||||||
|
- name: Publish verified main images
|
||||||
|
if: success() && gitea.ref == 'refs/heads/main'
|
||||||
|
env:
|
||||||
|
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
|
||||||
|
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||||
|
run: java generated/ci/Ci.java publish
|
||||||
|
- name: Collect diagnostics and clean up test containers
|
||||||
|
if: always()
|
||||||
|
run: java generated/ci/Ci.java cleanup
|
||||||
|
- name: Retain diagnostics for fourteen days
|
||||||
|
if: always()
|
||||||
|
uses: https://github.com/actions/upload-artifact@ff15f0306b3f739f7b6fd43fb5d26cd321bd4de5
|
||||||
|
with:
|
||||||
|
name: diagnostics-${{ gitea.run_id }}-${{ env.CI_RUN_ATTEMPT }}
|
||||||
|
path: ci/artifacts/
|
||||||
|
if-no-files-found: error
|
||||||
|
retention-days: 14
|
||||||
70
templates/common/.gitea/workflows/deploy.yaml
Normal file
70
templates/common/.gitea/workflows/deploy.yaml
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
name: Deploy production
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
mode:
|
||||||
|
description: 'Release mode'
|
||||||
|
required: true
|
||||||
|
type: choice
|
||||||
|
default: new
|
||||||
|
options:
|
||||||
|
- new
|
||||||
|
- retry
|
||||||
|
increment:
|
||||||
|
description: 'New release only: semantic version increment (first release is v1.0.0)'
|
||||||
|
required: true
|
||||||
|
type: choice
|
||||||
|
default: patch
|
||||||
|
options:
|
||||||
|
- patch
|
||||||
|
- minor
|
||||||
|
- major
|
||||||
|
version:
|
||||||
|
description: 'Retry only: existing version tag, e.g. v1.2.3; leave empty for new release'
|
||||||
|
required: false
|
||||||
|
type: string
|
||||||
|
|
||||||
|
# Gitea 1.25 ignores concurrency. The helper acquires a NAS-wide mkdir lock.
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
if: gitea.ref == 'refs/heads/main'
|
||||||
|
runs-on: @RUNNER@
|
||||||
|
env:
|
||||||
|
CI: 'true'
|
||||||
|
GITEA_SERVER_URL: ${{ gitea.server_url }}
|
||||||
|
CI_REPOSITORY: ${{ gitea.repository }}
|
||||||
|
CI_COMMIT: ${{ gitea.sha }}
|
||||||
|
CI_REF: ${{ gitea.ref }}
|
||||||
|
CI_EVENT: ${{ gitea.event_name }}
|
||||||
|
CI_RUN_ID: ${{ gitea.run_id }}
|
||||||
|
steps:
|
||||||
|
- name: Check out trusted main workflow revision
|
||||||
|
uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- name: Verify pinned engineering resources
|
||||||
|
run: python3 scripts/engineering.py --check
|
||||||
|
- name: Identify workflow attempt
|
||||||
|
run: java generated/ci/Ci.java attempt
|
||||||
|
- name: Release verified images
|
||||||
|
env:
|
||||||
|
RELEASE_MODE: ${{ inputs.mode }}
|
||||||
|
RELEASE_INCREMENT: ${{ inputs.increment }}
|
||||||
|
RELEASE_VERSION: ${{ inputs.version }}
|
||||||
|
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
|
||||||
|
DEPLOY_HOST: ${{ vars.DEPLOY_HOST }}
|
||||||
|
DEPLOY_USER: ${{ vars.DEPLOY_USER }}
|
||||||
|
DEPLOY_SSH_PORT: ${{ vars.DEPLOY_SSH_PORT }}
|
||||||
|
DEPLOY_DIRECTORY: ${{ vars.DEPLOY_DIRECTORY }}
|
||||||
|
DEPLOY_REGISTRY: ${{ vars.DEPLOY_REGISTRY }}
|
||||||
|
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
|
||||||
|
DEPLOY_KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
|
||||||
|
run: java generated/ci/Ci.java deploy
|
||||||
|
- name: Retain deployment diagnostics for fourteen days
|
||||||
|
if: always()
|
||||||
|
uses: https://github.com/actions/upload-artifact@ff15f0306b3f739f7b6fd43fb5d26cd321bd4de5
|
||||||
|
with:
|
||||||
|
name: deployment-${{ gitea.run_id }}-${{ env.CI_RUN_ATTEMPT }}
|
||||||
|
path: ci/artifacts/deployment/
|
||||||
|
if-no-files-found: warn
|
||||||
|
retention-days: 14
|
||||||
44
templates/common/.gitea/workflows/rollback.yaml
Normal file
44
templates/common/.gitea/workflows/rollback.yaml
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
name: Rollback production
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
rollback:
|
||||||
|
if: gitea.ref == 'refs/heads/main'
|
||||||
|
runs-on: @RUNNER@
|
||||||
|
env:
|
||||||
|
CI: 'true'
|
||||||
|
GITEA_SERVER_URL: ${{ gitea.server_url }}
|
||||||
|
CI_REPOSITORY: ${{ gitea.repository }}
|
||||||
|
CI_COMMIT: ${{ gitea.sha }}
|
||||||
|
CI_REF: ${{ gitea.ref }}
|
||||||
|
CI_EVENT: ${{ gitea.event_name }}
|
||||||
|
CI_RUN_ID: ${{ gitea.run_id }}
|
||||||
|
steps:
|
||||||
|
- name: Check out trusted main workflow revision
|
||||||
|
uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- name: Verify pinned engineering resources
|
||||||
|
run: python3 scripts/engineering.py --check
|
||||||
|
- name: Identify workflow attempt
|
||||||
|
run: java generated/ci/Ci.java attempt
|
||||||
|
- name: Restore previous successful release
|
||||||
|
env:
|
||||||
|
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
|
||||||
|
DEPLOY_HOST: ${{ vars.DEPLOY_HOST }}
|
||||||
|
DEPLOY_USER: ${{ vars.DEPLOY_USER }}
|
||||||
|
DEPLOY_SSH_PORT: ${{ vars.DEPLOY_SSH_PORT }}
|
||||||
|
DEPLOY_DIRECTORY: ${{ vars.DEPLOY_DIRECTORY }}
|
||||||
|
DEPLOY_REGISTRY: ${{ vars.DEPLOY_REGISTRY }}
|
||||||
|
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
|
||||||
|
DEPLOY_KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
|
||||||
|
run: java generated/ci/Ci.java rollback
|
||||||
|
- name: Retain deployment diagnostics for fourteen days
|
||||||
|
if: always()
|
||||||
|
uses: https://github.com/actions/upload-artifact@ff15f0306b3f739f7b6fd43fb5d26cd321bd4de5
|
||||||
|
with:
|
||||||
|
name: rollback-${{ gitea.run_id }}-${{ env.CI_RUN_ATTEMPT }}
|
||||||
|
path: ci/artifacts/deployment/
|
||||||
|
if-no-files-found: warn
|
||||||
|
retention-days: 14
|
||||||
6
templates/common/.gitignore
vendored
Normal file
6
templates/common/.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
target
|
||||||
|
**/node_modules/
|
||||||
|
**/dist/
|
||||||
|
**/__pycache__/
|
||||||
|
ci/artifacts/
|
||||||
|
.env
|
||||||
8
templates/common/AGENTS.md
Normal file
8
templates/common/AGENTS.md
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
# Project guidance
|
||||||
|
|
||||||
|
Read `generated/CODING_STANDARDS.md` before changing Java.
|
||||||
|
For reviews use `generated/skills/code-review/SKILL.md` with
|
||||||
|
`generated/docs/backend-review.md`.
|
||||||
|
Specifications and ADRs: @LANDSCAPE@.
|
||||||
|
Do not edit generated resources; use `mvn -N -Pregenerate-engineering validate`.
|
||||||
|
See README.md for setup and engineering upgrades. Skills are project-local.
|
||||||
20
templates/common/README.md
Normal file
20
templates/common/README.md
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
# @PROJECT@
|
||||||
|
|
||||||
|
Generated from the version recorded in engineering.lock.json.
|
||||||
|
Use JDK 25, Maven, Python 3, Git, Docker with Compose, and Node/npm for a frontend.
|
||||||
|
Run `mvn verify` for a backend and `java generated/ci/Ci.java verify` for the full recipe.
|
||||||
|
Local builds use committed generated files. Run
|
||||||
|
`mvn -N -Pregenerate-engineering validate` to regenerate, or
|
||||||
|
`python3 scripts/engineering.py --check` to check drift.
|
||||||
|
|
||||||
|
Edit engineering.properties for module paths, images (derived from PROJECT_NAME),
|
||||||
|
deployment project, required secrets and dependencies. Edit deploy/health.sh for
|
||||||
|
application readiness. Configure the workflow runner label and Gitea variables
|
||||||
|
and secrets as described in the engineering README. Repository creation and
|
||||||
|
secret installation are manual setup steps; the generator creates local files.
|
||||||
|
|
||||||
|
ADRs, specifications and onboarding: @LANDSCAPE@.
|
||||||
|
Engineering usage and upgrades: @ENGINEERING@.
|
||||||
|
Update engineering.lock.json's tag and commit together, regenerate and review
|
||||||
|
the diff and that release's upgrade instructions. Project-owned files outside
|
||||||
|
generated/ are never overwritten by an upgrade.
|
||||||
11
templates/common/ci/runner/config.yaml
Normal file
11
templates/common/ci/runner/config.yaml
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
log:
|
||||||
|
level: info
|
||||||
|
runner:
|
||||||
|
file: .runner
|
||||||
|
capacity: 1
|
||||||
|
timeout: 2h
|
||||||
|
shutdown_timeout: 2m
|
||||||
|
labels:
|
||||||
|
- @RUNNER@:host
|
||||||
|
cache:
|
||||||
|
enabled: false
|
||||||
3
templates/common/ci/runner/docker-daemon.json
Normal file
3
templates/common/ci/runner/docker-daemon.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"insecure-registries": ["192.168.2.28:3000"]
|
||||||
|
}
|
||||||
9
templates/common/ci/runner/docker-desktop.conf
Normal file
9
templates/common/ci/runner/docker-desktop.conf
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
# Desktop-specific override: start after graphical login, alongside Docker Desktop.
|
||||||
|
[Unit]
|
||||||
|
Requires=docker-desktop.service
|
||||||
|
After=graphical-session.target docker-desktop.service
|
||||||
|
PartOf=graphical-session.target docker-desktop.service
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=
|
||||||
|
WantedBy=graphical-session.target
|
||||||
19
templates/common/ci/runner/runner.service
Normal file
19
templates/common/ci/runner/runner.service
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=@PROJECT@ Gitea Actions runner
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
WorkingDirectory=%h/.local/share/@PROJECT@-runner
|
||||||
|
EnvironmentFile=%h/.config/@PROJECT@-runner/environment
|
||||||
|
ExecStartPre=/usr/bin/env java %h/.local/share/@PROJECT@-runner/WaitForDocker.java 120
|
||||||
|
TimeoutStartSec=150
|
||||||
|
ExecStart=%h/.local/bin/act_runner daemon --config %h/.local/share/@PROJECT@-runner/config.yaml
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=10
|
||||||
|
KillSignal=SIGTERM
|
||||||
|
TimeoutStopSec=150
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
4
templates/common/deploy/health.sh
Normal file
4
templates/common/deploy/health.sh
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# Sourced by the shared deployment mechanism, retained with each release.
|
||||||
|
application_health() {
|
||||||
|
curl --noproxy '*' --fail --silent --show-error --max-time 5 "http://$BIND_IP:$FRONTEND_PORT/" -o /dev/null
|
||||||
|
}
|
||||||
6
templates/common/deploy/production.env.example
Normal file
6
templates/common/deploy/production.env.example
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
# Copy to the deployment directory as production.env, chmod 600; never commit real secrets.
|
||||||
|
BIND_IP=127.0.0.1
|
||||||
|
FRONTEND_PORT=8088
|
||||||
|
DOCKER_BIN=/usr/bin/docker
|
||||||
|
DOCKER_SUDO=0
|
||||||
|
DOCKER_CONFIG=/absolute/deployment/path/docker-auth
|
||||||
177
templates/common/pom.xml
Normal file
177
templates/common/pom.xml
Normal file
@@ -0,0 +1,177 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||||
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||||
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
<groupId>org.example.@PROJECT@</groupId>
|
||||||
|
<artifactId>@PROJECT@</artifactId>
|
||||||
|
<version>1.0-SNAPSHOT</version>
|
||||||
|
<packaging>pom</packaging>
|
||||||
|
|
||||||
|
<name>@PROJECT@</name>
|
||||||
|
<description>Generated engineering starter</description>
|
||||||
|
|
||||||
|
<modules>@MODULES@</modules>
|
||||||
|
|
||||||
|
<!-- TODO Add SCM ??? -->
|
||||||
|
|
||||||
|
<properties>
|
||||||
|
<maven.compiler.release>25</maven.compiler.release>
|
||||||
|
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||||
|
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
|
||||||
|
<quarkus.platform.version>3.40.1</quarkus.platform.version>
|
||||||
|
|
||||||
|
<format.mode>apply</format.mode>
|
||||||
|
<format.skip>false</format.skip>
|
||||||
|
<palantir-java-format.version>2.101.0</palantir-java-format.version>
|
||||||
|
<rewrite-core.version>8.63.4</rewrite-core.version>
|
||||||
|
|
||||||
|
<ortools-java.version>9.15.6755</ortools-java.version>
|
||||||
|
<mapstruct.version>1.6.3</mapstruct.version>
|
||||||
|
<lombok-mapstruct-binding.version>0.2.0</lombok-mapstruct-binding.version>
|
||||||
|
<lombok.version>1.18.48</lombok.version>
|
||||||
|
|
||||||
|
<!-- test dependencies -->
|
||||||
|
<archunit.version>1.5.1</archunit.version>
|
||||||
|
<datafaker.version>2.7.0</datafaker.version>
|
||||||
|
</properties>
|
||||||
|
|
||||||
|
<!-- dependencies that can be included by child-project - without stating the version -->
|
||||||
|
<dependencyManagement><dependencies><dependency>
|
||||||
|
<groupId>io.quarkus.platform</groupId><artifactId>quarkus-bom</artifactId><version>${quarkus.platform.version}</version><type>pom</type><scope>import</scope>
|
||||||
|
</dependency></dependencies></dependencyManagement>
|
||||||
|
|
||||||
|
<!-- dependencies that will always be included by the child-project -->
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.projectlombok</groupId>
|
||||||
|
<artifactId>lombok</artifactId>
|
||||||
|
<version>${lombok.version}</version>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
|
||||||
|
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.codehaus.mojo</groupId>
|
||||||
|
<artifactId>exec-maven-plugin</artifactId>
|
||||||
|
<version>3.6.3</version>
|
||||||
|
<inherited>false</inherited>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<id>format</id>
|
||||||
|
<phase>validate</phase>
|
||||||
|
<goals><goal>exec</goal></goals>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
<configuration>
|
||||||
|
<skip>${format.skip}</skip>
|
||||||
|
<executable>${java.home}/bin/java</executable>
|
||||||
|
<includeProjectDependencies>false</includeProjectDependencies>
|
||||||
|
<includePluginDependencies>true</includePluginDependencies>
|
||||||
|
<arguments>
|
||||||
|
<argument>--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED</argument>
|
||||||
|
<argument>--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED</argument>
|
||||||
|
<argument>--add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED</argument>
|
||||||
|
<argument>--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED</argument>
|
||||||
|
<argument>--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED</argument>
|
||||||
|
<argument>--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED</argument>
|
||||||
|
<argument>--class-path</argument><classpath/>
|
||||||
|
<argument>${maven.multiModuleProjectDirectory}/generated/tooling/format/Format.java</argument>
|
||||||
|
<argument>${maven.multiModuleProjectDirectory}</argument>
|
||||||
|
<argument>${format.mode}</argument>
|
||||||
|
</arguments>
|
||||||
|
</configuration>
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>com.palantir.javaformat</groupId>
|
||||||
|
<artifactId>palantir-java-format</artifactId>
|
||||||
|
<version>${palantir-java-format.version}</version>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.openrewrite</groupId>
|
||||||
|
<artifactId>rewrite-core</artifactId>
|
||||||
|
<version>${rewrite-core.version}</version>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
</plugin>
|
||||||
|
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-compiler-plugin</artifactId>
|
||||||
|
<version>3.16.0</version>
|
||||||
|
<configuration>
|
||||||
|
<annotationProcessorPaths>
|
||||||
|
<path>
|
||||||
|
<groupId>org.projectlombok</groupId>
|
||||||
|
<artifactId>lombok</artifactId>
|
||||||
|
<version>${lombok.version}</version>
|
||||||
|
</path>
|
||||||
|
</annotationProcessorPaths>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-surefire-plugin</artifactId>
|
||||||
|
<version>3.6.1</version>
|
||||||
|
<configuration>
|
||||||
|
<systemPropertyVariables>
|
||||||
|
<java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
|
||||||
|
</systemPropertyVariables>
|
||||||
|
<includes>
|
||||||
|
<include>**/Test*.java</include>
|
||||||
|
<include>**/*Test.java</include>
|
||||||
|
<include>**/*Tests.java</include>
|
||||||
|
<include>**/*TestCase.java</include>
|
||||||
|
<include>**/*Sit.java</include>
|
||||||
|
</includes>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-clean-plugin</artifactId>
|
||||||
|
<version>3.5.0</version>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-resources-plugin</artifactId>
|
||||||
|
<version>3.5.0</version>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-jar-plugin</artifactId>
|
||||||
|
<version>3.5.1</version>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-install-plugin</artifactId>
|
||||||
|
<version>3.2.0</version>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-deploy-plugin</artifactId>
|
||||||
|
<version>3.2.0</version>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
|
||||||
|
|
||||||
|
<profiles>
|
||||||
|
<profile>
|
||||||
|
<id>regenerate-engineering</id>
|
||||||
|
<properties><format.skip>true</format.skip></properties>
|
||||||
|
<build><plugins><plugin>
|
||||||
|
<groupId>org.codehaus.mojo</groupId><artifactId>exec-maven-plugin</artifactId><version>3.6.3</version>
|
||||||
|
<inherited>false</inherited>
|
||||||
|
<executions><execution><id>regenerate-engineering</id><phase>validate</phase><goals><goal>exec</goal></goals>
|
||||||
|
<configuration combine.self="override"><executable>python3</executable>
|
||||||
|
<arguments><argument>${maven.multiModuleProjectDirectory}/scripts/engineering.py</argument></arguments>
|
||||||
|
</configuration>
|
||||||
|
</execution></executions>
|
||||||
|
</plugin></plugins></build>
|
||||||
|
</profile>
|
||||||
|
</profiles>
|
||||||
|
|
||||||
|
</project>
|
||||||
10
templates/frontend/frontend/Dockerfile
Normal file
10
templates/frontend/frontend/Dockerfile
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
FROM node:24-alpine AS build
|
||||||
|
WORKDIR /workspace
|
||||||
|
COPY . .
|
||||||
|
RUN npm ci && npm run lint && npm test && npm run build
|
||||||
|
FROM nginxinc/nginx-unprivileged:1.28-alpine AS runtime
|
||||||
|
EXPOSE 8080
|
||||||
|
FROM runtime AS prebuilt
|
||||||
|
COPY dist/ /usr/share/nginx/html/
|
||||||
|
FROM runtime AS standalone
|
||||||
|
COPY --from=build /workspace/dist/ /usr/share/nginx/html/
|
||||||
3
templates/frontend/frontend/build.mjs
Normal file
3
templates/frontend/frontend/build.mjs
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
import { mkdir, copyFile } from 'node:fs/promises';
|
||||||
|
await mkdir('dist', { recursive: true });
|
||||||
|
await copyFile('index.html', 'dist/index.html');
|
||||||
1
templates/frontend/frontend/index.html
Normal file
1
templates/frontend/frontend/index.html
Normal file
@@ -0,0 +1 @@
|
|||||||
|
<!doctype html><html lang="en"><meta charset="utf-8"><title>@PROJECT@</title><h1>@PROJECT@</h1></html>
|
||||||
1
templates/frontend/frontend/package-lock.json
generated
Normal file
1
templates/frontend/frontend/package-lock.json
generated
Normal file
@@ -0,0 +1 @@
|
|||||||
|
{"name":"@PROJECT@-frontend","version":"1.0.0","lockfileVersion":3,"requires":true,"packages":{"":{"name":"@PROJECT@-frontend","version":"1.0.0"}}}
|
||||||
4
templates/frontend/frontend/package.json
Normal file
4
templates/frontend/frontend/package.json
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"name": "@PROJECT@-frontend", "version": "1.0.0", "private": true,
|
||||||
|
"scripts": {"lint": "node --check build.mjs", "test": "node --test test.mjs", "build": "node build.mjs"}
|
||||||
|
}
|
||||||
4
templates/frontend/frontend/test.mjs
Normal file
4
templates/frontend/frontend/test.mjs
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
import { test } from 'node:test';
|
||||||
|
import { readFile } from 'node:fs/promises';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
test('starter page identifies the project', async () => { assert.match(await readFile('index.html', 'utf8'), /@PROJECT@/); });
|
||||||
97
tests/acceptance.py
Normal file
97
tests/acceptance.py
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Deploy two example revisions and roll back using the real shared remote script."""
|
||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import shutil
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
|
||||||
|
def run(args, *, cwd=None, env=None, input=None):
|
||||||
|
result = subprocess.run(args, cwd=cwd, env=env, input=input, text=True,
|
||||||
|
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=600)
|
||||||
|
if result.returncode:
|
||||||
|
raise RuntimeError('Command failed: ' + args[0] + '\n' + result.stdout)
|
||||||
|
return result.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument('project', type=Path)
|
||||||
|
parser.add_argument('--registry', required=True)
|
||||||
|
parser.add_argument('--namespace', required=True)
|
||||||
|
parser.add_argument('--user', required=True)
|
||||||
|
parser.add_argument('--token-file', type=Path, required=True)
|
||||||
|
args = parser.parse_args()
|
||||||
|
project = args.project.resolve()
|
||||||
|
name = 'recipe-acceptance-' + uuid.uuid4().hex[:10]
|
||||||
|
env = dict(os.environ)
|
||||||
|
env['DOCKER_HOST'] = run(['docker', 'context', 'inspect', '--format', '{{.Endpoints.docker.Host}}'])
|
||||||
|
env.pop('DOCKER_CONTEXT', None)
|
||||||
|
with tempfile.TemporaryDirectory(prefix='recipe-deploy-') as temporary:
|
||||||
|
root = Path(temporary)
|
||||||
|
auth = root / 'docker-auth'
|
||||||
|
auth.mkdir()
|
||||||
|
env['DOCKER_CONFIG'] = str(auth)
|
||||||
|
run(['docker', 'login', args.registry, '--username', args.user, '--password-stdin'],
|
||||||
|
env=env, input=args.token_file.read_text().strip() + '\n')
|
||||||
|
with socket.socket() as listener:
|
||||||
|
listener.bind(('127.0.0.1', 0))
|
||||||
|
port = listener.getsockname()[1]
|
||||||
|
(root / 'production.env').write_text(f'BIND_IP=127.0.0.1\nFRONTEND_PORT={port}\nDOCKER_BIN={shutil.which("docker")}\nDOCKER_SUDO=0\nDOCKER_CONFIG={auth}\n')
|
||||||
|
os.chmod(root / 'production.env', 0o600)
|
||||||
|
source = (project / 'generated/deploy/remote.sh').read_text()
|
||||||
|
prefix = f"COMPOSE_PROJECT={name}\nDEPLOY_ROLES='backend frontend'\nDEPENDENCY_SERVICES=''\nTARGET_PLATFORM=linux/amd64\nREQUIRED_SECRETS=''\n"
|
||||||
|
script = prefix + (project / 'deploy/health.sh').read_text() + '\n' + source
|
||||||
|
image_prefix = args.registry + '/' + args.namespace + '/' + name + '-'
|
||||||
|
|
||||||
|
def remote(action, owner, *values, input=None):
|
||||||
|
return run(['sh', '-c', script, '--', action, str(root), owner, *values],
|
||||||
|
env=env, input=input)
|
||||||
|
|
||||||
|
def apply(owner, version, revision):
|
||||||
|
for role in ('backend', 'frontend'):
|
||||||
|
image = image_prefix + role + ':' + revision
|
||||||
|
context = project if role == 'backend' else project / 'frontend'
|
||||||
|
run(['docker', 'build', '--target', 'prebuilt', '--label',
|
||||||
|
'org.opencontainers.image.revision=' + revision, '-t', image, str(context)], env=env)
|
||||||
|
run(['docker', 'push', image], env=env)
|
||||||
|
remote('lock', owner)
|
||||||
|
remote('preflight', owner)
|
||||||
|
remote('stage', owner, input=(project / 'deploy/compose.yaml').read_text())
|
||||||
|
remote('stage-health', owner, input=(project / 'deploy/health.sh').read_text())
|
||||||
|
prepared = remote('prepare', owner, version, revision, image_prefix)
|
||||||
|
assert prepared.count('@sha256:') == 2
|
||||||
|
remote('pending', owner, input='{"isolated_acceptance":true}\n')
|
||||||
|
assert remote('apply', owner, owner, '60').endswith('S6_HEALTHY')
|
||||||
|
remote('commit', owner, input='{"isolated_acceptance":true}\n')
|
||||||
|
remote('finish', owner)
|
||||||
|
remote('unlock', owner)
|
||||||
|
|
||||||
|
try:
|
||||||
|
apply('first', 'v1.0.0', 'a' * 40)
|
||||||
|
apply('second', 'v1.0.1', 'b' * 40)
|
||||||
|
remote('lock', 'rollback')
|
||||||
|
remote('pending', 'rollback', input='{"rollback":true}\n')
|
||||||
|
assert remote('apply', 'rollback', 'first', '60').endswith('S6_HEALTHY')
|
||||||
|
revisions = run(['docker', 'compose', '--project-name', name, '--env-file',
|
||||||
|
str(root / 'production.env'), '--env-file', str(root / 'releases/first/release.env'),
|
||||||
|
'-f', str(root / 'releases/first/compose.yaml'), 'ps', '-q'], env=env).splitlines()
|
||||||
|
assert len(revisions) == 2
|
||||||
|
for container in revisions:
|
||||||
|
assert run(['docker', 'inspect', '--format', '{{index .Config.Labels "nl.agilecc.revision"}}', container], env=env) == 'a' * 40
|
||||||
|
remote('finish', 'rollback')
|
||||||
|
remote('unlock', 'rollback')
|
||||||
|
print('PASS: actual digest preparation, two isolated deployments, readiness and rollback to the first revision')
|
||||||
|
finally:
|
||||||
|
config = root / 'releases/first/compose.yaml'
|
||||||
|
if config.exists():
|
||||||
|
run(['docker', 'compose', '--project-name', name, '--env-file', str(root / 'production.env'),
|
||||||
|
'--env-file', str(root / 'releases/first/release.env'), '-f', str(config), 'down', '--volumes'], env=env)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
51
tests/format_integration.py
Normal file
51
tests/format_integration.py
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
"""Integration checks against an isolated checkout; run python3 tooling/format/test.py."""
|
||||||
|
from pathlib import Path
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
root = Path(__file__).resolve().parents[1]
|
||||||
|
with tempfile.TemporaryDirectory(prefix="palantir-test-") as temporary:
|
||||||
|
fixture = Path(temporary)
|
||||||
|
pom = (root / "templates/common/pom.xml").read_text().replace("@PROJECT@", "format-fixture").replace("@MODULES@", "")
|
||||||
|
(fixture / "pom.xml").write_text(pom)
|
||||||
|
(fixture / "generated/tooling").mkdir(parents=True)
|
||||||
|
shutil.copytree(root / "resources/tooling/format", fixture / "generated/tooling/format")
|
||||||
|
(fixture / "engineering.properties").write_text("FORMAT_DIRECTORIES=backend/src,ci\n")
|
||||||
|
for name in ("backend/src/main/java", "backend/src/test/java", "ci/runner"):
|
||||||
|
(fixture / name).mkdir(parents=True)
|
||||||
|
source = fixture / "ci/runner/Example.java"
|
||||||
|
original = "import java.util.List;\nclass Example{String text=\"héllo\";int value(){return 1;}}\n"
|
||||||
|
source.write_text(original)
|
||||||
|
|
||||||
|
def run(mode, success=True):
|
||||||
|
result = subprocess.run(
|
||||||
|
["mvn", "-N", "-B", "-ntp", "exec:exec@format", "-Dformat.mode=" + mode],
|
||||||
|
cwd=fixture, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
|
||||||
|
assert (result.returncode == 0) == success, result.stdout
|
||||||
|
return result.stdout
|
||||||
|
|
||||||
|
output = run("check", False)
|
||||||
|
assert "ci/runner/Example.java" in output and "mvn -N exec:exec@format" in output
|
||||||
|
assert source.read_text() == original
|
||||||
|
run("apply")
|
||||||
|
formatted = source.read_text()
|
||||||
|
assert "import java.util.List" not in formatted and "héllo" in formatted
|
||||||
|
run("check")
|
||||||
|
run("apply")
|
||||||
|
assert source.read_text() == formatted
|
||||||
|
# Force a full (uncached) second pass to prove engine idempotence.
|
||||||
|
(fixture / "target/palantir-cache.properties").unlink()
|
||||||
|
run("check")
|
||||||
|
source.write_text(formatted + "class Broken { ??? }\n")
|
||||||
|
invalid = source.read_bytes()
|
||||||
|
output = run("apply", False)
|
||||||
|
assert "ci/runner/Example.java" in output
|
||||||
|
assert source.read_bytes() == invalid
|
||||||
|
# A new file is discovered even after a successful cached check.
|
||||||
|
source.write_text(formatted)
|
||||||
|
extra = fixture / "ci/New.java"
|
||||||
|
extra.write_text("class New{}")
|
||||||
|
run("check", False)
|
||||||
|
assert extra.read_text() == "class New{}"
|
||||||
|
print("PASS: check is read-only, apply/check agree, imports, UTF-8, idempotence, parser errors, new files")
|
||||||
95
tests/test_bootstrap.py
Normal file
95
tests/test_bootstrap.py
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
"""Exercise release pinning, drift detection, replacement and recovery without network."""
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
HOME = Path(__file__).resolve().parents[1]
|
||||||
|
spec = importlib.util.spec_from_file_location('bootstrap', HOME / 'bootstrap.py')
|
||||||
|
bootstrap = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(bootstrap)
|
||||||
|
|
||||||
|
|
||||||
|
class BootstrapTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.temporary = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(self.temporary.cleanup)
|
||||||
|
self.root = Path(self.temporary.name)
|
||||||
|
self.source = self.root / 'engineering'
|
||||||
|
self.source.mkdir()
|
||||||
|
self.git('init', '-q')
|
||||||
|
self.git('config', 'user.email', 'test@example.invalid')
|
||||||
|
self.git('config', 'user.name', 'Engineering test')
|
||||||
|
(self.source / 'resources').mkdir()
|
||||||
|
(self.source / 'resources/tool').write_text('version one\n')
|
||||||
|
self.commit = self.release('v1.0.0')
|
||||||
|
self.project = self.root / 'consumer'
|
||||||
|
self.project.mkdir()
|
||||||
|
self.lock = {'repository': str(self.source), 'tag': 'v1.0.0', 'commit': self.commit}
|
||||||
|
self.save_lock()
|
||||||
|
|
||||||
|
def git(self, *args):
|
||||||
|
return subprocess.check_output(['git', '-C', str(self.source), *args], text=True, stderr=subprocess.DEVNULL).strip()
|
||||||
|
|
||||||
|
def release(self, tag):
|
||||||
|
self.git('add', '.')
|
||||||
|
self.git('commit', '-qm', tag)
|
||||||
|
self.git('tag', tag)
|
||||||
|
return self.git('rev-parse', 'HEAD')
|
||||||
|
|
||||||
|
def save_lock(self):
|
||||||
|
(self.project / 'engineering.lock.json').write_text(json.dumps(self.lock))
|
||||||
|
|
||||||
|
def test_absent_generated_and_repeat_are_deterministic(self):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
first = bootstrap.files(self.project / 'generated')
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
self.assertEqual(first, bootstrap.files(self.project / 'generated'))
|
||||||
|
bootstrap.materialize(self.project, True)
|
||||||
|
|
||||||
|
def test_drift_includes_modified_added_and_missing_files(self):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
target = self.project / 'generated/tool'
|
||||||
|
for operation in ('modify', 'remove', 'add'):
|
||||||
|
with self.subTest(operation=operation):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
if operation == 'modify': target.write_text('hand edit')
|
||||||
|
elif operation == 'remove': target.unlink()
|
||||||
|
else: (target.parent / 'extra').write_text('untracked')
|
||||||
|
before = bootstrap.files(target.parent)
|
||||||
|
with self.assertRaisesRegex(ValueError, 'differ'):
|
||||||
|
bootstrap.materialize(self.project, True)
|
||||||
|
self.assertEqual(before, bootstrap.files(target.parent))
|
||||||
|
|
||||||
|
def test_update_removes_obsolete_files_and_preserves_project_files(self):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
(self.project / 'pom.xml').write_text('owned by project')
|
||||||
|
(self.source / 'resources/tool').unlink()
|
||||||
|
(self.source / 'resources/new').write_text('new release')
|
||||||
|
self.lock.update(tag='v1.1.0', commit=self.release('v1.1.0'))
|
||||||
|
self.save_lock()
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
self.assertFalse((self.project / 'generated/tool').exists())
|
||||||
|
self.assertEqual('owned by project', (self.project / 'pom.xml').read_text())
|
||||||
|
|
||||||
|
def test_moved_tag_fails_before_replacing_existing_content(self):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
before = bootstrap.files(self.project / 'generated')
|
||||||
|
(self.source / 'resources/tool').write_text('different commit')
|
||||||
|
self.release('v1.1.0')
|
||||||
|
self.git('tag', '-f', 'v1.0.0')
|
||||||
|
with self.assertRaisesRegex(ValueError, 'tag moved'):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
self.assertEqual(before, bootstrap.files(self.project / 'generated'))
|
||||||
|
|
||||||
|
def test_symlink_output_is_rejected(self):
|
||||||
|
(self.project / 'generated').symlink_to(self.source / 'resources', target_is_directory=True)
|
||||||
|
with self.assertRaisesRegex(ValueError, 'symlink'):
|
||||||
|
bootstrap.materialize(self.project)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
59
tests/test_generator.py
Normal file
59
tests/test_generator.py
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
"""Check component selection and real Maven bootstrapping with generated/ absent."""
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
HOME = Path(__file__).resolve().parents[1]
|
||||||
|
spec = importlib.util.spec_from_file_location('generator', HOME / 'create-project.py')
|
||||||
|
generator = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(generator)
|
||||||
|
|
||||||
|
|
||||||
|
class GeneratorTests(unittest.TestCase):
|
||||||
|
@classmethod
|
||||||
|
def setUpClass(cls):
|
||||||
|
cls.temporary = tempfile.TemporaryDirectory()
|
||||||
|
cls.addClassCleanup(cls.temporary.cleanup)
|
||||||
|
cls.root = Path(cls.temporary.name)
|
||||||
|
cls.source = cls.root / 'engineering'
|
||||||
|
shutil.copytree(HOME, cls.source, ignore=shutil.ignore_patterns('.git','target','__pycache__'))
|
||||||
|
for args in (['init', '-q'], ['config','user.name','Recipe test'],
|
||||||
|
['config','user.email','test@example.invalid'], ['add','.'],
|
||||||
|
['commit','-qm','Test release'], ['tag','v1.0.0']):
|
||||||
|
subprocess.run(['git','-C',str(cls.source),*args], check=True, stdout=subprocess.DEVNULL)
|
||||||
|
generator.HOME = cls.source
|
||||||
|
|
||||||
|
def test_component_choices_and_bootstrap_without_generated_files(self):
|
||||||
|
for components in (['backend'], ['frontend'], ['backend','frontend']):
|
||||||
|
with self.subTest(components=components):
|
||||||
|
project = self.root / ('consumer-' + '-'.join(components))
|
||||||
|
generator.create(project, 'sample', components, str(self.source), 'v1.0.0', 'test-runner', 'https://landscape.invalid')
|
||||||
|
self.assertEqual('backend' in components, (project / 'backend/pom.xml').exists())
|
||||||
|
self.assertEqual('frontend' in components, (project / 'frontend/package.json').exists())
|
||||||
|
self.assertIn('test-runner', (project / '.gitea/workflows/build.yaml').read_text())
|
||||||
|
shutil.rmtree(project / 'generated')
|
||||||
|
result = subprocess.run(['mvn','-N','-B','-ntp','-Pregenerate-engineering','validate'], cwd=project,
|
||||||
|
text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=180)
|
||||||
|
self.assertEqual(0, result.returncode, result.stdout)
|
||||||
|
self.assertTrue((project / 'generated/ci/Ci.java').exists())
|
||||||
|
subprocess.run(['python3','scripts/engineering.py','--check'], cwd=project, check=True, stdout=subprocess.DEVNULL)
|
||||||
|
if components == ['frontend']:
|
||||||
|
for args in (['npm','ci'], ['npm','run','lint'], ['npm','test'], ['npm','run','build']):
|
||||||
|
subprocess.run(args, cwd=project / 'frontend', check=True, stdout=subprocess.DEVNULL)
|
||||||
|
self.assertTrue((project / 'frontend/dist/index.html').exists())
|
||||||
|
|
||||||
|
def test_generator_preserves_existing_destination(self):
|
||||||
|
project = self.root / 'already-exists'
|
||||||
|
project.mkdir()
|
||||||
|
(project / 'important').write_text('preserve')
|
||||||
|
with self.assertRaisesRegex(ValueError, 'must not exist'):
|
||||||
|
generator.create(project, 'sample', ['backend'], str(self.source), 'v1.0.0', 'runner', 'landscape')
|
||||||
|
self.assertEqual('preserve', (project / 'important').read_text())
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
Reference in New Issue
Block a user