S19: establish engineering repository
This commit is contained in:
276
resources/CODING_STANDARDS.md
Normal file
276
resources/CODING_STANDARDS.md
Normal file
@@ -0,0 +1,276 @@
|
||||
# Backend coding standards
|
||||
|
||||
This document is the authoritative policy for handwritten production and test Java
|
||||
in `<backend-module>`. Generated code is out of scope. The Java conventions below also apply to
|
||||
handwritten Java helpers outside that module. Frontend conventions are owned by each consuming project.
|
||||
|
||||
## Scope and authority
|
||||
|
||||
- Cover handwritten production and test Java in `<backend-module>`. Exclude
|
||||
generated code. Dependency injection and production architecture rules apply
|
||||
where relevant. The Java conventions also cover Java helpers outside the backend; backend
|
||||
architecture, metrics, coverage and automated checks remain backend-only.
|
||||
Frontend code is out of scope.
|
||||
- [Java coding guidelines](https://www.geeksforgeeks.org/java/coding-guidelines-in-java/)
|
||||
supplied source material. Only the rules adopted in this local document apply;
|
||||
changes to the article do not change project policy.
|
||||
- The project's pinned Palantir formatter wins formatting conflicts, including
|
||||
comment layout. S8 remains responsible for formatter integration.
|
||||
- Do not adopt unrelated static-analysis rule bundles.
|
||||
|
||||
## General language guidance
|
||||
|
||||
- Always prefer newer stable official APIs, language features and officially
|
||||
recommended coding practices over older alternatives when they suit the task
|
||||
and are supported by the project's configured language/runtime version.
|
||||
Preserve required behavior and compatibility; document concrete reasons for
|
||||
retaining an older approach. This guidance applies to every language used in
|
||||
the project.
|
||||
|
||||
## Java conventions
|
||||
|
||||
- Types use UpperCamelCase; methods and variables use lowerCamelCase; constants
|
||||
use UPPER_SNAKE_CASE; package names are lowercase.
|
||||
- One top-level type per file; explicit member visibility; `@Override` on
|
||||
overriding methods. Do not invent visibility modifiers where Java forbids them.
|
||||
- Opening braces stay on the declaration/control-statement line. All `if`, `for`
|
||||
and `while` bodies require braces, including enhanced for and do/while bodies.
|
||||
- No wildcard imports. Static imports are allowed only in test classes.
|
||||
- Prefer `java.nio` APIs (including `java.nio.file` and `java.nio.charset`) over
|
||||
equivalent `java.io` APIs. Use stream APIs where no suitable NIO equivalent
|
||||
exists, such as process pipes or APIs requiring streams.
|
||||
- Use varargs (`Type... values`) when an array is the last method or constructor
|
||||
parameter, including `main(String... args)`. Preserve required override signatures
|
||||
and framework/API compatibility.
|
||||
- Prefer Java 25 `IO.print`/`IO.println` for direct standard output and `IO.readln`
|
||||
for line-oriented standard input. `IO` has no standard-error variant; retain
|
||||
`System.err` for error output. Use application logging where appropriate.
|
||||
- Comments explain intent or constraints. Require Javadoc on public classes and
|
||||
public methods that do more than basic POJO/data-carrier work. Trivial accessors
|
||||
and plain data carriers do not need boilerplate documentation. Describe behavior,
|
||||
constraints and failure conditions where relevant.
|
||||
- Block comments use the following layout, subject to Palantir formatting:
|
||||
|
||||
```java
|
||||
/*
|
||||
This is an example of a block comment.
|
||||
It may have more than one line.
|
||||
*/
|
||||
```
|
||||
- Prefer records for immutable data carriers. Classes are allowed when mutation,
|
||||
framework requirements or inheritance make records unsuitable. Preserve API and
|
||||
serialization behavior. Record immutability is shallow: protect mutable
|
||||
components appropriately.
|
||||
- Require `final` for parameters, local variables and fields wherever they can
|
||||
remain assigned once, including loop variables and catch parameters where Java
|
||||
permits it. Field finality must remain compatible with framework injection and
|
||||
lifecycle requirements. Permit reassignment when the algorithm needs it; do not
|
||||
introduce artificial holder objects to avoid reassignment. This rule does not
|
||||
require final classes or methods.
|
||||
- Use explicit types unless the initializer visibly identifies the type. `var` is
|
||||
allowed with `new School(...)` or an explicit cast, but not for method results,
|
||||
literals or other inferred expressions. For example,
|
||||
`final var school = new School(...)` is allowed, while a repository result uses
|
||||
`final School school = repository.find(id)`. Borderline cases require review.
|
||||
- No additional member-ordering or line-width policy beyond Palantir.
|
||||
|
||||
## Metrics
|
||||
|
||||
Targets guide development. Review signals require assessment, not automatic
|
||||
failure. Blocking limits fail verification. Strict inequalities matter: a value
|
||||
exactly at a blocking limit passes.
|
||||
|
||||
| Metric | Target | Review signal | Blocking limit |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Cyclomatic complexity per method/constructor | ≤ 5 | > 7 | > 10 |
|
||||
| Cognitive complexity per method/constructor | ≤ 10 | > 10 | > 15 |
|
||||
| Method/constructor length, code lines | ≤ 20 | > 30 | > 50 |
|
||||
| Parameters per method | ≤ 3 | > 4 | > 5 |
|
||||
| Parameters per explicitly written constructor | ≤ 4 | > 5 | > 7 |
|
||||
| Injected dependencies per class | ≤ 4 | > 5 | > 7 |
|
||||
| Fan-out per class, unique used types | ≤ 10 | > 15 | Review only |
|
||||
| Own superclass depth, excluding Object | ≤ 1 | > 2 | > 3 |
|
||||
| Control-flow nesting | ≤ 2 | > 3 | > 4 |
|
||||
| Production duplicated lines density | ≤ 1% | > 1% | > 3% |
|
||||
| Class length, code lines | ≤ 200 | > 300 | > 500 |
|
||||
| Declared public methods per class, excluding trivial accessors | ≤ 10 | > 15 | Review only |
|
||||
|
||||
### Measurement definitions
|
||||
|
||||
- Use the pinned PMD algorithms as authoritative for cyclomatic and cognitive
|
||||
complexity. Document their exact configuration, including switch and boolean
|
||||
treatment. Do not accidentally introduce a class-level aggregate complexity gate.
|
||||
- Measure each method, constructor and explicitly written record constructor
|
||||
independently. Lambda bodies contribute to their enclosing executable;
|
||||
methods in nested/local classes are separate executables.
|
||||
- Parameter limits exclude implicit record constructors and generated members.
|
||||
Record component counts remain a design review concern.
|
||||
- Length means physical source lines after Palantir formatting, excluding blank
|
||||
and comment-only lines. Declaration and brace-only lines count; a line containing
|
||||
both code and comments counts. Do not substitute statement counts or NCSS.
|
||||
Measure nested types separately and exclude them from enclosing class length.
|
||||
- Count distinct injected collaborators by resolved type and qualifier. Include
|
||||
constructor, field, setter, inherited and implicit constructor injection.
|
||||
Repeated injection of the same type and qualifier counts once; differently
|
||||
qualified instances of the same type count separately.
|
||||
- Fan-out counts distinct explicitly referenced resolved types, including JDK and
|
||||
framework types. Exclude primitives, the class itself and generated references.
|
||||
- Inheritance depth counts project-owned superclass layers; exclude Object,
|
||||
interfaces and external framework ancestry.
|
||||
- Nesting combines `if`, loops, `switch` and `catch`, with the first structure at
|
||||
depth one. An `else if` chain stays at one level. `else`, `try` and `finally`
|
||||
add no level themselves. Lambdas retain surrounding depth; methods in nested
|
||||
classes start fresh.
|
||||
- Public-method counts include explicitly declared public instance and static
|
||||
methods; exclude constructors, inherited methods and trivial accessors. A
|
||||
trivial accessor only reads or assigns its corresponding field, with no
|
||||
validation, branching or other work.
|
||||
- Duplication limits apply only to production code. Duplication in tests is allowed
|
||||
and has no target, review signal or blocking limit.
|
||||
- Detect production clones with CPD, minimum 100 tokens, without ignoring
|
||||
identifier or literal differences. Density is
|
||||
`100 × distinct duplicated code-line locations / all production code-line locations`.
|
||||
Exclude blank/comment-only lines from numerator and denominator. Count every
|
||||
occurrence, including the first; merge overlapping ranges to count each source
|
||||
line once. This project metric need not equal a future Sonar metric.
|
||||
|
||||
## Enforcement and developer workflow
|
||||
|
||||
- Use explicitly pinned Java-25-compatible Checkstyle and PMD/CPD versions.
|
||||
Retain ArchUnit for architecture checks and add small targeted custom checks
|
||||
where standard rules cannot accurately express the agreed semantics.
|
||||
- Automate every blocking metric and explicit rule where reliable checking is
|
||||
available. Identify rules requiring manual review explicitly; do not claim
|
||||
they are automated. Review signals must appear in reports even when builds pass.
|
||||
- Run standards checks during `mvn verify`; provide documented standalone commands.
|
||||
Ordinary compile/test commands retain the current S8 formatting behavior.
|
||||
- Standards checks report violations without rewriting code. CI passes these
|
||||
checks before image publication. Parser failures or unsupported Java constructs
|
||||
fail with the affected path and actionable diagnostics, rather than being skipped.
|
||||
- Add a root AGENTS.md instruction requiring agents to read CODING_STANDARDS.md
|
||||
before changing backend Java. Project review instructions must explicitly cover
|
||||
manual rules, review signals in changed code and approved waivers, without
|
||||
repeating checks already enforced by tooling.
|
||||
- Keep upstream skill files unchanged. Put project-specific code-review guidance
|
||||
in a companion file referenced by project instructions and used alongside
|
||||
`skills/code-review/SKILL.md`.
|
||||
- Waivers must be explicit, narrowly scoped and carry a recorded reason alongside
|
||||
code or configuration. Agents need maintainer agreement before introducing one.
|
||||
No blanket suppressions, exclusions of difficult files or weakened thresholds
|
||||
merely to make verification pass.
|
||||
|
||||
## Coverage
|
||||
|
||||
- Require at least 80% backend line coverage across handwritten production code
|
||||
for the module as a whole, not separately per class or package. Compare the
|
||||
unrounded percentage against 80%.
|
||||
- Local `mvn verify` must reach 80% using backend unit/integration tests alone.
|
||||
This intentionally tightens the original combined-tests-only requirement.
|
||||
- CI additionally instruments the backend exercised by Playwright, merges that
|
||||
coverage with backend unit/integration coverage and gates the combined result.
|
||||
The Playwright contribution need not separately reach 80%.
|
||||
- Exclude generated code only; do not broadly exclude DTOs, configuration or
|
||||
difficult code. Publish line and branch coverage reports; branch coverage has
|
||||
no agreed blocking percentage.
|
||||
- PIT measures mutation-test effectiveness, not line coverage, and is outside
|
||||
S7's required implementation. Configuring Sonar in the pipeline remains out of
|
||||
scope for S14. FindBugs/SpotBugs is not an additional required check suite.
|
||||
|
||||
|
||||
## Implemented checks and manual review
|
||||
|
||||
Checkstyle **12.3.1**, PMD/CPD **7.28.0**, and JaCoCo **0.8.14** are pinned.
|
||||
`generated/tooling/standards/checkstyle.xml` selects individual naming, structure, import and brace checks; no general analysis bundle is enabled.
|
||||
`generated/tooling/standards/src/main/java/Standards.java` supplies the targeted measurements.
|
||||
ArchUnit remains in the backend test suite for architectural boundaries.
|
||||
|
||||
PMD uses Java 25, the running JDK's `jrt-fs.jar`, compiled backend classes and the
|
||||
resolved test dependency classpath. Cyclomatic complexity uses PMD's default
|
||||
options: boolean paths count, assertions do not. Decisions include if/loops,
|
||||
catch, ternary and throw; non-default switch alternatives count individually,
|
||||
including comma-separated labels. There is no aggregate class-complexity gate.
|
||||
Cognitive complexity uses PMD's boolean-operation sequences, nesting, switch,
|
||||
recursion and flow rules. The scoped visitors retain those algorithms while
|
||||
including lambdas in their enclosing executable and excluding nested/local type
|
||||
bodies, whose methods are measured independently. Compact record constructors
|
||||
use the same visitors and start with cyclomatic complexity one.
|
||||
|
||||
Custom checks measure physical code lines from regular parser tokens, including
|
||||
brace/declaration lines and mixed code/comment lines. They count multiline literal
|
||||
lines while excluding physically blank lines. Class length excludes nested-type
|
||||
tokens, preserving lines shared with enclosing code. Other custom measurements
|
||||
implement the definitions above: nesting, resolved type fan-out, project-owned
|
||||
superclass depth, public non-accessor methods, and injected collaborators.
|
||||
Injection includes inherited fields/setters, explicit `@Inject` constructors,
|
||||
single constructors on bean classes, records' implicit canonical constructors,
|
||||
and Lombok required/all-argument constructor injection. Qualifier keys use resolved
|
||||
annotation names and values, including defaults; repeat sites are deduplicated.
|
||||
The CPD configuration preserves identifiers/literals, uses 100 tokens and disables
|
||||
skip blocks. Its line-location union counts overlapping occurrences only once.
|
||||
|
||||
Checkstyle and targeted checks enforce import/brace/naming/visibility rules,
|
||||
`@Override`, visible constructor/cast/new-array `var` initializer forms, final parameters, locals (including
|
||||
loops/catches and lambdas) and initialized fields. PMD's three selected finality
|
||||
rules additionally cover control-flow assignment and private constructor-initialized
|
||||
fields. Framework/lifecycle requirements, generated Lombok setters, and external
|
||||
writes to public/protected fields need review before changing field finality.
|
||||
Comment intent, Javadoc adequacy for nontrivial public contracts, record suitability
|
||||
and protection of mutable components require manual review. These are **manual
|
||||
rules**, as are ambiguous type inference, architecture/DI design decisions, NIO
|
||||
API selection, trailing-array varargs, console I/O selection and the POJO
|
||||
documentation boundary. Review all `DESIGN` record-component entries
|
||||
and `REVIEW` signals, even when a build passes, using `docs/backend-review.md`.
|
||||
|
||||
Reports never rewrite Java. Formatter application remains S8's separate validate
|
||||
execution. Generated output under `target` and generated source directories are
|
||||
outside the handwritten `src/main/java` and `src/test/java` scope. No maintained
|
||||
file, DTO, configuration class or package is excluded from standards or coverage.
|
||||
JaCoCo filters compiler/Lombok-generated members; no handwritten-code exclusions
|
||||
are configured. Line and branch results are published, with only lines gated.
|
||||
|
||||
Exact metric waivers live in `tooling/standards/waivers.tsv`; each records a metric,
|
||||
fully resolved executable signature (or class), exact retained value and the
|
||||
maintainer agreement/reason. No wildcard waivers are allowed, and a changed value
|
||||
fails again. A waived blocking result remains visible as `WAIVER`. PMD suppression
|
||||
annotations/comments cannot silently waive a rule. Adding a waiver always requires maintainer agreement.
|
||||
|
||||
## Commands and reports
|
||||
|
||||
Run from the root with JDK 25:
|
||||
|
||||
```sh
|
||||
mvn -Dformat.mode=check clean verify
|
||||
# Prepare standalone analysis without running tests:
|
||||
mvn -pl <backend-module> test-compile dependency:build-classpath@standards-classpath
|
||||
mvn -f generated/tooling/standards/pom.xml test exec:java
|
||||
# Exact local coverage gate after a test/report build:
|
||||
java generated/tooling/coverage/Coverage.java <backend-module>/target/site/jacoco/jacoco.xml
|
||||
```
|
||||
|
||||
`verify` runs backend tests, builds resolved analysis inputs, generates JaCoCo
|
||||
reports, runs checker regression tests and standards analysis, then gates exact
|
||||
module-wide line coverage. `compile` and `test` retain existing formatting behavior
|
||||
and do not run the verify-only standards checks. Standalone analysis requires its
|
||||
classpath preparation; unresolved types fail with the path and preparation advice.
|
||||
|
||||
Full metrics and findings: `<backend-module>/target/standards/report.txt`.
|
||||
Checker regression results: `generated/tooling/standards/target/surefire-reports`.
|
||||
Coverage HTML/XML/CSV: `<backend-module>/target/site/jacoco` (local backend tests)
|
||||
and `<backend-module>/target/site/jacoco-combined` (merged browser contribution).
|
||||
The gate compares integer line counts (`5 * covered >= 4 * total`), rejecting an
|
||||
empty report and percentages that would round up to 80%.
|
||||
|
||||
CI first passes local verify. It mounts the pinned JaCoCo agent into its disposable
|
||||
backend test container, exposes the agent only on the isolated Docker network,
|
||||
and uses the pinned CLI to dump `target/playwright.exec` after Playwright. A missing
|
||||
or empty dump fails. The following commands merge both datasets and gate the result
|
||||
before the verified-commit marker can authorize publication:
|
||||
|
||||
```sh
|
||||
mvn -pl <backend-module> jacoco:merge@combined jacoco:report@combined-report exec:exec@combined-coverage
|
||||
```
|
||||
|
||||
CI retains both coverage reports, execution data, standards findings and regression
|
||||
results with its usual diagnostics artifacts. The agent is mounted only for tests;
|
||||
published images contain no agent. PIT and Sonar configuration remain outside S7.
|
||||
Application validation evidence and retained-signal assessments stay in the consuming project.
|
||||
62
resources/ci/Ci.java
Normal file
62
resources/ci/Ci.java
Normal file
@@ -0,0 +1,62 @@
|
||||
import java.nio.file.*;
|
||||
import java.time.*;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.*;
|
||||
|
||||
/** Run from the repository root with JDK 25: java ci/Ci.java <command>. */
|
||||
public class Ci {
|
||||
public static void main(String[] args) {
|
||||
try {
|
||||
if (Runtime.version().feature() < 25) throw new IllegalArgumentException("JDK 25 or newer is required");
|
||||
if (args.length == 0) throw new IllegalArgumentException(usage());
|
||||
Path root = Path.of("").toAbsolutePath();
|
||||
var env = Recipe.environment(root, System.getenv());
|
||||
var commands = new Commands(Duration.ofMinutes(args[0].equals("publish") ? 15 : 75));
|
||||
switch (args[0]) {
|
||||
case "verify", "publish", "cleanup" -> {
|
||||
var pipeline = new Pipeline(root, env, commands);
|
||||
switch (args[0]) {
|
||||
case "verify" -> pipeline.verify();
|
||||
case "publish" -> pipeline.publish();
|
||||
default -> pipeline.cleanup();
|
||||
}
|
||||
}
|
||||
case "attempt" -> {
|
||||
Path file = Path.of(Setup.required(env, "GITHUB_ENV"));
|
||||
String id = DateTimeFormatter.ofPattern("yyyyMMddHHmmssSSSSSSSSS")
|
||||
.withZone(ZoneOffset.UTC)
|
||||
.format(Instant.now());
|
||||
Files.writeString(
|
||||
file, "CI_RUN_ATTEMPT=" + id + "\n", StandardOpenOption.CREATE, StandardOpenOption.APPEND);
|
||||
}
|
||||
case "configure-gitea" -> Setup.configureGitea(env);
|
||||
case "configure-docker" -> {
|
||||
if (args.length != 2) throw new IllegalArgumentException("Supply the daemon.json path");
|
||||
Setup.configureDocker(Path.of(args[1]), Setup.required(env, "INSECURE_REGISTRY"));
|
||||
}
|
||||
case "install-runner" ->
|
||||
Setup.installRunner(root, env, commands, args.length == 2 && args[1].equals("--prepare-only"));
|
||||
case "test" -> CiTests.run();
|
||||
case "deploy", "rollback" -> Release.execute(root, env, args[0]);
|
||||
case "release-test" -> ReleaseTests.run();
|
||||
case "configure-deployment" -> ReleaseSetup.configure(root, env);
|
||||
default -> throw new IllegalArgumentException(usage());
|
||||
}
|
||||
} catch (Commands.Failure e) {
|
||||
System.err.println(e.getMessage());
|
||||
System.exit(e.exitCode);
|
||||
} catch (Exception e) {
|
||||
// Never dump HTTP bodies, process arguments, environment or tokens.
|
||||
System.err.println(
|
||||
e instanceof IllegalArgumentException || e instanceof Setup.ApiFailure
|
||||
? e.getMessage()
|
||||
: "CI failed (" + e.getClass().getSimpleName() + "). Check diagnostics and configuration.");
|
||||
System.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
static String usage() {
|
||||
return "Usage: java ci/Ci.java verify|publish|cleanup|attempt|test|configure-gitea|configure-docker"
|
||||
+ " <path>|install-runner [--prepare-only]|deploy|rollback|release-test|configure-deployment";
|
||||
}
|
||||
}
|
||||
358
resources/ci/CiTests.java
Normal file
358
resources/ci/CiTests.java
Normal file
@@ -0,0 +1,358 @@
|
||||
import java.nio.file.*;
|
||||
import java.time.Duration;
|
||||
import java.util.*;
|
||||
|
||||
/** Dependency-free behavior tests: java ci/Ci.java test. */
|
||||
class CiTests {
|
||||
static final String SHA = "a".repeat(40);
|
||||
static int count;
|
||||
|
||||
interface Test {
|
||||
void run() throws Exception;
|
||||
}
|
||||
|
||||
static void test(String name, Test test) throws Exception {
|
||||
test.run();
|
||||
count++;
|
||||
System.out.println("PASS " + name);
|
||||
}
|
||||
|
||||
static void equal(Object expected, Object actual) {
|
||||
if (!Objects.equals(expected, actual)) throw new AssertionError("Expected " + expected + ", got " + actual);
|
||||
}
|
||||
|
||||
static void failure(Test action) throws Exception {
|
||||
try {
|
||||
action.run();
|
||||
} catch (Exception expected) {
|
||||
return;
|
||||
}
|
||||
throw new AssertionError("Expected failure");
|
||||
}
|
||||
|
||||
static void run() throws Exception {
|
||||
count = 0;
|
||||
test("recipe supports backend-only and frontend-only projects", () -> {
|
||||
equal(List.of("backend"), Recipe.roles(Map.of("BACKEND_MODULE", "server")));
|
||||
equal(List.of("frontend"), Recipe.roles(Map.of("FRONTEND_MODULE", "web")));
|
||||
failure(() -> Recipe.roles(Map.of()));
|
||||
failure(() -> Recipe.project(Map.of("PROJECT_NAME", "../unsafe")));
|
||||
});
|
||||
test("project configuration is overridden by environment and rejects escaping modules", () -> {
|
||||
final Path root = Files.createTempDirectory("recipe-config-");
|
||||
try {
|
||||
Files.writeString(
|
||||
root.resolve("engineering.properties"), "PROJECT_NAME=sample\nBACKEND_MODULE=server\n");
|
||||
equal(
|
||||
"override",
|
||||
Recipe.environment(root, Map.of("PROJECT_NAME", "override"))
|
||||
.get("PROJECT_NAME"));
|
||||
failure(() -> Recipe.environment(root, Map.of("BACKEND_MODULE", "../outside")));
|
||||
} finally {
|
||||
Pipeline.deleteTree(root);
|
||||
}
|
||||
});
|
||||
test("JSON preserves configuration values and escaped text", () -> {
|
||||
String json = "{\"text\":\"line\\nquote\\\"\\u20ac\",\"values\":[null,true,false,1,-2.5e2],\"nested\":{}}";
|
||||
Object data = Json.parse(json);
|
||||
equal(data, Json.parse(Json.write(data)));
|
||||
equal("line\nquote\"€", Json.object(data).get("text"));
|
||||
});
|
||||
test("JSON rejects malformed/ambiguous input", () -> {
|
||||
for (String value : List.of(
|
||||
"{\"a\":null,\"a\":1}", "{} []", "[1,]", "01", "\"\\uZZZZ\"", "\"line\n\"", "truefalse", "+1"))
|
||||
failure(() -> Json.parse(value));
|
||||
});
|
||||
test("Docker configuration merge is idempotent and preserves backup", () -> {
|
||||
Path dir = Files.createTempDirectory("ci-docker-test-");
|
||||
try {
|
||||
Path file = dir.resolve("daemon.json");
|
||||
String original = "{\"features\":{\"buildkit\":true},\"insecure-registries\":[\"existing:5000\"]}";
|
||||
Files.writeString(file, original);
|
||||
Setup.configureDocker(file, "registry.example:3000");
|
||||
String first = Files.readString(file);
|
||||
Setup.configureDocker(file, "registry.example:3000");
|
||||
equal(first, Files.readString(file));
|
||||
equal(
|
||||
List.of("existing:5000", "registry.example:3000"),
|
||||
Json.object(Json.parse(first)).get("insecure-registries"));
|
||||
equal(original, Files.readString(dir.resolve("daemon.json.before-engineering")));
|
||||
} finally {
|
||||
Pipeline.deleteTree(dir);
|
||||
}
|
||||
});
|
||||
test("Invalid Docker configuration is unchanged", () -> {
|
||||
Path dir = Files.createTempDirectory("ci-docker-test-");
|
||||
try {
|
||||
Path file = dir.resolve("daemon.json");
|
||||
String original = "{\"insecure-registries\":\"invalid\"}";
|
||||
Files.writeString(file, original);
|
||||
failure(() -> Setup.configureDocker(file, "registry.example:3000"));
|
||||
equal(original, Files.readString(file));
|
||||
} finally {
|
||||
Pipeline.deleteTree(dir);
|
||||
}
|
||||
});
|
||||
test("Feature branches never contact registry", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.pipeline.env.put("CI_REF", "refs/heads/feature/test");
|
||||
f.pipeline.publish();
|
||||
equal(0L, f.calls("login"));
|
||||
equal(0L, f.calls("push"));
|
||||
}
|
||||
});
|
||||
test("Unverified commit cannot publish", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
Files.delete(f.root.resolve("ci/artifacts/verified-commit"));
|
||||
failure(f.pipeline::publish);
|
||||
equal(0L, f.calls("push"));
|
||||
}
|
||||
});
|
||||
test("Publication pullback and rerun preserve commit tags", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.pipeline.publish();
|
||||
equal(4L, f.calls("push"));
|
||||
f.pipeline.publish();
|
||||
equal(6L, f.calls("push"));
|
||||
equal(8L, f.calls("pull"));
|
||||
}
|
||||
});
|
||||
test("Partial publication is completed without overwriting backend", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.pushFailure = "frontend:" + SHA;
|
||||
failure(f.pipeline::publish);
|
||||
equal(
|
||||
0L,
|
||||
f.fake.calls.stream()
|
||||
.filter(a -> a.contains("push") && a.getLast().endsWith(":main"))
|
||||
.count());
|
||||
f.fake.pushFailure = "";
|
||||
f.pipeline.publish();
|
||||
equal(
|
||||
1L,
|
||||
f.fake.calls.stream()
|
||||
.filter(a -> a.contains("push") && a.getLast().endsWith("backend:" + SHA))
|
||||
.count());
|
||||
}
|
||||
});
|
||||
test("Authentication error is not a missing tag", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.manifestError = true;
|
||||
failure(f.pipeline::publish);
|
||||
equal(0L, f.calls("push"));
|
||||
}
|
||||
});
|
||||
test("Pull failure blocks moving tags", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.pullFailure = "backend:" + SHA;
|
||||
failure(f.pipeline::publish);
|
||||
equal(
|
||||
0L,
|
||||
f.fake.calls.stream()
|
||||
.filter(a -> a.contains("push") && a.getLast().endsWith(":main"))
|
||||
.count());
|
||||
}
|
||||
});
|
||||
test("Older commit cannot move main", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.head = "c".repeat(40);
|
||||
f.pipeline.publish();
|
||||
equal(2L, f.calls("push"));
|
||||
}
|
||||
});
|
||||
test("Unavailable Docker fails before build with a concrete cause", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.dockerAvailable = false;
|
||||
try {
|
||||
f.pipeline.verify();
|
||||
throw new AssertionError("Expected failure");
|
||||
} catch (IllegalArgumentException error) {
|
||||
equal(true, error.getMessage().contains("Docker is unavailable"));
|
||||
}
|
||||
equal(
|
||||
0L,
|
||||
f.fake.calls.stream()
|
||||
.filter(a -> a.getFirst().equals("mvn"))
|
||||
.count());
|
||||
}
|
||||
});
|
||||
test("Architecture mismatch blocks publication", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.pipeline.env.put("TARGET_PLATFORM", "linux/arm64");
|
||||
failure(f.pipeline::publish);
|
||||
equal(0L, f.calls("push"));
|
||||
}
|
||||
});
|
||||
test("Failed verification removes success marker and preserves exit status", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.mavenFailure = true;
|
||||
try {
|
||||
f.pipeline.verify();
|
||||
throw new AssertionError("Expected failure");
|
||||
} catch (Commands.Failure error) {
|
||||
equal(42, error.exitCode);
|
||||
}
|
||||
equal(
|
||||
List.of("mvn", "-B", "-ntp", "-Dformat.mode=check", "clean", "verify"),
|
||||
f.fake.calls.stream()
|
||||
.filter(a -> a.getFirst().equals("mvn"))
|
||||
.findFirst()
|
||||
.orElseThrow());
|
||||
equal(false, Files.exists(f.root.resolve("ci/artifacts/verified-commit")));
|
||||
equal(0L, f.calls("build"));
|
||||
equal(0L, f.calls("push"));
|
||||
}
|
||||
});
|
||||
test("Frontend lint runs after install and blocks tests and build on failure", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.fake.lintFailure = true;
|
||||
try {
|
||||
f.pipeline.verifyFrontend();
|
||||
throw new AssertionError("Expected failure");
|
||||
} catch (Commands.Failure error) {
|
||||
equal(1, error.exitCode);
|
||||
}
|
||||
equal(List.of(List.of("npm", "ci"), List.of("npm", "run", "lint")), f.fake.calls);
|
||||
}
|
||||
});
|
||||
test("Frontend verification checks before testing and building", () -> {
|
||||
try (Fixture f = new Fixture()) {
|
||||
f.pipeline.verifyFrontend();
|
||||
equal(
|
||||
List.of(
|
||||
"ci",
|
||||
"run lint",
|
||||
"test -- --reporters=default --reporters=junit --output-file=unit-results/junit.xml",
|
||||
"run build"),
|
||||
f.fake.calls.stream()
|
||||
.map(a -> String.join(" ", a.subList(1, a.size())))
|
||||
.toList());
|
||||
}
|
||||
});
|
||||
test("Process execution enforces its deadline", () -> {
|
||||
Path dir = Files.createTempDirectory("ci-process-test-");
|
||||
try {
|
||||
Path file = dir.resolve("Sleep.java");
|
||||
Files.writeString(
|
||||
file,
|
||||
"class Sleep { public static void main(String[] args) throws Exception { Thread.sleep(10000);"
|
||||
+ " } }");
|
||||
Commands runner = new Commands(Duration.ofMillis(100));
|
||||
try {
|
||||
runner.execute(
|
||||
List.of(System.getProperty("java.home") + "/bin/java", file.toString()),
|
||||
dir,
|
||||
System.getenv(),
|
||||
null,
|
||||
false);
|
||||
throw new AssertionError("Expected timeout");
|
||||
} catch (Commands.Failure error) {
|
||||
equal(124, error.exitCode);
|
||||
}
|
||||
} finally {
|
||||
Pipeline.deleteTree(dir);
|
||||
}
|
||||
});
|
||||
System.out.println(count + " CI tests passed");
|
||||
}
|
||||
|
||||
static class Fixture implements AutoCloseable {
|
||||
final Path root = Files.createTempDirectory("ci-publication-test-");
|
||||
final FakeCommands fake = new FakeCommands();
|
||||
final Pipeline pipeline;
|
||||
|
||||
Fixture() throws Exception {
|
||||
Map<String, String> env = new HashMap<>(System.getenv());
|
||||
env.putAll(Map.of(
|
||||
"CI_COMMIT",
|
||||
SHA,
|
||||
"CI_RUN_ID",
|
||||
"test",
|
||||
"CI_RUN_ATTEMPT",
|
||||
"1",
|
||||
"CI_REF",
|
||||
"refs/heads/main",
|
||||
"CI_REPOSITORY",
|
||||
"owner/recipe-demo",
|
||||
"TARGET_PLATFORM",
|
||||
"linux/amd64",
|
||||
"GITEA_SERVER_URL",
|
||||
"http://registry.example:3000",
|
||||
"REGISTRY_USER",
|
||||
"test",
|
||||
"REGISTRY_TOKEN",
|
||||
"dummy-token",
|
||||
"CI_PUBLISH_LOCK_DIR",
|
||||
root.resolve("lock").toString()));
|
||||
env.put("PROJECT_NAME", "recipe-demo");
|
||||
env.put("BACKEND_MODULE", "backend");
|
||||
env.put("FRONTEND_MODULE", "frontend");
|
||||
pipeline = new Pipeline(root, env, fake) {
|
||||
@Override
|
||||
String mainHead() {
|
||||
return fake.head;
|
||||
}
|
||||
};
|
||||
Files.writeString(root.resolve("ci/artifacts/verified-commit"), SHA + "\n");
|
||||
}
|
||||
|
||||
long calls(String verb) {
|
||||
return fake.calls.stream()
|
||||
.filter(a -> a.size() > 1
|
||||
&& a.getFirst().equals("docker")
|
||||
&& a.get(1).equals(verb))
|
||||
.count();
|
||||
}
|
||||
|
||||
public void close() throws Exception {
|
||||
pipeline.closeDockerConfig();
|
||||
Pipeline.deleteTree(root);
|
||||
}
|
||||
}
|
||||
|
||||
static class FakeCommands extends Commands {
|
||||
final List<List<String>> calls = new ArrayList<>();
|
||||
final Map<String, String> images = new HashMap<>(), tags = new HashMap<>();
|
||||
String head = SHA, pushFailure = "", pullFailure = "";
|
||||
boolean manifestError, mavenFailure, lintFailure;
|
||||
boolean dockerAvailable = true;
|
||||
|
||||
FakeCommands() {
|
||||
super(Duration.ofMinutes(1));
|
||||
}
|
||||
|
||||
@Override
|
||||
Result execute(List<String> args, Path cwd, Map<String, String> env, String input, boolean echo) {
|
||||
calls.add(List.copyOf(args));
|
||||
if (args.getFirst().equals("git")) return new Result(0, SHA);
|
||||
if (args.getFirst().equals("mvn") && mavenFailure) return new Result(42, "deliberate failure");
|
||||
if (args.equals(List.of("npm", "run", "lint")) && lintFailure)
|
||||
return new Result(1, "deliberate lint failure");
|
||||
if (!args.getFirst().equals("docker")) return new Result(0, "");
|
||||
String command = args.get(1), ref = args.getLast();
|
||||
if (command.equals("info"))
|
||||
return new Result(dockerAvailable ? 0 : 1, args.contains("{{.Architecture}}") ? "amd64" : "linux");
|
||||
if (command.equals("context")) return new Result(0, "unix:///tmp/test-docker.sock");
|
||||
if (command.equals("manifest")) {
|
||||
if (manifestError) return new Result(1, "unauthorized");
|
||||
return images.containsKey(ref) ? new Result(0, "{}") : new Result(1, "manifest unknown");
|
||||
}
|
||||
if (command.equals("tag")) tags.put(ref, images.getOrDefault(args.get(2), SHA));
|
||||
if (command.equals("push")) {
|
||||
if (!pushFailure.isEmpty() && ref.contains(pushFailure)) return new Result(1, "push failed");
|
||||
images.put(ref, tags.getOrDefault(ref, SHA));
|
||||
}
|
||||
if (command.equals("pull") && !pullFailure.isEmpty() && ref.contains(pullFailure))
|
||||
return new Result(1, "pull failed");
|
||||
if (command.equals("image"))
|
||||
return new Result(
|
||||
0,
|
||||
args.stream().anyMatch(a -> a.contains("RepoDigests"))
|
||||
? ref.substring(0, ref.lastIndexOf(':')) + "@sha256:" + "b".repeat(64)
|
||||
: images.getOrDefault(ref, "missing"));
|
||||
if (command.equals("container") || command.equals("network") && args.contains("inspect"))
|
||||
return new Result(1, "");
|
||||
return new Result(0, "");
|
||||
}
|
||||
}
|
||||
}
|
||||
96
resources/ci/Commands.java
Normal file
96
resources/ci/Commands.java
Normal file
@@ -0,0 +1,96 @@
|
||||
import java.io.*;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.*;
|
||||
import java.time.*;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.*;
|
||||
|
||||
/** Executes argument lists directly, without shell interpolation or credential logging. */
|
||||
class Commands {
|
||||
record Result(int code, String output) {
|
||||
String checked() {
|
||||
if (code != 0) throw new Failure(code, "Command failed with exit status " + code);
|
||||
return output.strip();
|
||||
}
|
||||
}
|
||||
|
||||
static class Failure extends RuntimeException {
|
||||
final int exitCode;
|
||||
|
||||
Failure(int exitCode, String message) {
|
||||
super(message);
|
||||
this.exitCode = exitCode;
|
||||
}
|
||||
}
|
||||
|
||||
Instant deadline;
|
||||
volatile Process active;
|
||||
Path log;
|
||||
|
||||
void renew(Duration timeout) {
|
||||
deadline = Instant.now().plus(timeout);
|
||||
}
|
||||
|
||||
void cancel() {
|
||||
Process process = active;
|
||||
if (process != null) stop(process);
|
||||
}
|
||||
|
||||
Commands(Duration timeout) {
|
||||
deadline = Instant.now().plus(timeout);
|
||||
}
|
||||
|
||||
Result execute(List<String> args, Path cwd, Map<String, String> env, String input, boolean echo) throws Exception {
|
||||
if (!Instant.now().isBefore(deadline)) throw new Failure(124, "CI time limit exceeded");
|
||||
ProcessBuilder builder =
|
||||
new ProcessBuilder(args).directory(cwd.toFile()).redirectErrorStream(true);
|
||||
builder.environment().clear();
|
||||
builder.environment().putAll(env);
|
||||
Process process = builder.start();
|
||||
active = process;
|
||||
ByteArrayOutputStream captured = new ByteArrayOutputStream();
|
||||
Path destination = log;
|
||||
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
|
||||
Future<?> reader = executor.submit(() -> {
|
||||
try (InputStream stream = process.getInputStream();
|
||||
OutputStream file = destination == null
|
||||
? OutputStream.nullOutputStream()
|
||||
: Files.newOutputStream(
|
||||
destination, StandardOpenOption.CREATE, StandardOpenOption.APPEND)) {
|
||||
byte[] buffer = new byte[8192];
|
||||
int count;
|
||||
while ((count = stream.read(buffer)) != -1) {
|
||||
captured.write(buffer, 0, count);
|
||||
if (echo) {
|
||||
System.out.write(buffer, 0, count);
|
||||
System.out.flush();
|
||||
}
|
||||
file.write(buffer, 0, count);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
throw new UncheckedIOException(e);
|
||||
}
|
||||
});
|
||||
try {
|
||||
try (OutputStream stdin = process.getOutputStream()) {
|
||||
if (input != null) stdin.write(input.getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
if (!process.waitFor(
|
||||
Math.max(1, Duration.between(Instant.now(), deadline).toMillis()), TimeUnit.MILLISECONDS)) {
|
||||
stop(process);
|
||||
throw new Failure(124, "CI time limit exceeded");
|
||||
}
|
||||
reader.get(10, TimeUnit.SECONDS);
|
||||
return new Result(process.exitValue(), captured.toString(StandardCharsets.UTF_8));
|
||||
} finally {
|
||||
if (process.isAlive()) stop(process);
|
||||
active = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static void stop(Process process) {
|
||||
process.descendants().forEach(ProcessHandle::destroyForcibly);
|
||||
process.destroyForcibly();
|
||||
}
|
||||
}
|
||||
154
resources/ci/Json.java
Normal file
154
resources/ci/Json.java
Normal file
@@ -0,0 +1,154 @@
|
||||
import java.math.BigDecimal;
|
||||
import java.util.*;
|
||||
|
||||
/** Small dependency-free JSON codec for API/configuration objects, not application data. */
|
||||
final class Json {
|
||||
static Object parse(String source) {
|
||||
Parser parser = new Parser(source);
|
||||
Object result = parser.value();
|
||||
parser.space();
|
||||
if (parser.pos != source.length()) throw parser.invalid();
|
||||
return result;
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
static Map<String, Object> object(Object value) {
|
||||
if (!(value instanceof Map<?, ?>)) throw new IllegalArgumentException("Expected a JSON object");
|
||||
return (Map<String, Object>) value;
|
||||
}
|
||||
|
||||
static String write(Object value) {
|
||||
if (value == null) return "null";
|
||||
if (value instanceof String text) {
|
||||
StringBuilder out = new StringBuilder("\"");
|
||||
for (char c : text.toCharArray()) {
|
||||
switch (c) {
|
||||
case '"' -> out.append("\\\"");
|
||||
case '\\' -> out.append("\\\\");
|
||||
case '\n' -> out.append("\\n");
|
||||
case '\r' -> out.append("\\r");
|
||||
case '\t' -> out.append("\\t");
|
||||
default -> {
|
||||
if (c < 32) out.append(String.format("\\u%04x", (int) c));
|
||||
else out.append(c);
|
||||
}
|
||||
}
|
||||
}
|
||||
return out.append('"').toString();
|
||||
}
|
||||
if (value instanceof Boolean
|
||||
|| value instanceof BigDecimal
|
||||
|| value instanceof Integer
|
||||
|| value instanceof Long) return value.toString();
|
||||
if (value instanceof Map<?, ?> map) {
|
||||
StringJoiner out = new StringJoiner(",", "{", "}");
|
||||
map.forEach((k, v) -> out.add(write(Objects.toString(k)) + ":" + write(v)));
|
||||
return out.toString();
|
||||
}
|
||||
if (value instanceof List<?> list) {
|
||||
StringJoiner out = new StringJoiner(",", "[", "]");
|
||||
list.forEach(v -> out.add(write(v)));
|
||||
return out.toString();
|
||||
}
|
||||
throw new IllegalArgumentException("Unsupported JSON value");
|
||||
}
|
||||
|
||||
private static final class Parser {
|
||||
final String source;
|
||||
int pos;
|
||||
|
||||
Parser(String source) {
|
||||
this.source = source;
|
||||
}
|
||||
|
||||
IllegalArgumentException invalid() {
|
||||
return new IllegalArgumentException("Invalid JSON at offset " + pos);
|
||||
}
|
||||
|
||||
void space() {
|
||||
while (pos < source.length() && " \r\n\t".indexOf(source.charAt(pos)) >= 0) pos++;
|
||||
}
|
||||
|
||||
boolean take(char c) {
|
||||
space();
|
||||
if (pos < source.length() && source.charAt(pos) == c) {
|
||||
pos++;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
Object value() {
|
||||
space();
|
||||
if (pos >= source.length()) throw invalid();
|
||||
char c = source.charAt(pos);
|
||||
if (c == '"') return string();
|
||||
if (take('{')) {
|
||||
Map<String, Object> out = new LinkedHashMap<>();
|
||||
if (take('}')) return out;
|
||||
do {
|
||||
space();
|
||||
if (pos >= source.length() || source.charAt(pos) != '"') throw invalid();
|
||||
String key = string();
|
||||
if (!take(':') || out.containsKey(key)) throw invalid();
|
||||
out.put(key, value());
|
||||
} while (take(','));
|
||||
if (!take('}')) throw invalid();
|
||||
return out;
|
||||
}
|
||||
if (take('[')) {
|
||||
List<Object> out = new ArrayList<>();
|
||||
if (take(']')) return out;
|
||||
do {
|
||||
out.add(value());
|
||||
} while (take(','));
|
||||
if (!take(']')) throw invalid();
|
||||
return out;
|
||||
}
|
||||
for (String token : List.of("true", "false", "null")) {
|
||||
if (source.startsWith(token, pos)) {
|
||||
pos += token.length();
|
||||
return token.equals("null") ? null : Boolean.valueOf(token);
|
||||
}
|
||||
}
|
||||
int start = pos;
|
||||
while (pos < source.length() && "-+0123456789.eE".indexOf(source.charAt(pos)) >= 0) pos++;
|
||||
String number = source.substring(start, pos);
|
||||
if (!number.matches("-?(0|[1-9][0-9]*)(\\.[0-9]+)?([eE][+-]?[0-9]+)?")) throw invalid();
|
||||
return new BigDecimal(number);
|
||||
}
|
||||
|
||||
String string() {
|
||||
pos++;
|
||||
StringBuilder out = new StringBuilder();
|
||||
while (pos < source.length()) {
|
||||
char c = source.charAt(pos++);
|
||||
if (c == '"') return out.toString();
|
||||
if (c < 32) throw invalid();
|
||||
if (c != '\\') {
|
||||
out.append(c);
|
||||
continue;
|
||||
}
|
||||
if (pos >= source.length()) throw invalid();
|
||||
char escaped = source.charAt(pos++);
|
||||
switch (escaped) {
|
||||
case '"', '\\', '/' -> out.append(escaped);
|
||||
case 'b' -> out.append('\b');
|
||||
case 'f' -> out.append('\f');
|
||||
case 'n' -> out.append('\n');
|
||||
case 'r' -> out.append('\r');
|
||||
case 't' -> out.append('\t');
|
||||
case 'u' -> {
|
||||
if (pos + 4 > source.length()) throw invalid();
|
||||
String digits = source.substring(pos, pos + 4);
|
||||
if (!digits.matches("[0-9a-fA-F]{4}")) throw invalid();
|
||||
out.append((char) Integer.parseInt(digits, 16));
|
||||
pos += 4;
|
||||
}
|
||||
default -> throw invalid();
|
||||
}
|
||||
}
|
||||
throw invalid();
|
||||
}
|
||||
}
|
||||
}
|
||||
513
resources/ci/Pipeline.java
Normal file
513
resources/ci/Pipeline.java
Normal file
@@ -0,0 +1,513 @@
|
||||
import java.io.*;
|
||||
import java.nio.channels.*;
|
||||
import java.nio.file.*;
|
||||
import java.time.*;
|
||||
import java.util.*;
|
||||
|
||||
/** Verification, publication and cleanup shared by local use and Gitea Actions. */
|
||||
class Pipeline {
|
||||
final Path root, artifacts;
|
||||
final Map<String, String> env;
|
||||
final Commands commands;
|
||||
final String commit, runName, backendImage, frontendImage;
|
||||
Path isolatedConfig;
|
||||
|
||||
Pipeline(Path root, Map<String, String> environment, Commands commands) throws Exception {
|
||||
this.root = root;
|
||||
this.env = Recipe.environment(root, environment);
|
||||
this.commands = commands;
|
||||
commit = value("CI_COMMIT", () -> capture("git", "rev-parse", "HEAD"));
|
||||
String id = value("CI_RUN_ID", () -> "local"), attempt = value("CI_RUN_ATTEMPT", () -> "1");
|
||||
check(commit.matches("[0-9a-f]{40}"), "Invalid commit SHA");
|
||||
check((id + "-" + attempt).matches("[a-zA-Z0-9-]+"), "Invalid run identifier");
|
||||
runName = Recipe.project(env) + "-" + id + "-" + attempt;
|
||||
backendImage = Recipe.project(env) + "-ci-backend:" + commit + "-" + id + "-" + attempt;
|
||||
frontendImage = Recipe.project(env) + "-ci-frontend:" + commit + "-" + id + "-" + attempt;
|
||||
artifacts = root.resolve("ci/artifacts");
|
||||
Files.createDirectories(artifacts);
|
||||
}
|
||||
|
||||
interface Value {
|
||||
String get() throws Exception;
|
||||
}
|
||||
|
||||
String value(String key, Value defaultValue) throws Exception {
|
||||
if (env.getOrDefault(key, "").isBlank()) env.put(key, defaultValue.get());
|
||||
return env.get(key);
|
||||
}
|
||||
|
||||
static void check(boolean condition, String message) {
|
||||
if (!condition) throw new IllegalArgumentException(message);
|
||||
}
|
||||
|
||||
Commands.Result result(boolean echo, String... args) throws Exception {
|
||||
return commands.execute(List.of(args), root, env, null, echo);
|
||||
}
|
||||
|
||||
String capture(String... args) throws Exception {
|
||||
return result(false, args).checked();
|
||||
}
|
||||
|
||||
void run(String... args) throws Exception {
|
||||
result(true, args).checked();
|
||||
}
|
||||
|
||||
void requirePlatform() throws Exception {
|
||||
String target = Setup.required(env, "TARGET_PLATFORM");
|
||||
var docker = result(false, "docker", "info", "--format", "{{.Architecture}}");
|
||||
check(
|
||||
docker.code() == 0,
|
||||
"Docker is unavailable; start Docker Desktop before running CI (details in verify.log)");
|
||||
String arch = docker.output().strip();
|
||||
arch = switch (arch) {
|
||||
case "x86_64" -> "amd64";
|
||||
case "aarch64" -> "arm64";
|
||||
default -> arch;
|
||||
};
|
||||
String nativePlatform = capture("docker", "info", "--format", "{{.OSType}}") + "/" + arch;
|
||||
check(target.equals(nativePlatform), "Target " + target + " differs from Docker host " + nativePlatform);
|
||||
}
|
||||
|
||||
void isolateDocker() throws Exception {
|
||||
value("DOCKER_HOST", () -> capture("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}"));
|
||||
env.remove("DOCKER_CONTEXT");
|
||||
isolatedConfig = Files.createTempDirectory(Recipe.project(env) + "-docker-");
|
||||
env.put("DOCKER_CONFIG", isolatedConfig.toString());
|
||||
}
|
||||
|
||||
void verify() throws Exception {
|
||||
Files.deleteIfExists(artifacts.resolve("verified-commit"));
|
||||
env.put("CI", "true");
|
||||
commands.log = artifacts.resolve("verify.log");
|
||||
Files.writeString(commands.log, "");
|
||||
Thread shutdown = new Thread(
|
||||
() -> {
|
||||
commands.cancel();
|
||||
try {
|
||||
cleanup();
|
||||
closeDockerConfig();
|
||||
} catch (Exception e) {
|
||||
System.err.println("CI shutdown cleanup failed");
|
||||
}
|
||||
},
|
||||
"ci-cleanup");
|
||||
Runtime.getRuntime().addShutdownHook(shutdown);
|
||||
Exception failure = null;
|
||||
try {
|
||||
requirePlatform();
|
||||
isolateDocker();
|
||||
check(commit.equals(capture("git", "rev-parse", "HEAD")), "Checkout does not match pushed commit");
|
||||
run("java", "-version");
|
||||
if (Recipe.roles(env).contains("frontend")) run("node", "--version");
|
||||
if (Recipe.roles(env).contains("backend"))
|
||||
run("mvn", "-B", "-ntp", "-Dformat.mode=check", "clean", "verify");
|
||||
CiTests.run();
|
||||
ReleaseTests.run();
|
||||
Path frontend = root.resolve(env.getOrDefault("FRONTEND_MODULE", "."));
|
||||
if (Recipe.roles(env).contains("frontend")) verifyFrontend();
|
||||
String platform = env.get("TARGET_PLATFORM");
|
||||
for (String role : Recipe.roles(env)) {
|
||||
run(
|
||||
"docker",
|
||||
"build",
|
||||
"--platform",
|
||||
platform,
|
||||
"--target",
|
||||
"prebuilt",
|
||||
"--label",
|
||||
"org.opencontainers.image.revision=" + commit,
|
||||
"-t",
|
||||
role.equals("backend") ? backendImage : frontendImage,
|
||||
env.getOrDefault(
|
||||
role.toUpperCase() + "_CONTEXT",
|
||||
role.equals("backend") ? "." : env.get("FRONTEND_MODULE")));
|
||||
}
|
||||
if (Recipe.enabled(env, "BROWSER_TESTS")) {
|
||||
Files.writeString(artifacts.resolve("containers-started"), "");
|
||||
run("docker", "network", "create", "--label", Recipe.project(env) + ".ci.run=" + runName, runName);
|
||||
if (Recipe.enabled(env, "CI_DATABASE")) {
|
||||
run(
|
||||
"docker",
|
||||
"run",
|
||||
"-d",
|
||||
"--name",
|
||||
runName + "-database",
|
||||
"--network",
|
||||
runName,
|
||||
"--network-alias",
|
||||
"database",
|
||||
"--env",
|
||||
"POSTGRES_USER=" + env.get("DATABASE_USER"),
|
||||
"--env",
|
||||
"POSTGRES_PASSWORD=ci-only",
|
||||
"--env",
|
||||
"POSTGRES_DB=" + env.get("DATABASE_NAME"),
|
||||
"postgres:17.6");
|
||||
for (int attempt = 0; attempt < 60; attempt++) {
|
||||
if (result(
|
||||
false,
|
||||
"docker",
|
||||
"exec",
|
||||
runName + "-database",
|
||||
"pg_isready",
|
||||
"-U",
|
||||
env.get("DATABASE_USER"))
|
||||
.code()
|
||||
== 0) break;
|
||||
Thread.sleep(1000);
|
||||
}
|
||||
}
|
||||
run(
|
||||
"docker",
|
||||
"run",
|
||||
"-d",
|
||||
"--name",
|
||||
runName + "-backend",
|
||||
"--label",
|
||||
Recipe.project(env) + ".ci.run=" + runName,
|
||||
"--platform",
|
||||
platform,
|
||||
"--network",
|
||||
runName,
|
||||
"--network-alias",
|
||||
"backend",
|
||||
"--memory",
|
||||
"1g",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--volume",
|
||||
root.resolve(env.get("BACKEND_MODULE") + "/target/coverage-tools/agent.jar")
|
||||
+ ":/jacoco-agent.jar:ro",
|
||||
"--env",
|
||||
"JAVA_TOOL_OPTIONS=-javaagent:/jacoco-agent.jar=output=tcpserver,address=*,port=6300",
|
||||
"--env",
|
||||
"DATABASE_URL=jdbc:postgresql://database:5432/" + env.get("DATABASE_NAME"),
|
||||
"--env",
|
||||
"DATABASE_USER=" + env.get("DATABASE_USER"),
|
||||
"--env",
|
||||
"DATABASE_PASSWORD=ci-only",
|
||||
backendImage);
|
||||
run(
|
||||
"docker",
|
||||
"run",
|
||||
"-d",
|
||||
"--name",
|
||||
runName + "-frontend",
|
||||
"--label",
|
||||
Recipe.project(env) + ".ci.run=" + runName,
|
||||
"--platform",
|
||||
platform,
|
||||
"--network",
|
||||
runName,
|
||||
"--network-alias",
|
||||
"frontend",
|
||||
"--memory",
|
||||
"256m",
|
||||
"--cpus",
|
||||
"1",
|
||||
frontendImage);
|
||||
waitForApplication();
|
||||
String user = capture("id", "-u") + ":" + capture("id", "-g");
|
||||
run(
|
||||
"docker",
|
||||
"run",
|
||||
"--name",
|
||||
runName + "-browser",
|
||||
"--label",
|
||||
Recipe.project(env) + ".ci.run=" + runName,
|
||||
"--platform",
|
||||
platform,
|
||||
"--network",
|
||||
runName,
|
||||
"--memory",
|
||||
"2g",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--shm-size=1g",
|
||||
"--user",
|
||||
user,
|
||||
"--env",
|
||||
"HOME=/tmp",
|
||||
"--env",
|
||||
"CI=true",
|
||||
"--env",
|
||||
"PLAYWRIGHT_BASE_URL=http://frontend:8080",
|
||||
"--volume",
|
||||
frontend + ":/workspace",
|
||||
"--workdir",
|
||||
"/workspace",
|
||||
env.getOrDefault("PLAYWRIGHT_IMAGE", "mcr.microsoft.com/playwright:v1.64.1-noble"),
|
||||
"npm",
|
||||
"run",
|
||||
"e2e");
|
||||
run(
|
||||
"docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--network",
|
||||
runName,
|
||||
"--user",
|
||||
user,
|
||||
"--volume",
|
||||
root.resolve(env.get("BACKEND_MODULE") + "/target") + ":/coverage",
|
||||
"eclipse-temurin:25-jre",
|
||||
"java",
|
||||
"-jar",
|
||||
"/coverage/coverage-tools/cli.jar",
|
||||
"dump",
|
||||
"--address",
|
||||
"backend",
|
||||
"--destfile",
|
||||
"/coverage/playwright.exec");
|
||||
check(
|
||||
Files.size(root.resolve(env.get("BACKEND_MODULE") + "/target/playwright.exec")) > 0,
|
||||
"Playwright backend coverage is missing");
|
||||
run(
|
||||
"mvn",
|
||||
"-B",
|
||||
"-ntp",
|
||||
"-pl",
|
||||
env.get("BACKEND_MODULE"),
|
||||
"jacoco:merge@combined",
|
||||
"jacoco:report@combined-report",
|
||||
"exec:exec@combined-coverage");
|
||||
}
|
||||
Files.writeString(artifacts.resolve("verified-commit"), commit + "\n");
|
||||
} catch (Exception e) {
|
||||
failure = e;
|
||||
throw e;
|
||||
} finally {
|
||||
try {
|
||||
cleanup();
|
||||
} catch (Exception e) {
|
||||
if (failure != null) failure.addSuppressed(e);
|
||||
else throw e;
|
||||
} finally {
|
||||
closeDockerConfig();
|
||||
try {
|
||||
Runtime.getRuntime().removeShutdownHook(shutdown);
|
||||
} catch (IllegalStateException ignored) {
|
||||
/* shutting down */
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void verifyFrontend() throws Exception {
|
||||
Path frontend = root.resolve(env.get("FRONTEND_MODULE"));
|
||||
commands.execute(List.of("npm", "ci"), frontend, env, null, true).checked();
|
||||
commands.execute(List.of("npm", "run", "lint"), frontend, env, null, true)
|
||||
.checked();
|
||||
final Object configured = Json.parse(
|
||||
env.getOrDefault(
|
||||
"FRONTEND_TEST_COMMAND",
|
||||
"[\"npm\",\"test\",\"--\",\"--reporters=default\",\"--reporters=junit\",\"--output-file=unit-results/junit.xml\"]"));
|
||||
final List<String> test =
|
||||
((List<?>) configured).stream().map(Object::toString).toList();
|
||||
commands.execute(test, frontend, env, null, true).checked();
|
||||
commands.execute(List.of("npm", "run", "build"), frontend, env, null, true)
|
||||
.checked();
|
||||
}
|
||||
|
||||
void waitForApplication() throws Exception {
|
||||
for (int i = 0; i < 60; i++) {
|
||||
if (result(
|
||||
false,
|
||||
"docker",
|
||||
"exec",
|
||||
runName + "-frontend",
|
||||
"wget",
|
||||
"-q",
|
||||
"-O",
|
||||
"-",
|
||||
Setup.required(env, "CI_READINESS_URL"))
|
||||
.code()
|
||||
== 0) return;
|
||||
Thread.sleep(2000);
|
||||
}
|
||||
throw new IllegalArgumentException("Application readiness timed out");
|
||||
}
|
||||
|
||||
synchronized void cleanup() throws Exception {
|
||||
// A separate cleanup budget remains available after the build deadline.
|
||||
commands.renew(Duration.ofMinutes(2));
|
||||
List<String> failures = new ArrayList<>();
|
||||
for (String path : List.of(
|
||||
env.get("BACKEND_MODULE") + "/target/surefire-reports",
|
||||
env.get("BACKEND_MODULE") + "/target/standards",
|
||||
"generated/tooling/standards/target/surefire-reports",
|
||||
env.get("BACKEND_MODULE") + "/target/site/jacoco",
|
||||
env.get("BACKEND_MODULE") + "/target/site/jacoco-combined",
|
||||
env.get("BACKEND_MODULE") + "/target/failsafe-reports",
|
||||
env.get("FRONTEND_MODULE") + "/unit-results",
|
||||
env.get("FRONTEND_MODULE") + "/test-results",
|
||||
env.get("FRONTEND_MODULE") + "/playwright-report")) {
|
||||
if (Files.isDirectory(root.resolve(path))) {
|
||||
try {
|
||||
copyTree(root.resolve(path), artifacts.resolve(path));
|
||||
} catch (IOException e) {
|
||||
failures.add(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (String data : List.of("jacoco.exec", "playwright.exec", "jacoco-combined.exec")) {
|
||||
final Path source =
|
||||
root.resolve(env.get("BACKEND_MODULE") + "/target").resolve(data);
|
||||
if (Files.isRegularFile(source))
|
||||
Files.copy(source, artifacts.resolve(data), StandardCopyOption.REPLACE_EXISTING);
|
||||
}
|
||||
if (result(false, "docker", "info").code() == 0) {
|
||||
for (String role : List.of("backend", "frontend", "browser", "database")) {
|
||||
String container = runName + "-" + role;
|
||||
if (result(false, "docker", "container", "inspect", container).code() == 0) {
|
||||
var logs = result(false, "docker", "logs", container);
|
||||
Files.writeString(artifacts.resolve(role + ".log"), logs.output());
|
||||
if (logs.code() != 0) failures.add(role + " logs");
|
||||
if (result(false, "docker", "rm", "-f", "-v", container).code() != 0)
|
||||
failures.add(role + " cleanup");
|
||||
}
|
||||
}
|
||||
if (result(false, "docker", "network", "inspect", runName).code() == 0
|
||||
&& result(false, "docker", "network", "rm", runName).code() != 0) failures.add("network cleanup");
|
||||
if (failures.isEmpty()) Files.deleteIfExists(artifacts.resolve("containers-started"));
|
||||
} else if (Files.exists(artifacts.resolve("containers-started")))
|
||||
failures.add("Docker unavailable during cleanup");
|
||||
check(failures.isEmpty(), "Cleanup incomplete: " + String.join(", ", failures));
|
||||
}
|
||||
|
||||
void publish() throws Exception {
|
||||
if (!"refs/heads/main".equals(env.get("CI_REF"))) {
|
||||
System.out.println("Branch is not main; no publication");
|
||||
return;
|
||||
}
|
||||
check(
|
||||
Files.exists(artifacts.resolve("verified-commit"))
|
||||
&& Files.readString(artifacts.resolve("verified-commit"))
|
||||
.strip()
|
||||
.equals(commit),
|
||||
"This commit has not passed verification");
|
||||
String user = Setup.required(env, "REGISTRY_USER"), token = Setup.required(env, "REGISTRY_TOKEN");
|
||||
String server = Setup.required(env, "GITEA_SERVER_URL");
|
||||
String repository = Setup.required(env, "CI_REPOSITORY");
|
||||
requirePlatform();
|
||||
commands.log = artifacts.resolve("publish.log");
|
||||
Files.writeString(commands.log, "");
|
||||
var uri = java.net.URI.create(server);
|
||||
check(
|
||||
Set.of("http", "https").contains(uri.getScheme())
|
||||
&& (uri.getPath().isEmpty() || uri.getPath().equals("/")),
|
||||
"Registry must be hosted at the Gitea server root");
|
||||
String registry = uri.getRawAuthority();
|
||||
String owner = repository.split("/")[0].toLowerCase(Locale.ROOT);
|
||||
check(owner.matches("[a-z0-9][a-z0-9_.-]*"), "Invalid registry namespace");
|
||||
Path lockDir = Path.of(env.getOrDefault(
|
||||
"CI_PUBLISH_LOCK_DIR",
|
||||
env.getOrDefault(
|
||||
"XDG_STATE_HOME",
|
||||
Setup.home().resolve(".local/state").toString()) + "/" + Recipe.project(env) + "-ci"));
|
||||
Files.createDirectories(lockDir);
|
||||
// Shared host lock: Gitea 1.25 ignores workflow concurrency.
|
||||
try (FileChannel channel = FileChannel.open(
|
||||
lockDir.resolve("publish.lock"), StandardOpenOption.CREATE, StandardOpenOption.WRITE);
|
||||
FileLock lock = awaitLock(channel)) {
|
||||
isolateDocker();
|
||||
try {
|
||||
commands.execute(
|
||||
List.of("docker", "login", registry, "--username", user, "--password-stdin"),
|
||||
root,
|
||||
env,
|
||||
token,
|
||||
true)
|
||||
.checked();
|
||||
String prefix = registry + "/" + owner + "/" + Recipe.project(env) + "-";
|
||||
for (String role : Recipe.roles(env)) {
|
||||
ensureImage(prefix + role, role.equals("backend") ? backendImage : frontendImage);
|
||||
}
|
||||
for (String role : Recipe.roles(env)) {
|
||||
if (!commit.equals(mainHead())) {
|
||||
System.out.println("Newer main exists; moving tags unchanged");
|
||||
return;
|
||||
}
|
||||
String image = prefix + role;
|
||||
run("docker", "tag", image + ":" + commit, image + ":main");
|
||||
run("docker", "push", image + ":main");
|
||||
run("docker", "pull", "--platform", env.get("TARGET_PLATFORM"), image + ":main");
|
||||
}
|
||||
} finally {
|
||||
closeDockerConfig();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
FileLock awaitLock(FileChannel channel) throws Exception {
|
||||
Instant end = Instant.now().plusSeconds(600);
|
||||
while (Instant.now().isBefore(end)) {
|
||||
FileLock lock = channel.tryLock();
|
||||
if (lock != null) return lock;
|
||||
Thread.sleep(250);
|
||||
}
|
||||
throw new IllegalArgumentException("Publication lock timed out");
|
||||
}
|
||||
|
||||
String mainHead() throws Exception {
|
||||
Map<String, Object> branch = Json.object(Setup.api(
|
||||
env.get("GITEA_SERVER_URL"),
|
||||
env.get("REGISTRY_TOKEN"),
|
||||
"GET",
|
||||
"/api/v1/repos/" + env.get("CI_REPOSITORY") + "/branches/main",
|
||||
null));
|
||||
return Objects.toString(Json.object(branch.get("commit")).get("id"));
|
||||
}
|
||||
|
||||
void ensureImage(String repository, String source) throws Exception {
|
||||
String ref = repository + ":" + commit;
|
||||
var manifest = result(false, "docker", "manifest", "inspect", "--insecure", ref);
|
||||
if (manifest.code() == 0) System.out.println("Preserving existing " + ref);
|
||||
else if (manifest.output().toLowerCase(Locale.ROOT).matches("(?s).*(manifest unknown|no such manifest).*")) {
|
||||
run("docker", "tag", source, ref);
|
||||
run("docker", "push", ref);
|
||||
} else
|
||||
throw new IllegalArgumentException(
|
||||
"Registry manifest lookup failed; authentication/network errors are not missing tags");
|
||||
run("docker", "pull", "--platform", env.get("TARGET_PLATFORM"), ref);
|
||||
check(
|
||||
commit.equals(capture(
|
||||
"docker",
|
||||
"image",
|
||||
"inspect",
|
||||
"--format",
|
||||
"{{index .Config.Labels \"org.opencontainers.image.revision\"}}",
|
||||
ref)),
|
||||
"Existing image has an unexpected source revision");
|
||||
String digests =
|
||||
capture("docker", "image", "inspect", "--format", "{{range .RepoDigests}}{{println .}}{{end}}", ref);
|
||||
check(digests.contains(repository + "@sha256:"), "Missing registry digest");
|
||||
Files.writeString(
|
||||
artifacts.resolve(repository.endsWith("backend") ? "backend-digests.txt" : "frontend-digests.txt"),
|
||||
digests + "\n");
|
||||
}
|
||||
|
||||
synchronized void closeDockerConfig() throws IOException {
|
||||
if (isolatedConfig != null) {
|
||||
deleteTree(isolatedConfig);
|
||||
isolatedConfig = null;
|
||||
}
|
||||
}
|
||||
|
||||
static void copyTree(Path source, Path destination) throws IOException {
|
||||
try (var paths = Files.walk(source)) {
|
||||
for (Path path : paths.toList()) {
|
||||
Path target = destination.resolve(source.relativize(path));
|
||||
if (Files.isDirectory(path)) Files.createDirectories(target);
|
||||
else Files.copy(path, target, StandardCopyOption.REPLACE_EXISTING);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void deleteTree(Path root) throws IOException {
|
||||
if (!Files.exists(root)) return;
|
||||
try (var paths = Files.walk(root)) {
|
||||
for (Path path : paths.sorted(Comparator.reverseOrder()).toList()) Files.delete(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
61
resources/ci/Recipe.java
Normal file
61
resources/ci/Recipe.java
Normal file
@@ -0,0 +1,61 @@
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Properties;
|
||||
|
||||
/** Project-owned configuration for the shared engineering mechanisms. */
|
||||
final class Recipe {
|
||||
private Recipe() {}
|
||||
|
||||
static Map<String, String> environment(final Path root, final Map<String, String> input) throws Exception {
|
||||
final var result = new HashMap<String, String>(input);
|
||||
final Path file = root.resolve("engineering.properties");
|
||||
if (Files.exists(file)) {
|
||||
final var properties = new Properties();
|
||||
try (final var stream = Files.newInputStream(file)) {
|
||||
properties.load(stream);
|
||||
}
|
||||
for (final String key : properties.stringPropertyNames()) {
|
||||
result.putIfAbsent(
|
||||
key, properties.getProperty(key).replace("${user.home}", System.getProperty("user.home")));
|
||||
}
|
||||
}
|
||||
if (result.containsKey("PROJECT_NAME")) {
|
||||
project(result);
|
||||
for (final String role : roles(result)) {
|
||||
final Path module = Path.of(result.get(role.toUpperCase() + "_MODULE"));
|
||||
Pipeline.check(
|
||||
!module.isAbsolute() && !module.normalize().startsWith(".."),
|
||||
"Module must stay in the project");
|
||||
}
|
||||
Pipeline.check(
|
||||
!enabled(result, "BROWSER_TESTS") || roles(result).size() == 2,
|
||||
"Browser coverage requires both backend and frontend");
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
static String project(final Map<String, String> env) {
|
||||
final String name = Setup.required(env, "PROJECT_NAME");
|
||||
Pipeline.check(name.matches("[a-z][a-z0-9-]*"), "Invalid project name");
|
||||
return name;
|
||||
}
|
||||
|
||||
static List<String> roles(final Map<String, String> env) {
|
||||
final var roles = new ArrayList<String>();
|
||||
for (final String role : List.of("backend", "frontend")) {
|
||||
if (!env.getOrDefault(role.toUpperCase() + "_MODULE", "").isBlank()) {
|
||||
roles.add(role);
|
||||
}
|
||||
}
|
||||
Pipeline.check(!roles.isEmpty(), "Configure BACKEND_MODULE, FRONTEND_MODULE, or both");
|
||||
return roles;
|
||||
}
|
||||
|
||||
static boolean enabled(final Map<String, String> env, final String key) {
|
||||
return Boolean.parseBoolean(env.getOrDefault(key, "false"));
|
||||
}
|
||||
}
|
||||
236
resources/ci/Release.java
Normal file
236
resources/ci/Release.java
Normal file
@@ -0,0 +1,236 @@
|
||||
import java.net.URI;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
|
||||
/** Manual release coordinator. All mutation decisions are made under the NAS lock. */
|
||||
class Release {
|
||||
final Path root, artifacts;
|
||||
final Map<String, String> env;
|
||||
final ReleaseApi api;
|
||||
final ReleaseRemote remote;
|
||||
final String mode, increment, retry, pinned, attempt;
|
||||
ReleaseState.Deployment selected;
|
||||
boolean locked, pending;
|
||||
|
||||
static void execute(Path root, Map<String, String> env, String operation) throws Exception {
|
||||
validate(env, operation);
|
||||
String attempt = "release-" + env.get("CI_RUN_ID") + "-" + UUID.randomUUID();
|
||||
Path artifacts = root.resolve("ci/artifacts/deployment");
|
||||
Files.createDirectories(artifacts);
|
||||
var commands = new Commands(Duration.ofMinutes(15));
|
||||
commands.log = artifacts.resolve(attempt + ".log");
|
||||
Files.writeString(commands.log, "");
|
||||
try (var remote = new ReleaseRemote(root, env, commands, attempt)) {
|
||||
new Release(root, env, new ReleaseApi(env), remote, operation, attempt).run();
|
||||
}
|
||||
}
|
||||
|
||||
static void validate(Map<String, String> env, String operation) {
|
||||
Pipeline.check("refs/heads/main".equals(env.get("CI_REF")), "Production workflows must run from main");
|
||||
Pipeline.check(
|
||||
"workflow_dispatch".equals(env.get("CI_EVENT")), "Production requires a manual workflow dispatch");
|
||||
Pipeline.check(Setup.required(env, "CI_RUN_ID").matches("[0-9]+"), "Invalid Gitea run ID");
|
||||
ReleaseApi.sha(env.get("CI_COMMIT"));
|
||||
Pipeline.check(List.of("deploy", "rollback").contains(operation), "Unknown release operation");
|
||||
if (operation.equals("rollback")) return;
|
||||
String mode = Setup.required(env, "RELEASE_MODE");
|
||||
String increment = env.getOrDefault("RELEASE_INCREMENT", "");
|
||||
String retry = env.getOrDefault("RELEASE_VERSION", "");
|
||||
if (mode.equals("new")) {
|
||||
Pipeline.check(List.of("patch", "minor", "major").contains(increment), "Choose patch, minor or major");
|
||||
Pipeline.check(retry.isBlank(), "New release cannot specify an existing version");
|
||||
} else {
|
||||
Pipeline.check(mode.equals("retry"), "Choose new or retry release mode");
|
||||
ReleaseVersion.parse(retry);
|
||||
// Gitea shows all inputs; the new-release increment is ignored for retries.
|
||||
}
|
||||
}
|
||||
|
||||
Release(Path root, Map<String, String> env, ReleaseApi api, ReleaseRemote remote, String operation, String attempt)
|
||||
throws Exception {
|
||||
this.root = root;
|
||||
this.env = env;
|
||||
this.api = api;
|
||||
this.remote = remote;
|
||||
this.attempt = attempt;
|
||||
artifacts = root.resolve("ci/artifacts/deployment");
|
||||
Files.createDirectories(artifacts);
|
||||
mode = operation.equals("rollback") ? "rollback" : env.get("RELEASE_MODE");
|
||||
increment = env.getOrDefault("RELEASE_INCREMENT", "");
|
||||
retry = env.getOrDefault("RELEASE_VERSION", "");
|
||||
pinned = ReleaseApi.sha(env.get("CI_COMMIT"));
|
||||
}
|
||||
|
||||
void run() throws Exception {
|
||||
String outcome = "rejected";
|
||||
try {
|
||||
remote.acquire();
|
||||
locked = true;
|
||||
remote.call("preflight", null);
|
||||
ReleaseState before = ReleaseState.parse(remote.call("state", null));
|
||||
selected = select(before);
|
||||
System.out.println("Selected " + selected.version() + " at " + selected.sha());
|
||||
// Validate the recovery pair before any replacement too.
|
||||
if (before.current() != null) ensure(before.current());
|
||||
if (mode.equals("new")) {
|
||||
api.create(selected.version(), selected.sha(), attempt);
|
||||
Pipeline.check(
|
||||
selected.sha().equals(api.head()),
|
||||
"Main advanced; release tagged but production unchanged. Retry explicitly.");
|
||||
}
|
||||
var journal = new LinkedHashMap<String, Object>();
|
||||
journal.put("attempt", attempt);
|
||||
journal.put("mode", mode);
|
||||
journal.put("candidate", selected.json());
|
||||
journal.put("before", Json.parse(before.current() == null ? "{}" : before.json()));
|
||||
remote.call("pending", Json.write(journal));
|
||||
pending = true;
|
||||
outcome = "replacement-started";
|
||||
try {
|
||||
remote.call("apply", null, selected.directory());
|
||||
} catch (Exception failure) {
|
||||
diagnose(selected);
|
||||
outcome = recover(before);
|
||||
throw new IllegalArgumentException(
|
||||
"Deployment failed: " + outcome + ". See deployment diagnostics and recovery runbook.");
|
||||
}
|
||||
// A lost reply here is ambiguous: retain journal and lock, never guess
|
||||
// whether a state commit happened and launch an overlapping recovery.
|
||||
outcome = "healthy-state-commit-pending";
|
||||
remote.call("commit", before.success(selected).json());
|
||||
outcome = "success";
|
||||
record(outcome);
|
||||
remote.call("finish", null);
|
||||
pending = false;
|
||||
System.out.println("Production " + selected.version() + " is healthy (" + selected.sha() + ")");
|
||||
} finally {
|
||||
try {
|
||||
record(outcome);
|
||||
} finally {
|
||||
if (locked && !pending) remote.release();
|
||||
else if (locked)
|
||||
System.err.println("NAS lock and pending journal retained. Follow docs/production-deployment.md for"
|
||||
+ " recovery.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ReleaseState.Deployment select(ReleaseState before) throws Exception {
|
||||
if (mode.equals("rollback")) {
|
||||
Pipeline.check(before.previous() != null, "No previous successful production release is available");
|
||||
ensure(before.previous());
|
||||
return before.previous();
|
||||
}
|
||||
List<ReleaseApi.Tag> tags = api.tags();
|
||||
ReleaseVersion version;
|
||||
String commit;
|
||||
if (mode.equals("new")) {
|
||||
Pipeline.check(
|
||||
pinned.equals(api.head()), "Main advanced since this dispatch; start a new release workflow");
|
||||
Pipeline.check(
|
||||
tags.stream().noneMatch(t -> t.sha().equals(pinned)),
|
||||
"This commit already has a release tag; use retry or rollback");
|
||||
version = ReleaseVersion.next(
|
||||
tags.stream().map(ReleaseApi.Tag::version).toList(), increment);
|
||||
commit = pinned;
|
||||
} else {
|
||||
version = ReleaseVersion.parse(retry);
|
||||
var tag = tags.stream()
|
||||
.filter(t -> t.version().equals(version))
|
||||
.findFirst()
|
||||
.orElseThrow(() -> new IllegalArgumentException("Retry release tag does not exist"));
|
||||
api.annotated(tag);
|
||||
commit = tag.sha();
|
||||
}
|
||||
Pipeline.check(
|
||||
before.highest() == null || version.compareTo(before.highest()) >= (mode.equals("new") ? 1 : 0),
|
||||
"A newer release has already succeeded; use explicit rollback for older versions");
|
||||
String build = api.verified(commit);
|
||||
remote.call("stage", Files.readString(root.resolve("deploy/compose.yaml")));
|
||||
remote.call("stage-health", Files.readString(root.resolve("deploy/health.sh")));
|
||||
String prepared = remote.call("prepare", null, version.toString(), commit, prefix());
|
||||
return new ReleaseState.Deployment(
|
||||
version, commit, digest(prepared, "BACKEND"), digest(prepared, "FRONTEND"), attempt, build);
|
||||
}
|
||||
|
||||
String prefix() {
|
||||
var uri = URI.create(api.server);
|
||||
String authority = uri.getRawAuthority();
|
||||
String registry = env.getOrDefault("DEPLOY_REGISTRY", "").strip();
|
||||
if (registry.isEmpty()) registry = authority;
|
||||
int port = uri.getPort() >= 0 ? uri.getPort() : uri.getScheme().equals("https") ? 443 : 80;
|
||||
Pipeline.check(
|
||||
registry.equals(authority) || registry.equals("127.0.0.1:" + port),
|
||||
"Deployment registry must be the existing Gitea registry or its NAS loopback alias");
|
||||
String owner = Setup.required(env, "CI_REPOSITORY").split("/")[0].toLowerCase(java.util.Locale.ROOT);
|
||||
return registry + "/" + owner + "/" + Recipe.project(env) + "-";
|
||||
}
|
||||
|
||||
void ensure(ReleaseState.Deployment release) throws Exception {
|
||||
remote.call(
|
||||
"ensure",
|
||||
null,
|
||||
release.directory(),
|
||||
release.version().toString(),
|
||||
release.sha(),
|
||||
release.backend(),
|
||||
release.frontend());
|
||||
}
|
||||
|
||||
String digest(String response, String role) {
|
||||
if (!Recipe.roles(env).contains(role.toLowerCase(java.util.Locale.ROOT))) return "-";
|
||||
String value = response.lines()
|
||||
.filter(line -> line.startsWith("S6_" + role + "="))
|
||||
.map(line -> line.substring(("S6_" + role + "=").length()))
|
||||
.reduce((a, b) -> {
|
||||
throw new IllegalArgumentException("Ambiguous prepared image digest");
|
||||
})
|
||||
.orElseThrow(() -> new IllegalArgumentException("Missing prepared image digest"));
|
||||
Pipeline.check(
|
||||
value.startsWith(prefix() + role.toLowerCase(java.util.Locale.ROOT) + "@sha256:"),
|
||||
"Unexpected image repository");
|
||||
return value;
|
||||
}
|
||||
|
||||
String recover(ReleaseState before) throws Exception {
|
||||
if (before.current() == null) return "first-release-failed-manual-recovery-required";
|
||||
try {
|
||||
remote.call("apply", null, before.current().directory());
|
||||
} catch (Exception failure) {
|
||||
diagnose(before.current());
|
||||
return "recovery-failed-manual-recovery-required";
|
||||
}
|
||||
record("failed-previous-release-restored");
|
||||
remote.call("finish", null);
|
||||
pending = false;
|
||||
return "previous-release-restored";
|
||||
}
|
||||
|
||||
void diagnose(ReleaseState.Deployment release) {
|
||||
try {
|
||||
Files.writeString(
|
||||
artifacts.resolve(attempt + "-" + release.directory() + "-containers.log"),
|
||||
remote.call("diagnose", null, release.directory()));
|
||||
} catch (Exception ignored) {
|
||||
System.err.println("Container diagnostics unavailable; use the NAS recovery runbook");
|
||||
}
|
||||
}
|
||||
|
||||
void record(String outcome) throws Exception {
|
||||
var data = new LinkedHashMap<String, Object>();
|
||||
data.put("attempt", attempt);
|
||||
data.put("mode", mode);
|
||||
data.put("outcome", outcome);
|
||||
data.put("time", Instant.now().toString());
|
||||
data.put("release", selected == null ? null : selected.json());
|
||||
String json = Json.write(data) + "\n";
|
||||
Files.writeString(artifacts.resolve(attempt + ".json"), json);
|
||||
if (locked) remote.call("event", json);
|
||||
}
|
||||
}
|
||||
124
resources/ci/ReleaseApi.java
Normal file
124
resources/ci/ReleaseApi.java
Normal file
@@ -0,0 +1,124 @@
|
||||
import java.net.URI;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
/** Gitea 1.25 API adapter. Never include response bodies or tokens in errors. */
|
||||
class ReleaseApi {
|
||||
record Tag(ReleaseVersion version, String sha, String object) {}
|
||||
|
||||
final String server, token, base;
|
||||
|
||||
ReleaseApi(Map<String, String> env) {
|
||||
server = Setup.required(env, "GITEA_SERVER_URL");
|
||||
var uri = URI.create(server);
|
||||
Pipeline.check(
|
||||
List.of("http", "https").contains(uri.getScheme())
|
||||
&& uri.getUserInfo() == null
|
||||
&& uri.getQuery() == null
|
||||
&& uri.getFragment() == null
|
||||
&& (uri.getPath().isEmpty() || uri.getPath().equals("/")),
|
||||
"Gitea must use a server-root URL");
|
||||
String repository = Setup.required(env, "CI_REPOSITORY");
|
||||
Pipeline.check(repository.matches("[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+"), "Invalid repository name");
|
||||
base = "/api/v1/repos/" + repository;
|
||||
token = Setup.required(env, "RELEASE_TOKEN");
|
||||
}
|
||||
|
||||
Object request(String method, String path, Object body) throws Exception {
|
||||
return Setup.api(server, token, method, base + path, body);
|
||||
}
|
||||
|
||||
String head() throws Exception {
|
||||
return sha(
|
||||
Json.object(Json.object(request("GET", "/branches/main", null)).get("commit"))
|
||||
.get("id"));
|
||||
}
|
||||
|
||||
static String sha(Object value) {
|
||||
String result = Objects.toString(value, "");
|
||||
Pipeline.check(result.matches("[0-9a-f]{40}"), "Invalid release commit SHA");
|
||||
return result;
|
||||
}
|
||||
|
||||
List<Tag> tags() throws Exception {
|
||||
List<Tag> tags = new ArrayList<>();
|
||||
var seen = new java.util.HashSet<String>();
|
||||
for (int page = 1; ; page++) {
|
||||
var entries = (List<?>) request("GET", "/tags?limit=50&page=" + page, null);
|
||||
if (entries.isEmpty()) return tags;
|
||||
for (Object entry : entries) {
|
||||
var tag = Json.object(entry);
|
||||
String name = Objects.toString(tag.get("name"), "");
|
||||
Pipeline.check(
|
||||
seen.add(name),
|
||||
"Remote tag pagination repeated a tag; cannot establish a safe version baseline");
|
||||
// Unknown version-looking tags require explicit reconciliation, not a second baseline.
|
||||
if (!name.matches("v?[0-9].*")) continue;
|
||||
ReleaseVersion version = ReleaseVersion.parse(name);
|
||||
String commit = sha(Json.object(tag.get("commit")).get("sha"));
|
||||
String object = sha(tag.get("id"));
|
||||
tags.add(new Tag(version, commit, object));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void annotated(Tag tag) throws Exception {
|
||||
var data = Json.object(request("GET", "/git/tags/" + tag.object(), null));
|
||||
Pipeline.check(
|
||||
tag.version().toString().equals(data.get("tag"))
|
||||
&& tag.sha().equals(Json.object(data.get("object")).get("sha")),
|
||||
"Release tag must be annotated and point directly to its recorded commit");
|
||||
}
|
||||
|
||||
void create(ReleaseVersion version, String commit, String run) throws Exception {
|
||||
request(
|
||||
"POST",
|
||||
"/tags",
|
||||
Map.of(
|
||||
"tag_name",
|
||||
version.toString(),
|
||||
"target",
|
||||
commit,
|
||||
"message",
|
||||
"Release " + version + "\nCommit: " + commit + "\nDeployment attempt: " + run));
|
||||
Tag tag = tags().stream()
|
||||
.filter(t -> t.version().equals(version))
|
||||
.findFirst()
|
||||
.orElseThrow();
|
||||
Pipeline.check(tag.sha().equals(commit), "Created release tag does not match selected commit");
|
||||
annotated(tag);
|
||||
}
|
||||
|
||||
String verified(String commit) throws Exception {
|
||||
// A successful build.yaml main-push run includes verification AND successful image publication.
|
||||
// Do not accept another workflow, a branch build, or merely an existing image/tag.
|
||||
var seen = new java.util.HashSet<String>();
|
||||
for (int page = 1; ; page++) {
|
||||
var response = Json.object(request(
|
||||
"GET",
|
||||
"/actions/runs?head_sha=" + commit + "&event=push&branch=main&status=success&limit=50&page=" + page,
|
||||
null));
|
||||
var runs = (List<?>) response.get("workflow_runs");
|
||||
if (runs.isEmpty()) break;
|
||||
boolean fresh = false;
|
||||
for (Object entry : runs) {
|
||||
var run = Json.object(entry);
|
||||
fresh |= seen.add(Objects.toString(run.get("id")));
|
||||
if (commit.equals(run.get("head_sha"))
|
||||
&& "push".equals(run.get("event"))
|
||||
&& List.of("main", "refs/heads/main").contains(Objects.toString(run.get("head_branch")))
|
||||
&& "completed".equals(run.get("status"))
|
||||
&& "success".equals(run.get("conclusion"))
|
||||
&& "build.yaml@refs/heads/main".equals(run.get("path"))) {
|
||||
return Objects.toString(run.get("id"));
|
||||
}
|
||||
}
|
||||
// Older Gitea installations may ignore some list filters/pagination.
|
||||
// Validate each entry locally and stop if a page makes no progress.
|
||||
if (!fresh) break;
|
||||
}
|
||||
throw new IllegalArgumentException("No successful build.yaml main-push run verifies and publishes this SHA");
|
||||
}
|
||||
}
|
||||
125
resources/ci/ReleaseRemote.java
Normal file
125
resources/ci/ReleaseRemote.java
Normal file
@@ -0,0 +1,125 @@
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.attribute.PosixFilePermissions;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
/** Argument-safe SSH adapter; server-side mkdir lock is shared by every runner. */
|
||||
class ReleaseRemote implements AutoCloseable {
|
||||
final Path root, credentials;
|
||||
final Map<String, String> env;
|
||||
final Commands commands;
|
||||
final String owner, directory, host, user, port, script;
|
||||
|
||||
ReleaseRemote(Path root, Map<String, String> env, Commands commands, String owner) throws Exception {
|
||||
this.root = root;
|
||||
this.env = env;
|
||||
this.commands = commands;
|
||||
this.owner = owner;
|
||||
directory = Setup.required(env, "DEPLOY_DIRECTORY");
|
||||
host = Setup.required(env, "DEPLOY_HOST");
|
||||
user = Setup.required(env, "DEPLOY_USER");
|
||||
port = env.getOrDefault("DEPLOY_SSH_PORT", "22");
|
||||
Pipeline.check(
|
||||
directory.matches("/[a-zA-Z0-9_./-]+") && !directory.contains("..") && !directory.equals("/"),
|
||||
"Use an absolute dedicated deployment directory without spaces or parent traversal");
|
||||
Pipeline.check(
|
||||
host.matches("[a-zA-Z0-9][a-zA-Z0-9.-]*")
|
||||
&& user.matches("[a-zA-Z0-9_][a-zA-Z0-9_-]*")
|
||||
&& port.matches("[0-9]{1,5}")
|
||||
&& Integer.parseInt(port) > 0
|
||||
&& Integer.parseInt(port) <= 65535,
|
||||
"Invalid deployment SSH host, user or port");
|
||||
script = "COMPOSE_PROJECT=" + quote(Setup.required(env, "COMPOSE_PROJECT")) + "\n"
|
||||
+ "DEPLOY_ROLES=" + quote(String.join(" ", Recipe.roles(env))) + "\n"
|
||||
+ "DEPENDENCY_SERVICES=" + quote(env.getOrDefault("DEPENDENCY_SERVICES", "")) + "\n"
|
||||
+ "TARGET_PLATFORM=" + quote(env.getOrDefault("TARGET_PLATFORM", "linux/amd64")) + "\n"
|
||||
+ "REQUIRED_SECRETS=" + quote(env.getOrDefault("REQUIRED_DEPLOY_SECRETS", "")) + "\n"
|
||||
+ Files.readString(root.resolve("deploy/health.sh")) + "\n"
|
||||
+ Files.readString(root.resolve("generated/deploy/remote.sh"));
|
||||
credentials = Files.createTempDirectory("engineering-deploy-ssh-");
|
||||
try {
|
||||
writePrivate("identity", Setup.required(env, "DEPLOY_SSH_KEY") + "\n");
|
||||
writePrivate("known_hosts", Setup.required(env, "DEPLOY_KNOWN_HOSTS") + "\n");
|
||||
} catch (Exception e) {
|
||||
close();
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
void writePrivate(String name, String value) throws Exception {
|
||||
Path file = credentials.resolve(name);
|
||||
Files.createFile(file, PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString("rw-------")));
|
||||
Files.writeString(file, value);
|
||||
}
|
||||
|
||||
static String quote(String value) {
|
||||
return "'" + value.replace("'", "'\"'\"'") + "'";
|
||||
}
|
||||
|
||||
String call(String action, String input, String... arguments) throws Exception {
|
||||
int seconds = switch (action) {
|
||||
case "apply" -> 300;
|
||||
case "prepare", "ensure" -> 600;
|
||||
default -> 60;
|
||||
};
|
||||
var parameters = new ArrayList<>(List.of(action, directory, owner));
|
||||
parameters.addAll(List.of(arguments));
|
||||
return executeScript(script, input, seconds, parameters);
|
||||
}
|
||||
|
||||
String executeScript(String source, String input, int seconds, List<String> parameters) throws Exception {
|
||||
String remote = "timeout -k 5 " + seconds + " sh -c " + quote(source) + " --";
|
||||
for (String argument : parameters) remote += " " + quote(argument);
|
||||
var args = new ArrayList<>(List.of(
|
||||
"ssh",
|
||||
"-F",
|
||||
"/dev/null",
|
||||
"-T",
|
||||
"-o",
|
||||
"BatchMode=yes",
|
||||
"-o",
|
||||
"StrictHostKeyChecking=yes",
|
||||
"-o",
|
||||
"IdentitiesOnly=yes",
|
||||
"-o",
|
||||
"IdentityAgent=none",
|
||||
"-o",
|
||||
"ForwardAgent=no",
|
||||
"-o",
|
||||
"ConnectTimeout=10",
|
||||
"-o",
|
||||
"ServerAliveInterval=10",
|
||||
"-o",
|
||||
"ServerAliveCountMax=3",
|
||||
"-o",
|
||||
"LogLevel=ERROR",
|
||||
"-o",
|
||||
"UserKnownHostsFile=" + credentials.resolve("known_hosts"),
|
||||
"-o",
|
||||
"GlobalKnownHostsFile=/dev/null",
|
||||
"-i",
|
||||
credentials.resolve("identity").toString(),
|
||||
"-p",
|
||||
port,
|
||||
user + "@" + host,
|
||||
remote));
|
||||
commands.renew(Duration.ofSeconds(seconds + 30));
|
||||
return commands.execute(args, root, env, input, false).checked();
|
||||
}
|
||||
|
||||
void acquire() throws Exception {
|
||||
call("lock", null);
|
||||
}
|
||||
|
||||
void release() throws Exception {
|
||||
call("unlock", null);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
Pipeline.deleteTree(credentials);
|
||||
}
|
||||
}
|
||||
105
resources/ci/ReleaseSetup.java
Normal file
105
resources/ci/ReleaseSetup.java
Normal file
@@ -0,0 +1,105 @@
|
||||
import java.net.URI;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.time.Duration;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
/** One-time provisioning; secrets stay in owner-only files or Gitea secret storage. */
|
||||
class ReleaseSetup {
|
||||
static void configure(Path root, Map<String, String> input) throws Exception {
|
||||
var env = new HashMap<>(input);
|
||||
env.putIfAbsent("DEPLOY_SSH_PORT", "22");
|
||||
Path config = Setup.home().resolve(".config/" + Recipe.project(env) + "-runner");
|
||||
env.putIfAbsent("GITEA_ADMIN_TOKEN_FILE", config.resolve("admin-token").toString());
|
||||
env.putIfAbsent("RELEASE_TOKEN_FILE", config.resolve("release-token").toString());
|
||||
env.putIfAbsent("PULL_TOKEN_FILE", config.resolve("pull-token").toString());
|
||||
env.putIfAbsent("DEPLOY_SSH_KEY_FILE", config.resolve("deploy-key").toString());
|
||||
String admin = Setup.secret(env, "GITEA_ADMIN_TOKEN_FILE");
|
||||
String pull = Setup.secret(env, "PULL_TOKEN_FILE");
|
||||
env.put("RELEASE_TOKEN", Setup.secret(env, "RELEASE_TOKEN_FILE"));
|
||||
env.put("DEPLOY_SSH_KEY", Setup.secret(env, "DEPLOY_SSH_KEY_FILE"));
|
||||
String bind = env.get("DEPLOY_BIND_IP"), port = env.get("DEPLOY_FRONTEND_PORT");
|
||||
Pipeline.check(
|
||||
bind.matches("[0-9.]+")
|
||||
&& port.matches("[0-9]{4,5}")
|
||||
&& Integer.parseInt(port) >= 1024
|
||||
&& Integer.parseInt(port) <= 65535,
|
||||
"Invalid private bind address or port");
|
||||
var commands = new Commands(Duration.ofMinutes(15));
|
||||
Path diagnostics = root.resolve("ci/artifacts/deployment");
|
||||
Files.createDirectories(diagnostics);
|
||||
commands.log = diagnostics.resolve("setup.log");
|
||||
Files.writeString(commands.log, "");
|
||||
String host = env.get("DEPLOY_HOST");
|
||||
String lookup = env.get("DEPLOY_SSH_PORT").equals("22") ? host : "[" + host + "]:" + env.get("DEPLOY_SSH_PORT");
|
||||
env.put(
|
||||
"DEPLOY_KNOWN_HOSTS",
|
||||
commands.execute(
|
||||
List.of(
|
||||
"ssh-keygen",
|
||||
"-F",
|
||||
lookup,
|
||||
"-f",
|
||||
Setup.home().resolve(".ssh/known_hosts").toString()),
|
||||
root,
|
||||
env,
|
||||
null,
|
||||
false)
|
||||
.checked());
|
||||
Pipeline.check(!env.get("DEPLOY_KNOWN_HOSTS").isBlank(), "A previously trusted NAS hostkey is required");
|
||||
var api = new ReleaseApi(env);
|
||||
api.verified(api.head());
|
||||
var uri = URI.create(api.server);
|
||||
String registry = env.get("DEPLOY_REGISTRY");
|
||||
int registryPort = uri.getPort() >= 0 ? uri.getPort() : uri.getScheme().equals("https") ? 443 : 80;
|
||||
Pipeline.check(
|
||||
registry.equals(uri.getRawAuthority()) || registry.equals("127.0.0.1:" + registryPort),
|
||||
"Deployment registry must be the Gitea registry or its NAS loopback alias");
|
||||
String registryUser =
|
||||
env.getOrDefault("PULL_USER", env.get("CI_REPOSITORY").split("/")[0]);
|
||||
Pipeline.check(registryUser.matches("[a-zA-Z0-9_.-]+"), "Invalid registry user");
|
||||
try (var remote = new ReleaseRemote(root, env, commands, "setup")) {
|
||||
System.out.println(remote.executeScript(
|
||||
Files.readString(root.resolve("generated/deploy/provision.sh")),
|
||||
pull + "\n",
|
||||
300,
|
||||
List.of(
|
||||
remote.directory,
|
||||
registryUser,
|
||||
registry,
|
||||
bind,
|
||||
port,
|
||||
Setup.required(env, "DEPLOY_DOCKER_BIN"))));
|
||||
remote.acquire();
|
||||
try {
|
||||
remote.call("preflight", null);
|
||||
} finally {
|
||||
remote.release();
|
||||
}
|
||||
}
|
||||
// Provision credentials only after successful NAS login and release-read verification.
|
||||
for (String secret : List.of("RELEASE_TOKEN", "DEPLOY_SSH_KEY", "DEPLOY_KNOWN_HOSTS"))
|
||||
Setup.api(
|
||||
api.server, admin, "PUT", api.base + "/actions/secrets/" + secret, Map.of("data", env.get(secret)));
|
||||
for (String variable :
|
||||
List.of("DEPLOY_HOST", "DEPLOY_USER", "DEPLOY_SSH_PORT", "DEPLOY_DIRECTORY", "DEPLOY_REGISTRY"))
|
||||
variable(api, admin, variable, env.get(variable));
|
||||
System.out.println(
|
||||
"Production configuration and Gitea deployment secrets/variables installed; no release was created.");
|
||||
}
|
||||
|
||||
static void variable(ReleaseApi api, String admin, String name, String value) throws Exception {
|
||||
String path = api.base + "/actions/variables/" + name;
|
||||
boolean exists;
|
||||
try {
|
||||
Setup.api(api.server, admin, "GET", path, null);
|
||||
exists = true;
|
||||
} catch (Setup.ApiFailure error) {
|
||||
if (error.status != 404) throw error;
|
||||
exists = false;
|
||||
}
|
||||
Setup.api(api.server, admin, exists ? "PUT" : "POST", path, Map.of("value", value));
|
||||
}
|
||||
}
|
||||
244
resources/ci/ReleaseShellTests.java
Normal file
244
resources/ci/ReleaseShellTests.java
Normal file
@@ -0,0 +1,244 @@
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.attribute.PosixFilePermissions;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
/** Runs the actual NAS script with disposable command adapters, no Docker daemon. */
|
||||
class ReleaseShellTests {
|
||||
static void run() throws Exception {
|
||||
ReleaseTests.test("actual NAS script prepares digest images and checks the published endpoint", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.prepare();
|
||||
f.call("pending", "{}", "one").checked();
|
||||
CiTests.equal("S6_HEALTHY", f.call("apply", null, "one", "one").checked());
|
||||
String calls = Files.readString(f.directory.resolve("calls"));
|
||||
CiTests.equal(true, calls.contains("http://backend:8080/q/health/ready"));
|
||||
CiTests.equal(true, calls.contains("http://127.0.0.1:8088/api/items"));
|
||||
CiTests.equal(false, calls.contains(" down"));
|
||||
CiTests.equal(false, calls.contains("prune"));
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script rejects a missing database password before pulling images", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
Path config = f.directory.resolve("production.env");
|
||||
Files.writeString(
|
||||
config, Files.readString(config).replace("DATABASE_PASSWORD=fixture-only-password\n", ""));
|
||||
f.call("lock", null, "one").checked();
|
||||
Commands.Result result = f.call("preflight", null, "one");
|
||||
CiTests.equal(1, result.code());
|
||||
CiTests.equal(true, result.output().contains("Missing DATABASE_PASSWORD"));
|
||||
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script passes database configuration through sudo", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.executable("sudo", "#!/bin/sh\n[ \"$1\" != -n ] || shift\nunset DATABASE_PASSWORD\nexec \"$@\"\n");
|
||||
Path config = f.directory.resolve("production.env");
|
||||
Files.writeString(config, Files.readString(config).replace("DOCKER_SUDO=0", "DOCKER_SUDO=1"));
|
||||
f.prepare();
|
||||
CiTests.equal(
|
||||
false,
|
||||
Files.readString(f.directory.resolve("releases/one/release.env"))
|
||||
.contains("PASSWORD"));
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script rejects wrong image revision before replacement", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.env.put("WRONG_REVISION", "1");
|
||||
CiTests.failure(f::prepare);
|
||||
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script rejects unavailable image before replacement", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.env.put("FAIL_PULL", "1");
|
||||
CiTests.failure(f::prepare);
|
||||
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script rejects a changed retained release record", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.prepare();
|
||||
CiTests.failure(() -> f.call(
|
||||
"ensure", null, "one", "one", "v9.0.0", ReleaseTests.SHA, f.backend(), f.frontend())
|
||||
.checked());
|
||||
f.call("ensure", null, "one", "one", "v1.0.0", ReleaseTests.SHA, f.backend(), f.frontend())
|
||||
.checked();
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS replacement timeout is bounded and leaves recovery journal", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.prepare();
|
||||
f.call("pending", "{}", "one").checked();
|
||||
f.env.put("FAIL_HEALTH", "1");
|
||||
CiTests.equal(1, f.call("apply", null, "one", "one", "1").code());
|
||||
CiTests.equal(true, Files.exists(f.directory.resolve("pending.json")));
|
||||
// Termination releases only the operation fence, not the deployment lock.
|
||||
CiTests.equal(false, Files.exists(f.directory.resolve("deploy.lock/operation")));
|
||||
CiTests.equal(true, Files.exists(f.directory.resolve("deploy.lock/owner")));
|
||||
f.env.remove("FAIL_HEALTH");
|
||||
f.call("apply", null, "one", "one").checked();
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script fences overlapping remote replacements", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.prepare();
|
||||
Files.createDirectory(f.directory.resolve("deploy.lock/operation"));
|
||||
CiTests.failure(() -> f.call("apply", null, "one", "one").checked());
|
||||
CiTests.equal(false, Files.exists(f.directory.resolve("live.env")));
|
||||
}
|
||||
});
|
||||
ReleaseTests.test("actual NAS script stores state atomically and archives resolved journal", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.call("lock", null, "one").checked();
|
||||
f.call("pending", "{\"attempt\":\"one\"}", "one").checked();
|
||||
f.call("commit", "{\"current\":\"one\"}", "one").checked();
|
||||
CiTests.equal(
|
||||
"{\"current\":\"one\"}", f.call("state", null, "one").checked());
|
||||
f.call("event", "{\"outcome\":\"success\"}", "one").checked();
|
||||
f.call("finish", null, "one").checked();
|
||||
f.call("unlock", null, "one").checked();
|
||||
CiTests.equal(true, Files.exists(f.directory.resolve("history/one.json")));
|
||||
CiTests.equal(false, Files.exists(f.directory.resolve("pending.json")));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
static class Fixture implements AutoCloseable {
|
||||
final Path directory = Files.createTempDirectory("nas-script-test-");
|
||||
final Map<String, String> env = new HashMap<>(System.getenv());
|
||||
final String health = "application_health() {\n"
|
||||
+ "frontend=$(compose ps -q frontend)\n"
|
||||
+ "docker_cmd exec \"$frontend\" wget -q -T 5 -O /dev/null http://backend:8080/q/health/ready || return"
|
||||
+ " 1\n"
|
||||
+ "curl --fail http://127.0.0.1:8088/api/items -o /dev/null\n"
|
||||
+ "}\n";
|
||||
final String script = "COMPOSE_PROJECT=recipe-demo-test\n"
|
||||
+ "DEPLOY_ROLES='backend frontend'\n"
|
||||
+ "DEPENDENCY_SERVICES=database\n"
|
||||
+ "TARGET_PLATFORM=linux/amd64\n"
|
||||
+ "REQUIRED_SECRETS=DATABASE_PASSWORD\n"
|
||||
+ health + "\n"
|
||||
+ Files.readString(Path.of("generated/deploy/remote.sh"));
|
||||
final Commands commands = new Commands(Duration.ofMinutes(1));
|
||||
|
||||
Fixture() throws Exception {
|
||||
Files.createDirectory(directory.resolve("bin"));
|
||||
executable("docker", DOCKER);
|
||||
executable(
|
||||
"curl",
|
||||
"#!/bin/sh\nprintf 'curl %s\\n' \"$*\" >> \"$FIXTURE/calls\"\n[ \"${FAIL_HEALTH:-0}\" = 0 ]\n");
|
||||
env.put("PATH", directory.resolve("bin") + ":" + env.get("PATH"));
|
||||
env.put("FIXTURE", directory.toString());
|
||||
Files.createDirectory(directory.resolve("docker-auth"));
|
||||
Files.writeString(directory.resolve("docker-auth/config.json"), "{}");
|
||||
Files.writeString(
|
||||
directory.resolve("production.env"),
|
||||
"DATABASE_PASSWORD=fixture-only-password\n"
|
||||
+ "BIND_IP=127.0.0.1\n"
|
||||
+ "FRONTEND_PORT=8088\n"
|
||||
+ "DOCKER_SUDO=0\n"
|
||||
+ "DOCKER_BIN="
|
||||
+ directory.resolve("bin/docker") + "\nDOCKER_CONFIG=" + directory.resolve("docker-auth")
|
||||
+ "\n");
|
||||
}
|
||||
|
||||
void executable(String name, String body) throws Exception {
|
||||
Path path = directory.resolve("bin").resolve(name);
|
||||
Files.writeString(path, body);
|
||||
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rwx------"));
|
||||
}
|
||||
|
||||
Commands.Result call(String action, String input, String owner, String... args) throws Exception {
|
||||
var command = new ArrayList<>(
|
||||
List.of("timeout", "-k", "1", "10", "sh", "-c", script, "--", action, directory.toString(), owner));
|
||||
command.addAll(List.of(args));
|
||||
commands.renew(Duration.ofSeconds(15));
|
||||
return commands.execute(command, directory, env, input, false);
|
||||
}
|
||||
|
||||
String backend() {
|
||||
return "nas:3000/owner/recipe-demo-backend@sha256:" + "1".repeat(64);
|
||||
}
|
||||
|
||||
String frontend() {
|
||||
return "nas:3000/owner/recipe-demo-frontend@sha256:" + "2".repeat(64);
|
||||
}
|
||||
|
||||
void prepare() throws Exception {
|
||||
call("lock", null, "one").checked();
|
||||
call("preflight", null, "one").checked();
|
||||
call("stage", "services: {}\n", "one").checked();
|
||||
call("stage-health", health, "one").checked();
|
||||
String result = call("prepare", null, "one", "v1.0.0", ReleaseTests.SHA, "nas:3000/owner/recipe-demo-")
|
||||
.checked();
|
||||
CiTests.equal(true, result.contains("S6_BACKEND=" + backend()));
|
||||
CiTests.equal(true, result.contains("S6_FRONTEND=" + frontend()));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
Pipeline.deleteTree(directory);
|
||||
}
|
||||
}
|
||||
|
||||
static final String DOCKER = """
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
printf 'docker %s\n' "$*" >> "$FIXTURE/calls"
|
||||
[ "$1" != --config ] || shift 2
|
||||
case "$1" in
|
||||
info) echo linux/x86_64;;
|
||||
pull) [ "${FAIL_PULL:-0}" = 0 ];;
|
||||
image)
|
||||
format=$4; image=$5
|
||||
case "$image" in *backend*) role=backend; digit=1;; *) role=frontend; digit=2;; esac
|
||||
case "$format" in
|
||||
*revision*)
|
||||
if [ "${WRONG_REVISION:-0}" = 1 ]; then echo wrong; else printf '%040d\n' 0 | tr 0 a; fi;;
|
||||
*Architecture*) echo linux/amd64;;
|
||||
*RepoDigests*)
|
||||
printf '%s@sha256:' "${image%:*}"; printf '%064d\n' 0 | tr 0 "$digit";;
|
||||
*Id*) echo "sha256:$role";;
|
||||
*) exit 1;;
|
||||
esac;;
|
||||
compose)
|
||||
shift
|
||||
[ "$1" != version ] || exit 0
|
||||
shift 4
|
||||
[ "$1" = --env-file ] || exit 1
|
||||
production=$2
|
||||
shift 2
|
||||
[ "$1" = --env-file ] || exit 1
|
||||
file=$2
|
||||
shift 4
|
||||
# Compose must read the secret from disk, even after sudo clears the environment.
|
||||
test -f "$production"
|
||||
. "$production"
|
||||
[ "$DATABASE_PASSWORD" = fixture-only-password ] || exit 15
|
||||
case "$1" in
|
||||
config) test -f "$file";;
|
||||
up) cp "$file" "$FIXTURE/live.env";;
|
||||
ps) eval 'role=${'"$#"'}'; echo "$role";;
|
||||
logs) echo 'sample container diagnostics';;
|
||||
*) exit 1;;
|
||||
esac;;
|
||||
inspect)
|
||||
. "$FIXTURE/live.env"
|
||||
format=$3; id=$4
|
||||
case "$format" in
|
||||
*State.Running*) echo true;;
|
||||
*Image*) echo "sha256:$id";;
|
||||
*revision*) echo "$RELEASE_SHA";;
|
||||
*release*) echo "$RELEASE_VERSION";;
|
||||
*) exit 1;;
|
||||
esac;;
|
||||
exec) [ "${FAIL_HEALTH:-0}" = 0 ];;
|
||||
*) exit 1;;
|
||||
esac
|
||||
""";
|
||||
}
|
||||
83
resources/ci/ReleaseState.java
Normal file
83
resources/ci/ReleaseState.java
Normal file
@@ -0,0 +1,83 @@
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
/** Successful deployment state is separate from immutable allocated version tags. */
|
||||
record ReleaseState(ReleaseState.Deployment current, ReleaseState.Deployment previous, ReleaseVersion highest) {
|
||||
record Deployment(
|
||||
ReleaseVersion version, String sha, String backend, String frontend, String directory, String build) {
|
||||
Deployment {
|
||||
ReleaseApi.sha(sha);
|
||||
Pipeline.check(directory.matches("[a-zA-Z0-9-]+"), "Invalid retained release directory");
|
||||
Pipeline.check(!backend.equals("-") || !frontend.equals("-"), "A release must contain an image");
|
||||
for (String image : new String[] {backend, frontend})
|
||||
Pipeline.check(
|
||||
(image.equals("-") || image.matches("[a-zA-Z0-9.:/_-]+@sha256:[0-9a-f]{64}")),
|
||||
"Invalid retained image digest");
|
||||
}
|
||||
|
||||
Map<String, Object> json() {
|
||||
return Map.of(
|
||||
"version",
|
||||
version.toString(),
|
||||
"sha",
|
||||
sha,
|
||||
"backend",
|
||||
backend,
|
||||
"frontend",
|
||||
frontend,
|
||||
"directory",
|
||||
directory,
|
||||
"build",
|
||||
build);
|
||||
}
|
||||
|
||||
static Deployment parse(Object value) {
|
||||
if (value == null) return null;
|
||||
var data = Json.object(value);
|
||||
return new Deployment(
|
||||
ReleaseVersion.parse(text(data, "version")),
|
||||
text(data, "sha"),
|
||||
text(data, "backend"),
|
||||
text(data, "frontend"),
|
||||
text(data, "directory"),
|
||||
text(data, "build"));
|
||||
}
|
||||
}
|
||||
|
||||
static String text(Map<String, Object> data, String key) {
|
||||
return Objects.toString(data.get(key), "");
|
||||
}
|
||||
|
||||
static ReleaseState parse(String value) {
|
||||
var data = Json.object(Json.parse(value));
|
||||
if (data.isEmpty()) return new ReleaseState(null, null, null);
|
||||
Pipeline.check("1".equals(text(data, "schema")), "Unsupported deployment state schema");
|
||||
Deployment current = Deployment.parse(data.get("current"));
|
||||
Deployment previous = Deployment.parse(data.get("previous"));
|
||||
ReleaseVersion highest = ReleaseVersion.parse(text(data, "highest"));
|
||||
Pipeline.check(
|
||||
current != null
|
||||
&& highest.compareTo(current.version()) >= 0
|
||||
&& (previous == null || highest.compareTo(previous.version()) >= 0),
|
||||
"Inconsistent successful release state");
|
||||
return new ReleaseState(current, previous, highest);
|
||||
}
|
||||
|
||||
String json() {
|
||||
var data = new LinkedHashMap<String, Object>();
|
||||
data.put("schema", "1");
|
||||
data.put("current", current == null ? null : current.json());
|
||||
data.put("previous", previous == null ? null : previous.json());
|
||||
data.put("highest", highest == null ? null : highest.toString());
|
||||
return Json.write(data) + "\n";
|
||||
}
|
||||
|
||||
ReleaseState success(Deployment selected) {
|
||||
ReleaseVersion watermark =
|
||||
highest == null || selected.version().compareTo(highest) > 0 ? selected.version() : highest;
|
||||
// A redeploy of the same release must not discard the distinct rollback target.
|
||||
Deployment fallback = current != null && current.version().equals(selected.version()) ? previous : current;
|
||||
return new ReleaseState(selected, fallback, watermark);
|
||||
}
|
||||
}
|
||||
555
resources/ci/ReleaseTests.java
Normal file
555
resources/ci/ReleaseTests.java
Normal file
@@ -0,0 +1,555 @@
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
/** Release state-machine and real POSIX remote-script tests, without server writes. */
|
||||
class ReleaseTests {
|
||||
static final String SHA = "a".repeat(40), OLD = "b".repeat(40);
|
||||
static int count;
|
||||
|
||||
static void test(String name, CiTests.Test test) throws Exception {
|
||||
test.run();
|
||||
count++;
|
||||
System.out.println("PASS release: " + name);
|
||||
}
|
||||
|
||||
static Map<String, String> environment() {
|
||||
var env = new HashMap<>(System.getenv());
|
||||
env.putAll(Map.of(
|
||||
"CI_REF",
|
||||
"refs/heads/main",
|
||||
"CI_EVENT",
|
||||
"workflow_dispatch",
|
||||
"CI_RUN_ID",
|
||||
"123",
|
||||
"CI_COMMIT",
|
||||
SHA,
|
||||
"RELEASE_MODE",
|
||||
"new",
|
||||
"RELEASE_INCREMENT",
|
||||
"patch",
|
||||
"RELEASE_VERSION",
|
||||
"",
|
||||
"GITEA_SERVER_URL",
|
||||
"http://nas:3000",
|
||||
"CI_REPOSITORY",
|
||||
"owner/recipe-demo",
|
||||
"RELEASE_TOKEN",
|
||||
"test"));
|
||||
env.putAll(Map.of(
|
||||
"DEPLOY_HOST",
|
||||
"nas",
|
||||
"DEPLOY_USER",
|
||||
"deploy",
|
||||
"DEPLOY_DIRECTORY",
|
||||
"/tmp/production",
|
||||
"DEPLOY_SSH_KEY",
|
||||
"test-private-key",
|
||||
"DEPLOY_KNOWN_HOSTS",
|
||||
"test-pinned-host"));
|
||||
return env;
|
||||
}
|
||||
|
||||
static ReleaseState.Deployment deployment(String version, String sha, String directory) {
|
||||
return new ReleaseState.Deployment(
|
||||
ReleaseVersion.parse(version),
|
||||
sha,
|
||||
"nas:3000/owner/recipe-demo-backend@sha256:" + "1".repeat(64),
|
||||
"nas:3000/owner/recipe-demo-frontend@sha256:" + "2".repeat(64),
|
||||
directory,
|
||||
"10");
|
||||
}
|
||||
|
||||
static ReleaseState prior() {
|
||||
return new ReleaseState(
|
||||
deployment("v1.0.1", OLD, "previous-current"),
|
||||
deployment("v1.0.0", OLD, "previous-old"),
|
||||
ReleaseVersion.parse("v1.0.1"));
|
||||
}
|
||||
|
||||
static void run() throws Exception {
|
||||
count = 0;
|
||||
test("first release and all increments reset the correct components", () -> {
|
||||
CiTests.equal("v1.0.0", ReleaseVersion.next(List.of(), "major").toString());
|
||||
var versions = List.of(ReleaseVersion.parse("v1.2.3"), ReleaseVersion.parse("v1.1.9"));
|
||||
CiTests.equal("v1.2.4", ReleaseVersion.next(versions, "patch").toString());
|
||||
CiTests.equal("v1.3.0", ReleaseVersion.next(versions, "minor").toString());
|
||||
CiTests.equal("v2.0.0", ReleaseVersion.next(versions, "major").toString());
|
||||
CiTests.equal(
|
||||
"v1.0.100000000000000000000",
|
||||
ReleaseVersion.next(List.of(ReleaseVersion.parse("v1.0.99999999999999999999")), "patch")
|
||||
.toString());
|
||||
for (String invalid : List.of("1.0.0", "v01.0.0", "v1.0.0-rc1", "v1.0.0+build", "v1.0", "v-1.0.0"))
|
||||
CiTests.failure(() -> ReleaseVersion.parse(invalid));
|
||||
});
|
||||
test("branch, nonmanual, malformed input and conflicting new-release inputs are rejected", () -> {
|
||||
var env = environment();
|
||||
Release.validate(env, "deploy");
|
||||
env.put("CI_REF", "refs/heads/feature/S6");
|
||||
CiTests.failure(() -> Release.validate(env, "deploy"));
|
||||
env.put("CI_REF", "refs/heads/main");
|
||||
env.put("CI_EVENT", "push");
|
||||
CiTests.failure(() -> Release.validate(env, "rollback"));
|
||||
env.put("CI_EVENT", "workflow_dispatch");
|
||||
env.put("RELEASE_VERSION", "v1.0.0");
|
||||
CiTests.failure(() -> Release.validate(env, "deploy"));
|
||||
env.put("RELEASE_MODE", "retry");
|
||||
Release.validate(env, "deploy");
|
||||
env.put("RELEASE_VERSION", "bad");
|
||||
CiTests.failure(() -> Release.validate(env, "deploy"));
|
||||
});
|
||||
test("successful release stages images before tag, checks main again and commits state", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.release.run();
|
||||
CiTests.equal("v1.0.0", f.remote.state.current().version().toString());
|
||||
CiTests.equal(false, f.remote.locked);
|
||||
CiTests.equal(false, f.remote.pending);
|
||||
CiTests.equal(true, f.events.indexOf("prepare") < f.events.indexOf("tag"));
|
||||
CiTests.equal(true, f.events.indexOf("tag") < f.events.lastIndexOf("head"));
|
||||
CiTests.equal(true, f.events.lastIndexOf("head") < f.events.indexOf("apply:attempt"));
|
||||
}
|
||||
});
|
||||
test("unverified commit never creates a tag or replaces production", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.api.verified = false;
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("tag"));
|
||||
CiTests.equal(false, f.events.contains("apply:attempt"));
|
||||
CiTests.equal(false, f.remote.locked);
|
||||
}
|
||||
});
|
||||
test("missing images prevent tag allocation and replacement", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.remote.failAction = "prepare";
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("tag"));
|
||||
CiTests.equal(false, f.remote.pending);
|
||||
}
|
||||
});
|
||||
test("a tagged commit cannot receive another new version", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.api.tags.add(new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.0"), SHA, OLD));
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("prepare"));
|
||||
}
|
||||
});
|
||||
test("stale dispatch is rejected before pulling or tagging", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.api.head = OLD;
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("prepare"));
|
||||
}
|
||||
});
|
||||
test("main advancing after tag creation preserves tag without changing production", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.api.advanceAfterTag = true;
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(1, f.api.tags.size());
|
||||
CiTests.equal(false, f.remote.pending);
|
||||
CiTests.equal(false, f.events.contains("apply:attempt"));
|
||||
}
|
||||
});
|
||||
test("unavailable recovery images block replacement", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.withPrior();
|
||||
f.remote.failAction = "ensure";
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("tag"));
|
||||
CiTests.equal(false, f.remote.pending);
|
||||
}
|
||||
});
|
||||
test("failed candidate restores prior version and still fails the workflow", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.withPrior();
|
||||
f.remote.failedDirectories.add("attempt");
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(
|
||||
List.of("apply:attempt", "apply:previous-current"),
|
||||
f.events.stream().filter(e -> e.startsWith("apply:")).toList());
|
||||
CiTests.equal(prior(), f.remote.state);
|
||||
CiTests.equal(false, f.remote.pending);
|
||||
CiTests.equal(false, f.remote.locked);
|
||||
CiTests.equal(2, f.api.tags.size());
|
||||
}
|
||||
});
|
||||
test("failed first release retains journal and lock for manual recovery", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.remote.failedDirectories.add("attempt");
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(true, f.remote.pending);
|
||||
CiTests.equal(true, f.remote.locked);
|
||||
CiTests.equal(null, f.remote.state.current());
|
||||
}
|
||||
});
|
||||
test("failed automatic recovery retains journal and lock without retry loop", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.withPrior();
|
||||
f.remote.failedDirectories.addAll(List.of("attempt", "previous-current"));
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(
|
||||
2L,
|
||||
f.events.stream().filter(e -> e.startsWith("apply:")).count());
|
||||
CiTests.equal(true, f.remote.pending);
|
||||
CiTests.equal(true, f.remote.locked);
|
||||
}
|
||||
});
|
||||
test("ambiguous state commit never launches speculative rollback", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.withPrior();
|
||||
f.remote.failAction = "commit";
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(
|
||||
1L,
|
||||
f.events.stream().filter(e -> e.startsWith("apply:")).count());
|
||||
CiTests.equal(true, f.remote.pending);
|
||||
CiTests.equal(true, f.remote.locked);
|
||||
}
|
||||
});
|
||||
test("retry keeps existing version and SHA after main advances", () -> {
|
||||
try (var f = new Fixture("retry")) {
|
||||
f.api.head = OLD;
|
||||
f.release.run();
|
||||
CiTests.equal(SHA, f.remote.state.current().sha());
|
||||
CiTests.equal("v1.0.0", f.remote.state.current().version().toString());
|
||||
CiTests.equal(false, f.events.contains("tag"));
|
||||
CiTests.equal(true, f.events.contains("annotated"));
|
||||
}
|
||||
});
|
||||
test("retry cannot cross the successful-version high watermark after rollback", () -> {
|
||||
try (var f = new Fixture("retry")) {
|
||||
f.remote.state = prior().success(prior().previous());
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("apply:attempt"));
|
||||
}
|
||||
});
|
||||
test("manual rollback preserves version baseline and creates no tag", () -> {
|
||||
try (var f = new Fixture("rollback")) {
|
||||
f.remote.state = prior();
|
||||
f.release.run();
|
||||
CiTests.equal("v1.0.0", f.remote.state.current().version().toString());
|
||||
CiTests.equal("v1.0.1", f.remote.state.previous().version().toString());
|
||||
CiTests.equal("v1.0.1", f.remote.state.highest().toString());
|
||||
CiTests.equal(false, f.events.contains("tag"));
|
||||
}
|
||||
});
|
||||
test("redeploying current version preserves a distinct rollback target", () -> {
|
||||
var state = prior();
|
||||
CiTests.equal(state.previous(), state.success(state.current()).previous());
|
||||
CiTests.equal(state, ReleaseState.parse(state.json()));
|
||||
});
|
||||
test("NAS lock contention cannot allocate tags or replace production", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.remote.locked = true;
|
||||
CiTests.failure(f.release::run);
|
||||
CiTests.equal(false, f.events.contains("tag"));
|
||||
CiTests.equal(true, f.remote.locked);
|
||||
}
|
||||
});
|
||||
apiTests();
|
||||
shellTests();
|
||||
ReleaseShellTests.run();
|
||||
System.out.println(count + " release behavior tests passed");
|
||||
}
|
||||
|
||||
static class FakeApi extends ReleaseApi {
|
||||
final List<String> events;
|
||||
final List<Tag> tags = new ArrayList<>();
|
||||
String head = SHA;
|
||||
boolean verified = true, advanceAfterTag;
|
||||
|
||||
FakeApi(Map<String, String> env, List<String> events) {
|
||||
super(env);
|
||||
this.events = events;
|
||||
}
|
||||
|
||||
@Override
|
||||
String head() {
|
||||
events.add("head");
|
||||
return head;
|
||||
}
|
||||
|
||||
@Override
|
||||
List<Tag> tags() {
|
||||
return List.copyOf(tags);
|
||||
}
|
||||
|
||||
@Override
|
||||
String verified(String sha) {
|
||||
Pipeline.check(verified, "Unverified test commit");
|
||||
events.add("verified");
|
||||
return "10";
|
||||
}
|
||||
|
||||
@Override
|
||||
void annotated(Tag tag) {
|
||||
events.add("annotated");
|
||||
}
|
||||
|
||||
@Override
|
||||
void create(ReleaseVersion version, String sha, String attempt) {
|
||||
events.add("tag");
|
||||
tags.add(new Tag(version, sha, OLD));
|
||||
if (advanceAfterTag) head = OLD;
|
||||
}
|
||||
}
|
||||
|
||||
static class FakeRemote extends ReleaseRemote {
|
||||
final List<String> events;
|
||||
final List<String> failedDirectories = new ArrayList<>();
|
||||
ReleaseState state = new ReleaseState(null, null, null);
|
||||
boolean locked, pending;
|
||||
String failAction = "";
|
||||
|
||||
FakeRemote(Path root, Map<String, String> env, List<String> events) throws Exception {
|
||||
super(root, env, new Commands(Duration.ofMinutes(1)), "attempt");
|
||||
this.events = events;
|
||||
}
|
||||
|
||||
@Override
|
||||
String call(String action, String input, String... args) {
|
||||
events.add(action.equals("apply") ? action + ":" + args[0] : action);
|
||||
Pipeline.check(!action.equals(failAction), "Injected remote failure");
|
||||
switch (action) {
|
||||
case "lock" -> {
|
||||
Pipeline.check(!locked && !pending, "Locked");
|
||||
locked = true;
|
||||
}
|
||||
case "unlock" -> {
|
||||
Pipeline.check(!pending, "Pending");
|
||||
locked = false;
|
||||
}
|
||||
case "state" -> {
|
||||
return state.current() == null ? "{}" : state.json();
|
||||
}
|
||||
case "pending" -> pending = true;
|
||||
case "finish" -> pending = false;
|
||||
case "commit" -> state = ReleaseState.parse(input);
|
||||
case "prepare" -> {
|
||||
return "S6_BACKEND=" + deployment("v1.0.0", SHA, "attempt").backend() + "\nS6_FRONTEND="
|
||||
+ deployment("v1.0.0", SHA, "attempt").frontend();
|
||||
}
|
||||
case "apply" -> Pipeline.check(!failedDirectories.contains(args[0]), "Injected unhealthy release");
|
||||
default -> {}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
static class Fixture implements AutoCloseable {
|
||||
final Path root = Files.createTempDirectory("release-test-");
|
||||
final Map<String, String> env = environment();
|
||||
final List<String> events = new ArrayList<>();
|
||||
final FakeApi api;
|
||||
final FakeRemote remote;
|
||||
final Release release;
|
||||
|
||||
Fixture() throws Exception {
|
||||
this("new");
|
||||
}
|
||||
|
||||
Fixture(String mode) throws Exception {
|
||||
Files.createDirectories(root.resolve("deploy"));
|
||||
Files.createDirectories(root.resolve("generated/deploy"));
|
||||
env.put("PROJECT_NAME", "recipe-demo");
|
||||
env.put("BACKEND_MODULE", "backend");
|
||||
env.put("FRONTEND_MODULE", "frontend");
|
||||
env.put("COMPOSE_PROJECT", "recipe-demo-test");
|
||||
Files.writeString(root.resolve("generated/deploy/remote.sh"), "");
|
||||
Files.writeString(root.resolve("deploy/compose.yaml"), "services: {}\n");
|
||||
Files.writeString(root.resolve("deploy/health.sh"), "application_health() { true; }\n");
|
||||
env.put("RELEASE_MODE", mode);
|
||||
if (mode.equals("retry")) env.put("RELEASE_VERSION", "v1.0.0");
|
||||
api = new FakeApi(env, events);
|
||||
if (mode.equals("retry")) api.tags.add(new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.0"), SHA, OLD));
|
||||
remote = new FakeRemote(root, env, events);
|
||||
release = new Release(root, env, api, remote, mode.equals("rollback") ? "rollback" : "deploy", "attempt");
|
||||
}
|
||||
|
||||
void withPrior() {
|
||||
remote.state = prior();
|
||||
api.tags.add(new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.1"), OLD, OLD));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
remote.close();
|
||||
Pipeline.deleteTree(root);
|
||||
}
|
||||
}
|
||||
|
||||
static void apiTests() throws Exception {
|
||||
test("NAS registry alias keeps the same repository and rejects unrelated registries", () -> {
|
||||
try (var f = new Fixture()) {
|
||||
f.env.put("DEPLOY_REGISTRY", "127.0.0.1:3000");
|
||||
CiTests.equal("127.0.0.1:3000/owner/recipe-demo-", f.release.prefix());
|
||||
f.env.put("DEPLOY_REGISTRY", "other-registry:3000");
|
||||
CiTests.failure(f.release::prefix);
|
||||
}
|
||||
});
|
||||
test("repeating remote tag pages fail closed instead of allocating from incomplete history", () -> {
|
||||
var api = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return List.of(Map.of("name", "v1.0.0", "id", OLD, "commit", Map.of("sha", SHA)));
|
||||
}
|
||||
};
|
||||
CiTests.failure(api::tags);
|
||||
});
|
||||
test("annotated release tag must identify the recorded version and exact SHA", () -> {
|
||||
var tag = new ReleaseApi.Tag(ReleaseVersion.parse("v1.0.0"), SHA, OLD);
|
||||
var api = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return Map.of("tag", "v1.0.0", "object", Map.of("sha", SHA));
|
||||
}
|
||||
};
|
||||
api.annotated(tag);
|
||||
var wrong = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return Map.of("tag", "v1.0.0", "object", Map.of("sha", OLD));
|
||||
}
|
||||
};
|
||||
CiTests.failure(() -> wrong.annotated(tag));
|
||||
});
|
||||
test("Gitea API eligibility validates SHA, event, branch, workflow path and successful conclusion", () -> {
|
||||
var valid = Map.<String, Object>of(
|
||||
"id",
|
||||
1,
|
||||
"head_sha",
|
||||
SHA,
|
||||
"event",
|
||||
"push",
|
||||
"head_branch",
|
||||
"main",
|
||||
"path",
|
||||
"build.yaml@refs/heads/main",
|
||||
"status",
|
||||
"completed",
|
||||
"conclusion",
|
||||
"success");
|
||||
for (String field : List.of("head_sha", "event", "head_branch", "path", "status", "conclusion")) {
|
||||
var invalid = new HashMap<>(valid);
|
||||
invalid.put(field, "wrong");
|
||||
var api = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return Map.of("workflow_runs", List.of(invalid));
|
||||
}
|
||||
};
|
||||
CiTests.failure(() -> api.verified(SHA));
|
||||
}
|
||||
var api = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return Map.of("workflow_runs", List.of(valid));
|
||||
}
|
||||
};
|
||||
CiTests.equal("1", api.verified(SHA));
|
||||
});
|
||||
test("remote tag pagination includes failed-release baseline and rejects ambiguous tag formats", () -> {
|
||||
var api = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return path.endsWith("page=1")
|
||||
? List.of(Map.of("name", "v1.2.3", "id", OLD, "commit", Map.of("sha", SHA)))
|
||||
: List.of();
|
||||
}
|
||||
};
|
||||
CiTests.equal(
|
||||
"v1.2.4",
|
||||
ReleaseVersion.next(
|
||||
api.tags().stream()
|
||||
.map(ReleaseApi.Tag::version)
|
||||
.toList(),
|
||||
"patch")
|
||||
.toString());
|
||||
var invalid = new ReleaseApi(environment()) {
|
||||
@Override
|
||||
Object request(String method, String path, Object body) {
|
||||
return List.of(Map.of("name", "1.0", "id", OLD, "commit", Map.of("sha", SHA)));
|
||||
}
|
||||
};
|
||||
CiTests.failure(invalid::tags);
|
||||
});
|
||||
}
|
||||
|
||||
static void shellTests() throws Exception {
|
||||
test("real remote shell atomically excludes a second owner and retains unresolved journal", () -> {
|
||||
Path directory = Files.createTempDirectory("release-shell-test-");
|
||||
try {
|
||||
String script = Files.readString(Path.of("generated/deploy/remote.sh"));
|
||||
var commands = new Commands(Duration.ofSeconds(30));
|
||||
var env = environment();
|
||||
var first = commands.execute(
|
||||
List.of("sh", "-c", script, "--", "lock", directory.toString(), "one"),
|
||||
directory,
|
||||
env,
|
||||
null,
|
||||
false);
|
||||
first.checked();
|
||||
CiTests.equal(
|
||||
1,
|
||||
commands.execute(
|
||||
List.of("sh", "-c", script, "--", "lock", directory.toString(), "two"),
|
||||
directory,
|
||||
env,
|
||||
null,
|
||||
false)
|
||||
.code());
|
||||
commands.execute(
|
||||
List.of("sh", "-c", script, "--", "pending", directory.toString(), "one"),
|
||||
directory,
|
||||
env,
|
||||
"{\"candidate\":\"test\"}",
|
||||
false)
|
||||
.checked();
|
||||
CiTests.equal(
|
||||
1,
|
||||
commands.execute(
|
||||
List.of("sh", "-c", script, "--", "unlock", directory.toString(), "one"),
|
||||
directory,
|
||||
env,
|
||||
null,
|
||||
false)
|
||||
.code());
|
||||
CiTests.equal(true, Files.exists(directory.resolve("pending.json")));
|
||||
commands.execute(
|
||||
List.of("sh", "-c", script, "--", "finish", directory.toString(), "one"),
|
||||
directory,
|
||||
env,
|
||||
null,
|
||||
false)
|
||||
.checked();
|
||||
commands.execute(
|
||||
List.of("sh", "-c", script, "--", "unlock", directory.toString(), "one"),
|
||||
directory,
|
||||
env,
|
||||
null,
|
||||
false)
|
||||
.checked();
|
||||
CiTests.equal(false, Files.exists(directory.resolve("deploy.lock")));
|
||||
CiTests.equal(true, Files.exists(directory.resolve("history/one.json")));
|
||||
} finally {
|
||||
Pipeline.deleteTree(directory);
|
||||
}
|
||||
});
|
||||
test("SSH shell quoting preserves metacharacters as data", () -> {
|
||||
String value = "quote' dollar$ backtick` newline\n";
|
||||
var commands = new Commands(Duration.ofSeconds(5));
|
||||
CiTests.equal(
|
||||
value,
|
||||
commands.execute(
|
||||
List.of("sh", "-c", "printf '%s' " + ReleaseRemote.quote(value)),
|
||||
Path.of("."),
|
||||
environment(),
|
||||
null,
|
||||
false)
|
||||
.output());
|
||||
});
|
||||
}
|
||||
}
|
||||
39
resources/ci/ReleaseVersion.java
Normal file
39
resources/ci/ReleaseVersion.java
Normal file
@@ -0,0 +1,39 @@
|
||||
import java.math.BigInteger;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
/** Strict stable release versions; arbitrary-size integers avoid overflow/reuse. */
|
||||
record ReleaseVersion(BigInteger major, BigInteger minor, BigInteger patch) implements Comparable<ReleaseVersion> {
|
||||
private static final Pattern FORMAT = Pattern.compile("v(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)");
|
||||
|
||||
static ReleaseVersion parse(String value) {
|
||||
var match = FORMAT.matcher(value);
|
||||
Pipeline.check(match.matches(), "Expected release version vMAJOR.MINOR.PATCH");
|
||||
return new ReleaseVersion(
|
||||
new BigInteger(match.group(1)), new BigInteger(match.group(2)), new BigInteger(match.group(3)));
|
||||
}
|
||||
|
||||
static ReleaseVersion next(List<ReleaseVersion> versions, String increment) {
|
||||
Pipeline.check(List.of("patch", "minor", "major").contains(increment), "Choose patch, minor or major");
|
||||
var highest = versions.stream().max(ReleaseVersion::compareTo);
|
||||
if (highest.isEmpty()) return parse("v1.0.0");
|
||||
var current = highest.get();
|
||||
return switch (increment) {
|
||||
case "major" -> new ReleaseVersion(current.major.add(BigInteger.ONE), BigInteger.ZERO, BigInteger.ZERO);
|
||||
case "minor" -> new ReleaseVersion(current.major, current.minor.add(BigInteger.ONE), BigInteger.ZERO);
|
||||
default -> new ReleaseVersion(current.major, current.minor, current.patch.add(BigInteger.ONE));
|
||||
};
|
||||
}
|
||||
|
||||
@Override
|
||||
public int compareTo(ReleaseVersion other) {
|
||||
int value = major.compareTo(other.major);
|
||||
if (value == 0) value = minor.compareTo(other.minor);
|
||||
return value == 0 ? patch.compareTo(other.patch) : value;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "v" + major + "." + minor + "." + patch;
|
||||
}
|
||||
}
|
||||
287
resources/ci/Setup.java
Normal file
287
resources/ci/Setup.java
Normal file
@@ -0,0 +1,287 @@
|
||||
import java.io.*;
|
||||
import java.net.*;
|
||||
import java.net.http.*;
|
||||
import java.nio.file.*;
|
||||
import java.nio.file.attribute.*;
|
||||
import java.security.MessageDigest;
|
||||
import java.time.Duration;
|
||||
import java.util.*;
|
||||
|
||||
class Setup {
|
||||
static Path home() {
|
||||
return Path.of(System.getProperty("user.home"));
|
||||
}
|
||||
|
||||
static String required(Map<String, String> env, String key) {
|
||||
String value = env.get(key);
|
||||
if (value == null || value.isBlank()) throw new IllegalArgumentException("Missing " + key);
|
||||
return value;
|
||||
}
|
||||
|
||||
static String secret(Map<String, String> env, String key) throws IOException {
|
||||
Path path = Path.of(required(env, key));
|
||||
Set<PosixFilePermission> permissions = Files.getPosixFilePermissions(path);
|
||||
if (permissions.stream()
|
||||
.anyMatch(p -> p.name().startsWith("GROUP_") || p.name().startsWith("OTHERS_")))
|
||||
throw new IllegalArgumentException(key + " must refer to an owner-only file (mode 600)");
|
||||
String token = Files.readString(path).strip();
|
||||
if (token.isEmpty()) throw new IllegalArgumentException(key + " is empty");
|
||||
return token;
|
||||
}
|
||||
|
||||
static class ApiFailure extends IOException {
|
||||
final int status;
|
||||
|
||||
ApiFailure(int status) {
|
||||
super("Gitea HTTP " + status);
|
||||
this.status = status;
|
||||
}
|
||||
}
|
||||
|
||||
static Object api(String server, String token, String method, String path, Object body) throws Exception {
|
||||
Pipeline.check(token.matches("[a-zA-Z0-9_-]+"), "Invalid token format");
|
||||
HttpRequest request = HttpRequest.newBuilder(URI.create(server.replaceAll("/+$", "") + path))
|
||||
.timeout(Duration.ofSeconds(30))
|
||||
.header("Authorization", "token " + token)
|
||||
.header("Content-Type", "application/json")
|
||||
.method(
|
||||
method,
|
||||
body == null
|
||||
? HttpRequest.BodyPublishers.noBody()
|
||||
: HttpRequest.BodyPublishers.ofString(Json.write(body)))
|
||||
.build();
|
||||
try (HttpClient client =
|
||||
HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(30)).build()) {
|
||||
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
|
||||
if (response.statusCode() < 200 || response.statusCode() >= 300)
|
||||
throw new ApiFailure(response.statusCode());
|
||||
return response.body().isBlank() ? null : Json.parse(response.body());
|
||||
}
|
||||
}
|
||||
|
||||
static void configureGitea(Map<String, String> env) throws Exception {
|
||||
String admin = secret(env, "GITEA_ADMIN_TOKEN_FILE"), registry = secret(env, "REGISTRY_TOKEN_FILE");
|
||||
String user = required(env, "REGISTRY_USER"), platform = required(env, "TARGET_PLATFORM");
|
||||
Pipeline.check(
|
||||
Set.of("linux/amd64", "linux/arm64").contains(platform),
|
||||
"Set TARGET_PLATFORM from the verified Synology architecture");
|
||||
String server = required(env, "GITEA_SERVER_URL"), base = "/api/v1/repos/" + required(env, "CI_REPOSITORY");
|
||||
api(server, admin, "PATCH", base, Map.of("has_actions", true));
|
||||
api(server, admin, "PUT", base + "/actions/secrets/REGISTRY_USER", Map.of("data", user));
|
||||
api(server, admin, "PUT", base + "/actions/secrets/REGISTRY_TOKEN", Map.of("data", registry));
|
||||
String variable = base + "/actions/variables/TARGET_PLATFORM";
|
||||
boolean exists;
|
||||
try {
|
||||
api(server, admin, "GET", variable, null);
|
||||
exists = true;
|
||||
} catch (ApiFailure e) {
|
||||
if (e.status != 404) throw e;
|
||||
exists = false;
|
||||
}
|
||||
api(server, admin, exists ? "PUT" : "POST", variable, Map.of("value", platform));
|
||||
Map<String, Object> response =
|
||||
Json.object(api(server, admin, "POST", base + "/actions/runners/registration-token", null));
|
||||
Path directory = home().resolve(".config/" + Recipe.project(env) + "-runner");
|
||||
privateDirectory(directory);
|
||||
privateWrite(directory.resolve("registration-token"), Objects.toString(response.get("token")));
|
||||
System.out.println("Actions, registry secrets and platform configured; runner token saved outside Git.");
|
||||
}
|
||||
|
||||
static void configureDocker(Path file, String registry) throws IOException {
|
||||
Map<String, Object> data =
|
||||
Files.exists(file) ? Json.object(Json.parse(Files.readString(file))) : new LinkedHashMap<>();
|
||||
Object entry = data.getOrDefault("insecure-registries", new ArrayList<>());
|
||||
if (!(entry instanceof List<?>))
|
||||
throw new IllegalArgumentException("Existing insecure-registries must be an array; unchanged");
|
||||
List<Object> registries = new ArrayList<>((List<?>) entry);
|
||||
if (registries.contains(registry)) {
|
||||
System.out.println("Internal registry already configured; no change");
|
||||
return;
|
||||
}
|
||||
registries.add(registry);
|
||||
data.put("insecure-registries", registries);
|
||||
Path absolute = file.toAbsolutePath(), parent = absolute.getParent();
|
||||
Files.createDirectories(parent);
|
||||
if (Files.exists(file)) {
|
||||
Path backup = file.resolveSibling(file.getFileName() + ".before-engineering");
|
||||
if (!Files.exists(backup)) Files.copy(file, backup, StandardCopyOption.COPY_ATTRIBUTES);
|
||||
}
|
||||
Path temp = Files.createTempFile(parent, "daemon-", ".json");
|
||||
try {
|
||||
Files.writeString(temp, Json.write(data) + "\n");
|
||||
if (Files.exists(file)) Files.setPosixFilePermissions(temp, Files.getPosixFilePermissions(file));
|
||||
Files.move(temp, absolute, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
|
||||
} finally {
|
||||
Files.deleteIfExists(temp);
|
||||
}
|
||||
System.out.println("Internal registry added; restart/apply Docker Engine configuration to activate");
|
||||
}
|
||||
|
||||
static void installRunner(Path root, Map<String, String> env, Commands commands, boolean prepareOnly)
|
||||
throws Exception {
|
||||
Pipeline.check(
|
||||
System.getProperty("os.name").equals("Linux")
|
||||
&& Set.of("amd64", "x86_64").contains(System.getProperty("os.arch")),
|
||||
"Installer targets the existing Linux x86_64 mini-PC");
|
||||
Path runnerDir = home().resolve(".local/share/" + Recipe.project(env) + "-runner"),
|
||||
configDir = home().resolve(".config/" + Recipe.project(env) + "-runner");
|
||||
Path bin = home().resolve(".local/bin"), units = home().resolve(".config/systemd/user");
|
||||
for (Path path : List.of(runnerDir, configDir, bin, units)) privateDirectory(path);
|
||||
for (String tool : List.of("node", "java", "mvn", "docker", "git", "systemctl")) executable(env, tool);
|
||||
String javaHome = System.getProperty("java.home");
|
||||
String version = commands.execute(List.of("node", "--version"), root, env, null, false)
|
||||
.checked();
|
||||
String[] parts = version.replaceFirst("^v", "").split("\\.");
|
||||
int major = Integer.parseInt(parts[0]), minor = Integer.parseInt(parts[1]), patch = Integer.parseInt(parts[2]);
|
||||
Pipeline.check(
|
||||
(major == 22 && (minor > 22 || minor == 22 && patch >= 3))
|
||||
|| (major == 24 && minor >= 15)
|
||||
|| major >= 26,
|
||||
"Supported Node version required");
|
||||
Path runner = bin.resolve("act_runner");
|
||||
if (!Files.isExecutable(runner)) {
|
||||
URI uri = URI.create("https://dl.gitea.com/act_runner/0.2.13/act_runner-0.2.13-linux-amd64");
|
||||
byte[] bytes;
|
||||
try (HttpClient client = HttpClient.newBuilder()
|
||||
.followRedirects(HttpClient.Redirect.NORMAL)
|
||||
.build()) {
|
||||
var response = client.send(
|
||||
HttpRequest.newBuilder(uri)
|
||||
.timeout(Duration.ofMinutes(2))
|
||||
.build(),
|
||||
HttpResponse.BodyHandlers.ofByteArray());
|
||||
Pipeline.check(response.statusCode() == 200, "Runner download failed");
|
||||
bytes = response.body();
|
||||
}
|
||||
String hash = HexFormat.of()
|
||||
.formatHex(MessageDigest.getInstance("SHA-256").digest(bytes));
|
||||
Pipeline.check(
|
||||
hash.equals("3acac8b506ac8cadc88a55155b5d6378f0fab0b8f62d1e0c0450f4ccd69733e2"),
|
||||
"Runner checksum mismatch");
|
||||
Files.write(runner, bytes);
|
||||
Files.setPosixFilePermissions(runner, PosixFilePermissions.fromString("rwxr-xr-x"));
|
||||
}
|
||||
String actual = commands.execute(List.of(runner.toString(), "--version"), root, env, null, false)
|
||||
.checked();
|
||||
Pipeline.check(actual.contains("v0.2.13"), "Expected runner v0.2.13");
|
||||
privateWrite(runnerDir.resolve("config.yaml"), Files.readString(root.resolve("ci/runner/config.yaml")));
|
||||
Files.copy(
|
||||
root.resolve("ci/runner/runner.service"),
|
||||
units.resolve(Recipe.project(env) + "-runner.service"),
|
||||
StandardCopyOption.REPLACE_EXISTING);
|
||||
Files.copy(
|
||||
root.resolve("generated/ci/runner/WaitForDocker.java"),
|
||||
runnerDir.resolve("WaitForDocker.java"),
|
||||
StandardCopyOption.REPLACE_EXISTING);
|
||||
String dockerHost = env.get("DOCKER_HOST");
|
||||
if (dockerHost == null || dockerHost.isBlank())
|
||||
dockerHost = commands.execute(
|
||||
List.of("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}"),
|
||||
root,
|
||||
env,
|
||||
null,
|
||||
false)
|
||||
.checked();
|
||||
String path = javaHome + "/bin:" + executable(env, "node").getParent() + ":"
|
||||
+ executable(env, "mvn").getParent() + ":/usr/local/bin:/usr/bin:/bin";
|
||||
privateWrite(
|
||||
configDir.resolve("environment"),
|
||||
"JAVA_HOME=" + quoted(javaHome) + "\nPATH=" + quoted(path) + "\nDOCKER_HOST=" + quoted(dockerHost)
|
||||
+ "\n");
|
||||
boolean desktop = dockerHost.endsWith("/.docker/desktop/docker.sock");
|
||||
Path overrides = units.resolve(Recipe.project(env) + "-runner.service.d");
|
||||
Path desktopOverride = overrides.resolve("10-docker-desktop.conf");
|
||||
if (desktop) {
|
||||
Files.createDirectories(overrides);
|
||||
Files.copy(
|
||||
root.resolve("ci/runner/docker-desktop.conf"),
|
||||
desktopOverride,
|
||||
StandardCopyOption.REPLACE_EXISTING);
|
||||
} else {
|
||||
Files.deleteIfExists(desktopOverride);
|
||||
}
|
||||
commands.execute(List.of("systemctl", "--user", "daemon-reload"), root, env, null, true)
|
||||
.checked();
|
||||
if (prepareOnly) {
|
||||
System.out.println("Runner installed; registration and startup still required.");
|
||||
return;
|
||||
}
|
||||
commands.execute(List.of("docker", "info"), root, env, null, false).checked();
|
||||
if (!Files.exists(runnerDir.resolve(".runner"))) {
|
||||
Path tokenPath = Path.of(env.getOrDefault(
|
||||
"RUNNER_REGISTRATION_TOKEN_FILE",
|
||||
configDir.resolve("registration-token").toString()));
|
||||
Map<String, String> tokenEnv = Map.of("RUNNER_TOKEN_FILE", tokenPath.toString());
|
||||
String token = secret(tokenEnv, "RUNNER_TOKEN_FILE");
|
||||
commands.execute(
|
||||
List.of(
|
||||
runner.toString(),
|
||||
"register",
|
||||
"--no-interactive",
|
||||
"--instance",
|
||||
required(env, "GITEA_SERVER_URL"),
|
||||
"--name",
|
||||
required(env, "RUNNER_LABEL"),
|
||||
"--labels",
|
||||
required(env, "RUNNER_LABEL") + ":host",
|
||||
"--token",
|
||||
token),
|
||||
runnerDir,
|
||||
env,
|
||||
null,
|
||||
true)
|
||||
.checked();
|
||||
Files.setPosixFilePermissions(runnerDir.resolve(".runner"), PosixFilePermissions.fromString("rw-------"));
|
||||
}
|
||||
if (desktop) {
|
||||
commands.execute(
|
||||
List.of("systemctl", "--user", "enable", "--now", "docker-desktop.service"),
|
||||
root,
|
||||
env,
|
||||
null,
|
||||
true)
|
||||
.checked();
|
||||
// Replace any previous default.target link with the desktop-login target.
|
||||
commands.execute(
|
||||
List.of("systemctl", "--user", "disable", Recipe.project(env) + "-runner.service"),
|
||||
root,
|
||||
env,
|
||||
null,
|
||||
true)
|
||||
.checked();
|
||||
}
|
||||
commands.execute(
|
||||
List.of("systemctl", "--user", "enable", "--now", Recipe.project(env) + "-runner.service"),
|
||||
root,
|
||||
env,
|
||||
null,
|
||||
true)
|
||||
.checked();
|
||||
System.out.println("Runner registered and started; keep the mini-PC and Docker available.");
|
||||
}
|
||||
|
||||
static Path executable(Map<String, String> env, String name) {
|
||||
for (String dir : env.getOrDefault("PATH", "").split(File.pathSeparator)) {
|
||||
Path path = Path.of(dir, name);
|
||||
if (Files.isRegularFile(path) && Files.isExecutable(path)) return path.toAbsolutePath();
|
||||
}
|
||||
throw new IllegalArgumentException("Missing executable: " + name);
|
||||
}
|
||||
|
||||
static String quoted(String value) {
|
||||
Pipeline.check(!value.contains("\n") && !value.contains("\r"), "Invalid service environment value");
|
||||
return "\"" + value.replace("\\", "\\\\").replace("\"", "\\\"") + "\"";
|
||||
}
|
||||
|
||||
static void privateDirectory(Path path) throws IOException {
|
||||
Files.createDirectories(path);
|
||||
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rwx------"));
|
||||
}
|
||||
|
||||
static void privateWrite(Path path, String text) throws IOException {
|
||||
if (!Files.exists(path))
|
||||
Files.createFile(path, PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString("rw-------")));
|
||||
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rw-------"));
|
||||
Files.writeString(path, text);
|
||||
}
|
||||
}
|
||||
35
resources/ci/runner/WaitForDocker.java
Normal file
35
resources/ci/runner/WaitForDocker.java
Normal file
@@ -0,0 +1,35 @@
|
||||
import java.time.Duration;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
/** Source-launcher startup probe; uses the service's configured DOCKER_HOST. */
|
||||
class WaitForDocker {
|
||||
public static void main(String[] args) throws Exception {
|
||||
long seconds = args.length == 0 ? 120 : Long.parseLong(args[0]);
|
||||
if (seconds < 1 || seconds > 300) throw new IllegalArgumentException("Timeout must be 1–300 seconds");
|
||||
long deadline = System.nanoTime() + Duration.ofSeconds(seconds).toNanos();
|
||||
System.out.println("Waiting for the configured Docker engine...");
|
||||
while (System.nanoTime() < deadline) {
|
||||
Process process = new ProcessBuilder("docker", "info", "--format", "{{.ServerVersion}}")
|
||||
.redirectOutput(ProcessBuilder.Redirect.DISCARD)
|
||||
.redirectError(ProcessBuilder.Redirect.DISCARD)
|
||||
.start();
|
||||
try {
|
||||
long remaining = Math.max(1, deadline - System.nanoTime());
|
||||
if (process.waitFor(Math.min(remaining, TimeUnit.SECONDS.toNanos(5)), TimeUnit.NANOSECONDS)
|
||||
&& process.exitValue() == 0) {
|
||||
System.out.println("Docker is available; starting the Gitea runner.");
|
||||
return;
|
||||
}
|
||||
} finally {
|
||||
if (process.isAlive()) {
|
||||
process.destroyForcibly();
|
||||
process.waitFor();
|
||||
}
|
||||
}
|
||||
long remaining = deadline - System.nanoTime();
|
||||
if (remaining > 0) TimeUnit.NANOSECONDS.sleep(Math.min(remaining, TimeUnit.SECONDS.toNanos(2)));
|
||||
}
|
||||
System.err.println("Docker did not become available within " + seconds + " seconds; runner not started.");
|
||||
System.exit(1);
|
||||
}
|
||||
}
|
||||
30
resources/deploy/enable-docker.sh
Normal file
30
resources/deploy/enable-docker.sh
Normal file
@@ -0,0 +1,30 @@
|
||||
#!/bin/sh
|
||||
# One-time DSM administrator action, run with sudo; does not restart any service.
|
||||
set -eu
|
||||
test "$(id -u)" = 0 || { echo 'Run this setup with sudo' >&2; exit 1; }
|
||||
account=${1:?Supply deployment account}
|
||||
docker=${2:?Supply absolute Docker executable}
|
||||
case "$account" in ''|*[!a-zA-Z0-9_-]*) exit 1;; esac
|
||||
case "$docker" in /*) ;; *) exit 1;; esac
|
||||
target=/etc/sudoers.d/"$account"
|
||||
rule="$account ALL=(root) NOPASSWD: $docker"
|
||||
test -x "$docker" && id "$account" >/dev/null
|
||||
test -d /etc/sudoers.d || { echo 'Missing sudoers.d; no changes made' >&2; exit 1; }
|
||||
if [ -e "$target" ]; then
|
||||
test "$(cat "$target")" = "$rule" || { echo 'Existing sudoers entry differs; no changes made' >&2; exit 1; }
|
||||
else
|
||||
umask 077
|
||||
temporary=$(mktemp /etc/sudoers.d/"$account".XXXXXX)
|
||||
trap 'rm -f "$temporary"' EXIT
|
||||
printf '%s\n' "$rule" > "$temporary"
|
||||
chmod 440 "$temporary"
|
||||
if command -v visudo >/dev/null 2>&1; then visudo -cf "$temporary"; fi
|
||||
# A dot in the temporary name keeps sudo from loading the uncommitted file.
|
||||
mv "$temporary" "$target"
|
||||
if ! sudo -n -u "$account" sudo -n "$docker" info --format '{{.OSType}}/{{.Architecture}}'; then
|
||||
rm "$target"
|
||||
echo 'Docker permission check failed; new sudoers entry removed' >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
echo 'Dedicated Docker sudo permission is configured; no services were restarted.'
|
||||
38
resources/deploy/provision.sh
Normal file
38
resources/deploy/provision.sh
Normal file
@@ -0,0 +1,38 @@
|
||||
#!/bin/sh
|
||||
# Account-level provisioning. Docker sudo must already be enabled separately.
|
||||
set -eu
|
||||
umask 077
|
||||
root=$1
|
||||
registry_user=$2
|
||||
registry=$3
|
||||
bind=$4
|
||||
port=$5
|
||||
docker=${6:?Supply the Docker executable path}
|
||||
test -x "$docker"
|
||||
sudo -n "$docker" compose version
|
||||
sudo -n "$docker" info --format '{{.OSType}}/{{.Architecture}}'
|
||||
test ! -L "$root"
|
||||
mkdir -p "$root" "$root/docker-auth"
|
||||
chmod 700 "$root" "$root/docker-auth"
|
||||
if [ ! -f "$root/production.env" ]; then
|
||||
# First provisioning must not take a port already occupied by another service.
|
||||
command -v netstat >/dev/null
|
||||
if netstat -lnt | awk -v port=":$port" '$4 ~ (port "$") { found=1 } END { exit !found }'; then
|
||||
echo 'Frontend port is already occupied; production configuration unchanged' >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'BIND_IP=%s\nFRONTEND_PORT=%s\nDOCKER_BIN=%s\nDOCKER_SUDO=1\nDOCKER_CONFIG=%s/docker-auth\n' \
|
||||
"$bind" "$port" "$docker" "$root" > "$root/production.env"
|
||||
chmod 600 "$root/production.env"
|
||||
else
|
||||
# Preserve existing config; refuse to silently rebind a configured deployment.
|
||||
. "$root/production.env"
|
||||
test "$BIND_IP" = "$bind" && test "$FRONTEND_PORT" = "$port" \
|
||||
&& test "$DOCKER_CONFIG" = "$root/docker-auth" || {
|
||||
echo 'Existing production.env differs; review it before provisioning' >&2
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
# Credential enters only via stdin. No password, environment or auth config is logged.
|
||||
sudo -n "$docker" --config "$root/docker-auth" login "$registry" --username "$registry_user" --password-stdin
|
||||
echo 'Production directory and dedicated registry login prepared; no containers were changed.'
|
||||
204
resources/deploy/remote.sh
Normal file
204
resources/deploy/remote.sh
Normal file
@@ -0,0 +1,204 @@
|
||||
#!/bin/sh
|
||||
# Invoked over a pinned SSH connection by ReleaseRemote. No secrets in arguments.
|
||||
# mkdir is the NAS-wide lock; a disconnected runner leaves it for explicit recovery.
|
||||
set -eu
|
||||
umask 077
|
||||
action=$1
|
||||
root=$2
|
||||
owner=$3
|
||||
shift 3
|
||||
die() { echo "Deployment: $*" >&2; exit 1; }
|
||||
case "$root" in /*) ;; *) die 'Deployment directory must be absolute';; esac
|
||||
case "$owner" in ''|*[!a-zA-Z0-9-]*) die 'Invalid lock owner';; esac
|
||||
test -d "$root" || die 'Provision the deployment directory first'
|
||||
cd "$root"
|
||||
lock="$root/deploy.lock"
|
||||
release="$root/releases/$owner"
|
||||
|
||||
if [ "$action" = lock ]; then
|
||||
test ! -e pending.json || die 'Unresolved deployment: follow manual recovery runbook'
|
||||
mkdir "$lock" 2>/dev/null || die 'Deployment locked: another run or manual recovery owns the NAS'
|
||||
printf '%s\n' "$owner" > "$lock/owner"
|
||||
# Check again after acquiring, covering races with another lock holder.
|
||||
if [ -e pending.json ]; then
|
||||
rm "$lock/owner"
|
||||
rmdir "$lock"
|
||||
die 'Unresolved deployment: follow manual recovery runbook'
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
test -f "$lock/owner" && [ "$(cat "$lock/owner")" = "$owner" ] || die 'Deployment lock ownership mismatch'
|
||||
|
||||
case "$action" in
|
||||
unlock)
|
||||
test ! -e pending.json || die 'Pending deployment retained; manual recovery required'
|
||||
rm "$lock/owner"
|
||||
rmdir "$lock"
|
||||
exit 0;;
|
||||
state)
|
||||
if [ -f state.json ]; then cat state.json; else printf '{}\n'; fi
|
||||
exit 0;;
|
||||
pending)
|
||||
test ! -e pending.json || die 'Deployment already pending'
|
||||
cat > pending.json.tmp
|
||||
mv pending.json.tmp pending.json
|
||||
exit 0;;
|
||||
commit)
|
||||
test -f pending.json || die 'No pending deployment'
|
||||
cat > state.json.tmp
|
||||
mv state.json.tmp state.json
|
||||
exit 0;;
|
||||
finish)
|
||||
test -f pending.json || die 'No pending deployment'
|
||||
mkdir -p history
|
||||
mv pending.json "history/$owner.json"
|
||||
exit 0;;
|
||||
event)
|
||||
mkdir -p history
|
||||
cat > "history/$owner-outcome.json.tmp"
|
||||
mv "history/$owner-outcome.json.tmp" "history/$owner-outcome.json"
|
||||
exit 0;;
|
||||
esac
|
||||
|
||||
test -f production.env || die 'Missing production.env; follow provisioning runbook'
|
||||
# This account-owned file is trusted shell configuration, never a workflow input.
|
||||
. "$root/production.env"
|
||||
: "${BIND_IP:?}" "${FRONTEND_PORT:?}" "${DOCKER_BIN:?}" "${DOCKER_CONFIG:?}"
|
||||
export DOCKER_CONFIG
|
||||
docker_cmd() {
|
||||
if [ "${DOCKER_SUDO:-1}" = 1 ]; then
|
||||
sudo -n "$DOCKER_BIN" --config "$DOCKER_CONFIG" "$@"
|
||||
else
|
||||
"$DOCKER_BIN" --config "$DOCKER_CONFIG" "$@"
|
||||
fi
|
||||
}
|
||||
: "${COMPOSE_PROJECT:?}" "${DEPLOY_ROLES:?}" "${TARGET_PLATFORM:?}"
|
||||
case "$COMPOSE_PROJECT" in ''|*[!a-z0-9_-]*) die 'Invalid Compose project';; esac
|
||||
case "$DEPENDENCY_SERVICES" in *[!a-zA-Z0-9_\ -]*) die 'Invalid dependency services';; esac
|
||||
case "$DEPLOY_ROLES" in backend|frontend|'backend frontend') ;; *) die 'Invalid deployment roles';; esac
|
||||
case "$TARGET_PLATFORM" in linux/amd64|linux/arm64) ;; *) die 'Invalid target platform';; esac
|
||||
compose() {
|
||||
docker_cmd compose --project-name "$COMPOSE_PROJECT" --project-directory "$root" \
|
||||
--env-file "$root/production.env" --env-file "$release/release.env" -f "$release/compose.yaml" "$@"
|
||||
}
|
||||
identity() {
|
||||
image=$1
|
||||
sha=$2
|
||||
test "$(docker_cmd image inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$image")" = "$sha" \
|
||||
|| die 'Image revision does not match selected SHA'
|
||||
test "$(docker_cmd image inspect --format '{{.Os}}/{{.Architecture}}' "$image")" = "$TARGET_PLATFORM" \
|
||||
|| die 'Image platform differs from target'
|
||||
}
|
||||
use_release() {
|
||||
case "$1" in ''|*[!a-zA-Z0-9-]*) die 'Invalid release directory';; esac
|
||||
release="$root/releases/$1"
|
||||
test -f "$release/compose.yaml" && test -f "$release/release.env" || die 'Missing retained release configuration'
|
||||
}
|
||||
health() {
|
||||
# Compare image IDs, revision labels and the container release labels.
|
||||
for role in $DEPLOY_ROLES; do
|
||||
id=$(compose ps -q "$role")
|
||||
if [ "$role" = backend ]; then expected=$BACKEND_IMAGE; else expected=$FRONTEND_IMAGE; fi
|
||||
test "$(docker_cmd inspect --format '{{.State.Running}}' "$id")" = true || return 1
|
||||
test "$(docker_cmd inspect --format '{{.Image}}' "$id")" = "$(docker_cmd image inspect --format '{{.Id}}' "$expected")" || return 1
|
||||
test "$(docker_cmd inspect --format '{{index .Config.Labels "nl.agilecc.revision"}}' "$id")" = "$RELEASE_SHA" || return 1
|
||||
test "$(docker_cmd inspect --format '{{index .Config.Labels "nl.agilecc.release"}}' "$id")" = "$RELEASE_VERSION" || return 1
|
||||
done
|
||||
if [ -f "$release/health.sh" ]; then . "$release/health.sh"; fi
|
||||
application_health
|
||||
}
|
||||
|
||||
case "$action" in
|
||||
preflight)
|
||||
for secret in ${REQUIRED_SECRETS:-}; do
|
||||
case "$secret" in ''|*[!A-Z0-9_]*) die 'Invalid required secret name';; esac
|
||||
eval 'value=${'"$secret"':-}'
|
||||
test -n "$value" || die "Missing $secret in production.env"
|
||||
done
|
||||
command -v timeout >/dev/null || die 'timeout is required for bounded remote operations'
|
||||
command -v curl >/dev/null || die 'curl is required'
|
||||
test -x "$DOCKER_BIN" || die 'Docker executable missing'
|
||||
docker_cmd compose version >/dev/null
|
||||
actual=$(docker_cmd info --format '{{.OSType}}/{{.Architecture}}')
|
||||
case "$actual" in linux/x86_64) actual=linux/amd64;; linux/aarch64) actual=linux/arm64;; esac
|
||||
test "$actual" = "$TARGET_PLATFORM" || die 'Docker platform differs from target'
|
||||
test -d "$DOCKER_CONFIG" && test -f "$DOCKER_CONFIG/config.json" || die 'Provision registry pull login first'
|
||||
# Explicit private IPv4 binding; prevent accidental 0.0.0.0 publication.
|
||||
case "$BIND_IP" in
|
||||
10.*|192.168.*|172.16.*|172.17.*|172.18.*|172.19.*|172.2[0-9].*|172.30.*|172.31.*|127.*) ;;
|
||||
*) die 'Bind frontend to a private IPv4 address';;
|
||||
esac
|
||||
case "$FRONTEND_PORT" in ''|*[!0-9]*) die 'Invalid frontend port';; esac
|
||||
test "$FRONTEND_PORT" -ge 1024 && test "$FRONTEND_PORT" -le 65535 || die 'Use port 1024..65535'
|
||||
;;
|
||||
stage)
|
||||
mkdir -p "$release"
|
||||
test ! -e "$release/compose.yaml" || die 'Attempt configuration already exists'
|
||||
cat > "$release/compose.yaml"
|
||||
;;
|
||||
stage-health)
|
||||
test -f "$release/compose.yaml" || die 'Stage Compose first'
|
||||
test ! -e "$release/health.sh" || die 'Attempt health configuration already exists'
|
||||
cat > "$release/health.sh"
|
||||
;;
|
||||
prepare)
|
||||
version=$1; sha=$2; prefix=$3
|
||||
backend='-'; frontend='-'
|
||||
for role in $DEPLOY_ROLES; do
|
||||
repository="$prefix$role"
|
||||
image="$repository:$sha"
|
||||
docker_cmd pull --platform "$TARGET_PLATFORM" "$image" >&2
|
||||
identity "$image" "$sha"
|
||||
digest=$(docker_cmd image inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$image" \
|
||||
| awk -v prefix="$repository@sha256:" 'index($0,prefix)==1 { print; exit }')
|
||||
test -n "$digest" || die 'Missing registry digest'
|
||||
if [ "$role" = backend ]; then backend=$digest; else frontend=$digest; fi
|
||||
done
|
||||
printf 'RELEASE_VERSION=%s\nRELEASE_SHA=%s\nBACKEND_IMAGE=%s\nFRONTEND_IMAGE=%s\nBIND_IP=%s\nFRONTEND_PORT=%s\n' \
|
||||
"$version" "$sha" "$backend" "$frontend" "$BIND_IP" "$FRONTEND_PORT" > "$release/release.env"
|
||||
printf 'COMPOSE_PROJECT=%s\nDEPLOY_ROLES="%s"\nDEPENDENCY_SERVICES="%s"\nTARGET_PLATFORM=%s\n' \
|
||||
"$COMPOSE_PROJECT" "$DEPLOY_ROLES" "${DEPENDENCY_SERVICES:-}" "$TARGET_PLATFORM" >> "$release/release.env"
|
||||
compose config --quiet
|
||||
printf 'S6_BACKEND=%s\nS6_FRONTEND=%s\n' "$backend" "$frontend"
|
||||
;;
|
||||
ensure)
|
||||
use_release "$1"
|
||||
. "$release/release.env"
|
||||
test "$RELEASE_VERSION" = "$2" && test "$RELEASE_SHA" = "$3" \
|
||||
&& test "$BACKEND_IMAGE" = "$4" && test "$FRONTEND_IMAGE" = "$5" \
|
||||
|| die 'Retained configuration differs from recorded successful release'
|
||||
for image in "$BACKEND_IMAGE" "$FRONTEND_IMAGE"; do
|
||||
[ "$image" != - ] || continue
|
||||
docker_cmd pull --platform "$TARGET_PLATFORM" "$image" >&2
|
||||
identity "$image" "$RELEASE_SHA"
|
||||
done
|
||||
compose config --quiet
|
||||
;;
|
||||
apply)
|
||||
mkdir "$lock/operation" 2>/dev/null || die 'A remote replacement is still active; manual recovery required'
|
||||
trap 'rmdir "$lock/operation"' EXIT
|
||||
use_release "$1"
|
||||
. "$release/release.env"
|
||||
health_seconds=${2:-280}
|
||||
case "$health_seconds" in ''|*[!0-9]*) die 'Invalid readiness window';; esac
|
||||
test "$health_seconds" -ge 1 && test "$health_seconds" -le 280 || die 'Invalid readiness window'
|
||||
end=$(( $(date +%s) + health_seconds ))
|
||||
# Always recreate the frontend too: nginx must resolve the new backend IP.
|
||||
# Recreate application containers only; the database and volume survive releases.
|
||||
if [ -n "${DEPENDENCY_SERVICES:-}" ]; then
|
||||
compose up -d --no-build --pull missing --timeout 20 $DEPENDENCY_SERVICES
|
||||
fi
|
||||
compose up -d --no-deps --no-build --pull never --force-recreate --timeout 20 $DEPLOY_ROLES
|
||||
while ! health; do
|
||||
test "$(date +%s)" -lt "$end" || die 'Readiness checks timed out'
|
||||
sleep 1
|
||||
done
|
||||
echo 'S6_HEALTHY'
|
||||
;;
|
||||
diagnose)
|
||||
use_release "$1"
|
||||
compose ps
|
||||
compose logs --no-color --tail 200
|
||||
;;
|
||||
*) die 'Unknown remote deployment operation';;
|
||||
esac
|
||||
19
resources/docs/backend-review.md
Normal file
19
resources/docs/backend-review.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Backend review companion
|
||||
|
||||
Use alongside `generated/skills/code-review/SKILL.md` and `generated/CODING_STANDARDS.md`.
|
||||
|
||||
Review changed code for intent-revealing comments, missing public-contract Javadoc,
|
||||
appropriate record use, defensive copying of mutable record components, and
|
||||
compatibility with APIs, JSON and persistence. Assess ambiguous `var` initializers
|
||||
and framework/lifecycle exceptions to field finality. Confirm architecture and
|
||||
injection decisions make sense for production and tests. Check NIO API preference,
|
||||
varargs for trailing array parameters, and Java 25 IO use for direct console
|
||||
input/output; retain System.err for diagnostics. Require Javadoc for public
|
||||
classes and methods beyond basic POJO/data-carrier behavior.
|
||||
|
||||
Read standards reports even when verification passes. Assess every review signal
|
||||
in changed code, including fan-out, public methods and record component counts.
|
||||
Record a rationale for retained signals in application validation evidence or the change description.
|
||||
Check narrowly scoped waivers against recorded maintainer agreement and reason.
|
||||
Never introduce blanket exclusions or weaken thresholds to pass a build.
|
||||
Do not repeat mechanical checks already enforced by tooling; verify their result.
|
||||
23
resources/skills/README.md
Normal file
23
resources/skills/README.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Engineering skills
|
||||
|
||||
Reusable agent instructions for consuming repositories. Invoke them explicitly in the
|
||||
conversation, for example:
|
||||
|
||||
> Gebruik grill-me om de JSON-import en -export met mij uit te werken.
|
||||
|
||||
## Installed skills
|
||||
|
||||
| Skill | Purpose |
|
||||
| --- | --- |
|
||||
| [code-review](code-review/SKILL.md) | Review changes against documented coding standards and the originating spec. |
|
||||
| [grill-me](grill-me/SKILL.md) | Entry point for reviewing a plan or design through questions. |
|
||||
| [grilling](grilling/SKILL.md) | Interview instructions required by `grill-me`. |
|
||||
|
||||
The skills are copied from [Matt Pocock's skills.sh listing](https://skills.sh/mattpocock/skills/grill-me)
|
||||
and its [upstream repository](https://github.com/mattpocock/skills).
|
||||
The exact upstream revisions and paths are recorded in [upstream.json](upstream.json).
|
||||
The upstream [MIT license](LICENSE.mattpocock) is included.
|
||||
|
||||
These project-local files are available to the agent from the next conversation
|
||||
turn through the instructions in the consuming project's root `AGENTS.md`. They are not installed
|
||||
globally. Update upstream sources in engineering, retaining licenses and revision records, then release a new engineering version. Consumers regenerate their pinned copies.
|
||||
24
resources/skills/adr-brainstorm/ATTRIBUTION.md
Normal file
24
resources/skills/adr-brainstorm/ATTRIBUTION.md
Normal file
@@ -0,0 +1,24 @@
|
||||
# License and attribution
|
||||
|
||||
ADR brainstorm is maintained by Wanja Krah and is distributed under the MIT
|
||||
License. See [LICENSE.wanjakrah](LICENSE.wanjakrah) for the license covering
|
||||
Wanja Krah's original contributions.
|
||||
|
||||
The design-tree approach and related interviewing guidance in `SKILL.md` were
|
||||
adapted from Matt Pocock's `grilling` skill, invoked by his `grill-me` skill.
|
||||
This includes tracking dependent decisions, settling prerequisites before
|
||||
asking dependent questions, investigating accessible facts, and leaving
|
||||
decisions with the user. The wording and workflow have been adapted for ADR
|
||||
brainstorming; this is not an unmodified copy of the upstream skill.
|
||||
|
||||
Upstream source:
|
||||
- Repository: https://github.com/mattpocock/skills
|
||||
- Revision: `59db609eb1e18aaf59b63284d0e52255aff0c63c`
|
||||
- Paths: `skills/productivity/grilling` and `skills/productivity/grill-me`
|
||||
- License: MIT, copyright (c) 2026 Matt Pocock
|
||||
|
||||
Matt Pocock's full copyright and license notice is preserved in
|
||||
[LICENSE.mattpocock](LICENSE.mattpocock) for the adapted material. Keep both
|
||||
license files with redistributed copies or substantial portions of this skill.
|
||||
The two notices preserve attribution under the same MIT terms; they do not
|
||||
introduce an additional license restriction or imply upstream endorsement.
|
||||
21
resources/skills/adr-brainstorm/LICENSE.mattpocock
Normal file
21
resources/skills/adr-brainstorm/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Matt Pocock
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
21
resources/skills/adr-brainstorm/LICENSE.wanjakrah
Normal file
21
resources/skills/adr-brainstorm/LICENSE.wanjakrah
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Wanja Krah
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
103
resources/skills/adr-brainstorm/SKILL.md
Normal file
103
resources/skills/adr-brainstorm/SKILL.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
name: adr-brainstorm
|
||||
description: "Brainstorm with the user about a new ADR. Use when the user wants to stress-test their thinking, or uses any combination of 'adr' and 'brainstorm' in their trigger phrase."
|
||||
---
|
||||
|
||||
An ADR is an Architecture Decision Record. Such a record contains a decision on the architecture or design for the application landscape the user is maintaining.
|
||||
If an ADR-folder exists, read the README.md of that folder to get a full understanding of any specific rules and properties of how the user works with ADR's that might not be listed in this Skill.
|
||||
|
||||
The goal of the ADR is to provide an architectural solution or a design solution for a given context. Therefor, two to four different approaches must be worked out with their pro's and con's, from which the user should select one as their preferred solution.
|
||||
|
||||
Below, the brainstorm session is mapped out in multiple steps. Each step must be completed succesfully, before continuing to the next step. Success means reaching a shared understanding and whatever the step entitles as its success criteria.
|
||||
|
||||
If the user, or the relevant ADR, indicates that an ADR is to supersede another ADR, read that ADR first.
|
||||
|
||||
## Explore decisions with the user
|
||||
|
||||
### Follow a design tree
|
||||
|
||||
Map the architectural question as a design tree: each decision may
|
||||
introduce dependent decisions. Track which decisions are agreed,
|
||||
open, awaiting factual investigation, or explicitly deferred.
|
||||
|
||||
Ask dependent questions only after their prerequisites are settled.
|
||||
Revisit earlier decisions when new evidence changes their assumptions,
|
||||
and explain which dependent decisions need reconsideration.
|
||||
|
||||
Keep the tree within the ADR's agreed scope. Suggest follow-up work
|
||||
for independent topics rather than continually expanding the ADR.
|
||||
A deferred topic must have a clear boundary and must not leave the
|
||||
main decision ambiguous.
|
||||
|
||||
Finish when the user has agreed on the architectural decision and its material tradeoffs. Identify remaining implementation questions as follow-up work. Do not leave questions open that could materially change the chosen approach.
|
||||
|
||||
### Establish facts before asking for decisions
|
||||
|
||||
Finding accessible facts is the agent's responsibility. Inspect relevant
|
||||
code, configuration, specifications, ADRs, and authoritative documentation
|
||||
before asking the user questions those sources can answer.
|
||||
|
||||
Distinguish observed facts, assumptions, recommendations, and decisions.
|
||||
Cite sources for material factual claims and explain uncertainty where
|
||||
evidence is unavailable or conflicting. Ask the user for requirements,
|
||||
priorities, constraints, and knowledge unavailable from accessible sources.
|
||||
|
||||
Evaluate alternatives against the agreed requirements. Present credible
|
||||
alternatives fairly, including drawbacks of the recommended approach.
|
||||
Do not infer that an option is unsuitable merely because the current
|
||||
implementation uses something else.
|
||||
|
||||
### Keep the conversation collaborative
|
||||
|
||||
Use short rounds, usually one main question or a few closely related
|
||||
questions. Number questions when asking more than one. Do not ask every
|
||||
available question at once if that would hinder discussion.
|
||||
|
||||
For decision questions, normally provide a recommended answer and its
|
||||
reason. Where natural, phrase the question so "yes" clearly accepts that
|
||||
recommendation. Make its scope explicit; avoid combining independent
|
||||
decisions into one yes/no question.
|
||||
|
||||
Use open questions when discovering requirements or when a recommendation
|
||||
would prematurely narrow the discussion. Allow disagreement, alternatives,
|
||||
and back-and-forth exploration before seeking a decision.
|
||||
|
||||
### Leave architectural decisions with the user
|
||||
|
||||
The agent investigates, explains tradeoffs, challenges assumptions, and
|
||||
recommends. The user selects the architectural approach and accepts its
|
||||
material tradeoffs.
|
||||
|
||||
Do not treat silence, a factual answer, or agreement on one decision as
|
||||
approval of other decisions. Preserve earlier agreements without repeatedly
|
||||
requesting confirmation unless their meaning or supporting facts change.
|
||||
|
||||
At meaningful milestones, summarize the agreed decisions and remaining
|
||||
questions. Before finalizing the ADR, confirm shared understanding of the
|
||||
complete proposal. Keep recommendations and unresolved assumptions out of
|
||||
the Decisions section until the user accepts them.
|
||||
|
||||
### Guide the process through steps
|
||||
|
||||
Coöperate with the user via the following steps. Communicate each step in which step you are, and what the step entails. Use the template adr-template.md as a template for the final ADR.
|
||||
|
||||
The _first step_ is to get to a shared understanding of the probived context and the problem area the user is in need for a decision. Success for step one means you can describe the context in one to fifty sentences.
|
||||
|
||||
The _second step_ is to get to two to four different architectural solutions or design solutions for the described context. Every solution needs a short description of one to fifty sentences and a list of one to ten pro's and one to ten con's. allow discussion of consequences when they affect the choice. Record them formally in _step four_, but don’t postpone discovering migration costs or operational burdens until after selecting an option.
|
||||
Success for step two means there are more than one credible solutions with a set of relevant pro's and con's by which a user is able to choose, or not choose that solution for the next step.
|
||||
|
||||
The _third step_ is to pick the best solution. Ths user must decide what solution he prefers. The choosen solution will be presented as the ADR decision. A decision may branch into a small tree of related decisions, small meaning one to five levels deep. Make the architectural decision, its scope, and its material tradeoffs unambiguous; record remaining implementation questions as follow-up work. Success of step three means there is a shared understanding of the choosen solution and what it entails.
|
||||
|
||||
The _fourth step_ is to make clear what the consequences will be for the application landscape. Success of step four means there is a clear understanding of the changes that should be made to software landscape and the way futurue work should be implemented.
|
||||
|
||||
The _fifth step_ is fill out the other properties of the ADR that is required by the template. Succes of step five means the ID of the ADR is unique, the title is short and comprehandible, and gives the user an overall idea of what the ADR is about. If the ADR supersedes another ADR, that is listed as well.
|
||||
Set the Expiration Date of when the ADR should be reviewd. This should be a relative date, to be made absolute by the user as soon as the user moves the ADR to APPROVED.
|
||||
Finally, set the status to REFINED. It's up to the user to set the Status to either APPROVED - meaning the ADR is accepted and to be applied for all future work - or REJECTED - meaning the ADR is not accepted and should not be taken in consideration.
|
||||
|
||||
When updating the ADR with the result of the brainstorm, make it both easy to glanse through, as to read in full, meaning:
|
||||
- stick to the template
|
||||
- keep to the formatting as is custom for markdown-files
|
||||
- high use of headers and sub-headers
|
||||
- high use of bullit points, lists, and tables where applicable
|
||||
- keep sentences short and precise where possible
|
||||
- create images with ASCII-art where possible
|
||||
20
resources/skills/adr-brainstorm/adr-template.md
Normal file
20
resources/skills/adr-brainstorm/adr-template.md
Normal file
@@ -0,0 +1,20 @@
|
||||
**ID:** <insert ADR followed by a unique number> \
|
||||
**Title:** <insert short and descriptive title> \
|
||||
**Supersedes:** - \
|
||||
**Status:** NEW \
|
||||
**Expiration Date:** to be determined
|
||||
|
||||
## Context
|
||||
To be determined
|
||||
|
||||
## Decisions
|
||||
To be determined
|
||||
|
||||
## Options considered
|
||||
To be determined
|
||||
|
||||
## Consequences
|
||||
To be determined
|
||||
|
||||
## References
|
||||
<insert a list of references consulted, along with a timestamp of when it was consulted>
|
||||
3
resources/skills/adr-brainstorm/agents/openai.yaml
Normal file
3
resources/skills/adr-brainstorm/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "ADR Brainstorm"
|
||||
short_description: "Brainstorm with the user on an ADR to get it from NEW to REFINED"
|
||||
21
resources/skills/code-review/LICENSE.mattpocock
Normal file
21
resources/skills/code-review/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Matt Pocock
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
89
resources/skills/code-review/SKILL.md
Normal file
89
resources/skills/code-review/SKILL.md
Normal file
@@ -0,0 +1,89 @@
|
||||
---
|
||||
name: code-review
|
||||
description: "Review the changes since a fixed point (commit, branch, tag, or merge-base) along two axes: Standards (does the code follow this repo's documented coding standards?) and Spec (does the code match what the originating issue/spec asked for?). Runs both reviews in parallel sub-agents and reports them side by side. Use when the user wants to review a branch, a PR, work-in-progress changes, or asks to \"review since X\"."
|
||||
---
|
||||
|
||||
Two-axis review of the diff between `HEAD` and a fixed point the user supplies:
|
||||
|
||||
- **Standards**: does the code conform to this repo's documented coding standards?
|
||||
- **Spec**: does the code faithfully implement the originating issue / spec?
|
||||
|
||||
Both axes run as **parallel sub-agents** so they don't pollute each other's context, then this skill aggregates their findings. Start every sub-agent with a fresh context window.
|
||||
|
||||
The issue tracker should have been provided to you. If not, tell the user to run `/setup-matt-pocock-skills`.
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Pin the fixed point
|
||||
|
||||
Whatever the user said is the fixed point (a commit SHA, branch name, tag, `main`, `HEAD~5`, etc.). If they didn't specify one, ask for it.
|
||||
|
||||
Capture the diff command once: `git diff <fixed-point>...HEAD` (three-dot, so the comparison is against the merge-base). Also note the list of commits via `git log <fixed-point>..HEAD --oneline`.
|
||||
|
||||
Before going further, confirm the fixed point resolves (`git rev-parse <fixed-point>`) and the diff is non-empty. A bad ref or empty diff should fail here, not inside two parallel sub-agents.
|
||||
|
||||
### 2. Identify the spec source
|
||||
|
||||
Look for the originating spec, in this order:
|
||||
|
||||
1. Issue references in the commit messages (`#123`, `Closes #45`, GitLab `!67`, etc.), fetched via the workflow in the tracker doc.
|
||||
2. A path the user passed as an argument.
|
||||
3. A spec file under `docs/`, `specs/`, or `.scratch/` matching the branch name or feature.
|
||||
4. If nothing is found, ask the user where the spec is. If they say there isn't one, the **Spec** sub-agent will skip and report "no spec available".
|
||||
|
||||
### 3. Identify the standards sources
|
||||
|
||||
Search the repo for every file that documents how code should be written. When `CODING_STANDARDS.md` or `CONTRIBUTING.md` exists, it must be on the list.
|
||||
|
||||
On top of whatever the repo documents, the Standards axis always carries the **smell baseline** below: a fixed set of Fowler code smells (_Refactoring_, ch.3) that applies even when a repo documents nothing. Two rules bind it:
|
||||
|
||||
- **The repo overrides.** A documented repo standard always wins; where it endorses something the baseline would flag, suppress the smell.
|
||||
- **Always a judgement call.** Each smell is a labelled heuristic ("possible Feature Envy"), never a hard violation. Like any standard here, skip anything tooling already enforces.
|
||||
|
||||
Each smell reads *what it is* → *how to fix*; match it against the diff:
|
||||
|
||||
- **Mysterious Name**: a function, variable, or type whose name doesn't reveal what it does or holds. → rename it; if no honest name comes, the design's murky.
|
||||
- **Duplicated Code**: the same logic shape appears in more than one hunk or file in the change. → extract the shared shape, call it from both.
|
||||
- **Feature Envy**: a method that reaches into another object's data more than its own. → move the method onto the data it envies.
|
||||
- **Data Clumps**: the same few fields or params keep travelling together (a type wanting to be born). → bundle them into one type, pass that.
|
||||
- **Primitive Obsession**: a primitive or string standing in for a domain concept that deserves its own type. → give the concept its own small type.
|
||||
- **Repeated Switches**: the same `switch`/`if`-cascade on the same type recurs across the change. → replace with polymorphism, or one map both sites share.
|
||||
- **Shotgun Surgery**: one logical change forces scattered edits across many files in the diff. → gather what changes together into one module.
|
||||
- **Divergent Change**: one file or module is edited for several unrelated reasons. → split so each module changes for one reason.
|
||||
- **Speculative Generality**: abstraction, parameters, or hooks added for needs the spec doesn't have. → delete it; inline back until a real need shows.
|
||||
- **Message Chains**: long `a.b().c().d()` navigation the caller shouldn't depend on. → hide the walk behind one method on the first object.
|
||||
- **Middle Man**: a class or function that mostly just delegates onward. → cut it, call the real target direct.
|
||||
- **Refused Bequest**: a subclass or implementer that ignores or overrides most of what it inherits. → drop the inheritance, use composition.
|
||||
|
||||
### 4. Spawn both sub-agents in parallel
|
||||
|
||||
Issue both sub-agent calls together, in the foreground, and aggregate the reports they return.
|
||||
|
||||
**Standards sub-agent prompt** should include:
|
||||
|
||||
- The full diff command and commit list.
|
||||
- The list of standards-source files you found in step 3, **plus the smell baseline from step 3** pasted in full (the sub-agent has no other access to it).
|
||||
- The brief: "Report, per file/hunk where relevant, (a) every place the diff violates a documented standard: cite the standard (file + the rule); and (b) any baseline smell you spot: name it and quote the hunk. Distinguish hard violations from judgement calls: documented-standard breaches can be hard, but baseline smells are always judgement calls, and a documented repo standard overrides the baseline. Skip anything tooling enforces. Under 400 words."
|
||||
|
||||
**Spec sub-agent prompt** should include:
|
||||
|
||||
- The diff command and commit list.
|
||||
- The path or fetched contents of the spec.
|
||||
- The brief: "Report: (a) requirements the spec asked for that are missing or partial; (b) behaviour in the diff that wasn't asked for (scope creep); (c) requirements that look implemented but where the implementation looks wrong. Quote the spec line for each finding. Under 400 words."
|
||||
|
||||
If the spec is missing, skip the Spec sub-agent and note this in the final report.
|
||||
|
||||
### 5. Aggregate
|
||||
|
||||
Present the two reports under `## Standards` and `## Spec` headings, verbatim or lightly cleaned. Do **not** merge or rerank findings, because the two axes are deliberately separate (see _Why two axes_).
|
||||
|
||||
End with a one-line summary: total findings per axis, and the worst issue _within each axis_ (if any). Don't pick a single winner across axes: that's the reranking the separation exists to prevent.
|
||||
|
||||
## Why two axes
|
||||
|
||||
A change can pass one axis and fail the other:
|
||||
|
||||
- Code that follows every standard but implements the wrong thing → **Standards pass, Spec fail.**
|
||||
- Code that does exactly what the issue asked but breaks the project's conventions → **Spec pass, Standards fail.**
|
||||
|
||||
Reporting them separately stops one axis from masking the other.
|
||||
3
resources/skills/code-review/agents/openai.yaml
Normal file
3
resources/skills/code-review/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "Code Review"
|
||||
short_description: "Review a diff on standards and spec"
|
||||
21
resources/skills/grill-me/LICENSE.mattpocock
Normal file
21
resources/skills/grill-me/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Matt Pocock
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
7
resources/skills/grill-me/SKILL.md
Normal file
7
resources/skills/grill-me/SKILL.md
Normal file
@@ -0,0 +1,7 @@
|
||||
---
|
||||
name: grill-me
|
||||
description: A relentless interview to sharpen a plan or design.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
Call the Skill tool with "grilling".
|
||||
5
resources/skills/grill-me/agents/openai.yaml
Normal file
5
resources/skills/grill-me/agents/openai.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
interface:
|
||||
display_name: "Grill Me"
|
||||
short_description: "Sharpen a plan through interview"
|
||||
policy:
|
||||
allow_implicit_invocation: false
|
||||
21
resources/skills/grilling/LICENSE.mattpocock
Normal file
21
resources/skills/grilling/LICENSE.mattpocock
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Matt Pocock
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
30
resources/skills/grilling/SKILL.md
Normal file
30
resources/skills/grilling/SKILL.md
Normal file
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: grilling
|
||||
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
|
||||
---
|
||||
|
||||
Interview the user relentlessly until you reach a shared understanding. Map this as a **design tree**: every decision branches into the decisions that hang off it.
|
||||
|
||||
Work the tree in **rounds**. The **frontier** is every decision whose prerequisites are already settled: the questions you can ask _now_ without guessing at answers you haven't heard yet. Ask the whole frontier in one round: number each question and give your recommended answer. Then wait for the user's answers before the next round.
|
||||
|
||||
Format a round like so:
|
||||
|
||||
```
|
||||
❓ **Q1** - **<question title>**: <question body, might be multiple paragraphs, including multiple choices>
|
||||
|
||||
➡️ <your recommended answer>
|
||||
|
||||
---
|
||||
|
||||
❓ **Q2** - **<question title>**: <question body, might be multiple paragraphs, including multiple choices>
|
||||
|
||||
➡️ <your recommended answer>
|
||||
```
|
||||
|
||||
Word each question so "yes" accepts your recommended answer.
|
||||
|
||||
Each round the user answers reshapes the tree: settled decisions push the frontier outward and unblock questions that depended on them. Recompute the frontier and ask the next round. A question whose answer depends on another question still open in this round belongs to a _later_ round, not this one.
|
||||
|
||||
Finding _facts_ is your job, never the user's. When a frontier question needs a fact from the environment (filesystem, tools, etc.), dispatch a sub-agent to find it; don't ask the user for anything you could look up yourself. Don't block on it: a running exploration is an unsettled prerequisite, so only the questions downstream of it wait for the sub-agent to report; ask the rest of the frontier now. The _decisions_ are the user's: put each to them and wait.
|
||||
|
||||
The session is done when the frontier is empty: every branch of the design tree visited, nothing left silently assumed. Do not act on it until the user confirms you have reached a shared understanding.
|
||||
3
resources/skills/grilling/agents/openai.yaml
Normal file
3
resources/skills/grilling/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "Grilling"
|
||||
short_description: "Stress-test thinking a round of questions at a time"
|
||||
14
resources/skills/upstream.json
Normal file
14
resources/skills/upstream.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"repository": "https://github.com/mattpocock/skills",
|
||||
"commit": "59db609eb1e18aaf59b63284d0e52255aff0c63c",
|
||||
"source": "https://skills.sh/mattpocock/skills/grill-me",
|
||||
"skills": {
|
||||
"grill-me": "skills/productivity/grill-me",
|
||||
"grilling": "skills/productivity/grilling",
|
||||
"code-review": {
|
||||
"path": "skills/engineering/code-review",
|
||||
"commit": "a496f5513d4a4cf889bc48e189e579f4b0bdf6aa",
|
||||
"source": "https://www.skills.sh/mattpocock/skills/code-review"
|
||||
}
|
||||
}
|
||||
}
|
||||
30
resources/tooling/coverage/Coverage.java
Normal file
30
resources/tooling/coverage/Coverage.java
Normal file
@@ -0,0 +1,30 @@
|
||||
import java.nio.file.Path;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import org.w3c.dom.Element;
|
||||
|
||||
/** Exact module-wide line gate; consumes JaCoCo XML without rounding percentages. */
|
||||
public class Coverage {
|
||||
public static void main(String[] args) throws Exception {
|
||||
final var factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://xml.org/sax/features/external-general-entities", false);
|
||||
factory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
|
||||
factory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
|
||||
final Element report =
|
||||
factory.newDocumentBuilder().parse(Path.of(args[0]).toFile()).getDocumentElement();
|
||||
for (var child = report.getFirstChild(); child != null; child = child.getNextSibling()) {
|
||||
if (child instanceof Element counter
|
||||
&& counter.getTagName().equals("counter")
|
||||
&& counter.getAttribute("type").equals("LINE")) {
|
||||
final long covered = Long.parseLong(counter.getAttribute("covered"));
|
||||
final long missed = Long.parseLong(counter.getAttribute("missed"));
|
||||
final long total = covered + missed;
|
||||
System.out.println(
|
||||
"Backend line coverage: " + covered + "/" + total + " (" + 100.0 * covered / total + "%)");
|
||||
if (total == 0 || 5 * covered < 4 * total)
|
||||
throw new IllegalStateException("Backend line coverage must reach 80%: " + args[0]);
|
||||
return;
|
||||
}
|
||||
}
|
||||
throw new IllegalStateException("Missing module LINE counter: " + args[0]);
|
||||
}
|
||||
}
|
||||
136
resources/tooling/format/Format.java
Normal file
136
resources/tooling/format/Format.java
Normal file
@@ -0,0 +1,136 @@
|
||||
import com.palantir.javaformat.java.Formatter;
|
||||
import com.palantir.javaformat.java.JavaFormatterOptions;
|
||||
import java.nio.file.*;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.*;
|
||||
import org.openrewrite.*;
|
||||
import org.openrewrite.internal.InMemoryLargeSourceSet;
|
||||
import org.openrewrite.text.*;
|
||||
|
||||
/** Source-launcher bootstrap: no project-specific artifact is resolved or installed. */
|
||||
public class Format {
|
||||
public static void main(String[] args) throws Exception {
|
||||
Path root = Path.of(args[0]).toAbsolutePath().normalize();
|
||||
boolean check = switch (args[1]) {
|
||||
case "check" -> true;
|
||||
case "apply" -> false;
|
||||
default -> throw new IllegalArgumentException("format.mode must be apply or check");
|
||||
};
|
||||
Path cacheFile = root.resolve("target/palantir-cache.properties");
|
||||
var cache = new Properties();
|
||||
if (Files.exists(cacheFile)) {
|
||||
try (var input = Files.newInputStream(cacheFile)) {
|
||||
cache.load(input);
|
||||
}
|
||||
}
|
||||
String configuration = digest(Files.readString(root.resolve("pom.xml"))
|
||||
+ Files.readString(root.resolve("generated/tooling/format/Format.java"))
|
||||
+ System.getProperty("java.version")
|
||||
+ System.getProperty("java.class.path"));
|
||||
if (!configuration.equals(cache.getProperty("configuration"))) cache.clear();
|
||||
cache.setProperty("configuration", configuration);
|
||||
List<SourceFile> sources = new ArrayList<>();
|
||||
var settings = new Properties();
|
||||
try (var input = Files.newInputStream(root.resolve("engineering.properties"))) {
|
||||
settings.load(input);
|
||||
}
|
||||
for (String directory :
|
||||
settings.getProperty("FORMAT_DIRECTORIES", "backend/src").split(",")) {
|
||||
if (!Files.isDirectory(root.resolve(directory))) continue;
|
||||
try (var paths = Files.walk(root.resolve(directory))) {
|
||||
for (Path path : paths.filter(p -> p.toString().endsWith(".java"))
|
||||
.sorted()
|
||||
.toList()) {
|
||||
try {
|
||||
String content = Files.readString(path);
|
||||
String key = root.relativize(path).toString();
|
||||
if (!digest(content).equals(cache.getProperty(key))) {
|
||||
sources.add(PlainText.builder()
|
||||
.sourcePath(root.relativize(path))
|
||||
.text(content)
|
||||
.build());
|
||||
}
|
||||
} catch (Exception error) {
|
||||
throw new IllegalStateException(path + ": " + error.getMessage(), error);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
var context = new InMemoryExecutionContext(error -> {
|
||||
throw new IllegalStateException("Formatting failed: " + error.getMessage(), error);
|
||||
});
|
||||
var results = new PalantirRecipe()
|
||||
.run(new InMemoryLargeSourceSet(sources), context)
|
||||
.getChangeset()
|
||||
.getAllResults();
|
||||
for (Result result : results) {
|
||||
System.out.println((check ? "Needs formatting: " : "Formatted: ")
|
||||
+ result.getAfter().getSourcePath());
|
||||
}
|
||||
if (check && !results.isEmpty()) {
|
||||
throw new IllegalStateException("Run mvn -N exec:exec@format to apply Palantir formatting.");
|
||||
}
|
||||
// Complete parsing/formatting before writing any maintained file. Each replacement is atomic.
|
||||
if (!check) {
|
||||
for (Result result : results) {
|
||||
Path destination = root.resolve(result.getAfter().getSourcePath());
|
||||
Path temporary = Files.createTempFile(destination.getParent(), ".palantir-", ".tmp");
|
||||
try {
|
||||
Files.writeString(temporary, result.getAfter().printAll());
|
||||
Files.setPosixFilePermissions(temporary, Files.getPosixFilePermissions(destination));
|
||||
Files.move(
|
||||
temporary,
|
||||
destination,
|
||||
StandardCopyOption.ATOMIC_MOVE,
|
||||
StandardCopyOption.REPLACE_EXISTING);
|
||||
} finally {
|
||||
Files.deleteIfExists(temporary);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (SourceFile source : sources) {
|
||||
cache.setProperty(
|
||||
source.getSourcePath().toString(), digest(Files.readString(root.resolve(source.getSourcePath()))));
|
||||
}
|
||||
Files.createDirectories(cacheFile.getParent());
|
||||
try (var output = Files.newOutputStream(cacheFile)) {
|
||||
cache.store(output, "Successfully formatted content only");
|
||||
}
|
||||
System.out.println(
|
||||
"Palantir " + args[1] + ": " + sources.size() + " Java files, " + results.size() + " differences");
|
||||
}
|
||||
|
||||
static String digest(String value) throws Exception {
|
||||
return HexFormat.of()
|
||||
.formatHex(MessageDigest.getInstance("SHA-256")
|
||||
.digest(value.getBytes(java.nio.charset.StandardCharsets.UTF_8)));
|
||||
}
|
||||
|
||||
public static final class PalantirRecipe extends Recipe {
|
||||
public String getDisplayName() {
|
||||
return "Palantir Java Format";
|
||||
}
|
||||
|
||||
public String getDescription() {
|
||||
return "Apply the pinned Palantir engine and its native import handling.";
|
||||
}
|
||||
|
||||
public TreeVisitor<?, ExecutionContext> getVisitor() {
|
||||
return new PlainTextVisitor<ExecutionContext>() {
|
||||
final Formatter formatter = Formatter.createFormatter(JavaFormatterOptions.builder()
|
||||
.style(JavaFormatterOptions.Style.PALANTIR)
|
||||
.formatJavadoc(false)
|
||||
.build());
|
||||
|
||||
@Override
|
||||
public PlainText visitText(PlainText text, ExecutionContext context) {
|
||||
try {
|
||||
return text.withText(formatter.formatSourceAndFixImports(text.getText()));
|
||||
} catch (Exception | LinkageError error) {
|
||||
throw new IllegalStateException(text.getSourcePath() + ": " + error, error);
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
21
resources/tooling/standards/checkstyle.xml
Normal file
21
resources/tooling/standards/checkstyle.xml
Normal file
@@ -0,0 +1,21 @@
|
||||
<!DOCTYPE module PUBLIC "-//Checkstyle//DTD Checkstyle Configuration 1.3//EN" "https://checkstyle.org/dtds/configuration_1_3.dtd">
|
||||
<module name="Checker">
|
||||
<module name="TreeWalker">
|
||||
<module name="TypeName"/>
|
||||
<module name="MethodName"/>
|
||||
<module name="MemberName"/>
|
||||
<module name="ParameterName"/>
|
||||
<module name="LocalVariableName"/>
|
||||
<module name="LocalFinalVariableName"/>
|
||||
<module name="StaticVariableName"/>
|
||||
<module name="LambdaParameterName"/>
|
||||
<module name="RecordComponentName"/>
|
||||
<module name="LeftCurly"/>
|
||||
<module name="CatchParameterName"/>
|
||||
<module name="ConstantName"/>
|
||||
<module name="PackageName"><property name="format" value="^[a-z]+(\.[a-z][a-z0-9]*)*$"/></module>
|
||||
<module name="OneTopLevelClass"/>
|
||||
<module name="AvoidStarImport"/>
|
||||
<module name="NeedBraces"/>
|
||||
</module>
|
||||
</module>
|
||||
16
resources/tooling/standards/pom.xml
Normal file
16
resources/tooling/standards/pom.xml
Normal file
@@ -0,0 +1,16 @@
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<groupId>nl.agilecc</groupId><artifactId>backend-standards</artifactId><version>1</version>
|
||||
<properties><maven.compiler.release>25</maven.compiler.release><project.build.sourceEncoding>UTF-8</project.build.sourceEncoding></properties>
|
||||
<dependencies>
|
||||
<dependency><groupId>jakarta.inject</groupId><artifactId>jakarta.inject-api</artifactId><version>2.0.1</version><scope>test</scope></dependency>
|
||||
<dependency><groupId>net.sourceforge.pmd</groupId><artifactId>pmd-java</artifactId><version>7.28.0</version></dependency>
|
||||
<dependency><groupId>com.puppycrawl.tools</groupId><artifactId>checkstyle</artifactId><version>12.3.1</version></dependency>
|
||||
<dependency><groupId>org.junit.jupiter</groupId><artifactId>junit-jupiter</artifactId><version>5.14.1</version><scope>test</scope></dependency>
|
||||
</dependencies>
|
||||
<build><plugins>
|
||||
<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.16.0</version></plugin>
|
||||
<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-surefire-plugin</artifactId><version>3.6.1</version></plugin>
|
||||
<plugin><groupId>org.codehaus.mojo</groupId><artifactId>exec-maven-plugin</artifactId><version>3.6.3</version><configuration><mainClass>Standards</mainClass><arguments><argument>${project.basedir}/../../..</argument></arguments></configuration></plugin>
|
||||
</plugins></build>
|
||||
</project>
|
||||
590
resources/tooling/standards/src/main/java/Standards.java
Normal file
590
resources/tooling/standards/src/main/java/Standards.java
Normal file
@@ -0,0 +1,590 @@
|
||||
import com.puppycrawl.tools.checkstyle.*;
|
||||
import com.puppycrawl.tools.checkstyle.api.*;
|
||||
import java.io.*;
|
||||
import java.nio.file.*;
|
||||
import java.util.*;
|
||||
import net.sourceforge.pmd.*;
|
||||
import net.sourceforge.pmd.cpd.*;
|
||||
import net.sourceforge.pmd.lang.*;
|
||||
import net.sourceforge.pmd.lang.ast.*;
|
||||
import net.sourceforge.pmd.lang.java.ast.*;
|
||||
import net.sourceforge.pmd.lang.java.rule.AbstractJavaRule;
|
||||
import net.sourceforge.pmd.lang.java.symbols.*;
|
||||
import net.sourceforge.pmd.lang.java.types.*;
|
||||
import net.sourceforge.pmd.lang.metrics.MetricOptions;
|
||||
import net.sourceforge.pmd.lang.rule.RuleSet;
|
||||
|
||||
/** Deliberately independent of the application build and its annotation processors. */
|
||||
public class Standards {
|
||||
static final List<String> findings = new ArrayList<>();
|
||||
static final Map<Path, Set<Integer>> codeLines = new HashMap<>();
|
||||
static final Set<String> ownedTypes = new HashSet<>();
|
||||
static final Map<String, ASTTypeDeclaration> types = new HashMap<>();
|
||||
static final List<ASTCompilationUnit> units = new ArrayList<>();
|
||||
static Path root;
|
||||
static String module;
|
||||
|
||||
record Waiver(String metric, String executable, double value, String reason) {}
|
||||
|
||||
static final List<Waiver> waivers = new ArrayList<>();
|
||||
|
||||
public static void main(String[] args) throws Exception {
|
||||
findings.clear();
|
||||
codeLines.clear();
|
||||
types.clear();
|
||||
units.clear();
|
||||
ownedTypes.clear();
|
||||
root = Path.of(args[0]).toAbsolutePath().normalize();
|
||||
final var settings = new Properties();
|
||||
try (var input = Files.newInputStream(root.resolve("engineering.properties"))) {
|
||||
settings.load(input);
|
||||
}
|
||||
module = settings.getProperty("BACKEND_MODULE");
|
||||
if (module == null || module.isBlank()) throw new IllegalArgumentException("BACKEND_MODULE is required");
|
||||
waivers.clear();
|
||||
final Path waiverFile = root.resolve("tooling/standards/waivers.tsv");
|
||||
if (Files.exists(waiverFile))
|
||||
for (String line : Files.readAllLines(waiverFile)) {
|
||||
if (line.isBlank() || line.startsWith("#")) continue;
|
||||
final String[] fields = line.split("\t", -1);
|
||||
if (fields.length != 4 || fields[0].contains("*") || fields[1].contains("*") || fields[3].isBlank())
|
||||
throw new IllegalArgumentException("Invalid exact waiver in " + waiverFile + ": " + line);
|
||||
waivers.add(new Waiver(fields[0], fields[1], Double.parseDouble(fields[2]), fields[3]));
|
||||
}
|
||||
final Path output = root.resolve(module + "/target/standards");
|
||||
Files.createDirectories(output);
|
||||
final List<Path> files = new ArrayList<>();
|
||||
for (String scope : List.of("main", "test")) {
|
||||
try (var paths = Files.walk(root.resolve(module + "/src/" + scope + "/java"))) {
|
||||
paths.filter(p -> p.toString().endsWith(".java")).sorted().forEach(files::add);
|
||||
}
|
||||
}
|
||||
for (Path file : files) {
|
||||
final var matcher =
|
||||
java.util.regex.Pattern.compile("package\\s+([\\w.]+)\\s*;").matcher(Files.readString(file));
|
||||
if (matcher.find())
|
||||
ownedTypes.add(
|
||||
matcher.group(1) + "." + file.getFileName().toString().replace(".java", ""));
|
||||
}
|
||||
checkstyle(files);
|
||||
final PMDConfiguration config = new PMDConfiguration();
|
||||
config.setDefaultLanguageVersion(
|
||||
LanguageRegistry.PMD.getLanguageById("java").getVersion("25"));
|
||||
config.setThreads(1);
|
||||
config.prependAuxClasspath(
|
||||
Path.of(System.getProperty("java.home"), "lib/jrt-fs.jar").toString());
|
||||
config.setIgnoreIncrementalAnalysis(true);
|
||||
// Compiled application/test dependencies permit PMD to resolve external types.
|
||||
final Path cp = root.resolve(module + "/target/standards-classpath.txt");
|
||||
if (Files.exists(cp))
|
||||
config.prependAuxClasspath(
|
||||
Files.readString(cp) + File.pathSeparator + root.resolve(module + "/target/classes"));
|
||||
try (var analysis = PmdAnalysis.create(config)) {
|
||||
analysis.addRuleSet(RuleSet.forSingleRule(new Measurements()));
|
||||
for (String rule : List.of(
|
||||
"category/java/codestyle.xml/LocalVariableCouldBeFinal",
|
||||
"category/java/codestyle.xml/MethodArgumentCouldBeFinal",
|
||||
"category/java/design.xml/ImmutableField"))
|
||||
analysis.addRuleSet(analysis.newRuleSetLoader().loadFromResource(rule));
|
||||
for (Path file : files) analysis.files().addFile(file);
|
||||
final var report = analysis.performAnalysisAndCollectReport();
|
||||
for (var error : report.getProcessingErrors())
|
||||
findings.add("BLOCK parser " + error.getFileId() + " " + error.getMsg());
|
||||
for (var violation : report.getViolations())
|
||||
findings.add("BLOCK " + violation.getFileId() + ":" + violation.getBeginLine() + " "
|
||||
+ violation.getRule().getName() + " " + violation.getDescription());
|
||||
for (var violation : report.getSuppressedViolations())
|
||||
findings.add("BLOCK Unapproved suppression: " + violation);
|
||||
for (var error : report.getConfigurationErrors()) findings.add("BLOCK configuration " + error);
|
||||
}
|
||||
duplication(files);
|
||||
Collections.sort(findings);
|
||||
Files.write(output.resolve("report.txt"), findings);
|
||||
findings.stream().filter(f -> !f.startsWith("METRIC ")).forEach(System.out::println);
|
||||
System.out.println(
|
||||
"Backend standards: " + files.size() + " source files; full metrics: " + output.resolve("report.txt"));
|
||||
if (findings.stream().anyMatch(s -> s.startsWith("BLOCK")))
|
||||
throw new IllegalStateException("Backend standards failed; see " + output.resolve("report.txt"));
|
||||
}
|
||||
|
||||
static void checkstyle(List<Path> files) throws Exception {
|
||||
final Checker checker = new Checker();
|
||||
checker.setModuleClassLoader(Standards.class.getClassLoader());
|
||||
checker.configure(ConfigurationLoader.loadConfiguration(
|
||||
root.resolve("generated/tooling/standards/checkstyle.xml").toString(),
|
||||
new PropertiesExpander(new Properties())));
|
||||
checker.addListener(new AuditListener() {
|
||||
public void auditStarted(AuditEvent e) {}
|
||||
|
||||
public void auditFinished(AuditEvent e) {}
|
||||
|
||||
public void fileStarted(AuditEvent e) {}
|
||||
|
||||
public void fileFinished(AuditEvent e) {}
|
||||
|
||||
public void addError(AuditEvent e) {
|
||||
findings.add("BLOCK " + e.getFileName() + ":" + e.getLine() + " " + e.getMessage());
|
||||
}
|
||||
|
||||
public void addException(AuditEvent e, Throwable t) {
|
||||
findings.add("BLOCK " + e.getFileName() + " parser: " + t);
|
||||
}
|
||||
});
|
||||
try {
|
||||
checker.process(files.stream().map(Path::toFile).toList());
|
||||
} finally {
|
||||
checker.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
public static class Measurements extends AbstractJavaRule {
|
||||
public Measurements() {
|
||||
setName("BackendMeasurements");
|
||||
setMessage("Backend standards");
|
||||
setLanguage(LanguageRegistry.PMD.getLanguageById("java"));
|
||||
}
|
||||
|
||||
@Override
|
||||
public Object visit(ASTCompilationUnit unit, Object data) {
|
||||
units.add(unit);
|
||||
for (var type : all(unit, ASTTypeDeclaration.class))
|
||||
types.put(type.getSymbol().getBinaryName(), type);
|
||||
final Set<Integer> lines = lines(unit);
|
||||
codeLines.put(path(unit), lines);
|
||||
for (var node : all(unit, JavaNode.class)) {
|
||||
if (node instanceof ASTExecutableDeclaration executable) executable(executable);
|
||||
if (node instanceof ASTCompactConstructorDeclaration compact) {
|
||||
final var cyclo = new org.apache.commons.lang3.mutable.MutableInt(1);
|
||||
compact.acceptVisitor(new ScopedCyclo(compact), cyclo);
|
||||
final var cognitive =
|
||||
new net.sourceforge.pmd.lang.java.metrics.internal.CognitiveComplexityVisitor.State(
|
||||
compact);
|
||||
compact.acceptVisitor(new ScopedCognitive(), cognitive);
|
||||
metric(compact, "cyclomatic", cyclo.intValue(), 7, 10);
|
||||
metric(compact, "cognitive", cognitive.getComplexity(), 10, 15);
|
||||
metric(compact, "executableLines", lines(compact).size(), 30, 50);
|
||||
metric(compact, "nesting", nesting(compact.getBody(), 0), 3, 4);
|
||||
metric(
|
||||
compact,
|
||||
"parameters",
|
||||
compact.getEnclosingType().getRecordComponents().size(),
|
||||
5,
|
||||
7);
|
||||
visibility(compact);
|
||||
}
|
||||
if (node instanceof ASTVariableId variable) variable(variable);
|
||||
if (node instanceof ASTTypeDeclaration nested
|
||||
&& nested.isNested()
|
||||
&& !nested.isLocal()
|
||||
&& !nested.isAnonymous()) visibility(nested);
|
||||
if (node instanceof ASTImportDeclaration imp && imp.isStatic() && production(unit))
|
||||
block(imp, "Static imports are test-only");
|
||||
if (node instanceof ASTExecutableDeclaration || node instanceof ASTFieldDeclaration)
|
||||
visibility((ModifierOwner) node);
|
||||
if (node instanceof ASTMethodDeclaration method
|
||||
&& method.isOverride()
|
||||
&& !method.isAnnotationPresent("java.lang.Override"))
|
||||
block(method, "Overriding methods require @Override");
|
||||
}
|
||||
for (var type : all(unit, ASTTypeDeclaration.class)) typeMetrics(type);
|
||||
return data;
|
||||
}
|
||||
}
|
||||
|
||||
static Path path(Node node) {
|
||||
return Path.of(node.getTextDocument().getFileId().getAbsolutePath())
|
||||
.toAbsolutePath()
|
||||
.normalize();
|
||||
}
|
||||
|
||||
static boolean production(Node node) {
|
||||
return path(node).startsWith(root.resolve(module + "/src/main/java"));
|
||||
}
|
||||
|
||||
static <T> List<T> all(Node node, Class<T> clazz) {
|
||||
final List<T> result = new ArrayList<>();
|
||||
if (clazz.isInstance(node)) result.add(clazz.cast(node));
|
||||
for (int i = 0; i < node.getNumChildren(); i++) result.addAll(all(node.getChild(i), clazz));
|
||||
return result;
|
||||
}
|
||||
|
||||
static void block(Node node, String message) {
|
||||
findings.add("BLOCK " + location(node) + " " + message);
|
||||
}
|
||||
|
||||
static String location(Node node) {
|
||||
return root.relativize(path(node)) + ":" + node.getBeginLine();
|
||||
}
|
||||
|
||||
static void metric(Node node, String name, double value, double review, double limit) {
|
||||
if (value > limit)
|
||||
for (Waiver waiver : waivers) {
|
||||
if (waiver.metric().equals(name)
|
||||
&& waiver.executable().equals(identity(node))
|
||||
&& waiver.value() == value) {
|
||||
findings.add("WAIVER " + location(node) + " " + name + "=" + value + " " + waiver.reason());
|
||||
return;
|
||||
}
|
||||
}
|
||||
findings.add((value > limit ? "BLOCK " : value > review ? "REVIEW " : "METRIC ") + location(node) + " " + name
|
||||
+ "=" + value);
|
||||
}
|
||||
|
||||
static String identity(Node node) {
|
||||
if (node instanceof ASTCompactConstructorDeclaration compact) {
|
||||
final List<String> parameters = new ArrayList<>();
|
||||
for (var component : compact.getEnclosingType().getRecordComponents())
|
||||
parameters.add(component.getTypeMirror().toString());
|
||||
return compact.getEnclosingType().getBinaryName() + "#<init>(" + String.join(",", parameters) + ")";
|
||||
}
|
||||
if (node instanceof ASTExecutableDeclaration executable) {
|
||||
final List<String> parameters = new ArrayList<>();
|
||||
for (var parameter : executable.getFormalParameters())
|
||||
parameters.add(parameter.getTypeMirror().toString());
|
||||
return executable.getEnclosingType().getBinaryName() + "#"
|
||||
+ (executable instanceof ASTConstructorDeclaration ? "<init>" : executable.getName())
|
||||
+ "(" + String.join(",", parameters) + ")";
|
||||
}
|
||||
return node instanceof ASTTypeDeclaration type ? type.getBinaryName() : location(node);
|
||||
}
|
||||
|
||||
static Set<Integer> lines(JavaNode node) {
|
||||
final Set<Integer> result = new HashSet<>();
|
||||
var token = node.getFirstToken();
|
||||
while (token != null && !token.isEof()) {
|
||||
final var loc = token.getReportLocation();
|
||||
for (int line = loc.getStartLine(); line <= loc.getEndLine(); line++) result.add(line);
|
||||
if (token == node.getLastToken()) break;
|
||||
token = token.getNext();
|
||||
}
|
||||
final String[] sourceLines = node.getTextDocument().getText().toString().split("\\R", -1);
|
||||
result.removeIf(line -> line > sourceLines.length || sourceLines[line - 1].isBlank());
|
||||
return result;
|
||||
}
|
||||
|
||||
static void executable(ASTExecutableDeclaration method) {
|
||||
{
|
||||
metric(
|
||||
method,
|
||||
"parameters",
|
||||
method.getArity(),
|
||||
method instanceof ASTMethodDeclaration ? 4 : 5,
|
||||
method instanceof ASTMethodDeclaration ? 5 : 7);
|
||||
}
|
||||
if (method.getBody() == null) return;
|
||||
metric(method, "cyclomatic", cyclomatic(method), 7, 10);
|
||||
metric(method, "cognitive", cognitive(method), 10, 15);
|
||||
metric(method, "executableLines", lines(method).size(), 30, 50);
|
||||
metric(method, "nesting", nesting(method.getBody(), 0), 3, 4);
|
||||
}
|
||||
|
||||
static final class ScopedCyclo extends net.sourceforge.pmd.lang.java.metrics.internal.CycloVisitor {
|
||||
ScopedCyclo(JavaNode node) {
|
||||
super(MetricOptions.emptyOptions(), node);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Void visit(ASTLambdaExpression lambda, org.apache.commons.lang3.mutable.MutableInt value) {
|
||||
for (int i = 0; i < lambda.getNumChildren(); i++) lambda.getChild(i).acceptVisitor(this, value);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
static final class ScopedCognitive
|
||||
extends net.sourceforge.pmd.lang.java.metrics.internal.CognitiveComplexityVisitor {
|
||||
@Override
|
||||
public Void visit(ASTClassBody body, State state) {
|
||||
return null;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Void visitJavaNode(JavaNode node, State state) {
|
||||
return node instanceof ASTTypeDeclaration || node instanceof ASTTypeBody
|
||||
? null
|
||||
: super.visitJavaNode(node, state);
|
||||
}
|
||||
}
|
||||
|
||||
static int cyclomatic(ASTExecutableDeclaration method) {
|
||||
final var value = new org.apache.commons.lang3.mutable.MutableInt();
|
||||
method.acceptVisitor(new ScopedCyclo(method), value);
|
||||
return value.intValue();
|
||||
}
|
||||
|
||||
static int cognitive(ASTExecutableDeclaration method) {
|
||||
final var state = new net.sourceforge.pmd.lang.java.metrics.internal.CognitiveComplexityVisitor.State(method);
|
||||
method.acceptVisitor(new ScopedCognitive(), state);
|
||||
return state.getComplexity();
|
||||
}
|
||||
|
||||
static int nesting(Node node, int depth) {
|
||||
if (node instanceof ASTTypeDeclaration) return depth;
|
||||
final String kind = node.getXPathNodeName();
|
||||
final boolean control = Set.of(
|
||||
"IfStatement",
|
||||
"ForStatement",
|
||||
"ForeachStatement",
|
||||
"WhileStatement",
|
||||
"DoStatement",
|
||||
"SwitchStatement",
|
||||
"SwitchExpression",
|
||||
"CatchClause")
|
||||
.contains(kind);
|
||||
final boolean elseIf = node instanceof ASTIfStatement
|
||||
&& node.getParent() instanceof ASTIfStatement p
|
||||
&& p.getElseBranch() == node;
|
||||
final int current = depth + (control && !elseIf ? 1 : 0);
|
||||
int maximum = current;
|
||||
for (int i = 0; i < node.getNumChildren(); i++) maximum = Math.max(maximum, nesting(node.getChild(i), current));
|
||||
return maximum;
|
||||
}
|
||||
|
||||
static void visibility(ModifierOwner member) {
|
||||
final ASTTypeDeclaration owner = ((JavaNode) member).getEnclosingType();
|
||||
if (owner == null || owner.isAnnotation()) return;
|
||||
if (!member.getModifiers().hasAnyExplicitly(JModifier.PUBLIC, JModifier.PROTECTED, JModifier.PRIVATE))
|
||||
block(member, "Declare explicit member visibility");
|
||||
}
|
||||
|
||||
static void variable(ASTVariableId id) {
|
||||
if (id.isEnumConstant() && !id.getName().matches("[A-Z][A-Z0-9]*(_[A-Z0-9]+)*"))
|
||||
block(id, "Enum constants require UPPER_SNAKE_CASE");
|
||||
if (id.isRecordComponent() || id.isEnumConstant() || id.isUnnamed()) return;
|
||||
if (id.isTypeInferred() && !id.isLambdaParameter() && !id.isPatternBinding()) {
|
||||
final ASTExpression initializer = id.getInitializer();
|
||||
if (!(initializer instanceof ASTConstructorCall)
|
||||
&& !(initializer instanceof ASTCastExpression)
|
||||
&& !(initializer instanceof ASTArrayAllocation)) {
|
||||
block(id, "var requires a visible constructor or cast type");
|
||||
}
|
||||
}
|
||||
if (id.isLambdaParameter()
|
||||
&& id.getParent() instanceof ASTLambdaParameter parameter
|
||||
&& parameter.hasVarKeyword()) block(id, "Use an explicit lambda parameter type with final");
|
||||
if (id.isFinal()) return;
|
||||
final boolean written =
|
||||
id.getLocalUsages().stream().anyMatch(u -> u.getAccessType() != ASTAssignableExpr.AccessType.READ);
|
||||
if (written) return;
|
||||
if (id.isLambdaParameter()) {
|
||||
block(id, "Assigned-once lambda parameter requires final: " + id.getName());
|
||||
return;
|
||||
}
|
||||
if (id.isField()) {
|
||||
final ASTFieldDeclaration field =
|
||||
id.ancestors(ASTFieldDeclaration.class).first();
|
||||
if (field == null
|
||||
|| id.getInitializer() == null
|
||||
|| field.isAnnotationPresent("jakarta.inject.Inject")
|
||||
|| field.isAnnotationPresent("lombok.Setter")
|
||||
|| id.getEnclosingType().isAnnotationPresent("lombok.Setter")
|
||||
|| id.getEnclosingType().isAnnotationPresent("lombok.Data")) return;
|
||||
}
|
||||
block(id, "Assigned-once variable requires final: " + id.getName());
|
||||
}
|
||||
|
||||
static boolean owned(String name) {
|
||||
return types.containsKey(name) || ownedTypes.stream().anyMatch(n -> name.equals(n) || name.startsWith(n + "$"));
|
||||
}
|
||||
|
||||
static void typeMetrics(ASTTypeDeclaration type) {
|
||||
final Set<Integer> ownLines = new HashSet<>();
|
||||
final Set<Integer> scopeLines = codeLines.getOrDefault(path(type), lines(type));
|
||||
final var nestedTypes = all(type, ASTTypeDeclaration.class).stream()
|
||||
.filter(t -> t != type)
|
||||
.toList();
|
||||
var token = type.getFirstToken();
|
||||
while (token != null && !token.isEof()) {
|
||||
final var region = token.getRegion();
|
||||
if (nestedTypes.stream().noneMatch(t -> t.getTextRegion().contains(region))) {
|
||||
final var loc = token.getReportLocation();
|
||||
for (int line = loc.getStartLine(); line <= loc.getEndLine(); line++)
|
||||
if (scopeLines.contains(line)) ownLines.add(line);
|
||||
}
|
||||
if (token == type.getLastToken()) break;
|
||||
token = token.getNext();
|
||||
}
|
||||
metric(type, "classLines", ownLines.size(), 300, 500);
|
||||
if (type.isRecord())
|
||||
findings.add("DESIGN " + location(type) + " recordComponents="
|
||||
+ type.getRecordComponents().size() + " (manual design review)");
|
||||
int depth = 0;
|
||||
var superclass = type.getSymbol().getSuperclass();
|
||||
while (superclass != null && owned(superclass.getBinaryName())) {
|
||||
depth++;
|
||||
superclass = superclass.getSuperclass();
|
||||
}
|
||||
metric(type, "ownSuperclassDepth", depth, 2, 3);
|
||||
final Set<String> fanout = new HashSet<>();
|
||||
for (var ref : all(type, ASTClassType.class)) {
|
||||
if (ref.getEnclosingType() != type) continue;
|
||||
final var symbol = ref.getTypeMirror().getSymbol();
|
||||
if (symbol != null && symbol.isUnresolved())
|
||||
block(
|
||||
ref,
|
||||
"Cannot resolve type " + ref.getSimpleName()
|
||||
+ "; compile the backend and regenerate target/standards-classpath.txt before"
|
||||
+ " checking");
|
||||
if (symbol instanceof JClassSymbol resolved && !resolved.equals(type.getSymbol()))
|
||||
fanout.add(resolved.getBinaryName());
|
||||
}
|
||||
metric(type, "fanOut", fanout.size(), 15, Double.POSITIVE_INFINITY);
|
||||
int publicMethods = 0;
|
||||
for (var method : type.getOperations())
|
||||
if (method instanceof ASTMethodDeclaration
|
||||
&& method.hasExplicitModifiers(JModifier.PUBLIC)
|
||||
&& !trivial(method)) publicMethods++;
|
||||
metric(type, "publicMethods", publicMethods, 15, Double.POSITIVE_INFINITY);
|
||||
final Set<Dependency> dependencies = new HashSet<>();
|
||||
injected(type, dependencies, new HashSet<>());
|
||||
metric(type, "injectedDependencies", dependencies.size(), 5, 7);
|
||||
}
|
||||
|
||||
static boolean trivial(ASTExecutableDeclaration method) {
|
||||
if (method.getBody() == null || method.getBody().getNumChildren() != 1) return false;
|
||||
final JavaNode statement = method.getBody().getChild(0);
|
||||
final String name = method.getName();
|
||||
String property = name;
|
||||
if (name.startsWith("get") || name.startsWith("set")) property = name.substring(3);
|
||||
else if (name.startsWith("is")) property = name.substring(2);
|
||||
if (property.isEmpty()) return false;
|
||||
property = Character.toLowerCase(property.charAt(0)) + property.substring(1);
|
||||
final String fieldName = property;
|
||||
if (method.getEnclosingType().getSymbol().getDeclaredFields().stream()
|
||||
.noneMatch(f -> f.getSimpleName().equals(fieldName))) return false;
|
||||
if (method.getArity() == 0)
|
||||
return statement instanceof ASTReturnStatement returned
|
||||
&& accessorField(returned.getExpr(), method, property);
|
||||
if (method.getArity() != 1 || !name.startsWith("set")) return false;
|
||||
return statement instanceof ASTExpressionStatement expression
|
||||
&& expression.getExpr() instanceof ASTAssignmentExpression assignment
|
||||
&& !assignment.isCompound()
|
||||
&& accessorField(assignment.getLeftOperand(), method, property)
|
||||
&& assignment.getRightOperand() instanceof ASTVariableAccess value
|
||||
&& method.getFormalParameters().get(0).getVarId().getSymbol().equals(value.getReferencedSym());
|
||||
}
|
||||
|
||||
static boolean accessorField(ASTExpression expression, ASTExecutableDeclaration method, String property) {
|
||||
if (!(expression instanceof ASTAssignableExpr.ASTNamedReferenceExpr reference)) return false;
|
||||
if (expression instanceof ASTFieldAccess field && !(field.getQualifier() instanceof ASTThisExpression))
|
||||
return false;
|
||||
return method.getEnclosingType().getSymbol().getDeclaredFields().stream()
|
||||
.anyMatch(
|
||||
field -> field.getSimpleName().equals(property) && field.equals(reference.getReferencedSym()));
|
||||
}
|
||||
|
||||
record Dependency(JTypeMirror type, Set<String> qualifiers) {}
|
||||
|
||||
static boolean annotated(AnnotableSymbol symbol, String annotation) {
|
||||
return symbol.getDeclaredAnnotations().stream()
|
||||
.anyMatch(a -> a.getBinaryName().equals(annotation));
|
||||
}
|
||||
|
||||
static Dependency dependency(JVariableSymbol variable) {
|
||||
return dependency(variable, Substitution.EMPTY);
|
||||
}
|
||||
|
||||
static Dependency dependency(JVariableSymbol variable, Substitution substitution) {
|
||||
final Set<String> qualifiers = new HashSet<>();
|
||||
for (var annotation : variable.getDeclaredAnnotations()) {
|
||||
if (annotation.getBinaryName().equals("jakarta.inject.Named")
|
||||
|| annotated(annotation.getAnnotationSymbol(), "jakarta.inject.Qualifier")) {
|
||||
final Map<String, String> attributes = new TreeMap<>();
|
||||
for (String name : annotation.getAttributeNames())
|
||||
attributes.put(name, String.valueOf(annotation.getAttribute(name)));
|
||||
qualifiers.add(annotation.getBinaryName() + attributes);
|
||||
}
|
||||
}
|
||||
return new Dependency(variable.getTypeMirror(substitution), qualifiers);
|
||||
}
|
||||
|
||||
static void injected(ASTTypeDeclaration type, Set<Dependency> dependencies, Set<String> visited) {
|
||||
if (!visited.add(type.getBinaryName())) return;
|
||||
final var constructors = type.getSymbol().getConstructors();
|
||||
final boolean bean = type.getDeclaredAnnotations().any(a -> {
|
||||
final var symbol = a.getTypeMirror().getSymbol();
|
||||
return Set.of(
|
||||
"jakarta.inject.Singleton",
|
||||
"jakarta.enterprise.context.ApplicationScoped",
|
||||
"jakarta.enterprise.context.RequestScoped",
|
||||
"jakarta.enterprise.context.Dependent",
|
||||
"jakarta.enterprise.context.SessionScoped")
|
||||
.contains(symbol.getCanonicalName())
|
||||
|| annotated(symbol, "jakarta.inject.Scope")
|
||||
|| annotated(symbol, "jakarta.enterprise.context.NormalScope")
|
||||
|| annotated(symbol, "jakarta.enterprise.inject.Stereotype");
|
||||
});
|
||||
for (var constructor : constructors) {
|
||||
if (annotated(constructor, "jakarta.inject.Inject") || bean && constructors.size() == 1) {
|
||||
for (var parameter : constructor.getFormalParameters()) dependencies.add(dependency(parameter));
|
||||
}
|
||||
}
|
||||
// PMD deliberately sees handwritten source, so account for Lombok's generated constructor.
|
||||
if (bean
|
||||
&& type.getOperations()
|
||||
.filterIs(ASTConstructorDeclaration.class)
|
||||
.isEmpty()) {
|
||||
final boolean allArgs = type.isAnnotationPresent("lombok.AllArgsConstructor");
|
||||
final boolean requiredArgs = type.isAnnotationPresent("lombok.RequiredArgsConstructor");
|
||||
if (allArgs || requiredArgs)
|
||||
for (var field : type.getDeclarations(ASTFieldDeclaration.class)) {
|
||||
if (!field.isStatic())
|
||||
for (var id : field.getVarIds())
|
||||
if (allArgs
|
||||
|| id.getInitializer() == null
|
||||
&& (id.isFinal() || field.isAnnotationPresent("lombok.NonNull")))
|
||||
dependencies.add(dependency(id.getSymbol()));
|
||||
}
|
||||
}
|
||||
inheritedInjection(type.getTypeMirror(), dependencies, new HashSet<>());
|
||||
}
|
||||
|
||||
static void inheritedInjection(JClassType resolvedType, Set<Dependency> dependencies, Set<String> visited) {
|
||||
if (resolvedType == null) return;
|
||||
final JClassSymbol symbol = resolvedType.getSymbol();
|
||||
final Substitution substitution = resolvedType.getTypeParamSubst();
|
||||
if (!visited.add(symbol.getBinaryName())) return;
|
||||
for (var field : symbol.getDeclaredFields())
|
||||
if (annotated(field, "jakarta.inject.Inject")) dependencies.add(dependency(field, substitution));
|
||||
for (var method : symbol.getDeclaredMethods())
|
||||
if (annotated(method, "jakarta.inject.Inject"))
|
||||
for (var parameter : method.getFormalParameters())
|
||||
dependencies.add(dependency(parameter, substitution));
|
||||
inheritedInjection(resolvedType.getSuperClass(), dependencies, visited);
|
||||
}
|
||||
|
||||
static void duplication(List<Path> files) throws Exception {
|
||||
final var config = new CPDConfiguration();
|
||||
config.setMinimumTileSize(100);
|
||||
config.setIgnoreIdentifiers(false);
|
||||
config.setIgnoreLiterals(false);
|
||||
config.setSkipLexicalErrors(false);
|
||||
config.setNoSkipBlocks(true);
|
||||
final Set<String> duplicated = new HashSet<>();
|
||||
try (var cpd = CpdAnalysis.create(config)) {
|
||||
for (var file : files)
|
||||
if (file.startsWith(root.resolve(module + "/src/main/java")))
|
||||
cpd.files().addFile(file);
|
||||
cpd.performAnalysis(report -> {
|
||||
for (var error : report.getProcessingErrors()) findings.add("BLOCK CPD parser " + error);
|
||||
for (var match : report.getMatches())
|
||||
for (var mark : match) {
|
||||
final var loc = mark.getLocation();
|
||||
final Path path = Path.of(loc.getFileId().getAbsolutePath())
|
||||
.toAbsolutePath()
|
||||
.normalize();
|
||||
for (int line = loc.getStartLine(); line <= loc.getEndLine(); line++)
|
||||
if (codeLines.getOrDefault(path, Set.of()).contains(line))
|
||||
duplicated.add(path + ":" + line);
|
||||
findings.add("CLONE " + path + ":" + loc.getStartLine() + "-" + loc.getEndLine());
|
||||
}
|
||||
});
|
||||
}
|
||||
final long total = codeLines.entrySet().stream()
|
||||
.filter(e -> e.getKey().startsWith(root.resolve(module + "/src/main/java")))
|
||||
.mapToLong(e -> e.getValue().size())
|
||||
.sum();
|
||||
final double density = total == 0 ? 0 : 100.0 * duplicated.size() / total;
|
||||
findings.add((density > 3 ? "BLOCK " : density > 1 ? "REVIEW " : "METRIC ") + "productionDuplication=" + density
|
||||
+ "% (" + duplicated.size() + "/" + total + ")");
|
||||
}
|
||||
}
|
||||
372
resources/tooling/standards/src/test/java/StandardsTest.java
Normal file
372
resources/tooling/standards/src/test/java/StandardsTest.java
Normal file
@@ -0,0 +1,372 @@
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import java.nio.file.*;
|
||||
import java.util.*;
|
||||
import net.sourceforge.pmd.*;
|
||||
import net.sourceforge.pmd.lang.*;
|
||||
import net.sourceforge.pmd.lang.document.FileId;
|
||||
import net.sourceforge.pmd.lang.java.ast.*;
|
||||
import net.sourceforge.pmd.lang.rule.RuleSet;
|
||||
import org.junit.jupiter.api.*;
|
||||
|
||||
class StandardsTest {
|
||||
private List<String> analyze(String text) throws Exception {
|
||||
Standards.root = Path.of("").toAbsolutePath();
|
||||
Standards.module = "backend";
|
||||
Standards.ownedTypes.clear();
|
||||
Standards.findings.clear();
|
||||
Standards.units.clear();
|
||||
Standards.types.clear();
|
||||
Standards.codeLines.clear();
|
||||
final var config = new PMDConfiguration();
|
||||
config.setThreads(1);
|
||||
config.prependAuxClasspath(
|
||||
Path.of(System.getProperty("java.home"), "lib/jrt-fs.jar").toString());
|
||||
config.prependAuxClasspath(
|
||||
System.getProperty("surefire.test.class.path", System.getProperty("java.class.path")));
|
||||
config.setIgnoreIncrementalAnalysis(true);
|
||||
config.setDefaultLanguageVersion(
|
||||
LanguageRegistry.PMD.getLanguageById("java").getVersion("25"));
|
||||
try (var analysis = PmdAnalysis.create(config)) {
|
||||
analysis.addRuleSet(RuleSet.forSingleRule(new Standards.Measurements()));
|
||||
analysis.files().addSourceFile(FileId.fromPath(Standards.root.resolve("Fixture.java")), text);
|
||||
var report = analysis.performAnalysisAndCollectReport();
|
||||
assertTrue(
|
||||
report.getProcessingErrors().isEmpty(),
|
||||
report.getProcessingErrors().toString());
|
||||
}
|
||||
return List.copyOf(Standards.findings);
|
||||
}
|
||||
|
||||
private boolean contains(List<String> findings, String prefix, String metric) {
|
||||
return findings.stream().anyMatch(s -> s.startsWith(prefix) && s.endsWith(metric));
|
||||
}
|
||||
|
||||
@Test
|
||||
void cyclomaticAndCognitiveBoundaries() throws Exception {
|
||||
var at = analyze("class Fixture { public void run(final boolean b) { " + "if (b) {}".repeat(9) + "} }");
|
||||
assertTrue(contains(at, "REVIEW", "cyclomatic=10.0"));
|
||||
var over = analyze("class Fixture { public void run(final boolean b) { " + "if (b) {}".repeat(10) + "} }");
|
||||
assertTrue(contains(over, "BLOCK", "cyclomatic=11.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture { public void run(final boolean b) {" + "if (b) {}".repeat(15) + "} }"),
|
||||
"REVIEW",
|
||||
"cognitive=15.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture { public void run(final boolean b) {" + "if (b) {}".repeat(16) + "} }"),
|
||||
"BLOCK",
|
||||
"cognitive=16.0"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void methodAndConstructorParameterBoundariesAndImplicitRecords() throws Exception {
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||
+ " final int e) {} }"),
|
||||
"REVIEW",
|
||||
"parameters=5.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||
+ " final int e, final int f) {} }"),
|
||||
"BLOCK",
|
||||
"parameters=6.0"));
|
||||
var implicit = analyze("record Fixture(int a, int b, int c, int d, int e, int f, int g, int h) {}");
|
||||
assertFalse(implicit.stream().anyMatch(s -> s.contains("parameters=")));
|
||||
var compact = analyze(
|
||||
"record Fixture(int a, int b, int c, int d, int e, int f, int g, int h) { public Fixture" + " {} }");
|
||||
assertTrue(contains(compact, "BLOCK", "parameters=8.0"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void physicalLinesExcludeCommentsAndCountBraces() throws Exception {
|
||||
var findings = analyze("class Fixture {\n"
|
||||
+ "public void run() {\n"
|
||||
+ "// comment\n"
|
||||
+ "/* multi\n"
|
||||
+ " comment */\n\n"
|
||||
+ "System.out.println(1); // code\n"
|
||||
+ "}\n"
|
||||
+ "}");
|
||||
assertTrue(contains(findings, "METRIC", "executableLines=3.0"));
|
||||
assertTrue(contains(findings, "METRIC", "classLines=5.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture {\npublic void run() {\n" + "System.out.println(1);\n".repeat(48) + "}\n}"),
|
||||
"REVIEW",
|
||||
"executableLines=50.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture {\npublic void run() {\n" + "System.out.println(1);\n".repeat(49) + "}\n}"),
|
||||
"BLOCK",
|
||||
"executableLines=51.0"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void nestingElseIfCatchSwitchLambdaAndNestedMethods() throws Exception {
|
||||
var chain = analyze(
|
||||
"class Fixture { public void run(final boolean b) { if (b) {} else if (b) {} else if (b)" + " {} } }");
|
||||
assertTrue(contains(chain, "METRIC", "nesting=1.0"));
|
||||
var nested = analyze("class Fixture { public void run(final boolean b) { if (b) { while (b) { switch (1) {"
|
||||
+ " default -> { try {} catch (Exception e) { Runnable r = () -> { if (b) {} }; } } }"
|
||||
+ " } } } }");
|
||||
assertTrue(contains(nested, "BLOCK", "nesting=5.0"));
|
||||
var separate = analyze("class Fixture { public void run(final boolean b) { if (b) { class Inner { public void"
|
||||
+ " go() { if (b) {} } } } } }");
|
||||
assertFalse(separate.stream().anyMatch(s -> s.endsWith("nesting=2.0")));
|
||||
}
|
||||
|
||||
@Test
|
||||
void lambdaComplexityContributesAndNestedClassDoesNot() throws Exception {
|
||||
var findings = analyze("class Fixture { public void run(final boolean b) { Runnable r = () -> { if (b) {} };"
|
||||
+ " class Inner { public void nested() { if (b) {} } } } }");
|
||||
assertEquals(
|
||||
2, findings.stream().filter(s -> s.endsWith("cyclomatic=2.0")).count(), findings.toString());
|
||||
}
|
||||
|
||||
@Test
|
||||
void varAndFinalRules() throws Exception {
|
||||
var findings =
|
||||
analyze("class Fixture { public void run(int p) { var a = new Object(); var b = (Object) a; var c"
|
||||
+ " = name(); int count = 0; count++; } private String name() { return \"x\"; } }");
|
||||
assertEquals(
|
||||
1, findings.stream().filter(s -> s.contains("var requires")).count());
|
||||
assertTrue(findings.stream().anyMatch(s -> s.contains("requires final: p")));
|
||||
assertFalse(findings.stream().anyMatch(s -> s.contains("requires final: count")));
|
||||
}
|
||||
|
||||
@Test
|
||||
void patternBindingsRequireFinalOnlyWhenAssignedOnce() throws Exception {
|
||||
var findings = analyze("""
|
||||
record Pair(Object key, Object value) {}
|
||||
class Fixture {
|
||||
public void run(final Object input) {
|
||||
if (input instanceof String unchanged) { System.out.println(unchanged); }
|
||||
if (input instanceof final String protectedValue) { System.out.println(protectedValue); }
|
||||
if (input instanceof String reassigned) { reassigned = reassigned.trim(); System.out.println(reassigned); }
|
||||
switch (input) {
|
||||
case String switched -> System.out.println(switched);
|
||||
default -> {}
|
||||
}
|
||||
if (input instanceof Pair(var key, var value)) {
|
||||
System.out.println(key);
|
||||
System.out.println(value);
|
||||
}
|
||||
}
|
||||
}
|
||||
""");
|
||||
for (String binding : List.of("unchanged", "switched", "key", "value"))
|
||||
assertTrue(findings.stream().anyMatch(s -> s.contains("requires final: " + binding)), findings.toString());
|
||||
for (String binding : List.of("protectedValue", "reassigned"))
|
||||
assertFalse(findings.stream().anyMatch(s -> s.contains("requires final: " + binding)), findings.toString());
|
||||
assertFalse(findings.stream().anyMatch(s -> s.contains("var requires")), findings.toString());
|
||||
}
|
||||
|
||||
@Test
|
||||
void inheritanceCountsOnlyProjectSuperclasses() throws Exception {
|
||||
var findings = analyze("class A {} class B extends A {} class C extends B {} class D extends C {} class Fixture"
|
||||
+ " extends D {} class External extends java.util.ArrayList<String> {}");
|
||||
assertTrue(contains(findings, "REVIEW", "ownSuperclassDepth=3.0"));
|
||||
assertTrue(contains(findings, "BLOCK", "ownSuperclassDepth=4.0"));
|
||||
assertTrue(contains(findings, "METRIC", "ownSuperclassDepth=0.0"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void injectionUsesResolvedTypesQualifiersInheritedMembersAndImplicitConstructors() throws Exception {
|
||||
var findings = analyze("""
|
||||
class Parent {
|
||||
@jakarta.inject.Inject private String inherited;
|
||||
@jakarta.inject.Inject public void setValue(final Integer value) {}
|
||||
}
|
||||
@jakarta.inject.Singleton class Fixture extends Parent {
|
||||
private final String value;
|
||||
public Fixture(@jakarta.inject.Named("a") final String first,
|
||||
@jakarta.inject.Named("a") final String repeated,
|
||||
@jakarta.inject.Named("b") final String second) { value = first; }
|
||||
@jakarta.inject.Inject private String repeatedInherited;
|
||||
}
|
||||
""");
|
||||
assertTrue(contains(findings, "METRIC", "injectedDependencies=4.0"), findings.toString());
|
||||
String fields = "";
|
||||
for (int i = 0; i < 7; i++)
|
||||
fields += "@jakarta.inject.Inject @jakarta.inject.Named(\"" + i + "\") private String d" + i + ";";
|
||||
assertTrue(contains(analyze("class Fixture {" + fields + "}"), "REVIEW", "injectedDependencies=7.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture {" + fields + "@jakarta.inject.Inject private Integer eighth;}"),
|
||||
"BLOCK",
|
||||
"injectedDependencies=8.0"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void fanoutCountsDistinctJdkTypesAndPublicMethodsExcludeOnlyTrueAccessors() throws Exception {
|
||||
var findings = analyze("""
|
||||
class Fixture {
|
||||
private String name;
|
||||
private java.util.List<String> values;
|
||||
private java.util.List<String> repeated;
|
||||
public String getName() { return name; }
|
||||
public void setName(final String value) { this.name = value; }
|
||||
public String getOther() { return name; }
|
||||
public String getValidatedName() { if (name == null) { throw new IllegalStateException(); } return name; }
|
||||
public static int calculate() { return 1; }
|
||||
}
|
||||
""");
|
||||
assertTrue(contains(findings, "METRIC", "fanOut=3.0"), findings.toString());
|
||||
assertTrue(contains(findings, "METRIC", "publicMethods=3.0"), findings.toString());
|
||||
}
|
||||
|
||||
@Test
|
||||
void commentedAccessorsRemainTrivialWhileAdditionalWorkStillCounts() throws Exception {
|
||||
var findings = analyze("""
|
||||
class Fixture {
|
||||
private String name;
|
||||
private String trimmed;
|
||||
private String logged;
|
||||
private String calculated;
|
||||
private String appended;
|
||||
public String getName() { /* Intent comment. */ return name; }
|
||||
public void setName(final String value) { this./* Field comment. */name = value; }
|
||||
public String name() { // Read the stored value.
|
||||
return this.name;
|
||||
}
|
||||
public void setName(final String value, final boolean validate) { this.name = value; }
|
||||
public void setTrimmed(final String value) { this.trimmed = value.trim(); }
|
||||
public String getLogged() { System.out.println(logged); return logged; }
|
||||
public String getCalculated() { return calculated.trim(); }
|
||||
public void setAppended(final String value) { this.appended += value; }
|
||||
}
|
||||
""");
|
||||
assertTrue(contains(findings, "METRIC", "publicMethods=5.0"), findings.toString());
|
||||
}
|
||||
|
||||
@Test
|
||||
void classLengthBoundaryAndNestedTypeLines() throws Exception {
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture {\n" + "private int field;\n".repeat(498) + "}"), "REVIEW", "classLines=500.0"));
|
||||
assertTrue(contains(
|
||||
analyze("class Fixture {\n" + "private int field;\n".repeat(499) + "}"), "BLOCK", "classLines=501.0"));
|
||||
var separate = analyze("class Fixture {\nprivate class Inner {\nprivate int value;\n}\nprivate int other;\n}");
|
||||
assertEquals(
|
||||
2, separate.stream().filter(s -> s.endsWith("classLines=3.0")).count());
|
||||
assertTrue(contains(analyze("class Fixture { private class Inner {} }"), "METRIC", "classLines=1.0"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void switchBooleanAndThrowTreatmentIsPinned() throws Exception {
|
||||
var findings = analyze("class Fixture { public int run(final boolean a, final boolean b) { if (a && b) { throw"
|
||||
+ " new IllegalStateException(); } return switch (1) { case 0, 1 -> 1; default -> 2;"
|
||||
+ " }; } }");
|
||||
assertTrue(contains(findings, "METRIC", "cyclomatic=6.0"), findings.toString());
|
||||
}
|
||||
|
||||
private Path project(String production, String tests) throws Exception {
|
||||
Path directory = Files.createTempDirectory("s7-fixture-");
|
||||
Files.writeString(directory.resolve("engineering.properties"), "BACKEND_MODULE=backend\n");
|
||||
Path prod = directory.resolve("backend/src/main/java");
|
||||
Path test = directory.resolve("backend/src/test/java");
|
||||
Files.createDirectories(prod);
|
||||
Files.createDirectories(test);
|
||||
Path config = directory.resolve("generated/tooling/standards/checkstyle.xml");
|
||||
Files.createDirectories(config.getParent());
|
||||
Files.copy(Path.of(System.getProperty("basedir"), "checkstyle.xml"), config);
|
||||
if (production != null) Files.writeString(prod.resolve("Fixture.java"), production);
|
||||
if (tests != null) Files.writeString(test.resolve("ExampleTest.java"), tests);
|
||||
return directory;
|
||||
}
|
||||
|
||||
@Test
|
||||
void checksFailWithoutRewritingSourcesAndExposeReviewSignals() throws Exception {
|
||||
String source = "package example;\n"
|
||||
+ "import java.util.*;\n"
|
||||
+ "class Fixture { public void run(int x) { if (x == 0) System.out.println(x); } }";
|
||||
Path directory = project(source, null);
|
||||
assertThrows(IllegalStateException.class, () -> Standards.main(new String[] {directory.toString()}));
|
||||
assertEquals(source, Files.readString(directory.resolve("backend/src/main/java/Fixture.java")));
|
||||
String report = Files.readString(directory.resolve("backend/target/standards/report.txt"));
|
||||
assertTrue(report.contains("form of import"), report);
|
||||
assertTrue(report.contains("must use '{}'s"), report);
|
||||
assertTrue(report.contains("requires final: x"), report);
|
||||
}
|
||||
|
||||
@Test
|
||||
void productionCpdCountsEveryOccurrenceAndTestClonesDoNotGate() throws Exception {
|
||||
String method = "public void first() {\n" + "System.out.println(123);\n".repeat(30) + "}\n";
|
||||
String source = "package example;\nclass Fixture {\n" + method + method.replace("first", "second") + "}";
|
||||
Path directory = project(source, null);
|
||||
assertThrows(IllegalStateException.class, () -> Standards.main(new String[] {directory.toString()}));
|
||||
String report = Files.readString(directory.resolve("backend/target/standards/report.txt"));
|
||||
assertTrue(report.contains("BLOCK productionDuplication=") && report.contains("(64/67)"), report);
|
||||
assertTrue(report.lines().filter(l -> l.startsWith("CLONE")).count() >= 2, report);
|
||||
Path testProject =
|
||||
project("package example; class Fixture {}", source.replace("class Fixture", "class ExampleTest"));
|
||||
Standards.main(new String[] {testProject.toString()});
|
||||
String testReport = Files.readString(testProject.resolve("backend/target/standards/report.txt"));
|
||||
assertTrue(testReport.contains("productionDuplication=0.0%"), testReport);
|
||||
}
|
||||
|
||||
@Test
|
||||
void parserFailureNamesTheAffectedFileAndFails() throws Exception {
|
||||
Path directory = project("package example; class Fixture { public void broken( }", null);
|
||||
Exception failure = assertThrows(Exception.class, () -> Standards.main(new String[] {directory.toString()}));
|
||||
assertTrue(
|
||||
failure.toString().contains("Fixture.java")
|
||||
|| Standards.findings.stream().anyMatch(f -> f.contains("Fixture.java")),
|
||||
failure.toString());
|
||||
}
|
||||
|
||||
@Test
|
||||
void exactCoverageGateAcceptsEightyPercentAndRejectsRoundedNearMiss() throws Exception {
|
||||
Path directory = Files.createTempDirectory("s7-coverage-");
|
||||
Path xml = directory.resolve("report.xml");
|
||||
Path helper = Path.of(System.getProperty("basedir"), "../coverage/Coverage.java")
|
||||
.normalize();
|
||||
for (long[] counts : List.of(new long[] {4, 1, 0}, new long[] {799999, 200001, 1}, new long[] {0, 0, 1})) {
|
||||
Files.writeString(
|
||||
xml,
|
||||
"<report><counter type=\"LINE\" covered=\"" + counts[0] + "\" missed=\"" + counts[1]
|
||||
+ "\"/></report>");
|
||||
Process process = new ProcessBuilder(
|
||||
Path.of(System.getProperty("java.home"), "bin/java").toString(),
|
||||
helper.toString(),
|
||||
xml.toString())
|
||||
.redirectErrorStream(true)
|
||||
.redirectOutput(directory.resolve("output.log").toFile())
|
||||
.start();
|
||||
assertEquals(counts[2], process.waitFor(), Files.readString(directory.resolve("output.log")));
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void overridesAndImplicitRecordInjectionAreMeasured() throws Exception {
|
||||
var override = analyze("class Fixture extends java.util.ArrayList<String> { public void clear() {} }");
|
||||
assertTrue(override.stream().anyMatch(f -> f.contains("require @Override")), override.toString());
|
||||
var record = analyze("@jakarta.inject.Singleton record Fixture(String one, String repeated, Integer two) {}");
|
||||
assertTrue(contains(record, "METRIC", "injectedDependencies=2.0"), record.toString());
|
||||
var inherited = analyze("class Parent<T> { @jakarta.inject.Inject private T dependency; } class Fixture extends"
|
||||
+ " Parent<String> { @jakarta.inject.Inject private String repeated; }");
|
||||
assertEquals(
|
||||
2,
|
||||
inherited.stream()
|
||||
.filter(f -> f.endsWith("injectedDependencies=1.0"))
|
||||
.count(),
|
||||
inherited.toString());
|
||||
}
|
||||
|
||||
@Test
|
||||
void waiverMatchesOnlyTheExactMetricSignatureAndRetainedValue() throws Exception {
|
||||
Standards.waivers.add(new Standards.Waiver(
|
||||
"parameters",
|
||||
"Fixture#run(int,int,int,int,int,int)",
|
||||
6,
|
||||
"Test fixture agreement: preserve an example signature"));
|
||||
try {
|
||||
var waived = analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||
+ " final int e, final int f) {} }");
|
||||
assertTrue(
|
||||
waived.stream().anyMatch(f -> f.startsWith("WAIVER") && f.contains("parameters=6.0")),
|
||||
waived.toString());
|
||||
var changed = analyze("class Fixture { public void run(final int a, final int b, final int c, final int d,"
|
||||
+ " final int e, final int f, final int g) {} }");
|
||||
assertTrue(contains(changed, "BLOCK", "parameters=7.0"));
|
||||
} finally {
|
||||
Standards.waivers.clear();
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user