S19: consume versioned engineering resources and move landscape documentation
All checks were successful
Build, test and publish / verify (push) Successful in 8m54s
All checks were successful
Build, test and publish / verify (push) Successful in 8m54s
This commit is contained in:
98
docs/validation/S8-formatting-evidence.md
Normal file
98
docs/validation/S8-formatting-evidence.md
Normal file
@@ -0,0 +1,98 @@
|
||||
> Historical validation evidence from application commit `bd898e92d22f6c412e671908770f4d89c8ae12bc`. Commands retain their original paths; current tooling is documented in README.md.
|
||||
|
||||
# S8 implementation evidence
|
||||
|
||||
Measured on 2026-10-30 on Linux x86_64 (kernel 7.0.0-38-generic), AMD Ryzen 7
|
||||
7730U, 16 logical CPUs; Ubuntu OpenJDK 25.0.4.1, Maven 3.9.16. These are local
|
||||
wall-clock measurements, not universal performance guarantees.
|
||||
|
||||
## Integration and scope
|
||||
|
||||
- exec-maven-plugin 3.6.3; OpenRewrite core 8.63.4; Palantir Java Format 2.101.0.
|
||||
- Custom `Format.PalantirRecipe` processes OpenRewrite PlainText source files
|
||||
using Palantir's `formatSourceAndFixImports`, Palantir style, `formatJavadoc(false)`.
|
||||
No Java type attribution, OpenRewrite Java parser or AutoFormat is involved.
|
||||
- 73 maintained Java files: 65 backend production/test files, 7 CI/runner helpers,
|
||||
and the formatter helper. Recursive discovery excludes generated build output.
|
||||
- Root validate executes before compilation; compile/test/verify/install all reach
|
||||
validate. CI explicitly passes check mode to clean verify. Standalone commands
|
||||
run only the root exec execution and do not compile the backend.
|
||||
- Bootstrap was first executed with neither Palantir nor OpenRewrite in the local
|
||||
Maven repository. The helper is source-launched, with dependencies supplied by
|
||||
Maven's plugin classpath. There is no project recipe artifact to preinstall.
|
||||
The isolated regression fixture also has no compiled adapter or backend module.
|
||||
|
||||
## Correctness
|
||||
|
||||
- Full `mvn -B -ntp clean verify` passed after formatting: 33 backend tests,
|
||||
zero failures/errors/skips. Both benchmark clean verifications also pass.
|
||||
- `java ci/Ci.java test`: 17 passed, including assertion of the exact check-only
|
||||
Maven invocation and that Maven failure removes the success marker and prevents
|
||||
image build/publication.
|
||||
- `python3 tooling/format/test.py` checks an isolated source-launcher checkout:
|
||||
read-only check failure with path and apply command; native unused-import
|
||||
removal; UTF-8 non-ASCII preservation; check/apply agreement; second uncached
|
||||
check with no differences; malformed Java fails with its path and leaves the
|
||||
source bytes unchanged; a newly introduced file cannot evade the content cache.
|
||||
- All current Java files have been processed by the real engine on JDK 25,
|
||||
including records, switch expressions/multiple labels, pattern instanceof,
|
||||
lambdas and source-launcher CI classes. Tests compile/run the resulting sources.
|
||||
This demonstrates the syntax used here, not every possible Java 25 construct.
|
||||
- Palantir uses javac internals (explicit JVM exports are in the root POM) and its
|
||||
modern AST visitor. Recheck formatter compatibility when upgrading the JDK.
|
||||
Compact source files / instance main methods are a known upstream limitation
|
||||
([upstream issue](https://github.com/palantir/palantir-java-format/issues/1602));
|
||||
they are not used here. Unsupported future syntax fails, rather than being
|
||||
silently excluded. Extra Markdown/Javadoc formatting is disabled.
|
||||
- The initial Java rewrite is committed separately from integration changes;
|
||||
its only transformations are the formatter's native output and import handling.
|
||||
|
||||
## Performance
|
||||
|
||||
`python3 tooling/format/benchmark.py` runs sequentially, with normal dependencies
|
||||
and OS filesystem caches warm. Each configuration receives a compile warmup.
|
||||
The baseline temporarily removes the entire exec plugin declaration from the
|
||||
root POM, then restores its original bytes before the formatting build. Therefore
|
||||
these comparisons include plugin loading, dependency resolution, JVM launch,
|
||||
source-launcher compilation, hashing, cache IO and recipe execution. They do not
|
||||
compare only the inner formatter function or a skipped plugin against itself.
|
||||
|
||||
The successful-content cache is warm for unchanged builds; it is part of the
|
||||
integration. Changed/new files are processed by Palantir. Clean builds remove the
|
||||
cache and process every maintained Java file. Each run's stdout and elapsed time
|
||||
are saved under `target/`; the benchmark restores the root POM and changed helper.
|
||||
|
||||
| Repeated compile | Baseline (s) | Integration (s) | Added overhead (s) |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| 1 | 4.887 | 6.414 | 1.527 |
|
||||
| 2 | 4.821 | 5.964 | 1.143 |
|
||||
| 3 | 4.559 | 6.024 | 1.465 |
|
||||
|
||||
Maximum observed added overhead: **1.527 seconds**, below the agreed three-second
|
||||
limit. Baseline compile warmup: 5.307 s; integration warmup: 7.604 s.
|
||||
|
||||
The changed-file case appends a newline to `ci/runner/WaitForDocker.java`, then
|
||||
runs `mvn -B -ntp compile`: **5.506 s**, successful; Palantir processes that
|
||||
changed file. The script restores its original bytes afterward.
|
||||
|
||||
Clean baseline `mvn -B -ntp clean verify`: **24.797 s**. Clean integration
|
||||
`mvn -B -ntp -Dformat.mode=check clean verify`: **27.747 s**, all 73 files
|
||||
processed, no differences, all tests pass. The three-second criterion applies
|
||||
to repeated unchanged builds, not the clean-build comparison.
|
||||
|
||||
### First-use downloads (separate measurement)
|
||||
|
||||
A disposable root POM with the same pinned plugin/dependencies and an empty
|
||||
`-Dmaven.repo.local` replaces only the Java executable with `/usr/bin/true`.
|
||||
Thus dependency resolution runs without compiling the launcher or executing the
|
||||
formatter/application. Command:
|
||||
`mvn -N -B -ntp -Dmaven.repo.local=<empty-temp-repository> org.codehaus.mojo:exec-maven-plugin:3.6.3:exec@format`.
|
||||
Cold resolution took **9.041 s**; the same command with that repository populated
|
||||
took **1.826 s**. The **7.215 s** cold-versus-warm difference represents first-use
|
||||
network/repository population overhead in this environment, separately from the
|
||||
warm-cache build measurements above. Cold timing also includes root POM/BOM and
|
||||
plugin bootstrap resolution; it is not reported as pure formatter execution.
|
||||
Reproduce with `python3 tooling/format/benchmark-downloads.py` (Linux, network
|
||||
access). The initial real source-launcher bootstrap/check also successfully
|
||||
processed all files; its expected failure listed formatting differences, rather
|
||||
than a missing project recipe or parser/bootstrap error.
|
||||
Reference in New Issue
Block a user