diff options
| -rw-r--r-- | TODO.md | 53 |
1 files changed, 32 insertions, 21 deletions
@@ -4,29 +4,40 @@ Background: the scanner runs millions of virtual threads (one per port probe tod Per-probe allocation churn, not peak RSS, is the main memory pressure source (GC load). Hot path: per host -> `ScanHostTask` -> per port -> `ScanPortTask` / `ScanUdpPortTask`. -These items were reviewed and approved in this order. Nothing is implemented yet. Constraint (multi-scan): Scanner must remain able to run several parallel scans with different configurations and targets in one JVM. Anything static must be a true compile-time constant, never per-scan state. -## Phase 2 — approved small changes (do when green-lit, one commit each) +## Phase 2 — DONE (measured: ~31% less CPU per scan, ~4% faster wall, ~100 MB less peak RSS) -- [ ] `ScanPortTask` / `ScanUdpPortTask`: make `waitInMillis` / `waitInNanos` - `static final` (they are hardcoded 100 ms constants, not config — safe for - multi-scan). Saves 16+ bytes on every port task. -- [ ] `PortRangeIterator`: delete dead field `done` (never read). -- [ ] `PortRangeIterator`: remember last `nextSetBit` result so `next()` does not - rescan; `hasNext()` + `next()` currently both call `nextSetBit`. -- [ ] Parse port range **once per scan** (in `Scanner`/context), not per host: - remove `new PortRange(scan.getPorts())` from `ScanHostTask`; workers iterate - the shared immutable BitSet with a plain int cursor. Saves per host: one - string parse + up to two 8 KB BitSet copies. -- [ ] `CliPrinter`: pre-compile the `stty size` regex as a `static final Pattern`. - NOTE: the `getColumnWidth()` process spawn runs on purpose (terminal can be - resized) — do NOT cache the width itself. -- [ ] `CliPrinter.printErrors` / `printPortsResults` / `PortList.getPorts()`: - replace `String.format` + `stream().boxed().toList()` with direct - `StringBuilder` appends over the BitSet. +Workload: `java -jar target/jns-1.0-SNAPSHOT.jar -i - -p all` +(20 000 hosts × 65 535 ports), `/usr/bin/time -v`, single before-run + 4 stable after-runs. + +- [x] `ScanPortTask` / `ScanUdpPortTask`: `waitInMillis` / `waitInNanos` are now + `static final WAIT_IN_NANOS` (`8d355c7`). Saves 16+ bytes on every port task. +- [x] `PortRangeIterator`: dead field `done` deleted, last `nextSetBit` result + cached so `next()` no longer rescans; `clear()` removed with it (`4772393`). +- [x] Port range parsed **once per scan** in `Scanner`, carried in + `ScanExecutionContext`; `ScanHostTask` no longer re-parses per host; + `PortRangeIterator` iterates the shared BitSet read-only (`5fb3b80`). +- [x] `CliPrinter`: `stty size` regex is `static final TTY_SIZE_PATTERN` (`3ee01d8`). + (The `getColumnWidth()` process spawn is kept on purpose — terminal resize.) +- [x] `CliPrinter` output: `String.format` + `stream().boxed().toList()` replaced + by direct `StringBuilder` appends over the BitSet; + `PortList.nextSetBit(int)` accessor added (`1b79d8c`). + +Benchmark (before vs after-4, `before-optimization.txt` / `after-optimization-4.txt`): +| Metric | before | after-4 | delta | +|--------------|---------|---------|-----------| +| User CPU (s) | 167.22 | 111.22 | −33 % | +| Sys CPU (s) | 143.43 | 103.14 | −28 % | +| Total CPU (s)| 310.65 | 214.36 | −31 % | +| Wall (s) | 2114.63 | 2026.19 | −4.2 % | +| Max RSS (KB) | 1228920 | 1121484 | −8 % | + +Interpretation: biggest CPU saver is the iterator change (applied ×1.3 bn ports). +The scan is network-latency-bound, so wall dropped only ~4 % (CPU busy-fraction +14 % → 10 %, voluntary context switches −14 %). Remaining headroom is Phase 3. ## Phase 2b — deferred (user: not implemented now, revisit later) @@ -59,9 +70,9 @@ compile-time constant, never per-scan state. - Caveats: touches cancellation / "partial results on Ctrl-C" behavior (see `CompletionConsumer` contract). Keep `socketLimit` semaphore as the global throttle. Must stay compatible with parallel scanners. - - Only do if Phase 2 numbers still show GC pressure dominated by per-port - scaffolding. Measure with `-Xlog:gc*`, JFR `jdk.ObjectAllocationInNewTLAB`, - wall-time on a fixed target set (e.g. `ips-for-testing.txt` × `1-1024`). + - Gate: Phase 2 measured −31 % CPU; only do Phase 3 if further GC pressure + dominates (measure with `-Xlog:gc*`, JFR `jdk.ObjectAllocationInNewTLAB`, + wall-time on a fixed target set, e.g. `ips-for-testing.txt` × `1-1024`). ## Phase 4 — optional / measure first |