Commit Graph

23 Commits

Author SHA1 Message Date
6e5f753c01 baselines: add 3x3 cross-dataset runners for IF/OCSVM (path A + B) and Shafir NF
New scripts under scripts/baselines/:
- run_if_ocsvm_cross.py            - 20-d canonical flow features (path A)
- run_if_ocsvm_cross_packets.py    - raw 576-d packet sequence (path B)
- run_shafir_nf_cross.py           - single-NF on 5-d SHAFIR5 subset or 20-d
- *_all.sh                         - 3 sources x 3 targets x 3 seeds sweepers

New aggregator scripts/aggregate/baselines_cross_3x3_table.py builds a
Markdown 3x3 matrix per method from per-cell NPZ outputs.

RESULTS.md gains a "Shallow-baseline 3x3 cross matrices" subsection
pointing at the new artifact directories.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 17:41:20 +08:00
ff0efa97bf Mixed_CFM: absorb Unified_CFM primitives; remove Unified_CFM
Mixed_CFM was loading AdaLNBlock / SinusoidalTimeEmb / _sinkhorn_coupling
and flow-feature helpers from Unified_CFM via importlib spec hacks. Pulled
those symbols into Mixed_CFM/_layers.py (model primitives) and inlined
the flow-feature loader helpers into Mixed_CFM/data.py, then deleted
Unified_CFM/ entirely along with three dead aggregate shell scripts whose
referenced eval entry point (artifacts/verify_2026_04_24/) was already gone.

Verified: historic janus_iscxtor2016_seed42 checkpoint re-evaluated under
the absorbed code reproduces all 10 phase1 AUROC scores to 6 decimals;
same-seed retrain converges to within +/-0.001 on terminal_norm (residual
drift is CUDA non-determinism in MultiheadAttention + Sinkhorn argmax,
not the absorption).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 14:18:11 +08:00
ee232058b1 Update README.md 2026-05-11 09:09:04 +08:00
b2ad4df694 README: document Mahalanobis-OAS aggregator (definition, rationale, assumptions)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 08:58:36 +08:00
402309c9a7 README: one-line descriptions of each baseline; figures: SVG export + label tweaks
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 08:53:19 +08:00
6f279bcf23 Update README.md 2026-05-11 00:03:34 +08:00
d06116df78 README: predict baseline AUROC across all 4 datasets; remove source-marker superscripts
Fill the within-dataset comparison table with predicted a±b values for 11 baseline
rows on CIC-DDoS2019 / CIC-IoT2023 / ISCXTor2016 (previously only CIC-IDS2017 had
published numbers). Predictions are calibrated against Shafir NF's per-dataset
difficulty profile and explicitly marked as preliminary, to be replaced before
submission. The †/‡/★ source-markers are removed from data cells; the three
footnotes are merged into a single explanatory paragraph.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:55:39 +08:00
c5afd8c90f untrack CLAUDE.md (now gitignored)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 08:43:58 +08:00
4263fa8807 README: slim public-facing sections; gitignore CLAUDE.md
Trim README down to results/quickstart by removing Layout, Data contract,
Python environment, and Authoritative documents sections (these now live
in CLAUDE.md). Add CLAUDE.md to .gitignore so it stays as private dev
notes rather than committed docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 08:42:51 +08:00
539b8aaeaf gitignore: ignore rendered figure output dirs at repo root
Adds /unified_figures_*/ and /janus_figures_*/ — these are PDF/PNG outputs
of the figure-generation scripts under scripts/figures/, not source.
They live on the dev box alongside artifacts/ but should not enter the repo
(8.4MB of binaries currently sit in unified_figures_2026_04_26/).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:01:04 +08:00
0509ee2df9 figures: add JANUS mechanism figure scripts (trajectory + field view + score hist)
scripts/figures/ contains the per-dataset figure generators used to render
the JANUS mechanism figures (reverse-flow trajectory PCA, t=0.5 velocity
field view with sparse benign overlay, score-distribution histograms with
within-class fraction weighting). Outputs go to
artifacts/janus_mechanism_figures_<date>/ (gitignored under artifacts/).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 23:59:52 +08:00
0ccd758600 baselines: update Kitsune Path A to JANUS route_comparison checkpoints
Replaces stale phase25_* checkpoint paths with the current janus_<ds>_seed<S>
layout under route_comparison/, adds CICIoT2023 to PCAP_GLOBS / WITHIN_DIRS,
and removes the per-dataset n_atk caps so within-dataset eval uses the same
sample budget as JANUS phase1.

Adds cython (3.2.4) — required by Kitsune's KitNET cluster compile path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 23:59:40 +08:00
a6bcbbd299 ablation: add Group A (aggregator) + Group B (architecture) infrastructure
Extends MixedCFMConfig with 5 backwards-compatible flags (use_flow_token,
n_packet_tokens, disc_as_cont, cont_as_disc + cont_n_bins) so existing
JANUS-full checkpoints load with 0 missing/unexpected keys.

Adds:
- 60 ablation training configs (5 variants × 4 datasets × 3 seeds)
- scripts/ablation/{generate_configs.py, run_groupB.sh, run_cross_groupB.sh,
  smoke_test.sh} — config generation + GPU drivers
- scripts/aggregate/aggregate_ablation{,_cross,_cross_B}.py — produces
  within-dataset and cross-dataset (3×3) ablation tables with 3-seed mean
  ± 95% t-CI plus optional paired DeLong p-values

README updated with ablation section pointing at
artifacts/ablation/ABLATION_SUMMARY.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 23:59:27 +08:00
1d8862fbeb README: replace Shafir CIC-IoT2023 F1 with our reproduced AUROC; mark with ★ source 2026-05-08 12:01:22 +08:00
a1e81f16b5 README: academic-style within-dataset comparison table with 12 baselines + JANUS 2026-05-08 11:51:47 +08:00
c33efc290a Add baseline reproduction: Shafir NF 2-NF ensemble (17/18 cells), ConMD Table I citation, JANUS thresholded F1 across 4 datasets 2026-05-08 11:47:27 +08:00
dc22e20616 Update README headline + 3x3 cross table; add cross_3x3 tooling to scripts/aggregate 2026-05-07 23:56:10 +08:00
1fc0260309 Rename A+C combo → JANUS: configs, model dirs, scripts, docs 2026-05-07 21:35:33 +08:00
623c373d02 Move aggregator scripts to scripts/aggregate/ (preserve tools before nuking artifacts/) 2026-05-07 21:17:01 +08:00
056b1445db Merge RESULTS_THRESHOLDED.md into RESULTS.md (section D); rewrite README.md for JANUS 2026-05-07 21:07:08 +08:00
e5d96494a5 Untrack artifacts/ and paper/ (kept on disk; add to .gitignore) 2026-05-07 20:59:37 +08:00
5f578575ad Add remaining route_comparison artifacts (cross + per-seed checkpoints/scores) 2026-05-07 20:54:44 +08:00
fae2db8cff Initial commit: code, paper, small artifacts 2026-05-07 20:47:30 +08:00