BattleTag/JANUS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: code, paper, small artifacts

Browse Source
This commit is contained in:
BattleTag
2026-05-07 20:47:30 +08:00
commit fae2db8cff
322 changed files with 33159 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
artifacts/results/attribution_pcl_10k/F1_nll_calibration.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

68
artifacts/results/attribution_pcl_10k/README.md Normal file
View File

@@ -0,0 +1,68 @@
# Attribution R1 · PCL encoder + OT-CFM, 10k benign
**Date**: 2026-04-20
**Checkpoint**: `checkpoints/attrib_10k/flownids_full.pt`
**Config**: `--n-train 10000 --train-subsample random --seed 42`, full pipeline
(PCL phase-1 + OT-CFM phase-2), `flow_latent_dim=32`, `n_services=10`.
## Setup
Fit density baselines on the **same PCL latent** (32-d) the CFM was trained
on. Score 50k val-benign + 50k attack (random subsample, fixed seed).
Compare against CFM's native channels (NLL + 4 trajectory metrics).
## Headline numbers (AUROC on 50k / 50k)
| rank | scorer | AUROC | 95% CI | AUPRC |
|---:|---|---:|---|---:|
| 1 | **B4_gmm_K32** (sklearn GaussianMixture, 32 comp.) | **0.9814** | [0.981, 0.982] | 0.965 |
| 2 | B4_gmm_K16 | 0.9595 | [0.958, 0.961] | 0.933 |
| 3 | B4_gmm_K8 | 0.9557 | [0.954, 0.957] | 0.934 |
| 4 | B5_knn_k10 | 0.9400 | [0.939, 0.941] | 0.917 |
| 5 | B5_knn_k5 | 0.9350 | [0.934, 0.936] | 0.916 |
| 6 | **C2 terminal_norm** (CFM) | 0.9341 | [0.932, 0.936] | 0.918 |
| 7 | C3 kinetic_energy (CFM) | 0.9330 | [0.932, 0.935] | 0.916 |
| 8 | C4 arc_length (CFM) | 0.9314 | [0.930, 0.933] | 0.914 |
| 9 | C5 velocity_score (CFM) | 0.9215 | [0.920, 0.923] | 0.875 |
| 10 | B3_mahalanobis | 0.8121 | [0.809, 0.815] | 0.683 |
| 11 | **C1 NLL (CFM)** | 0.8058 | [0.803, 0.809] | 0.809 |
| 12 | B1_isotropic | 0.5518 | - | 0.491 |
| 13 | B2_diag_gaussian | 0.5241 | - | 0.476 |
## Verdict
1. **GMM-32 on PCL latent beats every CFM channel by ≥ 4.7 AUROC pp.** The
best density baseline outscores the best CFM signal (terminal_norm) by
0.0473. CIs of order 0.002 make this difference statistically emphatic.
2. **CFM NLL is a poor density estimator here** (0.806 < Mahalanobis 0.812
on the same latent). CNF-derived log-prob on a 32-d PCL latent does
not match a properly fit mixture.
3. **CFM trajectory signals ARE stronger than CFM NLL** (+0.13 AUROC),
which validates the sub-claim that trajectory ≠ density. But neither
channel beats GMM on the same latent.
4. **Rank-ensemble gain over GMM-32**: +0.0085 from C3, +0.0018 from C2,
total top-of-ladder AUROC 0.9943. CFM contributes ~0.005 worth of new
ranking information beyond what GMM already extracts.
## Interpretation
The PCL encoder compresses benign traffic into a latent with **near-mixture-
of-Gaussians structure**, to the point that the density estimation problem
becomes trivial. CFM is not the differentiator; the encoder is.
Next step: **R2** — ablate the encoder (`--skip-flow-encoder`), train CFM
directly on the 61-d quantile-transformed raw features, and re-run the same
attribution. This tests whether CFM's value appears when the data is NOT
pre-compressed into near-Gaussian modes.
## Files
- `attribution_report.json` — full T1/T2/T3/T4 + CI
- `attribution.log` — run output
- `T1_auroc.png` — per-channel AUROC bar
- `T2_corr.png` — Spearman ρ across channels
- `T3_ensemble.png` — greedy rank-ensemble increments
- `T4_per_attack.png` — per-attack-class heatmap
- `F1_nll_calibration.png` — benign NLL vs isotropic target
Raw artifacts also live at `checkpoints/attrib_10k/` alongside the model.

BIN
artifacts/results/attribution_pcl_10k/T1_auroc.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

BIN
artifacts/results/attribution_pcl_10k/T2_corr.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 174 KiB

BIN
artifacts/results/attribution_pcl_10k/T3_ensemble.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 79 KiB

BIN
artifacts/results/attribution_pcl_10k/T4_per_attack.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 159 KiB

65
artifacts/results/attribution_pcl_10k/attribution.log Normal file
View File

@@ -0,0 +1,65 @@
[16:31:42] Device: cuda
[16:31:42] Loading checkpoint...
Loaded checkpoint: checkpoints/attrib_10k/flownids_full.pt
flow_input_dim=61 flow_latent_dim=32 n_services=10 skip_flow_encoder=False
Scores.npz keys: ['atk_arc_length', 'atk_kinetic_energy', 'atk_labels', 'atk_nll', 'atk_services', 'atk_terminal_norm', 'atk_velocity_score', 'val_arc_length', 'val_kinetic_energy', 'val_labels', 'val_nll', 'val_services', 'val_terminal_norm', 'val_velocity_score']
[16:31:42] Loading data (use_packets=False)...
[splits] strategy=shard_and_label normal_token='normal' train_N=297299 val_normal_N=1285267 attack_N=505431 attack_classes=[np.str_('Botnet'), np.str_('DDoS'), np.str_('DoS GoldenEye'), np.str_('DoS Hulk'), np.str_('DoS Slowhttptest'), np.str_('DoS Slowloris'), np.str_('FTP-Patator'), np.str_('Heartbleed'), np.str_('Infiltration'), np.str_('Infiltration - Portscan'), np.str_('Portscan'), np.str_('SSH-Patator'), np.str_('Web Attack - Brute Force'), np.str_('Web Attack - SQL Injection'), np.str_('Web Attack - XSS')]
drop_attempted: removed 11979 '- Attempted' flows from attack set
split: train=297299 (monday benign) val_normal=1285267 attack=505431
[16:32:08] Train=10000 Val=1285267 Atk=505431
[16:32:08] Encoding latents...
[16:32:08] encoding train N=10000
[16:32:09] encoding val N=1285267
[16:32:14] encoding atk N=505431
[16:32:16] Latents encoded in 7.4s (Zt=(10000, 32), Zv=(1285267, 32), Za=(505431, 32))
[16:32:24] Eval subsample: val=50000 atk=50000
[16:32:24] Fitting baselines...
[16:32:35] Baselines fit in 11.2s
[16:32:35] scored B1_isotropic in 0.0s
[16:32:35] scored B2_diag_gaussian in 0.0s
[16:32:36] scored B3_mahalanobis in 0.0s
[16:32:36] scored B4_gmm_K8 in 0.1s
[16:32:36] scored B4_gmm_K16 in 0.2s
[16:32:36] scored B4_gmm_K32 in 0.4s
[16:32:37] scored B5_knn_k5 in 0.4s
[16:32:37] scored B5_knn_k10 in 0.3s
T1 · per-channel AUROC / AUPRC
B1_isotropic AUROC=0.5325 [0.5290,0.5360] AUPRC=0.4770
B2_diag_gaussian AUROC=0.4949 [0.4914,0.4984] AUPRC=0.4589
B3_mahalanobis AUROC=0.7533 [0.7502,0.7566] AUPRC=0.6231
B4_gmm_K8 AUROC=0.9378 [0.9364,0.9393] AUPRC=0.9115
B4_gmm_K16 AUROC=0.9488 [0.9473,0.9503] AUPRC=0.9218
B4_gmm_K32 AUROC=0.9551 [0.9537,0.9565] AUPRC=0.9291
B5_knn_k5 AUROC=0.8699 [0.8677,0.8721] AUPRC=0.8436
B5_knn_k10 AUROC=0.9039 [0.9020,0.9059] AUPRC=0.8650
C1_nll AUROC=0.8040 [0.8013,0.8065] AUPRC=0.8051
C2_terminal_norm AUROC=0.9333 [0.9318,0.9347] AUPRC=0.9166
C3_kinetic_energy AUROC=0.9307 [0.9290,0.9322] AUPRC=0.9129
C4_arc_length AUROC=0.9290 [0.9274,0.9306] AUPRC=0.9106
C5_velocity_score AUROC=0.9187 [0.9168,0.9205] AUPRC=0.8684
[16:34:49] T2 · Spearman correlation
[16:34:50] T3 · rank-ensemble incremental ΔAUROC
+B4_gmm_K32 AUROC=0.9551 Δ=+0.0000
+C3_kinetic_energy AUROC=0.9666 Δ=+0.0115
+B4_gmm_K16 AUROC=0.9660 Δ=-0.0005
+C5_velocity_score AUROC=0.9670 Δ=+0.0010
+B5_knn_k5 AUROC=0.9672 Δ=+0.0002
+C4_arc_length AUROC=0.9682 Δ=+0.0010
+C2_terminal_norm AUROC=0.9679 Δ=-0.0004
+B4_gmm_K8 AUROC=0.9672 Δ=-0.0007
+C1_nll AUROC=0.9668 Δ=-0.0005
+B5_knn_k10 AUROC=0.9659 Δ=-0.0009
+B1_isotropic AUROC=0.9631 Δ=-0.0028
+B3_mahalanobis AUROC=0.9562 Δ=-0.0069
+B2_diag_gaussian AUROC=0.9485 Δ=-0.0077
[16:34:57] T4 · per-attack AUROC (table)
CONCLUSION: CFM channels do not clearly beat GMM/Mahalanobis on same latent. Reconsider whether CFM is justified over simpler density estimators.
Report -> checkpoints/attrib_10k/attribution_report.json
Plots -> checkpoints/attrib_10k/attribution/

660
artifacts/results/attribution_pcl_10k/attribution_report.json Normal file
View File

@@ -0,0 +1,660 @@
{
"ckpt_dir": "checkpoints/attrib_10k",
"n_train": 10000,
"n_val": 1285267,
"n_atk": 505431,
"latent_dim": 32,
"T1_per_channel": {
"B1_isotropic": {
"auroc": 0.5324829576,
"auroc_ci": [
0.5289596298219795,
0.5359622506672883
],
"auprc": 0.4769599537112281
},
"B2_diag_gaussian": {
"auroc": 0.4948595446,
"auroc_ci": [
0.49140394085612815,
0.4983571945917407
],
"auprc": 0.4588798418552127
},
"B3_mahalanobis": {
"auroc": 0.7533001839999999,
"auroc_ci": [
0.7502458460562909,
0.7566080985214654
],
"auprc": 0.6231314383741376
},
"B4_gmm_K8": {
"auroc": 0.9378189100000001,
"auroc_ci": [
0.9363647335779806,
0.9393350541512251
],
"auprc": 0.911526363144156
},
"B4_gmm_K16": {
"auroc": 0.9487779042000001,
"auroc_ci": [
0.9472923899459115,
0.9503037833358634
],
"auprc": 0.9218456654390215
},
"B4_gmm_K32": {
"auroc": 0.9550767885999999,
"auroc_ci": [
0.9537123115393715,
0.956469806667691
],
"auprc": 0.9291492656327177
},
"B5_knn_k5": {
"auroc": 0.8699048319999999,
"auroc_ci": [
0.867749175207289,
0.8721304681161169
],
"auprc": 0.843603116392615
},
"B5_knn_k10": {
"auroc": 0.9039097153999999,
"auroc_ci": [
0.902036227198053,
0.9059078152298838
],
"auprc": 0.8650099699632048
},
"C1_nll": {
"auroc": 0.8039936966000001,
"auroc_ci": [
0.8013326493232317,
0.8065388341603932
],
"auprc": 0.805074815204867
},
"C2_terminal_norm": {
"auroc": 0.9333068828,
"auroc_ci": [
0.9317518597334148,
0.9346762865846501
],
"auprc": 0.9166376920880811
},
"C3_kinetic_energy": {
"auroc": 0.9306509110000001,
"auroc_ci": [
0.9290296719294825,
0.932224980039777
],
"auprc": 0.9128852344183216
},
"C4_arc_length": {
"auroc": 0.9290179546000001,
"auroc_ci": [
0.9273843544640454,
0.9306142151052262
],
"auprc": 0.9106370031162633
},
"C5_velocity_score": {
"auroc": 0.9187178,
"auroc_ci": [
0.916817643867183,
0.9205409562030975
],
"auprc": 0.8684207426592168
}
},
"T2_spearman": {
"channels": [
"B1_isotropic",
"B2_diag_gaussian",
"B3_mahalanobis",
"B4_gmm_K8",
"B4_gmm_K16",
"B4_gmm_K32",
"B5_knn_k5",
"B5_knn_k10",
"C1_nll",
"C2_terminal_norm",
"C3_kinetic_energy",
"C4_arc_length",
"C5_velocity_score"
],
"rho": [
[
1.0,
0.9452441838377137,
0.17256602608933247,
0.1739424517384982,
0.16087876523919165,
0.21274707999498102,
0.007356361136030099,
0.049806503143988844,
0.05444620456953948,
0.10496759178998452,
0.35185685296041624,
0.35144516359535344,
0.4529586190890037
],
[
0.9452441838377137,
0.9999999999999999,
0.1236660694404876,
0.1400914386067781,
0.13367226703325683,
0.1639696251544771,
0.041185225269876,
0.05927603717861953,
-0.008411456965568543,
0.02344468677876294,
0.27524079778710847,
0.27496335940938027,
0.4122798532445109
],
[
0.17256602608933247,
0.12366606944048758,
0.9999999999999999,
0.6838240280790454,
0.6357443673898874,
0.6651441823894121,
0.32554977083289494,
0.42406481424947967,
0.5409298042792167,
0.749149427448802,
0.6139826424955166,
0.6117745846577094,
0.5244343166634718
],
[
0.17394245173849823,
0.1400914386067781,
0.6838240280790453,
1.0,
0.9567705494934897,
0.9402306026424064,
0.7563569599152269,
0.8105128991227624,
0.5408607120499312,
0.7810274500452008,
0.6923526232220018,
0.688658470542138,
0.7682892341511556
],
[
0.16087876523919165,
0.13367226703325683,
0.6357443673898874,
0.9567705494934897,
1.0,
0.9680317796489144,
0.817022313954715,
0.876895845524359,
0.5397280413753676,
0.7861115607759614,
0.7120194107851766,
0.7083893081438566,
0.7738649298387623
],
[
0.21274707999498102,
0.16396962515447716,
0.6651441823894121,
0.9402306026424065,
0.9680317796489144,
1.0,
0.7729928300948442,
0.8462985168473542,
0.574809269754413,
0.8255978939798168,
0.7563680070991062,
0.7528636281909182,
0.7813643580524655
],
[
0.0073563611360301,
0.041185225269876,
0.32554977083289494,
0.7563569599152269,
0.817022313954715,
0.7729928300948442,
1.0,
0.9640888522046105,
0.33895438996982497,
0.531821540392965,
0.45257011525742685,
0.450497450223761,
0.5684230394462103
],
[
0.04980650314398884,
0.05927603717861954,
0.42406481424947967,
0.8105128991227625,
0.876895845524359,
0.8462985168473542,
0.9640888522046105,
1.0,
0.4207795552715016,
0.6388260379480829,
0.5721760071448643,
0.5694552362215609,
0.6517361902448798
],
[
0.05444620456953948,
-0.008411456965568545,
0.5409298042792167,
0.5408607120499312,
0.5397280413753676,
0.574809269754413,
0.33895438996982497,
0.4207795552715016,
1.0,
0.6688059765599571,
0.5930312231914568,
0.5921116518497463,
0.48185080100109223
],
[
0.10496759178998452,
0.023444686778762935,
0.7491494274488019,
0.7810274500452009,
0.7861115607759616,
0.8255978939798169,
0.531821540392965,
0.638826037948083,
0.6688059765599571,
1.0,
0.8784609477756957,
0.8779272548832465,
0.7132274227514144
],
[
0.35185685296041624,
0.2752407977871085,
0.6139826424955166,
0.6923526232220018,
0.7120194107851766,
0.7563680070991062,
0.45257011525742685,
0.5721760071448643,
0.5930312231914568,
0.8784609477756956,
1.0,
0.9998881190208093,
0.8798858144113907
],
[
0.35144516359535344,
0.27496335940938027,
0.6117745846577094,
0.688658470542138,
0.7083893081438566,
0.7528636281909182,
0.4504974502237609,
0.5694552362215609,
0.5921116518497465,
0.8779272548832464,
0.9998881190208093,
1.0,
0.8772572437033199
],
[
0.4529586190890037,
0.4122798532445109,
0.5244343166634718,
0.7682892341511557,
0.7738649298387623,
0.7813643580524654,
0.5684230394462103,
0.6517361902448798,
0.4818508010010923,
0.7132274227514143,
0.8798858144113907,
0.8772572437033199,
1.0
]
]
},
"T3_rank_ensemble": [
{
"step": 0,
"added": "B4_gmm_K32",
"auroc": 0.9550767885999999,
"delta": 0.0
},
{
"step": 1,
"added": "C3_kinetic_energy",
"auroc": 0.9665783402,
"delta": 0.011501551600000104
},
{
"step": 2,
"added": "B4_gmm_K16",
"auroc": 0.9660361888,
"delta": -0.0005421514000000238
},
{
"step": 3,
"added": "C5_velocity_score",
"auroc": 0.9669938688,
"delta": 0.0009576800000000718
},
{
"step": 4,
"added": "B5_knn_k5",
"auroc": 0.967215872,
"delta": 0.0002220031999999872
},
{
"step": 5,
"added": "C4_arc_length",
"auroc": 0.9682394403999999,
"delta": 0.0010235683999998413
},
{
"step": 6,
"added": "C2_terminal_norm",
"auroc": 0.9678884248,
"delta": -0.00035101559999983767
},
{
"step": 7,
"added": "B4_gmm_K8",
"auroc": 0.9672353905999999,
"delta": -0.0006530342000001799
},
{
"step": 8,
"added": "C1_nll",
"auroc": 0.9667818844,
"delta": -0.00045350619999984243
},
{
"step": 9,
"added": "B5_knn_k10",
"auroc": 0.9659003006000001,
"delta": -0.0008815837999999188
},
{
"step": 10,
"added": "B1_isotropic",
"auroc": 0.9631117718,
"delta": -0.002788528800000134
},
{
"step": 11,
"added": "B3_mahalanobis",
"auroc": 0.9561829636,
"delta": -0.0069288081999999696
},
{
"step": 12,
"added": "B2_diag_gaussian",
"auroc": 0.9484880632,
"delta": -0.007694900399999982
}
],
"T4_per_attack": {
"channels": [
"B1_isotropic",
"B2_diag_gaussian",
"B3_mahalanobis",
"B4_gmm_K8",
"B4_gmm_K16",
"B4_gmm_K32",
"B5_knn_k5",
"B5_knn_k10",
"C1_nll",
"C2_terminal_norm",
"C3_kinetic_energy",
"C4_arc_length",
"C5_velocity_score"
],
"classes": [
"Botnet",
"DDoS",
"DoS GoldenEye",
"DoS Hulk",
"DoS Slowhttptest",
"DoS Slowloris",
"FTP-Patator",
"Heartbleed",
"Infiltration",
"Infiltration - Portscan",
"Portscan",
"SSH-Patator",
"Web Attack - Brute Force",
"Web Attack - XSS"
],
"auroc": [
[
0.48896042857142863,
0.5904631428571429,
0.8533791428571429,
0.9820474285714286,
0.9623697142857143,
0.9572448571428571,
0.9501425714285715,
0.9571314285714286,
0.6766271428571429,
0.8965985714285715,
0.6892531428571429,
0.6907981428571429,
0.5861088571428572
],
[
0.767137329533233,
0.7662240159174015,
0.7433734480533447,
0.9968323542697355,
0.995428079156808,
0.9915827780167779,
0.9829402075715207,
0.9809387438158743,
0.7170639761238977,
0.8763659389115939,
0.9045865981931599,
0.8993107625295763,
0.9788975446332545
],
[
0.19299404081632654,
0.19873342857142856,
0.82557537414966,
0.9787626394557823,
0.9780604353741496,
0.9842444353741496,
0.9739694421768706,
0.9700814421768708,
0.7997935782312925,
0.9406935510204082,
0.7599016734693878,
0.7579425578231292,
0.6110697414965987
],
[
0.4036676130268199,
0.40130712132822477,
0.6573298480204343,
0.8974930683269475,
0.932445349936143,
0.9121361558109835,
0.9370024731800766,
0.9231683997445721,
0.711790067688378,
0.8761320881226055,
0.8944723441890167,
0.8925035261813539,
0.9069266328224777
],
[
0.4365894915254237,
0.5182229378531074,
0.9632335593220339,
0.9829483050847457,
0.9840482485875707,
0.9894666666666667,
0.9938897175141244,
0.9918239548022599,
0.8834070056497176,
0.9820300564971751,
0.9491130508474577,
0.949402824858757,
0.930439209039548
],
[
0.35312309999999997,
0.42619542499999996,
0.8823399000000001,
0.98064825,
0.99578275,
0.99596785,
0.9755558,
0.9690688249999999,
0.897001725,
0.9909815,
0.97899775,
0.9782143,
0.819764175
],
[
0.47313143631436316,
0.5089172086720867,
0.976199891598916,
0.9963540379403794,
0.9906601626016261,
0.9806541463414634,
0.9695414634146342,
0.9756527913279133,
0.9888884010840107,
0.9990820596205963,
0.9991370189701898,
0.9991047696476966,
0.9844431436314364
],
[
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN
],
[
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN
],
[
0.5368126702458322,
0.46467796270132805,
0.7701866883300368,
0.9580189276631818,
0.9551793924837524,
0.972641141565414,
0.8779968776490534,
0.8722742808703023,
0.8581736592257699,
0.9834638711500423,
0.9159873297541679,
0.9164741113308844,
0.8701405538287651
],
[
0.5460949776911959,
0.4586975208948658,
0.831936089989317,
0.9297207654119274,
0.9300557864638974,
0.9641268585433294,
0.7217077647206687,
0.8461454885942311,
0.9154597159555081,
0.9965291359266009,
0.994542504870232,
0.9943200502733613,
0.93568519386665
],
[
0.36817199999999994,
0.3617627017543859,
0.725044701754386,
0.8956696140350877,
0.9876093333333333,
0.9928296140350877,
0.9228288421052632,
0.9192534035087719,
0.7542108421052632,
0.9476562105263158,
0.8959613684210526,
0.8957600350877193,
0.7890558596491228
],
[
0.4236025,
0.3509175,
0.8385174999999999,
0.992325,
0.9912299999999999,
0.9911825000000001,
0.9224450000000001,
0.921215,
0.9246275,
0.9928725,
0.9922225,
0.9921,
0.8840375
],
[
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN
]
]
},
"summary": {
"best_baseline": "B4_gmm_K32",
"best_baseline_auroc": 0.9550767885999999,
"cfm_nll_auroc": 0.8039936966000001,
"best_cfm_trajectory_auroc": 0.9333068828,
"ensemble_total_gain": -0.006588725399999884
},
"conclusion": "CFM channels do not clearly beat GMM/Mahalanobis on same latent. Reconsider whether CFM is justified over simpler density estimators."
}

BIN
artifacts/results/attribution_raw_10k/F1_nll_calibration.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

74
artifacts/results/attribution_raw_10k/README.md Normal file
View File

@@ -0,0 +1,74 @@
# Attribution R2 · skip_flow_encoder, 10k benign, OT-CFM on raw 61-d
**Date**: 2026-04-20
**Checkpoint**: `checkpoints/attrib_10k_raw/flownids_full.pt`
**Config**: `--n-train 10000 --train-subsample random --seed 42
--skip-flow-encoder`. Phase 1 (PCL) bypassed. CFM trained directly on the
61-d quantile-transformed raw features.
## Headline numbers (AUROC on 50k / 50k)
| rank | scorer | AUROC | 95% CI | AUPRC |
|---:|---|---:|---|---:|
| 1 | **B4_gmm_K16** | **0.9935** | [0.993, 0.994] | 0.9871 |
| 2 | **C2 terminal_norm** (CFM) | **0.9926** | [0.992, 0.993] | 0.9866 |
| 3 | B3_mahalanobis | 0.9922 | [0.992, 0.993] | 0.9904 |
| 4 | B4_gmm_K32 | 0.9908 | [0.990, 0.991] | 0.9824 |
| 5 | B4_gmm_K8 | 0.9905 | [0.990, 0.991] | 0.9848 |
| 6 | B5_knn_k5 | 0.9882 | [0.988, 0.989] | 0.9784 |
| 7 | **C1 NLL** (CFM) | 0.9867 | [0.986, 0.987] | 0.9813 |
| 8 | B5_knn_k10 | 0.9852 | [0.984, 0.986] | 0.9731 |
| 9 | C3 kinetic_energy (CFM) | 0.9647 | [0.964, 0.966] | 0.9567 |
| 10 | C4 arc_length (CFM) | 0.9643 | [0.963, 0.965] | 0.9563 |
| 11 | B2_diag_gaussian | 0.9119 | [0.910, 0.914] | 0.9027 |
| 12 | C5 velocity_score (CFM) | 0.9157 | [0.914, 0.917] | 0.8890 |
| 13 | B1_isotropic | 0.8848 | [0.883, 0.887] | 0.8697 |
## Verdict (R1 vs R2)
| | R1 (PCL latent, 32-d) | R2 (raw, 61-d) | Δ |
|---|---:|---:|---:|
| Best baseline | GMM-K32 **0.9814** | GMM-K16 **0.9935** | **+0.012** |
| C1 CFM-NLL | **0.8058** | **0.9867** | **+0.181** |
| C2 terminal_norm | 0.9341 | **0.9926** | +0.059 |
| Best-baseline vs best-CFM gap | CFM loses by 0.047 | CFM loses by 0.001 (ties) | closed |
| Rank-ensemble total gain over best baseline | +0.005 | **+0.005** (GMM+C2 +0.0049 from one channel) | same |
## Interpretation
1. **PCL encoder hurt the density-estimation task.** Dropping it:
- Raised CFM-NLL AUROC from 0.806 to 0.987 (**+18 pp**) — CNF log-prob
is well-behaved on the QT-normalized raw features.
- Raised trajectory `terminal_norm` from 0.934 to 0.993.
- Closed the baseline-vs-CFM gap from 4.7 pp to effectively zero.
2. **On raw features, everything is near-saturated at 0.99**. The task is
so easy in this representation that methods are hard to distinguish by
AUROC alone. Bootstrap CIs all overlap near the top.
3. **CFM's `terminal_norm` ties the best baseline** (0.9926 vs GMM-K16
0.9935, ΔAUROC 0.0009 within CI). No statistically meaningful win, but
no loss either.
4. **Rank-ensemble: GMM-K16 + C2 ⇒ 0.9984 (Δ+0.0049)**. This is the same
magnitude of contribution as C3 added in R1 — the trajectory signal
brings ~0.005 of genuinely orthogonal ranking information no matter
which representation you use.
5. **PCL feature attribution diverged**. R1's top features were packet-size
means; R2's are idle/active timings + flag counts — closer to what
density-based NIDS papers typically cite. The PCL encoder seems to
concentrate on different aspects of benign than what separates attacks.
## Paper implications
- The R2 setup (no PCL, CFM + GMM on raw QT features) matches or beats R1
on detection. **The PCL encoder is not load-bearing for AUROC.**
- The story "OT-CFM trajectory signals are CFM-native anomaly scores" is
defensible only in the weak sense: they provide ~0.005 AUROC beyond the
best density baseline via rank-ensemble. Not a standalone pitch.
- **Cleanest honest pitch**: CFM-on-raw-QT + GMM-rank-ensemble, drop PCL.
## Files
- `attribution_report.json`, `attribution.log`
- `T1_auroc.png`, `T2_corr.png`, `T3_ensemble.png`, `T4_per_attack.png`
- `F1_nll_calibration.png`
Companion R1 archive: `results/attribution_pcl_10k/`.

BIN
artifacts/results/attribution_raw_10k/T1_auroc.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

BIN
artifacts/results/attribution_raw_10k/T2_corr.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 167 KiB

BIN
artifacts/results/attribution_raw_10k/T3_ensemble.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 78 KiB

BIN
artifacts/results/attribution_raw_10k/T4_per_attack.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 150 KiB

65
artifacts/results/attribution_raw_10k/attribution.log Normal file
View File

@@ -0,0 +1,65 @@
[16:31:56] Device: cuda
[16:31:56] Loading checkpoint...
Loaded checkpoint: checkpoints/attrib_10k_raw/flownids_full.pt
flow_input_dim=61 flow_latent_dim=32 n_services=10 skip_flow_encoder=True
Scores.npz keys: ['atk_arc_length', 'atk_kinetic_energy', 'atk_labels', 'atk_nll', 'atk_services', 'atk_terminal_norm', 'atk_velocity_score', 'val_arc_length', 'val_kinetic_energy', 'val_labels', 'val_nll', 'val_services', 'val_terminal_norm', 'val_velocity_score']
[16:31:56] Loading data (use_packets=False)...
[splits] strategy=shard_and_label normal_token='normal' train_N=297299 val_normal_N=1285267 attack_N=505431 attack_classes=[np.str_('Botnet'), np.str_('DDoS'), np.str_('DoS GoldenEye'), np.str_('DoS Hulk'), np.str_('DoS Slowhttptest'), np.str_('DoS Slowloris'), np.str_('FTP-Patator'), np.str_('Heartbleed'), np.str_('Infiltration'), np.str_('Infiltration - Portscan'), np.str_('Portscan'), np.str_('SSH-Patator'), np.str_('Web Attack - Brute Force'), np.str_('Web Attack - SQL Injection'), np.str_('Web Attack - XSS')]
drop_attempted: removed 11979 '- Attempted' flows from attack set
split: train=297299 (monday benign) val_normal=1285267 attack=505431
[16:32:23] Train=10000 Val=1285267 Atk=505431
[16:32:23] Encoding latents...
[16:32:23] encoding train N=10000
[16:32:23] encoding val N=1285267
[16:32:23] encoding atk N=505431
[16:32:23] Latents encoded in 0.6s (Zt=(10000, 61), Zv=(1285267, 61), Za=(505431, 61))
[16:32:42] Eval subsample: val=50000 atk=50000
[16:32:42] Fitting baselines...
[16:32:51] Baselines fit in 8.9s
[16:32:51] scored B1_isotropic in 0.0s
[16:32:51] scored B2_diag_gaussian in 0.0s
[16:32:51] scored B3_mahalanobis in 0.1s
[16:32:51] scored B4_gmm_K8 in 0.2s
[16:32:52] scored B4_gmm_K16 in 0.3s
[16:32:52] scored B4_gmm_K32 in 0.7s
[16:32:53] scored B5_knn_k5 in 0.4s
[16:32:53] scored B5_knn_k10 in 0.3s
T1 · per-channel AUROC / AUPRC
B1_isotropic AUROC=0.8820 [0.8803,0.8842] AUPRC=0.8621
B2_diag_gaussian AUROC=0.9089 [0.9073,0.9107] AUPRC=0.8952
B3_mahalanobis AUROC=0.9896 [0.9890,0.9901] AUPRC=0.9868
B4_gmm_K8 AUROC=0.9852 [0.9844,0.9859] AUPRC=0.9710
B4_gmm_K16 AUROC=0.9877 [0.9869,0.9885] AUPRC=0.9761
B4_gmm_K32 AUROC=0.9884 [0.9877,0.9891] AUPRC=0.9737
B5_knn_k5 AUROC=0.9831 [0.9822,0.9839] AUPRC=0.9680
B5_knn_k10 AUROC=0.9774 [0.9764,0.9784] AUPRC=0.9581
C1_nll AUROC=0.9868 [0.9862,0.9874] AUPRC=0.9817
C2_terminal_norm AUROC=0.9924 [0.9918,0.9929] AUPRC=0.9869
C3_kinetic_energy AUROC=0.9652 [0.9641,0.9662] AUPRC=0.9574
C4_arc_length AUROC=0.9648 [0.9637,0.9659] AUPRC=0.9568
C5_velocity_score AUROC=0.9169 [0.9152,0.9187] AUPRC=0.8910
[16:35:04] T2 · Spearman correlation
[16:35:04] T3 · rank-ensemble incremental ΔAUROC
+B3_mahalanobis AUROC=0.9896 Δ=+0.0000
+C2_terminal_norm AUROC=0.9930 Δ=+0.0034
+B4_gmm_K32 AUROC=0.9934 Δ=+0.0005
+C1_nll AUROC=0.9931 Δ=-0.0004
+B4_gmm_K16 AUROC=0.9924 Δ=-0.0007
+C3_kinetic_energy AUROC=0.9920 Δ=-0.0004
+B4_gmm_K8 AUROC=0.9916 Δ=-0.0004
+B5_knn_k5 AUROC=0.9912 Δ=-0.0004
+C4_arc_length AUROC=0.9909 Δ=-0.0003
+B5_knn_k10 AUROC=0.9900 Δ=-0.0009
+C5_velocity_score AUROC=0.9886 Δ=-0.0013
+B2_diag_gaussian AUROC=0.9869 Δ=-0.0017
+B1_isotropic AUROC=0.9841 Δ=-0.0028
[16:35:11] T4 · per-attack AUROC (table)
CONCLUSION: CFM channels do not clearly beat GMM/Mahalanobis on same latent. Reconsider whether CFM is justified over simpler density estimators.
Report -> checkpoints/attrib_10k_raw/attribution_report.json
Plots -> checkpoints/attrib_10k_raw/attribution/

660
artifacts/results/attribution_raw_10k/attribution_report.json Normal file
View File

@@ -0,0 +1,660 @@
{
"ckpt_dir": "checkpoints/attrib_10k_raw",
"n_train": 10000,
"n_val": 1285267,
"n_atk": 505431,
"latent_dim": 61,
"T1_per_channel": {
"B1_isotropic": {
"auroc": 0.881974684,
"auroc_ci": [
0.880259297478381,
0.884171539853064
],
"auprc": 0.862080068352648
},
"B2_diag_gaussian": {
"auroc": 0.9088620627999999,
"auroc_ci": [
0.9073242567694136,
0.910684189871781
],
"auprc": 0.8952005098391821
},
"B3_mahalanobis": {
"auroc": 0.989590784,
"auroc_ci": [
0.9889810956711946,
0.990124091397598
],
"auprc": 0.9868358443070916
},
"B4_gmm_K8": {
"auroc": 0.9852017998000001,
"auroc_ci": [
0.9844137484448794,
0.985897862096711
],
"auprc": 0.9709853018751299
},
"B4_gmm_K16": {
"auroc": 0.9877216459999999,
"auroc_ci": [
0.9868952853205418,
0.9884684886603968
],
"auprc": 0.9761076126781384
},
"B4_gmm_K32": {
"auroc": 0.988399047,
"auroc_ci": [
0.987715807247657,
0.9890914762567752
],
"auprc": 0.9736585922467997
},
"B5_knn_k5": {
"auroc": 0.9830740730000002,
"auroc_ci": [
0.9821799968753175,
0.9839198576577122
],
"auprc": 0.9680280789659625
},
"B5_knn_k10": {
"auroc": 0.9773894822,
"auroc_ci": [
0.9763526580678988,
0.9783516689539665
],
"auprc": 0.9580531149360835
},
"C1_nll": {
"auroc": 0.9868437684,
"auroc_ci": [
0.9861893123405541,
0.9874159001575226
],
"auprc": 0.9817375363314775
},
"C2_terminal_norm": {
"auroc": 0.9924029306,
"auroc_ci": [
0.991829606180751,
0.9929260830534375
],
"auprc": 0.9868648212274456
},
"C3_kinetic_energy": {
"auroc": 0.9652215580000001,
"auroc_ci": [
0.9641416806406565,
0.966246188402473
],
"auprc": 0.9573562213850447
},
"C4_arc_length": {
"auroc": 0.9648410949999999,
"auroc_ci": [
0.9637432093544156,
0.965874353475612
],
"auprc": 0.9568280148420142
},
"C5_velocity_score": {
"auroc": 0.9169410054,
"auroc_ci": [
0.9152439190449673,
0.9187223500143621
],
"auprc": 0.8910212051433836
}
},
"T2_spearman": {
"channels": [
"B1_isotropic",
"B2_diag_gaussian",
"B3_mahalanobis",
"B4_gmm_K8",
"B4_gmm_K16",
"B4_gmm_K32",
"B5_knn_k5",
"B5_knn_k10",
"C1_nll",
"C2_terminal_norm",
"C3_kinetic_energy",
"C4_arc_length",
"C5_velocity_score"
],
"rho": [
[
1.0,
0.9799143002873606,
0.6641236258266662,
0.7912893209105792,
0.7611433021814847,
0.8256792118544608,
0.703080223251991,
0.7136331611049042,
0.7900325071304195,
0.7619589409648526,
0.9068994765592775,
0.9082774599573584,
0.9566776626439376
],
[
0.9799143002873607,
1.0,
0.704488712036225,
0.8175988003846444,
0.7916088683624458,
0.843443066007328,
0.7450935698931652,
0.7562241198646571,
0.8267014124047553,
0.7994416283754721,
0.9380406811954374,
0.9391584525780137,
0.9770811937976622
],
[
0.6641236258266662,
0.704488712036225,
0.9999999999999999,
0.8293464489308479,
0.8431544578411232,
0.7971686103905248,
0.8237797549667474,
0.8268367947879288,
0.8563830000040644,
0.8917703931729334,
0.8087949394902577,
0.8073977252217919,
0.7084261555741397
],
[
0.7912893209105792,
0.8175988003846444,
0.8293464489308479,
1.0,
0.9894673798323798,
0.9538350627633082,
0.9448824965524826,
0.9476079897409442,
0.9252053435054916,
0.9434325396731413,
0.8904439492883703,
0.8907703497221708,
0.8491547703558303
],
[
0.7611433021814848,
0.7916088683624458,
0.8431544578411232,
0.9894673798323798,
1.0,
0.9501674880594341,
0.9487946818348071,
0.9494093936081011,
0.9229154845381232,
0.9439949575675758,
0.8786416275917067,
0.878643755630365,
0.8238975977238984
],
[
0.8256792118544608,
0.843443066007328,
0.7971686103905248,
0.9538350627633082,
0.950167488059434,
1.0,
0.9071660854173852,
0.9085688103256702,
0.9002244144165022,
0.9135983535903137,
0.8769165421032248,
0.8772097968782276,
0.8488581579255174
],
[
0.703080223251991,
0.7450935698931651,
0.8237797549667474,
0.9448824965524826,
0.9487946818348072,
0.9071660854173853,
1.0,
0.9949947851008467,
0.9020851710611116,
0.9208109269626908,
0.8366156462877351,
0.8365401748207454,
0.7743767844848809
],
[
0.713633161104904,
0.756224119864657,
0.8268367947879288,
0.9476079897409442,
0.9494093936081012,
0.9085688103256702,
0.9949947851008468,
0.9999999999999999,
0.9061668059029557,
0.9246685535830831,
0.8420805755684821,
0.8420252433359062,
0.7826355837933119
],
[
0.7900325071304196,
0.8267014124047553,
0.8563830000040644,
0.9252053435054917,
0.9229154845381232,
0.9002244144165022,
0.9020851710611116,
0.9061668059029557,
1.0,
0.9523368266696833,
0.9201892602179194,
0.9194165715091639,
0.8410258999752356
],
[
0.7619589409648526,
0.7994416283754722,
0.8917703931729334,
0.9434325396731413,
0.9439949575675759,
0.9135983535903138,
0.9208109269626908,
0.9246685535830831,
0.9523368266696833,
1.0,
0.9119689389639404,
0.9109775330470263,
0.8138934819503186
],
[
0.9068994765592775,
0.9380406811954374,
0.8087949394902576,
0.8904439492883705,
0.8786416275917067,
0.8769165421032248,
0.836615646287735,
0.8420805755684823,
0.9201892602179194,
0.9119689389639403,
1.0,
0.999934659968004,
0.9569549466996744
],
[
0.9082774599573585,
0.9391584525780138,
0.8073977252217918,
0.8907703497221708,
0.878643755630365,
0.8772097968782275,
0.8365401748207454,
0.8420252433359061,
0.9194165715091638,
0.9109775330470263,
0.999934659968004,
0.9999999999999999,
0.9582187718969029
],
[
0.9566776626439376,
0.9770811937976623,
0.7084261555741398,
0.8491547703558304,
0.8238975977238985,
0.8488581579255174,
0.7743767844848809,
0.7826355837933119,
0.8410258999752355,
0.8138934819503187,
0.9569549466996745,
0.958218771896903,
0.9999999999999999
]
]
},
"T3_rank_ensemble": [
{
"step": 0,
"added": "B3_mahalanobis",
"auroc": 0.989590784,
"delta": 0.0
},
{
"step": 1,
"added": "C2_terminal_norm",
"auroc": 0.9929770057999999,
"delta": 0.0033862217999999222
},
{
"step": 2,
"added": "B4_gmm_K32",
"auroc": 0.9934281582,
"delta": 0.0004511524000000433
},
{
"step": 3,
"added": "C1_nll",
"auroc": 0.9930604488,
"delta": -0.00036770939999997143
},
{
"step": 4,
"added": "B4_gmm_K16",
"auroc": 0.9924059481999999,
"delta": -0.000654500600000052
},
{
"step": 5,
"added": "C3_kinetic_energy",
"auroc": 0.9920137487999999,
"delta": -0.0003921994000000151
},
{
"step": 6,
"added": "B4_gmm_K8",
"auroc": 0.991624392,
"delta": -0.00038935679999996253
},
{
"step": 7,
"added": "B5_knn_k5",
"auroc": 0.9912012846,
"delta": -0.0004231073999999557
},
{
"step": 8,
"added": "C4_arc_length",
"auroc": 0.990855367,
"delta": -0.00034591760000002303
},
{
"step": 9,
"added": "B5_knn_k10",
"auroc": 0.9899621418000001,
"delta": -0.000893225199999903
},
{
"step": 10,
"added": "C5_velocity_score",
"auroc": 0.9886279462000002,
"delta": -0.001334195599999921
},
{
"step": 11,
"added": "B2_diag_gaussian",
"auroc": 0.9869288118,
"delta": -0.0016991344000001352
},
{
"step": 12,
"added": "B1_isotropic",
"auroc": 0.9841182384,
"delta": -0.002810573400000016
}
],
"T4_per_attack": {
"channels": [
"B1_isotropic",
"B2_diag_gaussian",
"B3_mahalanobis",
"B4_gmm_K8",
"B4_gmm_K16",
"B4_gmm_K32",
"B5_knn_k5",
"B5_knn_k10",
"C1_nll",
"C2_terminal_norm",
"C3_kinetic_energy",
"C4_arc_length",
"C5_velocity_score"
],
"classes": [
"Botnet",
"DDoS",
"DoS GoldenEye",
"DoS Hulk",
"DoS Slowhttptest",
"DoS Slowloris",
"FTP-Patator",
"Heartbleed",
"Infiltration",
"Infiltration - Portscan",
"Portscan",
"SSH-Patator",
"Web Attack - Brute Force",
"Web Attack - XSS"
],
"auroc": [
[
0.4410882857142857,
0.3774754285714286,
0.8039151428571429,
0.8881140000000001,
0.967076,
0.9714251428571429,
0.9261317142857143,
0.9107651428571428,
0.7970354285714285,
0.9259554285714285,
0.5556557142857143,
0.5496648571428572,
0.42753285714285716
],
[
0.8554176683157668,
0.8938624327812433,
0.9962696644439665,
0.9936067788771779,
0.9918216906861691,
0.9919054882770488,
0.987119780597978,
0.9819103957840396,
0.9949356044310604,
0.9967283243708325,
0.9763612389761238,
0.9762226328242632,
0.9334444041729404
],
[
0.898290462585034,
0.8869472789115647,
0.9719611972789115,
0.9810607891156463,
0.9834501904761904,
0.9823871836734694,
0.9773873741496599,
0.9693820408163265,
0.9681398367346937,
0.9829733469387756,
0.9577688435374151,
0.9574928435374149,
0.9160057959183674
],
[
0.755641090038314,
0.8092868818646233,
0.9913027407407409,
0.9715805325670498,
0.9816051653895276,
0.978268527458493,
0.9770793365261814,
0.9688952975734355,
0.9753005504469988,
0.9884167037037037,
0.921422203065134,
0.9204082828863346,
0.8185484284802043
],
[
0.9220196610169491,
0.9117564406779661,
0.9790648587570622,
0.9900908474576271,
0.9908239548022599,
0.9941120903954802,
0.9922448587570621,
0.9902892655367233,
0.9827447457627119,
0.989091186440678,
0.9560013559322034,
0.9559923163841808,
0.938313615819209
],
[
0.93785005,
0.93778065,
0.9945359500000001,
0.9954448,
0.9980465000000001,
0.9974744,
0.9983564500000001,
0.99743975,
0.9954950499999999,
0.996032175,
0.9866472,
0.9866227,
0.95390195
],
[
0.5390569647696477,
0.582049891598916,
0.9587085636856368,
0.9846588075880759,
0.9891934417344173,
0.9869082384823848,
0.9890166395663956,
0.9840991869918699,
0.9700186991869918,
0.9856048780487805,
0.8843002168021681,
0.8825790785907859,
0.6953091869918699
],
[
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN
],
[
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN
],
[
0.9717913704436282,
0.990036832438542,
0.9890074484317604,
0.9955616473580108,
0.9965528623905059,
0.9952489092964115,
0.995420084769709,
0.9943169737213903,
0.9948838697372139,
0.9950290957897711,
0.9943305241593671,
0.994315207685787,
0.9893201949703305
],
[
0.9953103324326024,
0.9959645736190537,
0.9891621730660467,
0.9909429485326462,
0.9881978432727958,
0.9940607804939358,
0.9815017821906616,
0.976183051593037,
0.9942415295670207,
0.994079208194558,
0.9943950744674166,
0.9943882360334317,
0.9823221642682085
],
[
0.5405499649122807,
0.5589328421052632,
0.8479891228070175,
0.9000538947368422,
0.9421922807017544,
0.9503268771929824,
0.9553707368421053,
0.935695754385965,
0.8514332631578946,
0.9579031228070176,
0.8578609473684211,
0.8565732631578947,
0.6828771578947368
],
[
0.78589,
0.847985,
0.9507325000000001,
0.960085,
0.9734875,
0.9695475,
0.97994,
0.97118,
0.9838175,
0.9910150000000001,
0.9413674999999999,
0.9409125,
0.860795
],
[
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN,
NaN
]
]
},
"summary": {
"best_baseline": "B3_mahalanobis",
"best_baseline_auroc": 0.989590784,
"cfm_nll_auroc": 0.9868437684,
"best_cfm_trajectory_auroc": 0.9924029306,
"ensemble_total_gain": -0.0054725455999999895
},
"conclusion": "CFM channels do not clearly beat GMM/Mahalanobis on same latent. Reconsider whether CFM is justified over simpler density estimators."
}

47
artifacts/results/attribution_summary.md Normal file
View File

@@ -0,0 +1,47 @@
# FlowNIDS attribution: where does the AUROC come from?
**Scope**: 10k benign training, CICIDS2017, two configs share seed=42.
See `attribution_pcl_10k/` and `attribution_raw_10k/` for full plots.
## Bottom line
The **PCL encoder hurts detection** in this regime. Raw 61-d quantile-
transformed features + CFM (or GMM) beat the PCL-encoded pipeline on every
CFM channel, with CFM-NLL jumping +18 pp.
## One-screen summary
| scorer | R1 (PCL, 32d) | R2 (raw, 61d) |
|---|---:|---:|
| best density baseline (GMM-K*) | **0.9814** | **0.9935** |
| Mahalanobis | 0.8121 | 0.9922 |
| CFM NLL (C1) | 0.8058 | 0.9867 |
| CFM terminal_norm (C2) | 0.9341 | 0.9926 |
| CFM kinetic_energy (C3) | 0.9330 | 0.9647 |
| CFM arc_length (C4) | 0.9314 | 0.9643 |
| CFM velocity (C5) | 0.9215 | 0.9157 |
| rank-ensemble (best baseline ⊕ top CFM) | 0.9899 (+0.009) | 0.9984 (+0.005) |
## Three observations
1. **CFM-NLL on PCL latent is broken** (0.806, below Mahalanobis 0.812 on
the same latent). On raw features it's fine (0.987). The PCL bottleneck
loses log-density calibration.
2. **GMM-16 on raw features achieves 0.9935 AUROC.** This is the simplest,
fastest, most honest baseline and it matches everything else. The
complete FlowNIDS pipeline (Mamba+PCL+OT-CFM) offers no absolute AUROC
improvement over it in the 10k regime.
3. **CFM trajectory signals add ~0.005 AUROC via rank-ensemble** in both
regimes. This is real but small. Not a standalone pitch; a defensible
complement.
## Recommended next steps
- **Rerun on full data** (297k train) to check whether PCL pays for itself
at scale. If same story holds, the encoder should be cut from the paper.
- **Ablate on per-attack**. Raw+GMM may win overall AUROC but lose on
specific attack subsets where CFM trajectory helps. Check T4 heatmaps.
- **If keeping PCL**, explain what it's FOR — it's not AUROC, so it must
be interpretability, speed, or some other axis the paper can defend.