Initial commit: code, paper, small artifacts
This commit is contained in:
103
artifacts/baselines/COMPARISON_TABLE.md
Normal file
103
artifacts/baselines/COMPARISON_TABLE.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# Unified_CFM vs Baselines — Performance Comparison Table
|
||||
|
||||
Live tracking. Last updated: 2026-04-30.
|
||||
|
||||
Two reproduction modes are tracked separately:
|
||||
|
||||
- **Method reproduction** (≡ "ours-method-only"): we use the baseline's
|
||||
model/training algorithm but feed it our 20-d packet-derived canonical flow
|
||||
features (or 9-d packet sequences for AT/Kitsune). Tests whether the
|
||||
baseline's architecture beats Unified_CFM on the **same data substrate**.
|
||||
|
||||
- **True reproduction** (≡ "ours-true-repro"): we use the baseline's full
|
||||
pipeline — including their feature engineering (CICFlowMeter CSV /
|
||||
AfterImage / image encoding). Tests whether their published numbers
|
||||
reproduce on our datasets.
|
||||
|
||||
"paper" = the number quoted in the source paper; not run by us.
|
||||
|
||||
## Headline AUROC table
|
||||
|
||||
Two columns per baseline:
|
||||
- "ours-method-only" = baseline model on our 20-d / 9-d features, 3-seed mean±std.
|
||||
- "ours-true-repro" = baseline's full pipeline (CICFlowMeter CSV + their feature subsets), single seed for now.
|
||||
|
||||
Raw AUROC reported. abs() = sign-agnostic max(AUROC, 1−AUROC) for inverted-signal baselines.
|
||||
|
||||
| Protocol | **Unified_CFM** | Shafir NF paper | Shafir NF method-only (ours) | **Shafir NF true-repro (ours)** | Kitsune paper | Kitsune Path B method-only (ours) | AT method-only (ours) |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| ISCXTor2016 within | **0.9945 ± 0.0011** | 0.8731 | 0.9422 ± 0.0075 | **0.7562** [^4] | 0.7800 | 0.5653 ± 0.0226 | 0.4122 ± 0.0503 (abs 0.59) |
|
||||
| CICIDS2017 within (σ=0.6) | **0.9858 ± 0.0021** | 0.9303 | 0.9256 ± 0.0188 | **0.8678** [^4] | 0.8500 | 0.7023 ± 0.0310 | 0.5009 ± 0.2107 (abs 0.66) |
|
||||
| CICDDoS2019 within | **0.9960 ± 0.0010** | 0.9300 | 0.8903 ± 0.0386 | 0.5926 [^5] | — | 0.4710 ± 0.0039 | 0.4777 ± 0.3325 (abs 0.75) |
|
||||
| IDS2017→DDoS2019 forward | 0.9109 ± 0.0032 | 0.8900 | **0.9210 ± 0.0111** | 0.7831 | — | 0.4905 ± 0.0751 | 0.5404 ± 0.1495 (abs 0.63) |
|
||||
| DDoS2019→IDS2017 reverse | 0.5999 (single) [^1] | **0.9300** | 0.7247 ± 0.0035 | 0.7473 | — | 0.7483 ± 0.0137 | 0.4767 ± 0.2597 (abs 0.70) |
|
||||
| CICIoT2023 within (single seed) | **0.9618** [^2] | F1=0.9951 [^3] | 0.8996 [^2] | 0.7398 [^4] | — | — | — |
|
||||
|
||||
**Bold** = best per row.
|
||||
|
||||
[^1]: Reverse `terminal_norm` 0.5999 single-seed. Our **PNA** score on this
|
||||
protocol is 0.9089 (3-seed mean) — beats both Shafir NF reproduced (0.7247)
|
||||
and Shafir paper (0.93).
|
||||
|
||||
[^2]: CICIoT2023 single seed. Our 20-d canonical features. Shafir NF (ours)
|
||||
used 5-d Shafir-selected features (HTTPS, Protocol_Type, Magnitude,
|
||||
Variance, fin_count) computed from our packets.
|
||||
|
||||
[^3]: Shafir paper Table VIII reports F1=0.9951 (not AUROC) on CICIoT2023.
|
||||
They used the official CICFlowMeter pipeline on full pcap; not directly
|
||||
AUROC-comparable. Our F1@P95 on this protocol = **0.9463** (with our 20-d
|
||||
canonical features).
|
||||
|
||||
[^4]: Shafir NF true-reproduction uses their **paper-specified SHAP-selected
|
||||
feature subsets**: ISCXTor 4 features (Flow IAT Std, Flow Bytes/s,
|
||||
Flow Packets/s, Bwd IAT Max — paper §V-A), CICIDS2017 5 features (Bwd
|
||||
Packet Length Mean, Fwd Packets/s, ACK Flag Count, Total Length of Bwd
|
||||
Packets, Flow Duration — paper §V-B), CICIoT2023 5 features (HTTPS,
|
||||
Protocol Type, Magnitude, Variance, fin_count — paper §V-D). Driver
|
||||
uses **single NF**; paper headline numbers (0.8731 / 0.9303 / 0.93) come
|
||||
from a **2-NF ensemble** which we don't reproduce. Expected single-NF
|
||||
underperformance vs ensemble: ~0.05-0.15 AUROC.
|
||||
|
||||
[^5]: CICDDoS2019 true-repro uses CICIDS2017 best-5 feature subset (paper
|
||||
§V-C: "comparable feature subset shared with CICIDS2017"). Result is
|
||||
weak (0.59) — those 5 features are tuned for CICIDS attacks, not DrDoS.
|
||||
Paper's 0.93 number is presumably with CICDDoS-specific feature
|
||||
selection (not specified in paper).
|
||||
|
||||
## CICIoT2023 thresholded F1 table
|
||||
|
||||
| Method | flow features | F1@P95 | F1@P99 | TPR@1%FPR |
|
||||
|---|---|---:|---:|---:|
|
||||
| **Unified_CFM 20-d canonical** | 20-d packet-derived | **0.9463** | **0.9004** | **0.8208** |
|
||||
| Unified_CFM Shafir-5 | 5-d Shafir SHAP-selected | 0.9266 | 0.8704 | 0.7726 |
|
||||
| Shafir NF (ours), Shafir-5 | 5-d Shafir SHAP-selected | 0.9053 | 0.0652 | — |
|
||||
| Shafir NF (paper) | 5-d via official CICFlowMeter | **0.9951** | — | — |
|
||||
|
||||
## Summary by within / cross direction
|
||||
|
||||
**Within-dataset (4 protocols)**: Unified_CFM beats Shafir NF reproduction on **4/4** (ISCXTor +0.052, CICIDS +0.060, CICDDoS +0.106, CICIoT2023 +0.062).
|
||||
**Forward cross (1 protocol)**: Shafir NF (ours) 0.921 narrowly edges Unified_CFM 0.911 (+0.010 — within noise).
|
||||
**Reverse cross (1 protocol)**: Shafir NF (ours) 0.725 beats Unified_CFM `terminal_norm` 0.600, but Unified_CFM `PNA` 0.909 beats both. Reverse score-of-record for Unified_CFM is PNA, not terminal_norm.
|
||||
|
||||
## Sources
|
||||
|
||||
- Unified_CFM main: `RESULTS.md`, `RESULTS_THRESHOLDED.md`
|
||||
- Unified_CFM CICIoT2023: `artifacts/runs/unified_cfm_ciciot2023_2026_04_29/{phase1/phase1_summary.json, phase1/thresholded.json}`
|
||||
- Unified_CFM PNA on reverse: `artifacts/phase_new_scores_2026_04_29/pna/reverse_cross_seed{42,43,44}.json`
|
||||
- Shafir NF (ours): `artifacts/baselines/shafir_nf_2026_04_29/summary.md`
|
||||
- Kitsune (ours): `artifacts/baselines/kitsune_2026_04_29/summary.md`
|
||||
- Shafir paper baselines: `artifacts/locked_baselines.md`
|
||||
|
||||
## Reproduction-mode coverage table
|
||||
|
||||
For each baseline, what's the achievable reproduction mode and current status:
|
||||
|
||||
| Baseline | Method-only (ours-features) | True-repro (their-features) | Comment |
|
||||
|---|---|---|---|
|
||||
| **Shafir NF** | ✅ 15 cells | ✅ 6 cells (single seed, CSV+SHAP-5) | True-repro single-NF; paper headline uses 2-NF ensemble |
|
||||
| **Kitsune** | ✅ 15 cells (Path B, KitNET on 9-d) | ❌ blocked (Path A) | Path A pcap streaming starts at 1471 pkt/s, slows to 581 pkt/s as AfterImage state grows. CICIDS2017 alone needs 7-9 hours; CICDDoS2019 / CICIoT2023 prohibitive. Single-pcap test on ISCXTor showed only 22 unique 5-tuples per pcap → flow coverage gaps. Documented as infeasible at our data scale. |
|
||||
| **Anomaly-Transformer** | ✅ 15 cells | n/a | AT has no domain-specific feature pipeline (raw time-series only); method-only IS the faithful reproduction here |
|
||||
| **ConMD** | not started | not started | Repo is just a 878-line model class — would require writing entire distillation training loop + image-like preprocessing from scratch. ~2 days work, no guaranteed paper-number reproduction. |
|
||||
| **9 image-AD methods** (FastFlow, PatchCore, PADIM, STFPM, DFKDE, DFM, DRAEM, CFlow, RD4AD via `anomalib`) | not started | not started | Requires flow→image encoder which Shafir didn't open-source. Any reproduction would be "our flow→image + their image-AD model" → method-only at best. ~2 days work. |
|
||||
| **TSLANet** | n/a | n/a | Vendored repo at the noted commit has no anomaly detection module. Skipped. |
|
||||
| **ganomaly / RD4AD / STFPM standalone** | not started | not started | Would be subsumed by anomalib path; standalone repos have no native traffic-data support. |
|
||||
@@ -0,0 +1,320 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.69,
|
||||
"loss_first_last": [
|
||||
0.14222308861303934,
|
||||
0.005586131493549181
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.36776777750000006,
|
||||
"auprc": 0.7022768182855437
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.34954356750000004,
|
||||
"auprc": 0.6604947928545277
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.4061339125,
|
||||
"auprc": 0.7025293226875595
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.36881044999999996,
|
||||
"auprc": 0.6932174063535016
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.19116588908450705
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.18985000000000002
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.18985000000000002
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.505333219470538
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.19054429674099488
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.18985000000000002
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.2037271978021978
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.20567407574391344
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.18985000000000002
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.18985000000000002
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.18985000000000002
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.19371738609112713
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.9442166468313002
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.23980361663652802
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.20412762111352134
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.8218587514585765
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.44220000000000004
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.19179731514084505
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.18974999999999997
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.18974999999999997
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.6647462852263023
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.19044361063464837
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.18974999999999997
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.2068688644688644
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.2098838142470694
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.18974999999999997
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.18974999999999997
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.18974999999999997
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.19352434052757791
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.8067483189526927
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.23951234177215186
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.20833101952277655
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.7022372812135356
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.44220000000000004
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.28356947053800163
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.2824230769230769
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.2818045536519387
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.2837037170263788
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.8967866557572151
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.288824773960217
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.28154999999999997
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.7609575845974329
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7525999999999999
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.2140569982394366
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.21365
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.21365
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.30573680614859094
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.21432414236706693
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.21365
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.22095155677655676
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.22327037871956718
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.21365
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.21365
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.21365
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.21740203836930455
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.9253638351681047
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.2627084086799277
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.22233470715835144
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.8148401400233373
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.5740000000000001
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,320 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 24.46,
|
||||
"loss_first_last": [
|
||||
0.1440289024310776,
|
||||
0.004007022972277637
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.16187093,
|
||||
"auprc": 0.6077831395684652
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.16731178000000002,
|
||||
"auprc": 0.6028034515372264
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.18686188499999998,
|
||||
"auprc": 0.556872781554051
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.1994999625,
|
||||
"auprc": 0.616710814385669
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.017933482542524647
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.005402806563039751
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.007452684859154955
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.27283804855275445
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.07414517657192077
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.009164126712328793
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.3974829535095715
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.42633196573489635
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.005454378648874089
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.007531764705882378
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.06480254614894973
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.06535786240786241
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.12263094156827128
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.5118133650519031
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.43651510494752627
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.2210940389294404
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.16269999999999996
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.017658281110116386
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.005101899827288433
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.007152772887323949
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.5034922035480859
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.06803212747631353
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.008623758561643841
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.3702902461257976
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.3996642470694319
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.005153628023352798
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.007226386554621853
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.060337873965626995
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.0621894348894349
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.12265498032092038
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.4962176903114187
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.4099449025487256
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.21177378345498785
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.16159999999999997
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.12521938227394808
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.11268424006908465
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.12095347711267607
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.12790480859010273
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.23087037037037036
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.11873750000000002
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.24098144940747496
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.24872605951307486
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.11225166805671392
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.12170260504201681
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.2215689688096754
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.22320724815724818
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.22364801695428396
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.2383819204152249
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.24979988755622184
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.21808619221411196
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.4455
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.05750152193375112
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.04537931778929188
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.05087596830985915
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.114456162464986
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.14196339362618432
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.05058343321917808
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.4329668641750228
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.46167132551848505
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.04534462051709758
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.051506092436974786
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.13326954169318905
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.13529152334152333
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.1905550408719346
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.5306939013840829
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.4702869190404797
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.28141763990267643
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.31999999999999995
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,320 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.12,
|
||||
"loss_first_last": [
|
||||
0.13797885122932965,
|
||||
0.004073698047100555
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6323407125,
|
||||
"auprc": 0.8247269083802768
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.5918001275,
|
||||
"auprc": 0.7644950552709932
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.8401825849999998,
|
||||
"auprc": 0.9044669345849929
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.6628086200000001,
|
||||
"auprc": 0.8249129023839272
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.38358046387154326
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.3384721048182587
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.8784359464627152
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.42309863195057373
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.9832278733031674
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.37689410714285715
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.9443937277580071
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.9635801412180053
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.34371287813310286
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.8914378446115288
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.9836372795969773
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.9855167865707434
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.20813423054417787
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.9565173021925643
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.9599579822616406
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.35599000000000003
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.6828000000000001
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.3705818911685994
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.3302398562975486
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.7660391013384321
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.616021359223301
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.8523205429864253
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.360296875
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.8770422153024912
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.899051279788173
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.3357694468452896
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.7734077694235588
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.8464110831234257
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.8579430455635492
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.21209106202457578
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.8926503336510964
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.8919618994826312
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.34583795321637434
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9594
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.951335816235504
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.9408696956889264
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.9935487571701721
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.49186902030008817
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.9981957466063348
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.9505358928571428
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.8902613879003559
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.8952235657546336
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.9460346585998273
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.9933953216374268
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.9987342569269522
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.9972292565947243
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.5606071679344644
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.8713183031458532
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.8944427937915743
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.6068018713450292
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.4114
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.5112844335414808
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.47495126796280646
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.8811490439770555
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.3321917034421889
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.9584939366515838
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.5054002232142857
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.9330716192170819
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.948352824360106
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.48074563526361275
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.8884563909774438
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.9560999370277078
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.9615719424460432
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.26969002340550025
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.9451080076263109
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.9463139320029565
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.3996357309941521
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.4225
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,256 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 22.45,
|
||||
"loss_first_last": [
|
||||
0.15159071482057812,
|
||||
0.004075585566814753
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6001296883333334,
|
||||
"auprc": 0.8383184815597481
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.547795575,
|
||||
"auprc": 0.7839841390353699
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.36235965666666664,
|
||||
"auprc": 0.7142202977874932
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.47742446,
|
||||
"auprc": 0.7638875860346089
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.8872391304347825
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.8693338404033378
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.8588349137931033
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.8672453141696944
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.8991410256410255
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.762110810810811
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.7410864406779661
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.46535
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.603313003492433
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.14459035543766577
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.6733217105263157
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.72576
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.7853
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.7783739130434782
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.7791971662030597
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.798263146551724
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.7894506197905534
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.8176192307692307
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.7589518918918919
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.754200847457627
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.56245
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.5414751222351571
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.14076252519893898
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.7641335526315789
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.87258
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.9582999999999999
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.20422608695652175
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.19559450625869262
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.17157586206896552
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.18297270784355632
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.190975
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.21840324324324326
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.15560000000000002
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.24850000000000003
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.7628175669383004
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.4828546206896552
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.15560000000000002
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.15560000000000002
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.15560000000000002
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.8840760869565217
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.5441651773296244
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.3994240301724138
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.4272720987390468
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.6002397435897436
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.38377540540540545
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.21240254237288134
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.33949999999999997
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.7373752735739231
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.3844263448275862
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.05923914473684209
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.05864999999999998
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.05864999999999998
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,272 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.73,
|
||||
"loss_first_last": [
|
||||
0.1467826651244224,
|
||||
0.0038189603681852923
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.25881962166666667,
|
||||
"auprc": 0.6678241607690786
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.25728142333333337,
|
||||
"auprc": 0.6590998108230994
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.29273710333333336,
|
||||
"auprc": 0.6397626176311149
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.2794696483333333,
|
||||
"auprc": 0.6818023724164499
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.3123346153846153
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.6342495588494794
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.7240725672877847
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.12875773550916603
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.31539722222222216
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.15719341317365268
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.03223434579439251
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9448
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.4443001539554714
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.05020893327711605
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.9247592896174862
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.9488333333333333
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.8862
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.9608599999999999
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.3178192307692307
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.6361651402858655
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.7227036231884058
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.1345415810109145
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.3220927777777778
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.15693502994011976
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.03367242990654206
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9434
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.4329837280909521
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.04342239907241488
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.9255346994535518
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.9508
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.8877499999999999
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.9626
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.30856794871794874
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.2570666401976355
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.3667913043478261
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.16186963017908235
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.22024166666666672
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.21525359281437126
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.14395000000000002
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.14395000000000002
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.5923504263382284
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.31523249710129647
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.1541614754098361
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.14395000000000002
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.48135
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.14395000000000002
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.34030512820512826
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.6269999823539791
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.7092333333333333
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.13366804598919146
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.2878455555555556
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.13084820359281435
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.033217757009345775
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.43779999999999997
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.4858389152060635
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.10328855802677346
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.6608251366120219
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.5921666666666667
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.93145
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.52772
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.79,
|
||||
"loss_first_last": [
|
||||
0.14415377727414988,
|
||||
0.0037091414420570754
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6436450233333333,
|
||||
"auprc": 0.8672315471451842
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.6878354799999999,
|
||||
"auprc": 0.8657705943772598
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.4987740683333334,
|
||||
"auprc": 0.8127357475982513
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.5558856366666667,
|
||||
"auprc": 0.8318653186696193
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.46004210526315786
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.5145650968544518
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.7046408296943232
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.4318485673352435
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.6630583333333333
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.5808367088607596
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.3321348214285714
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.5555688679245283
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.9853483215454448
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.19467950310559007
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.3236285714285714
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.19369999999999998
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.4877605263157895
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.63700774835614
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.8225126637554585
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.582189382362305
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.793025
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.7865386075949368
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.7114633928571429
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.5279190635066727
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.8887979731869522
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.6114167701863353
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.9434428571428571
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.18889999999999996
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.31109868421052633
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.23562989159409986
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.21711735807860263
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.22242115568290352
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.2532523809523809
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.31956424050632914
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.14125
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.44260834100322133
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.990073931172807
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.14250807453416148
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.14125
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.3135
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.5111026315789474
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.35984901368402344
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.342010807860262
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.27933460150695105
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.4002023809523809
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.39653987341772157
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.1268642857142857
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.5756708582604694
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.9719925208487281
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.054568944099378874
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.02589999999999998
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.26039999999999996
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,320 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 22.48,
|
||||
"loss_first_last": [
|
||||
0.1520787082329581,
|
||||
0.003957434124136462
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.4892238015437741,
|
||||
"auprc": 0.5018063179317858
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.4701616494007718,
|
||||
"auprc": 0.4553141881058038
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.6214469022953484,
|
||||
"auprc": 0.6244131030607472
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.5738190788137315,
|
||||
"auprc": 0.5600867877876071
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2418359693877551
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2234609693877551
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.37049260204081635
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6877126700680272
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.48235034013605443
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.22540357142857143
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5496690476190476
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5629209183673469
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.22110892857142855
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.38065323129251705
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4970219387755102
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.48847789115646256
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9474401360544217
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5612562925170068
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5692301870748299
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9008552721088435
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.378726598173516
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23988681972789117
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.22176845238095239
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.36733971088435374
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.739343112244898
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.477965731292517
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.22383358843537415
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5421444727891156
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5541784013605442
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.219068962585034
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3767042517006803
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4928329081632653
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4839550170068027
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.785248044217687
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5538645408163266
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5612477040816326
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7573638605442178
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.3706054794520548
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4279896258503401
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4071503401360545
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5898164965986394
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.36098630952380956
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7220323979591837
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4041826530612244
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7640134353741496
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7767905612244899
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.40308545918367344
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6017380952380953
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.737468962585034
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.727680612244898
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9082727040816327
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7536788265306124
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.781905612244898
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8426772108843539
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.26392294520547943
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3539172619047619
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3341599489795919
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4993686224489796
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.44331615646258504
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6230568027210884
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.33368554421768715
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6872853741496598
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7011593537414965
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.33136301020408165
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5099774659863945
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6386501700680273
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6296390306122449
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.950218962585034
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6960769557823131
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.707658843537415
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9208323979591837
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.3331678082191781
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,320 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 22.73,
|
||||
"loss_first_last": [
|
||||
0.1470018303658389,
|
||||
0.0038443528826030185
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.49693973694901483,
|
||||
"auprc": 0.49237919507338757
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.48891327950436725,
|
||||
"auprc": 0.48587457299186
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.6319110146252285,
|
||||
"auprc": 0.6692675938817987
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.5400531383302865,
|
||||
"auprc": 0.5541969096322785
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3203674319727891
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3218841836734694
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5871074829931973
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6272752551020409
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.564168537414966
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.33965544217687077
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6149467687074831
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6218015306122449
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3331803571428571
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5915131802721088
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5796848639455783
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5787874149659864
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.34615850340136056
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6246388605442177
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6211027210884353
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30889481292517007
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.45648915525114153
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3108757653061225
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.31266122448979594
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5745394557823129
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6311168367346939
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5498848639455782
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.32866666666666666
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6105522959183673
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6175331632653062
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3237489795918368
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5768287414965987
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5674460884353741
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5659906462585034
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3396127551020408
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6206904761904761
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6166451530612246
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30192755102040814
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.4538639269406393
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5342164115646258
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5300934523809524
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.811029761904762
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.18806403061224486
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8042083333333333
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5495919217687075
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7322251700680272
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7518767006802721
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5437292517006802
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8123817176870749
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8117076530612244
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.815749149659864
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.48128273809523814
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.677136649659864
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7469124149659864
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4573297619047618
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.448048401826484
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3825764455782313
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3856704931972789
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6547069727891157
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3307896258503401
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6298654761904762
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.402787925170068
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6847105442176871
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6919932823129251
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3971041666666666
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6574113095238094
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6470957482993198
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.645858843537415
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4046221088435374
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6932554421768707
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6913091836734694
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3709507653061225
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.49996963470319633
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,320 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.07,
|
||||
"loss_first_last": [
|
||||
0.1445360630750656,
|
||||
0.0036873004643859556
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.2586951960186878,
|
||||
"auprc": 0.38725463725704035
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.2539725878529352,
|
||||
"auprc": 0.37013045453745935
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.36782300426569164,
|
||||
"auprc": 0.4000846941316927
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.3272018941702214,
|
||||
"auprc": 0.409383950502048
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12040178571428571
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12222193877551019
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.1191062074829932
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4970311224489796
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.10205935374149659
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12043333333333331
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4376086734693878
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.43346173469387755
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12309634353741497
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12141343537414964
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.10477653061224489
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.10042551020408164
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3379833333333333
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5251631802721088
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4429054421768708
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.27171930272108846
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.47257134703196346
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12264251700680272
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12220187074829933
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.11917806122448979
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6788215986394558
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.10203945578231292
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12052142857142857
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.37424438775510205
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3764227891156462
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12306598639455782
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.12105663265306123
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.10475527210884356
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.10019710884353741
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3209952380952381
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4633068027210885
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3798562925170068
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2553731292517007
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.49411666666666665
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3529809523809524
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3595994897959184
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3543705782312925
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.20132899659863943
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.32573061224489797
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3585214285714286
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5194945578231294
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5092042517006803
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.36101071428571424
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3558926870748299
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3283911564625851
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3172437074829932
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.14559957482993197
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5893117346938775
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5245889455782313
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2092083333333333
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.46540730593607305
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23454574829931973
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23998571428571427
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23522534013605445
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.33707559523809527
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.20624557823129253
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2381486394557823
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5053664965986395
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5041442176870747
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2412746598639456
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23769591836734694
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.20951088435374152
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2017103741496598
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.34869957482993197
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5825056122448979
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.508903231292517
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30893324829931973
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.4550844748858447
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,64 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 21.2,
|
||||
"loss_first_last": [
|
||||
0.168125517100473,
|
||||
0.005558558188591011
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.49322682926829275,
|
||||
"auprc": 0.17516424930113023
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.5264396341463415,
|
||||
"auprc": 0.22898976744241134
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.47917290396341466,
|
||||
"auprc": 0.18524469826822748
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.4372799923780488,
|
||||
"auprc": 0.1486540527683511
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.49322682926829275
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5264396341463415
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.47917290396341466
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.4372799923780488
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,64 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 22.49,
|
||||
"loss_first_last": [
|
||||
0.1598462136108664,
|
||||
0.003827647895469696
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.41331875,
|
||||
"auprc": 0.19291385713283932
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.4134266768292683,
|
||||
"auprc": 0.22173333659746425
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.47364451219512194,
|
||||
"auprc": 0.12669803046008787
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.35424939024390245,
|
||||
"auprc": 0.11785068848066366
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.41331875
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.4134266768292683
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.47364451219512194
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.35424939024390245
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,64 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 20.76,
|
||||
"loss_first_last": [
|
||||
0.15891090356096438,
|
||||
0.006547442672750618
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.48698502286585366,
|
||||
"auprc": 0.1969998720953106
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.5150377286585366,
|
||||
"auprc": 0.22484916372486033
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5020458079268293,
|
||||
"auprc": 0.1872525491455697
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.4449618902439024,
|
||||
"auprc": 0.18297321346276874
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.48698502286585366
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5150377286585366
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5020458079268293
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.4449618902439024
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
315
artifacts/baselines/anomaly_transformer_2026_04_29/master.log
Normal file
315
artifacts/baselines/anomaly_transformer_2026_04_29/master.log
Normal file
@@ -0,0 +1,315 @@
|
||||
=== protocol=iscxtor_within seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=iscxtor_within seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=1,312 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0274 (7.7s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0131 (14.3s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0056 (21.2s elapsed)
|
||||
[train] 21.2s, final rec_loss=0.0056
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 0.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed42.json
|
||||
[best agg=max] AUROC=0.5264 AUPRC=0.2290
|
||||
max AUROC=0.5264 AUPRC=0.2290
|
||||
mean AUROC=0.4932 AUPRC=0.1752
|
||||
median AUROC=0.4792 AUPRC=0.1852
|
||||
p90 AUROC=0.4373 AUPRC=0.1487
|
||||
[done] elapsed=33s artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed42.json
|
||||
=== protocol=iscxtor_within seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=iscxtor_within seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=1,312 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0250 (7.6s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0078 (14.9s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0038 (22.5s elapsed)
|
||||
[train] 22.5s, final rec_loss=0.0038
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 0.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed43.json
|
||||
[best agg=median] AUROC=0.4736 AUPRC=0.1267
|
||||
median AUROC=0.4736 AUPRC=0.1267
|
||||
max AUROC=0.4134 AUPRC=0.2217
|
||||
mean AUROC=0.4133 AUPRC=0.1929
|
||||
p90 AUROC=0.3542 AUPRC=0.1179
|
||||
[done] elapsed=34s artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed43.json
|
||||
=== protocol=iscxtor_within seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=iscxtor_within seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=1,312 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0230 (7.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0071 (14.0s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0065 (20.8s elapsed)
|
||||
[train] 20.8s, final rec_loss=0.0065
|
||||
[score] benign in 2.2s
|
||||
[score] attack in 0.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed44.json
|
||||
[best agg=max] AUROC=0.5150 AUPRC=0.2248
|
||||
max AUROC=0.5150 AUPRC=0.2248
|
||||
median AUROC=0.5020 AUPRC=0.1873
|
||||
mean AUROC=0.4870 AUPRC=0.1970
|
||||
p90 AUROC=0.4450 AUPRC=0.1830
|
||||
[done] elapsed=33s artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed44.json
|
||||
=== protocol=cicids_within seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicids_within seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=30,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0256 (7.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0108 (14.6s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0041 (22.5s elapsed)
|
||||
[train] 22.5s, final rec_loss=0.0041
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 6.4s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed42.json
|
||||
[best agg=mean] AUROC=0.6001 AUPRC=0.8383
|
||||
mean AUROC=0.6001 AUPRC=0.8383
|
||||
max AUROC=0.5478 AUPRC=0.7840
|
||||
p90 AUROC=0.4774 AUPRC=0.7639
|
||||
median AUROC=0.3624 AUPRC=0.7142
|
||||
[done] elapsed=142s artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed42.json
|
||||
=== protocol=cicids_within seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicids_within seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=30,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0233 (8.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0081 (16.0s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0038 (23.7s elapsed)
|
||||
[train] 23.7s, final rec_loss=0.0038
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 6.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed43.json
|
||||
[best agg=median] AUROC=0.2927 AUPRC=0.6398
|
||||
median AUROC=0.2927 AUPRC=0.6398
|
||||
p90 AUROC=0.2795 AUPRC=0.6818
|
||||
mean AUROC=0.2588 AUPRC=0.6678
|
||||
max AUROC=0.2573 AUPRC=0.6591
|
||||
[done] elapsed=141s artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed43.json
|
||||
=== protocol=cicids_within seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicids_within seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=30,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0197 (8.3s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0097 (16.1s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0037 (23.8s elapsed)
|
||||
[train] 23.8s, final rec_loss=0.0037
|
||||
[score] benign in 2.2s
|
||||
[score] attack in 6.4s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed44.json
|
||||
[best agg=max] AUROC=0.6878 AUPRC=0.8658
|
||||
max AUROC=0.6878 AUPRC=0.8658
|
||||
mean AUROC=0.6436 AUPRC=0.8672
|
||||
p90 AUROC=0.5559 AUPRC=0.8319
|
||||
median AUROC=0.4988 AUPRC=0.8127
|
||||
[done] elapsed=141s artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed44.json
|
||||
=== protocol=cicddos_within seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicddos_within seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=20,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0306 (8.3s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0127 (15.9s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0056 (23.7s elapsed)
|
||||
[train] 23.7s, final rec_loss=0.0056
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 4.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed42.json
|
||||
[best agg=median] AUROC=0.4061 AUPRC=0.7025
|
||||
median AUROC=0.4061 AUPRC=0.7025
|
||||
p90 AUROC=0.3688 AUPRC=0.6932
|
||||
mean AUROC=0.3678 AUPRC=0.7023
|
||||
max AUROC=0.3495 AUPRC=0.6605
|
||||
[done] elapsed=45s artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed42.json
|
||||
=== protocol=cicddos_within seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicddos_within seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=20,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0222 (8.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0079 (16.2s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (24.5s elapsed)
|
||||
[train] 24.5s, final rec_loss=0.0040
|
||||
[score] benign in 2.0s
|
||||
[score] attack in 4.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed43.json
|
||||
[best agg=p90] AUROC=0.1995 AUPRC=0.6167
|
||||
p90 AUROC=0.1995 AUPRC=0.6167
|
||||
median AUROC=0.1869 AUPRC=0.5569
|
||||
max AUROC=0.1673 AUPRC=0.6028
|
||||
mean AUROC=0.1619 AUPRC=0.6078
|
||||
[done] elapsed=46s artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed43.json
|
||||
=== protocol=cicddos_within seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicddos_within seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=20,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0225 (8.0s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0078 (15.6s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0041 (23.1s elapsed)
|
||||
[train] 23.1s, final rec_loss=0.0041
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 4.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed44.json
|
||||
[best agg=median] AUROC=0.8402 AUPRC=0.9045
|
||||
median AUROC=0.8402 AUPRC=0.9045
|
||||
p90 AUROC=0.6628 AUPRC=0.8249
|
||||
mean AUROC=0.6323 AUPRC=0.8247
|
||||
max AUROC=0.5918 AUPRC=0.7645
|
||||
[done] elapsed=45s artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed44.json
|
||||
=== protocol=forward_cross seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=forward_cross seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=10,000 val=10,000 attack=9,846 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0256 (7.7s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0107 (15.0s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (22.5s elapsed)
|
||||
[train] 22.5s, final rec_loss=0.0040
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 2.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed42.json
|
||||
[best agg=median] AUROC=0.6214 AUPRC=0.6244
|
||||
median AUROC=0.6214 AUPRC=0.6244
|
||||
p90 AUROC=0.5738 AUPRC=0.5601
|
||||
mean AUROC=0.4892 AUPRC=0.5018
|
||||
max AUROC=0.4702 AUPRC=0.4553
|
||||
[done] elapsed=157s artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed42.json
|
||||
=== protocol=forward_cross seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=forward_cross seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=10,000 val=10,000 attack=9,846 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0231 (7.8s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0081 (15.3s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0038 (22.7s elapsed)
|
||||
[train] 22.7s, final rec_loss=0.0038
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 2.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed43.json
|
||||
[best agg=median] AUROC=0.6319 AUPRC=0.6693
|
||||
median AUROC=0.6319 AUPRC=0.6693
|
||||
p90 AUROC=0.5401 AUPRC=0.5542
|
||||
mean AUROC=0.4969 AUPRC=0.4924
|
||||
max AUROC=0.4889 AUPRC=0.4859
|
||||
[done] elapsed=157s artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed43.json
|
||||
=== protocol=forward_cross seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=forward_cross seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=10,000 val=10,000 attack=9,846 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0195 (8.1s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0100 (15.7s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0037 (23.1s elapsed)
|
||||
[train] 23.1s, final rec_loss=0.0037
|
||||
[score] benign in 2.2s
|
||||
[score] attack in 2.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed44.json
|
||||
[best agg=median] AUROC=0.3678 AUPRC=0.4001
|
||||
median AUROC=0.3678 AUPRC=0.4001
|
||||
p90 AUROC=0.3272 AUPRC=0.4094
|
||||
mean AUROC=0.2587 AUPRC=0.3873
|
||||
max AUROC=0.2540 AUPRC=0.3701
|
||||
[done] elapsed=157s artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed44.json
|
||||
=== protocol=reverse_cross seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=reverse_cross seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=6,772 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0299 (8.1s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0128 (15.7s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0056 (23.3s elapsed)
|
||||
[train] 23.3s, final rec_loss=0.0056
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 1.4s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed42.json
|
||||
[best agg=mean] AUROC=0.8442 AUPRC=0.7504
|
||||
mean AUROC=0.8442 AUPRC=0.7504
|
||||
max AUROC=0.8172 AUPRC=0.7065
|
||||
p90 AUROC=0.7700 AUPRC=0.6800
|
||||
median AUROC=0.6507 AUPRC=0.6041
|
||||
[done] elapsed=250s artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed42.json
|
||||
=== protocol=reverse_cross seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=reverse_cross seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=6,772 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0222 (8.1s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0077 (15.8s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (23.6s elapsed)
|
||||
[train] 23.6s, final rec_loss=0.0040
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 1.5s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed43.json
|
||||
[best agg=max] AUROC=0.6797 AUPRC=0.5524
|
||||
max AUROC=0.6797 AUPRC=0.5524
|
||||
mean AUROC=0.4566 AUPRC=0.4307
|
||||
p90 AUROC=0.3843 AUPRC=0.3849
|
||||
median AUROC=0.3337 AUPRC=0.4061
|
||||
[done] elapsed=247s artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed43.json
|
||||
=== protocol=reverse_cross seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=reverse_cross seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=6,772 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0225 (7.9s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0077 (15.2s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (22.7s elapsed)
|
||||
[train] 22.7s, final rec_loss=0.0040
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 1.5s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed44.json
|
||||
[best agg=max] AUROC=0.5801 AUPRC=0.6040
|
||||
max AUROC=0.5801 AUPRC=0.6040
|
||||
median AUROC=0.4205 AUPRC=0.3476
|
||||
mean AUROC=0.3775 AUPRC=0.4047
|
||||
p90 AUROC=0.2758 AUPRC=0.3123
|
||||
[done] elapsed=244s artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed44.json
|
||||
@@ -0,0 +1,316 @@
|
||||
=== protocol=iscxtor_within seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=iscxtor_within seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=1,312 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0274 (7.7s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0131 (14.3s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0056 (21.2s elapsed)
|
||||
[train] 21.2s, final rec_loss=0.0056
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 0.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed42.json
|
||||
[best agg=max] AUROC=0.5264 AUPRC=0.2290
|
||||
max AUROC=0.5264 AUPRC=0.2290
|
||||
mean AUROC=0.4932 AUPRC=0.1752
|
||||
median AUROC=0.4792 AUPRC=0.1852
|
||||
p90 AUROC=0.4373 AUPRC=0.1487
|
||||
[done] elapsed=33s artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed42.json
|
||||
=== protocol=iscxtor_within seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=iscxtor_within seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=1,312 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0250 (7.6s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0078 (14.9s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0038 (22.5s elapsed)
|
||||
[train] 22.5s, final rec_loss=0.0038
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 0.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed43.json
|
||||
[best agg=median] AUROC=0.4736 AUPRC=0.1267
|
||||
median AUROC=0.4736 AUPRC=0.1267
|
||||
max AUROC=0.4134 AUPRC=0.2217
|
||||
mean AUROC=0.4133 AUPRC=0.1929
|
||||
p90 AUROC=0.3542 AUPRC=0.1179
|
||||
[done] elapsed=34s artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed43.json
|
||||
=== protocol=iscxtor_within seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=iscxtor_within seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=1,312 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0230 (7.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0071 (14.0s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0065 (20.8s elapsed)
|
||||
[train] 20.8s, final rec_loss=0.0065
|
||||
[score] benign in 2.2s
|
||||
[score] attack in 0.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed44.json
|
||||
[best agg=max] AUROC=0.5150 AUPRC=0.2248
|
||||
max AUROC=0.5150 AUPRC=0.2248
|
||||
median AUROC=0.5020 AUPRC=0.1873
|
||||
mean AUROC=0.4870 AUPRC=0.1970
|
||||
p90 AUROC=0.4450 AUPRC=0.1830
|
||||
[done] elapsed=33s artifacts/baselines/anomaly_transformer_2026_04_29/iscxtor_within_seed44.json
|
||||
=== protocol=cicids_within seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicids_within seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=30,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0256 (7.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0108 (14.6s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0041 (22.5s elapsed)
|
||||
[train] 22.5s, final rec_loss=0.0041
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 6.4s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed42.json
|
||||
[best agg=mean] AUROC=0.6001 AUPRC=0.8383
|
||||
mean AUROC=0.6001 AUPRC=0.8383
|
||||
max AUROC=0.5478 AUPRC=0.7840
|
||||
p90 AUROC=0.4774 AUPRC=0.7639
|
||||
median AUROC=0.3624 AUPRC=0.7142
|
||||
[done] elapsed=142s artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed42.json
|
||||
=== protocol=cicids_within seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicids_within seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=30,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0233 (8.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0081 (16.0s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0038 (23.7s elapsed)
|
||||
[train] 23.7s, final rec_loss=0.0038
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 6.3s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed43.json
|
||||
[best agg=median] AUROC=0.2927 AUPRC=0.6398
|
||||
median AUROC=0.2927 AUPRC=0.6398
|
||||
p90 AUROC=0.2795 AUPRC=0.6818
|
||||
mean AUROC=0.2588 AUPRC=0.6678
|
||||
max AUROC=0.2573 AUPRC=0.6591
|
||||
[done] elapsed=141s artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed43.json
|
||||
=== protocol=cicids_within seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicids_within seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=10,000 val=10,000 attack=30,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0197 (8.3s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0097 (16.1s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0037 (23.8s elapsed)
|
||||
[train] 23.8s, final rec_loss=0.0037
|
||||
[score] benign in 2.2s
|
||||
[score] attack in 6.4s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed44.json
|
||||
[best agg=max] AUROC=0.6878 AUPRC=0.8658
|
||||
max AUROC=0.6878 AUPRC=0.8658
|
||||
mean AUROC=0.6436 AUPRC=0.8672
|
||||
p90 AUROC=0.5559 AUPRC=0.8319
|
||||
median AUROC=0.4988 AUPRC=0.8127
|
||||
[done] elapsed=141s artifacts/baselines/anomaly_transformer_2026_04_29/cicids_within_seed44.json
|
||||
=== protocol=cicddos_within seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicddos_within seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=20,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0306 (8.3s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0127 (15.9s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0056 (23.7s elapsed)
|
||||
[train] 23.7s, final rec_loss=0.0056
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 4.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed42.json
|
||||
[best agg=median] AUROC=0.4061 AUPRC=0.7025
|
||||
median AUROC=0.4061 AUPRC=0.7025
|
||||
p90 AUROC=0.3688 AUPRC=0.6932
|
||||
mean AUROC=0.3678 AUPRC=0.7023
|
||||
max AUROC=0.3495 AUPRC=0.6605
|
||||
[done] elapsed=45s artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed42.json
|
||||
=== protocol=cicddos_within seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicddos_within seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=20,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0222 (8.2s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0079 (16.2s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (24.5s elapsed)
|
||||
[train] 24.5s, final rec_loss=0.0040
|
||||
[score] benign in 2.0s
|
||||
[score] attack in 4.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed43.json
|
||||
[best agg=p90] AUROC=0.1995 AUPRC=0.6167
|
||||
p90 AUROC=0.1995 AUPRC=0.6167
|
||||
median AUROC=0.1869 AUPRC=0.5569
|
||||
max AUROC=0.1673 AUPRC=0.6028
|
||||
mean AUROC=0.1619 AUPRC=0.6078
|
||||
[done] elapsed=46s artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed43.json
|
||||
=== protocol=cicddos_within seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=cicddos_within seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=20,000 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0225 (8.0s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0078 (15.6s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0041 (23.1s elapsed)
|
||||
[train] 23.1s, final rec_loss=0.0041
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 4.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed44.json
|
||||
[best agg=median] AUROC=0.8402 AUPRC=0.9045
|
||||
median AUROC=0.8402 AUPRC=0.9045
|
||||
p90 AUROC=0.6628 AUPRC=0.8249
|
||||
mean AUROC=0.6323 AUPRC=0.8247
|
||||
max AUROC=0.5918 AUPRC=0.7645
|
||||
[done] elapsed=45s artifacts/baselines/anomaly_transformer_2026_04_29/cicddos_within_seed44.json
|
||||
=== protocol=forward_cross seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=forward_cross seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=10,000 val=10,000 attack=9,846 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0256 (7.7s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0107 (15.0s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (22.5s elapsed)
|
||||
[train] 22.5s, final rec_loss=0.0040
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 2.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed42.json
|
||||
[best agg=median] AUROC=0.6214 AUPRC=0.6244
|
||||
median AUROC=0.6214 AUPRC=0.6244
|
||||
p90 AUROC=0.5738 AUPRC=0.5601
|
||||
mean AUROC=0.4892 AUPRC=0.5018
|
||||
max AUROC=0.4702 AUPRC=0.4553
|
||||
[done] elapsed=157s artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed42.json
|
||||
=== protocol=forward_cross seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=forward_cross seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=10,000 val=10,000 attack=9,846 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0231 (7.8s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0081 (15.3s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0038 (22.7s elapsed)
|
||||
[train] 22.7s, final rec_loss=0.0038
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 2.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed43.json
|
||||
[best agg=median] AUROC=0.6319 AUPRC=0.6693
|
||||
median AUROC=0.6319 AUPRC=0.6693
|
||||
p90 AUROC=0.5401 AUPRC=0.5542
|
||||
mean AUROC=0.4969 AUPRC=0.4924
|
||||
max AUROC=0.4889 AUPRC=0.4859
|
||||
[done] elapsed=157s artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed43.json
|
||||
=== protocol=forward_cross seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=forward_cross seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=10,000 val=10,000 attack=9,846 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0195 (8.1s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0100 (15.7s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0037 (23.1s elapsed)
|
||||
[train] 23.1s, final rec_loss=0.0037
|
||||
[score] benign in 2.2s
|
||||
[score] attack in 2.1s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed44.json
|
||||
[best agg=median] AUROC=0.3678 AUPRC=0.4001
|
||||
median AUROC=0.3678 AUPRC=0.4001
|
||||
p90 AUROC=0.3272 AUPRC=0.4094
|
||||
mean AUROC=0.2587 AUPRC=0.3873
|
||||
max AUROC=0.2540 AUPRC=0.3701
|
||||
[done] elapsed=157s artifacts/baselines/anomaly_transformer_2026_04_29/forward_cross_seed44.json
|
||||
=== protocol=reverse_cross seed=42 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=reverse_cross seed=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=6,772 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0299 (8.1s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0128 (15.7s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0056 (23.3s elapsed)
|
||||
[train] 23.3s, final rec_loss=0.0056
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 1.4s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed42.json
|
||||
[best agg=mean] AUROC=0.8442 AUPRC=0.7504
|
||||
mean AUROC=0.8442 AUPRC=0.7504
|
||||
max AUROC=0.8172 AUPRC=0.7065
|
||||
p90 AUROC=0.7700 AUPRC=0.6800
|
||||
median AUROC=0.6507 AUPRC=0.6041
|
||||
[done] elapsed=250s artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed42.json
|
||||
=== protocol=reverse_cross seed=43 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=reverse_cross seed=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=6,772 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0222 (8.1s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0077 (15.8s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (23.6s elapsed)
|
||||
[train] 23.6s, final rec_loss=0.0040
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 1.5s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed43.json
|
||||
[best agg=max] AUROC=0.6797 AUPRC=0.5524
|
||||
max AUROC=0.6797 AUPRC=0.5524
|
||||
mean AUROC=0.4566 AUPRC=0.4307
|
||||
p90 AUROC=0.3843 AUPRC=0.3849
|
||||
median AUROC=0.3337 AUPRC=0.4061
|
||||
[done] elapsed=247s artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed43.json
|
||||
=== protocol=reverse_cross seed=44 epochs=15 batch=128 ===
|
||||
[run] anomaly_transformer protocol=reverse_cross seed=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=10,000 val=10,000 attack=6,772 D=9 device=cuda
|
||||
[model] params=305,941
|
||||
[epoch 5/15] rec_loss=0.0225 (7.9s elapsed)
|
||||
[epoch 10/15] rec_loss=0.0077 (15.2s elapsed)
|
||||
[epoch 15/15] rec_loss=0.0040 (22.7s elapsed)
|
||||
[train] 22.7s, final rec_loss=0.0040
|
||||
[score] benign in 2.1s
|
||||
[score] attack in 1.5s
|
||||
[saved] artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed44.json
|
||||
[best agg=max] AUROC=0.5801 AUPRC=0.6040
|
||||
max AUROC=0.5801 AUPRC=0.6040
|
||||
median AUROC=0.4205 AUPRC=0.3476
|
||||
mean AUROC=0.3775 AUPRC=0.4047
|
||||
p90 AUROC=0.2758 AUPRC=0.3123
|
||||
[done] elapsed=244s artifacts/baselines/anomaly_transformer_2026_04_29/reverse_cross_seed44.json
|
||||
ALL DONE
|
||||
@@ -0,0 +1,288 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.29,
|
||||
"loss_first_last": [
|
||||
0.14130105172531515,
|
||||
0.005604498141409853
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.8441902909037211,
|
||||
"auprc": 0.7503811509240934
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.8171573981098643,
|
||||
"auprc": 0.7064730252974024
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.6507296072061427,
|
||||
"auprc": 0.6041436533714446
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.7699723641464855,
|
||||
"auprc": 0.6800127316009327
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8937602102102102
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9210349849849848
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8860076576576577
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8927644144144142
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9019121621621621
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7949579579579579
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7788546546546548
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7892
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.7397142857142857
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8061200450450451
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8833273273273273
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.698478828828829
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.7296356164383562
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.9339384615384616
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.7209722222222222
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.780576951951952
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8745903903903903
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8431027027027026
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8299033033033032
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.889171996996997
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7773794294294294
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7797936936936937
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9548
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.8399714285714286
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7701466966966968
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8481421171171172
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7707084834834834
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.8702712328767123
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.8619307692307692
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.85105
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7698609609609609
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.485801876876877
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5896962462462464
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6884997747747748
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.565258033033033
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6203490240240241
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5560197447447448
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.3437
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.40904285714285715
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9600471471471471
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9599916666666667
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3526761261261261
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.3437
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.858123076923077
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.3437
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9052775525525526
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7179333333333333
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7656078078078079
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7565526276276275
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.805270870870871
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7739398648648649
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8422819819819819
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.1049
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.4063142857142857
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8906704204204202
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9116813813813812
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.42198918918918926
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.11386301369863014
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.9354615384615386
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.1049
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,288 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 23.63,
|
||||
"loss_first_last": [
|
||||
0.14317506532880325,
|
||||
0.003993322554079792
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.4565637551683402,
|
||||
"auprc": 0.43074154476845705
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.6796632088009451,
|
||||
"auprc": 0.552430354953035
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.3336748154164205,
|
||||
"auprc": 0.40610685321644757
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.3842701860602481,
|
||||
"auprc": 0.3849137591979363
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4569912912912913
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6351558558558559
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.42046006006006004
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3436268768768769
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.45092192192192193
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3478476726726727
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.17148513513513514
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.07469999999999999
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.12732857142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6950406906906906
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9706375375375375
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.11286516516516515
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.18231780821917806
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5425076923076922
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.1983611111111111
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6036139639639639
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.780161111111111
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6608007507507507
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6074217717717718
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6471046546546547
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5971674174174174
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6830364864864865
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.8371000000000001
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.639357142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4951987987987988
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9017625375375377
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7886006006006006
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.9095986301369864
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.6941076923076923
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.9189111111111111
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.250798048048048
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.26640420420420424
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.25962019519519514
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.16605758258258257
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.1765325825825826
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.2530582582582583
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.1632
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.1632
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.2160857142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6873358108108109
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9778397897897897
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.1634536036036036
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.1632
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.19192307692307692
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.1632
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4940503003003003
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3986143393393393
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.2924507507507508
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.25072905405405405
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.31573280780780777
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.26408123123123123
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.11244279279279279
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.04904999999999998
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.14922142857142856
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7294531531531532
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9646893393393393
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.06833258258258255
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.04904999999999998
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.43051538461538463
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.04904999999999998
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,288 @@
|
||||
{
|
||||
"method": "anomaly_transformer",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"D": 9,
|
||||
"epochs": 15,
|
||||
"lr": 0.0001,
|
||||
"k_disc": 3.0,
|
||||
"temperature": 50.0,
|
||||
"d_model": 128,
|
||||
"t_train_sec": 22.69,
|
||||
"loss_first_last": [
|
||||
0.14046703491218482,
|
||||
0.003991496433869381
|
||||
],
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.37751623597164796,
|
||||
"auprc": 0.4047034772236886
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.5801241952155936,
|
||||
"auprc": 0.6039728234817969
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.42049935026580026,
|
||||
"auprc": 0.3476409191499162
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.2757992690490254,
|
||||
"auprc": 0.3122648050899628
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4738412162162162
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7721778528528529
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5258142642642643
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.711262912912913
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5486725225225224
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3832174924924925
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.30577192192192193
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.6284000000000001
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.19322857142857142
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.013867492492492497
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.009455630630630627
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.06311981981981982
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.1339671232876712
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5136076923076923
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.13924999999999998
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.626911036036036
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9007268768768768
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.739242942942943
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8433722222222222
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7245804804804805
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6317326576576576
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7286666666666667
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9992
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.6157285714285714
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.0187463963963964
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.009331831831831828
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.534770945945946
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.8766520547945206
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.6754769230769231
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.8684611111111111
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5296717717717717
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3575990990990991
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3952307807807808
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3318165165165165
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.3367241741741741
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.35629737237237236
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.40581298798798804
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.30374999999999996
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.30374999999999996
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.522578978978979
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6302899399399399
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.35813018018018017
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.30374999999999996
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.32809615384615387
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.30374999999999996
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4574834084084084
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.43173205705705703
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.32524474474474474
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.29145908408408405
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.37425945945945943
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.21709729729729732
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.21281051051051048
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.05625000000000002
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.1657357142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.1499222972972973
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.14283513513513513
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.18387312312312312
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.05786849315068495
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.4016461538461539
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.06093611111111113
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
724
artifacts/baselines/anomaly_transformer_2026_04_29/summary.json
Normal file
724
artifacts/baselines/anomaly_transformer_2026_04_29/summary.json
Normal file
@@ -0,0 +1,724 @@
|
||||
{
|
||||
"rows": [
|
||||
{
|
||||
"protocol": "iscxtor_within",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "p90",
|
||||
"auroc_mean": 0.41216375762195123,
|
||||
"auroc_std": 0.050302170433342654,
|
||||
"abs_auroc_mean": 0.5878362423780489,
|
||||
"abs_auroc_std": 0.050302170433342654,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.46451020071138216,
|
||||
"auroc_std": 0.04444281163746598,
|
||||
"abs_auroc_mean": 0.5354897992886178,
|
||||
"abs_auroc_std": 0.044442811637466016
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.4849680132113821,
|
||||
"auroc_std": 0.062218349147918434,
|
||||
"abs_auroc_mean": 0.54268356199187,
|
||||
"abs_auroc_std": 0.03843480472750384
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.4849544080284553,
|
||||
"auroc_std": 0.015057481255194563,
|
||||
"abs_auroc_mean": 0.5164094639227642,
|
||||
"abs_auroc_std": 0.012742713175109128
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.41216375762195123,
|
||||
"auroc_std": 0.050302170433342654,
|
||||
"abs_auroc_mean": 0.5878362423780489,
|
||||
"abs_auroc_std": 0.050302170433342654
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "cicids_within",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "mean",
|
||||
"auroc_mean": 0.5008647777777778,
|
||||
"auroc_std": 0.2107434205204985,
|
||||
"abs_auroc_mean": 0.6616516966666667,
|
||||
"abs_auroc_std": 0.07222883427530209,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.5008647777777778,
|
||||
"auroc_std": 0.2107434205204985,
|
||||
"abs_auroc_mean": 0.6616516966666667,
|
||||
"abs_auroc_std": 0.07222883427530209
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.4976374927777778,
|
||||
"auroc_std": 0.2196157413237703,
|
||||
"abs_auroc_mean": 0.6594498772222223,
|
||||
"abs_auroc_std": 0.10051393425032756
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.38462360944444446,
|
||||
"auroc_std": 0.10480730142299796,
|
||||
"abs_auroc_mean": 0.6153763905555555,
|
||||
"abs_auroc_std": 0.10480730142299793
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.43759324833333335,
|
||||
"auroc_std": 0.14244768765655508,
|
||||
"abs_auroc_mean": 0.5996638427777777,
|
||||
"abs_auroc_std": 0.10599021352571453
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "cicddos_within",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "median",
|
||||
"auroc_mean": 0.47772612749999993,
|
||||
"auroc_std": 0.3324922077261016,
|
||||
"abs_auroc_mean": 0.7490622624999999,
|
||||
"abs_auroc_std": 0.13508234670144054,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.3873264733333334,
|
||||
"auroc_std": 0.23584393356290714,
|
||||
"abs_auroc_mean": 0.7009006683333333,
|
||||
"abs_auroc_std": 0.11884329434390078
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.369551825,
|
||||
"auroc_std": 0.2129503159588168,
|
||||
"abs_auroc_mean": 0.69164826,
|
||||
"abs_auroc_std": 0.12561585595244118
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.47772612749999993,
|
||||
"auroc_std": 0.3324922077261016,
|
||||
"abs_auroc_mean": 0.7490622624999999,
|
||||
"abs_auroc_std": 0.13508234670144054
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.4103730108333334,
|
||||
"auroc_std": 0.23443402670235713,
|
||||
"abs_auroc_mean": 0.6981660691666667,
|
||||
"abs_auroc_std": 0.09002289821513176
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "forward_cross",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "median",
|
||||
"auroc_mean": 0.5403936403954228,
|
||||
"auroc_std": 0.1495421102979965,
|
||||
"abs_auroc_mean": 0.6285116375516284,
|
||||
"abs_auroc_std": 0.00611968542235559,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.41495291150382557,
|
||||
"auroc_std": 0.1353781339481017,
|
||||
"abs_auroc_mean": 0.5850470884961744,
|
||||
"abs_auroc_std": 0.13537813394810166
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.40434917225269135,
|
||||
"auroc_std": 0.13056700869549043,
|
||||
"abs_auroc_mean": 0.5956508277473085,
|
||||
"abs_auroc_std": 0.13056700869549043
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.5403936403954228,
|
||||
"auroc_std": 0.1495421102979965,
|
||||
"abs_auroc_mean": 0.6285116375516284,
|
||||
"abs_auroc_std": 0.00611968542235559
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.4803580371047465,
|
||||
"auroc_std": 0.1337072839194574,
|
||||
"abs_auroc_mean": 0.5955567743245989,
|
||||
"abs_auroc_std": 0.06899059467567079
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "reverse_cross",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "p90",
|
||||
"auroc_mean": 0.4766806064185863,
|
||||
"auroc_std": 0.2597239425287057,
|
||||
"abs_auroc_mean": 0.7033009696790707,
|
||||
"abs_auroc_std": 0.07921673490801118,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.5594234273479031,
|
||||
"auroc_std": 0.2497623920996739,
|
||||
"abs_auroc_mean": 0.670036766587911,
|
||||
"abs_auroc_std": 0.15591412731533005
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.6923149340421343,
|
||||
"auroc_std": 0.11902199138084962,
|
||||
"abs_auroc_mean": 0.6923149340421343,
|
||||
"abs_auroc_std": 0.11902199138084962
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.4683012576294545,
|
||||
"auroc_std": 0.1638435290449658,
|
||||
"abs_auroc_mean": 0.6321851471746406,
|
||||
"abs_auroc_std": 0.04628766262566922
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.4766806064185863,
|
||||
"auroc_std": 0.2597239425287057,
|
||||
"abs_auroc_mean": 0.7033009696790707,
|
||||
"abs_auroc_std": 0.07921673490801118
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"per_class": {
|
||||
"iscxtor_within": {
|
||||
"tor": {
|
||||
"n": 1312,
|
||||
"aurocs": [
|
||||
0.49322682926829275,
|
||||
0.41331875,
|
||||
0.48698502286585366
|
||||
]
|
||||
}
|
||||
},
|
||||
"cicids_within": {
|
||||
"Botnet": {
|
||||
"n": 46,
|
||||
"aurocs": [
|
||||
0.8872391304347825,
|
||||
0.3123346153846153,
|
||||
0.46004210526315786
|
||||
]
|
||||
},
|
||||
"DDoS": {
|
||||
"n": 5752,
|
||||
"aurocs": [
|
||||
0.8693338404033378,
|
||||
0.6342495588494794,
|
||||
0.5145650968544518
|
||||
]
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"n": 464,
|
||||
"aurocs": [
|
||||
0.8588349137931033,
|
||||
0.7240725672877847,
|
||||
0.7046408296943232
|
||||
]
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"n": 9358,
|
||||
"aurocs": [
|
||||
0.8672453141696944,
|
||||
0.12875773550916603,
|
||||
0.4318485673352435
|
||||
]
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"n": 78,
|
||||
"aurocs": [
|
||||
0.8991410256410255,
|
||||
0.31539722222222216,
|
||||
0.6630583333333333
|
||||
]
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"n": 185,
|
||||
"aurocs": [
|
||||
0.762110810810811,
|
||||
0.15719341317365268,
|
||||
0.5808367088607596
|
||||
]
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"n": 236,
|
||||
"aurocs": [
|
||||
0.7410864406779661,
|
||||
0.03223434579439251,
|
||||
0.3321348214285714
|
||||
]
|
||||
},
|
||||
"Infiltration": {
|
||||
"n": 2,
|
||||
"aurocs": [
|
||||
0.46535,
|
||||
0.9448
|
||||
]
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"n": 4295,
|
||||
"aurocs": [
|
||||
0.603313003492433,
|
||||
0.4443001539554714,
|
||||
0.5555688679245283
|
||||
]
|
||||
},
|
||||
"Portscan": {
|
||||
"n": 9425,
|
||||
"aurocs": [
|
||||
0.14459035543766577,
|
||||
0.05020893327711605,
|
||||
0.9853483215454448
|
||||
]
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"n": 152,
|
||||
"aurocs": [
|
||||
0.6733217105263157,
|
||||
0.9247592896174862,
|
||||
0.19467950310559007
|
||||
]
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"n": 5,
|
||||
"aurocs": [
|
||||
0.72576,
|
||||
0.9488333333333333,
|
||||
0.3236285714285714
|
||||
]
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"n": 2,
|
||||
"aurocs": [
|
||||
0.7853,
|
||||
0.9608599999999999
|
||||
]
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"n": 2,
|
||||
"aurocs": [
|
||||
0.8862,
|
||||
0.19369999999999998
|
||||
]
|
||||
}
|
||||
},
|
||||
"cicddos_within": {
|
||||
"DrDoS_DNS": {
|
||||
"n": 1136,
|
||||
"aurocs": [
|
||||
0.19116588908450705,
|
||||
0.017933482542524647,
|
||||
0.38358046387154326
|
||||
]
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"n": 1152,
|
||||
"aurocs": [
|
||||
0.18985000000000002,
|
||||
0.005402806563039751,
|
||||
0.3384721048182587
|
||||
]
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"n": 1135,
|
||||
"aurocs": [
|
||||
0.18985000000000002,
|
||||
0.007452684859154955,
|
||||
0.8784359464627152
|
||||
]
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"n": 1171,
|
||||
"aurocs": [
|
||||
0.505333219470538,
|
||||
0.27283804855275445,
|
||||
0.42309863195057373
|
||||
]
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"n": 1166,
|
||||
"aurocs": [
|
||||
0.19054429674099488,
|
||||
0.07414517657192077,
|
||||
0.9832278733031674
|
||||
]
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"n": 1086,
|
||||
"aurocs": [
|
||||
0.18985000000000002,
|
||||
0.009164126712328793,
|
||||
0.37689410714285715
|
||||
]
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"n": 1092,
|
||||
"aurocs": [
|
||||
0.2037271978021978,
|
||||
0.3974829535095715,
|
||||
0.9443937277580071
|
||||
]
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"n": 1109,
|
||||
"aurocs": [
|
||||
0.20567407574391344,
|
||||
0.42633196573489635,
|
||||
0.9635801412180053
|
||||
]
|
||||
},
|
||||
"LDAP": {
|
||||
"n": 1105,
|
||||
"aurocs": [
|
||||
0.18985000000000002,
|
||||
0.005454378648874089,
|
||||
0.34371287813310286
|
||||
]
|
||||
},
|
||||
"MSSQL": {
|
||||
"n": 1184,
|
||||
"aurocs": [
|
||||
0.18985000000000002,
|
||||
0.007531764705882378,
|
||||
0.8914378446115288
|
||||
]
|
||||
},
|
||||
"NetBIOS": {
|
||||
"n": 1539,
|
||||
"aurocs": [
|
||||
0.18985000000000002,
|
||||
0.06480254614894973,
|
||||
0.9836372795969773
|
||||
]
|
||||
},
|
||||
"Portmap": {
|
||||
"n": 417,
|
||||
"aurocs": [
|
||||
0.19371738609112713,
|
||||
0.06535786240786241,
|
||||
0.9855167865707434
|
||||
]
|
||||
},
|
||||
"Syn": {
|
||||
"n": 3361,
|
||||
"aurocs": [
|
||||
0.9442166468313002,
|
||||
0.12263094156827128,
|
||||
0.20813423054417787
|
||||
]
|
||||
},
|
||||
"TFTP": {
|
||||
"n": 1106,
|
||||
"aurocs": [
|
||||
0.23980361663652802,
|
||||
0.5118133650519031,
|
||||
0.9565173021925643
|
||||
]
|
||||
},
|
||||
"UDP": {
|
||||
"n": 1383,
|
||||
"aurocs": [
|
||||
0.20412762111352134,
|
||||
0.43651510494752627,
|
||||
0.9599579822616406
|
||||
]
|
||||
},
|
||||
"UDPLag": {
|
||||
"n": 857,
|
||||
"aurocs": [
|
||||
0.8218587514585765,
|
||||
0.2210940389294404,
|
||||
0.35599000000000003
|
||||
]
|
||||
},
|
||||
"WebDDoS": {
|
||||
"n": 1,
|
||||
"aurocs": [
|
||||
0.44220000000000004,
|
||||
0.16269999999999996,
|
||||
0.6828000000000001
|
||||
]
|
||||
}
|
||||
},
|
||||
"forward_cross": {
|
||||
"DrDoS_DNS": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.2418359693877551,
|
||||
0.3203674319727891,
|
||||
0.12040178571428571
|
||||
]
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.2234609693877551,
|
||||
0.3218841836734694,
|
||||
0.12222193877551019
|
||||
]
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.37049260204081635,
|
||||
0.5871074829931973,
|
||||
0.1191062074829932
|
||||
]
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.6877126700680272,
|
||||
0.6272752551020409,
|
||||
0.4970311224489796
|
||||
]
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.48235034013605443,
|
||||
0.564168537414966,
|
||||
0.10205935374149659
|
||||
]
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.22540357142857143,
|
||||
0.33965544217687077,
|
||||
0.12043333333333331
|
||||
]
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.5496690476190476,
|
||||
0.6149467687074831,
|
||||
0.4376086734693878
|
||||
]
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.5629209183673469,
|
||||
0.6218015306122449,
|
||||
0.43346173469387755
|
||||
]
|
||||
},
|
||||
"LDAP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.22110892857142855,
|
||||
0.3331803571428571,
|
||||
0.12309634353741497
|
||||
]
|
||||
},
|
||||
"MSSQL": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.38065323129251705,
|
||||
0.5915131802721088,
|
||||
0.12141343537414964
|
||||
]
|
||||
},
|
||||
"NetBIOS": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.4970219387755102,
|
||||
0.5796848639455783,
|
||||
0.10477653061224489
|
||||
]
|
||||
},
|
||||
"Portmap": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.48847789115646256,
|
||||
0.5787874149659864,
|
||||
0.10042551020408164
|
||||
]
|
||||
},
|
||||
"Syn": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.9474401360544217,
|
||||
0.34615850340136056,
|
||||
0.3379833333333333
|
||||
]
|
||||
},
|
||||
"TFTP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.5612562925170068,
|
||||
0.6246388605442177,
|
||||
0.5251631802721088
|
||||
]
|
||||
},
|
||||
"UDP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.5692301870748299,
|
||||
0.6211027210884353,
|
||||
0.4429054421768708
|
||||
]
|
||||
},
|
||||
"UDPLag": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.9008552721088435,
|
||||
0.30889481292517007,
|
||||
0.27171930272108846
|
||||
]
|
||||
},
|
||||
"WebDDoS": {
|
||||
"n": 438,
|
||||
"aurocs": [
|
||||
0.378726598173516,
|
||||
0.45648915525114153,
|
||||
0.47257134703196346
|
||||
]
|
||||
}
|
||||
},
|
||||
"reverse_cross": {
|
||||
"Botnet": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8937602102102102,
|
||||
0.4569912912912913,
|
||||
0.4738412162162162
|
||||
]
|
||||
},
|
||||
"DDoS": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.9210349849849848,
|
||||
0.6351558558558559,
|
||||
0.7721778528528529
|
||||
]
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8860076576576577,
|
||||
0.42046006006006004,
|
||||
0.5258142642642643
|
||||
]
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8927644144144142,
|
||||
0.3436268768768769,
|
||||
0.711262912912913
|
||||
]
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.9019121621621621,
|
||||
0.45092192192192193,
|
||||
0.5486725225225224
|
||||
]
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.7949579579579579,
|
||||
0.3478476726726727,
|
||||
0.3832174924924925
|
||||
]
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.7788546546546548,
|
||||
0.17148513513513514,
|
||||
0.30577192192192193
|
||||
]
|
||||
},
|
||||
"Heartbleed": {
|
||||
"n": 1,
|
||||
"aurocs": [
|
||||
0.7892,
|
||||
0.07469999999999999,
|
||||
0.6284000000000001
|
||||
]
|
||||
},
|
||||
"Infiltration": {
|
||||
"n": 7,
|
||||
"aurocs": [
|
||||
0.7397142857142857,
|
||||
0.12732857142857143,
|
||||
0.19322857142857142
|
||||
]
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8061200450450451,
|
||||
0.6950406906906906,
|
||||
0.013867492492492497
|
||||
]
|
||||
},
|
||||
"Portscan": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8833273273273273,
|
||||
0.9706375375375375,
|
||||
0.009455630630630627
|
||||
]
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.698478828828829,
|
||||
0.11286516516516515,
|
||||
0.06311981981981982
|
||||
]
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"n": 73,
|
||||
"aurocs": [
|
||||
0.7296356164383562,
|
||||
0.18231780821917806,
|
||||
0.1339671232876712
|
||||
]
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"n": 13,
|
||||
"aurocs": [
|
||||
0.9339384615384616,
|
||||
0.5425076923076922,
|
||||
0.5136076923076923
|
||||
]
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"n": 18,
|
||||
"aurocs": [
|
||||
0.7209722222222222,
|
||||
0.1983611111111111,
|
||||
0.13924999999999998
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"baselines": {
|
||||
"terminal_norm": {
|
||||
"iscxtor_within": [
|
||||
0.9945,
|
||||
0.0011
|
||||
],
|
||||
"cicids_within": [
|
||||
0.9858,
|
||||
0.0021
|
||||
],
|
||||
"cicddos_within": [
|
||||
0.996,
|
||||
0.001
|
||||
],
|
||||
"forward_cross": [
|
||||
0.9109,
|
||||
0.0032
|
||||
],
|
||||
"reverse_cross": [
|
||||
0.5999,
|
||||
null
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
# Anomaly-Transformer (ICLR 2022) Baseline — On Our 5-Protocol Layout
|
||||
|
||||
Date: 2026-04-29
|
||||
|
||||
Method: ICLR 2022 Anomaly-Transformer (association-discrepancy minimax). Vendored model class from `baselines/Anomaly-Transformer/model/AnomalyTransformer.py`; training + scoring loop reimplemented to match our protocol (input shape [B, T=64, D=9] = our z-scored packet sequences, same train/val/attack splits as eval_new_scores.py).
|
||||
Hyperparams: d_model=128, n_heads=4, e_layers=3, batch=128, lr=1e-4, k_disc=3.0, temperature=50.0, epochs=15.
|
||||
Score: per-position softmax(-association_KL · T) · MSE(rec, x), then aggregated per flow (mean / max / median / p90).
|
||||
|
||||
## Headline AUROC (best aggregator per protocol, 3-seed mean ± std)
|
||||
|
||||
| Protocol | terminal_norm (Unified_CFM) | **AT (ours)** | abs AUROC | best agg | Δ vs terminal |
|
||||
|---|---:|---:|---:|---|---:|
|
||||
| ISCXTor2016 within | 0.9945 ± 0.0011 | **0.4122 ± 0.0503** | 0.5878 ± 0.0503 | `p90` | -0.5823 |
|
||||
| CICIDS2017 within (σ=0.6) | 0.9858 ± 0.0021 | **0.5009 ± 0.2107** | 0.6617 ± 0.0722 | `mean` | -0.4849 |
|
||||
| CICDDoS2019 within | 0.9960 ± 0.0010 | **0.4777 ± 0.3325** | 0.7491 ± 0.1351 | `median` | -0.5183 |
|
||||
| IDS2017→DDoS2019 forward | 0.9109 ± 0.0032 | **0.5404 ± 0.1495** | 0.6285 ± 0.0061 | `median` | -0.3705 |
|
||||
| DDoS2019→IDS2017 reverse | 0.5999 | **0.4767 ± 0.2597** | 0.7033 ± 0.0792 | `p90` | -0.1232 |
|
||||
|
||||
## All aggregators (3-seed mean ± std)
|
||||
|
||||
| Protocol | mean | max | median | p90 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| ISCXTor2016 within | 0.4645 ± 0.0444 | 0.4850 ± 0.0622 | 0.4850 ± 0.0151 | 0.4122 ± 0.0503 |
|
||||
| CICIDS2017 within (σ=0.6) | 0.5009 ± 0.2107 | 0.4976 ± 0.2196 | 0.3846 ± 0.1048 | 0.4376 ± 0.1424 |
|
||||
| CICDDoS2019 within | 0.3873 ± 0.2358 | 0.3696 ± 0.2130 | 0.4777 ± 0.3325 | 0.4104 ± 0.2344 |
|
||||
| IDS2017→DDoS2019 forward | 0.4150 ± 0.1354 | 0.4043 ± 0.1306 | 0.5404 ± 0.1495 | 0.4804 ± 0.1337 |
|
||||
| DDoS2019→IDS2017 reverse | 0.5594 ± 0.2498 | 0.6923 ± 0.1190 | 0.4683 ± 0.1638 | 0.4767 ± 0.2597 |
|
||||
|
||||
## Per-attack (forward + reverse, mean aggregator)
|
||||
|
||||
### IDS2017→DDoS2019 forward
|
||||
| attack | n | AT AUROC mean ± std |
|
||||
|---|---:|---:|
|
||||
| `DrDoS_DNS` | 588 | 0.2275 ± 0.1007 |
|
||||
| `DrDoS_LDAP` | 588 | 0.2225 ± 0.0998 |
|
||||
| `DrDoS_MSSQL` | 588 | 0.3589 ± 0.2342 |
|
||||
| `DrDoS_NTP` | 588 | 0.6040 ± 0.0974 |
|
||||
| `DrDoS_NetBIOS` | 588 | 0.3829 ± 0.2466 |
|
||||
| `DrDoS_SNMP` | 588 | 0.2285 ± 0.1096 |
|
||||
| `DrDoS_SSDP` | 588 | 0.5341 ± 0.0897 |
|
||||
| `DrDoS_UDP` | 588 | 0.5394 ± 0.0963 |
|
||||
| `LDAP` | 588 | 0.2258 ± 0.1051 |
|
||||
| `MSSQL` | 588 | 0.3645 ± 0.2355 |
|
||||
| `NetBIOS` | 588 | 0.3938 ± 0.2537 |
|
||||
| `Portmap` | 588 | 0.3892 ± 0.2542 |
|
||||
| `Syn` | 588 | 0.5439 ± 0.3495 |
|
||||
| `TFTP` | 588 | 0.5704 ± 0.0504 |
|
||||
| `UDP` | 588 | 0.5444 ± 0.0917 |
|
||||
| `UDPLag` | 588 | 0.4938 ± 0.3530 |
|
||||
| `WebDDoS` | 438 | 0.4359 ± 0.0502 |
|
||||
|
||||
### DDoS2019→IDS2017 reverse
|
||||
| attack | n | AT AUROC mean ± std |
|
||||
|---|---:|---:|
|
||||
| `Botnet` | 666 | 0.6082 ± 0.2474 |
|
||||
| `DDoS` | 666 | 0.7761 ± 0.1430 |
|
||||
| `DoS GoldenEye` | 666 | 0.6108 ± 0.2441 |
|
||||
| `DoS Hulk` | 666 | 0.6492 ± 0.2798 |
|
||||
| `DoS Slowhttptest` | 666 | 0.6338 ± 0.2373 |
|
||||
| `DoS Slowloris` | 666 | 0.5087 ± 0.2486 |
|
||||
| `FTP-Patator` | 666 | 0.4187 ± 0.3190 |
|
||||
| `Heartbleed` | 1 | 0.4974 ± 0.3748 |
|
||||
| `Infiltration` | 7 | 0.3534 ± 0.3362 |
|
||||
| `Infiltration - Portscan` | 666 | 0.5050 ± 0.4290 |
|
||||
| `Portscan` | 666 | 0.6211 ± 0.5315 |
|
||||
| `SSH-Patator` | 666 | 0.2915 ± 0.3533 |
|
||||
| `Web Attack - Brute Force` | 73 | 0.3486 ± 0.3308 |
|
||||
| `Web Attack - SQL Injection` | 13 | 0.6634 ± 0.2348 |
|
||||
| `Web Attack - XSS` | 18 | 0.3529 ± 0.3202 |
|
||||
85
artifacts/baselines/extract_ciciot2023.log
Normal file
85
artifacts/baselines/extract_ciciot2023.log
Normal file
@@ -0,0 +1,85 @@
|
||||
[discover] 34 pcap files across 34 labels
|
||||
backdoor_malware 1 pcap(s)
|
||||
browserhijacking 1 pcap(s)
|
||||
commandinjection 1 pcap(s)
|
||||
ddos-ack_fragmentation 1 pcap(s)
|
||||
ddos-http_flood 1 pcap(s)
|
||||
ddos-icmp_flood 1 pcap(s)
|
||||
ddos-icmp_fragmentation 1 pcap(s)
|
||||
ddos-pshack_flood 1 pcap(s)
|
||||
ddos-rstfinflood 1 pcap(s)
|
||||
ddos-slowloris 1 pcap(s)
|
||||
ddos-syn_flood 1 pcap(s)
|
||||
ddos-synonymousip_flood 1 pcap(s)
|
||||
ddos-tcp_flood 1 pcap(s)
|
||||
ddos-udp_flood 1 pcap(s)
|
||||
ddos-udp_fragmentation 1 pcap(s)
|
||||
dictionarybruteforce 1 pcap(s)
|
||||
dns_spoofing 1 pcap(s)
|
||||
dos-http_flood 1 pcap(s)
|
||||
dos-syn_flood 1 pcap(s)
|
||||
dos-tcp_flood 1 pcap(s)
|
||||
dos-udp_flood 1 pcap(s)
|
||||
mirai-greeth_flood 1 pcap(s)
|
||||
mirai-greip_flood 1 pcap(s)
|
||||
mirai-udpplain 1 pcap(s)
|
||||
mitm-arpspoofing 1 pcap(s)
|
||||
normal 1 pcap(s)
|
||||
recon-hostdiscovery 1 pcap(s)
|
||||
recon-osscan 1 pcap(s)
|
||||
recon-pingsweep 1 pcap(s)
|
||||
recon-portscan 1 pcap(s)
|
||||
sqlinjection 1 pcap(s)
|
||||
uploading_attack 1 pcap(s)
|
||||
vulnerabilityscan 1 pcap(s)
|
||||
xss 1 pcap(s)
|
||||
[extract_labeled_pcaps] n_pcaps=34 T_full=256 extra_cols=('class_folder',)
|
||||
backdoor_malware Backdoor_Malware.pcap extra={'class_folder': 'Backdoor_Malware'}
|
||||
normal BenignTraffic.pcap extra={'class_folder': 'Benign_Final'}
|
||||
browserhijacking BrowserHijacking.pcap extra={'class_folder': 'BrowserHijacking'}
|
||||
commandinjection CommandInjection.pcap extra={'class_folder': 'CommandInjection'}
|
||||
ddos-ack_fragmentation DDoS-ACK_Fragmentation.pcap extra={'class_folder': 'DDoS-ACK_Fragmentation'}
|
||||
ddos-http_flood DDoS-HTTP_Flood-.pcap extra={'class_folder': 'DDoS-HTTP_Flood'}
|
||||
ddos-icmp_flood DDoS-ICMP_Flood.pcap extra={'class_folder': 'DDoS-ICMP_Flood'}
|
||||
ddos-icmp_fragmentation DDoS-ICMP_Fragmentation.pcap extra={'class_folder': 'DDoS-ICMP_Fragmentation'}
|
||||
ddos-pshack_flood DDoS-PSHACK_Flood.pcap extra={'class_folder': 'DDoS-PSHACK_Flood'}
|
||||
ddos-rstfinflood DDoS-RSTFINFlood.pcap extra={'class_folder': 'DDoS-RSTFINFlood'}
|
||||
... (24 more)
|
||||
[extract_labeled_pcaps] running 34 pcap(s) with 4 worker(s)
|
||||
[extract_labeled_pcaps] sharded output enabled: datasets/ciciot2023/processed/full_store shard_size=100,000
|
||||
[extract_labeled_pcaps] worker spool=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/.full_store.spool._q0mmt_4 flush_size=10,000
|
||||
[pcap:Backdoor_Malware.pcap] label=backdoor_malware 29,155 pkts → 2,325 flows in 0.6s (0.05M pkts/s)
|
||||
[pcap:CommandInjection.pcap] label=commandinjection 49,515 pkts → 3,784 flows in 1.0s (0.05M pkts/s)
|
||||
[pcap:BrowserHijacking.pcap] label=browserhijacking 55,181 pkts → 2,800 flows in 1.0s (0.05M pkts/s)
|
||||
[pcap:BenignTraffic.pcap] label=normal 2,000,000 pkts → 130,266 flows in 36.7s (0.05M pkts/s)
|
||||
[pcap:DDoS-HTTP_Flood-.pcap] label=ddos-http_flood 2,000,000 pkts → 424,632 flows in 51.0s (0.04M pkts/s)
|
||||
[pcap:DDoS-ICMP_Fragmentation.pcap] label=ddos-icmp_fragmentation 91,881 pkts → 15,315 flows in 24.2s (0.00M pkts/s)
|
||||
[pcap:DDoS-ACK_Fragmentation.pcap] label=ddos-ack_fragmentation 1,421,801 pkts → 1,199,853 flows in 72.1s (0.02M pkts/s)
|
||||
[pcap:DDoS-PSHACK_Flood.pcap] label=ddos-pshack_flood 2,000,000 pkts → 472,916 flows in 47.0s (0.04M pkts/s)
|
||||
[pcap:DDoS-SYN_Flood.pcap] label=ddos-syn_flood 2,000,000 pkts → 445,305 flows in 46.9s (0.04M pkts/s)
|
||||
[pcap:DDoS-SlowLoris.pcap] label=ddos-slowloris 2,000,000 pkts → 170,596 flows in 40.9s (0.05M pkts/s)
|
||||
[pcap:DDoS-RSTFINFlood.pcap] label=ddos-rstfinflood 2,000,000 pkts → 1,989,762 flows in 87.8s (0.02M pkts/s)
|
||||
[pcap:DDoS-SynonymousIP_Flood.pcap] label=ddos-synonymousip_flood 2,000,000 pkts → 66,126 flows in 36.4s (0.05M pkts/s)
|
||||
[pcap:DDoS-UDP_Flood.pcap] label=ddos-udp_flood 2,000,000 pkts → 3,021 flows in 22.8s (0.09M pkts/s)
|
||||
[pcap:DDoS-UDP_Fragmentation.pcap] label=ddos-udp_fragmentation 1,141,302 pkts → 10,202 flows in 26.7s (0.04M pkts/s)
|
||||
[pcap:DDoS-TCP_Flood.pcap] label=ddos-tcp_flood 2,000,000 pkts → 459,439 flows in 44.7s (0.04M pkts/s)
|
||||
[pcap:DictionaryBruteForce.pcap] label=dictionarybruteforce 121,861 pkts → 7,910 flows in 2.3s (0.05M pkts/s)
|
||||
[pcap:DNS_Spoofing.pcap] label=dns_spoofing 1,717,375 pkts → 83,761 flows in 29.3s (0.06M pkts/s)
|
||||
[pcap:DoS-SYN_Flood.pcap] label=dos-syn_flood 2,000,000 pkts → 332,245 flows in 45.3s (0.04M pkts/s)
|
||||
[pcap:DoS-HTTP_Flood.pcap] label=dos-http_flood 2,000,000 pkts → 426,432 flows in 51.0s (0.04M pkts/s)
|
||||
[pcap:DoS-TCP_Flood.pcap] label=dos-tcp_flood 2,000,000 pkts → 404,258 flows in 46.6s (0.04M pkts/s)
|
||||
[pcap:DoS-UDP_Flood.pcap] label=dos-udp_flood 2,000,000 pkts → 67,459 flows in 33.9s (0.06M pkts/s)
|
||||
[pcap:DDoS-ICMP_Flood.pcap] label=ddos-icmp_flood 78,905 pkts → 12,441 flows in 267.1s (0.00M pkts/s)
|
||||
[pcap:MITM-ArpSpoofing.pcap] label=mitm-arpspoofing 2,000,000 pkts → 55,312 flows in 32.1s (0.06M pkts/s)
|
||||
[pcap:Mirai-udpplain.pcap] label=mirai-udpplain 2,000,000 pkts → 4,351 flows in 21.6s (0.09M pkts/s)
|
||||
[pcap:Recon-HostDiscovery.pcap] label=recon-hostdiscovery 1,253,455 pkts → 663,947 flows in 39.1s (0.03M pkts/s)
|
||||
[pcap:Recon-PingSweep.pcap] label=recon-pingsweep 19,361 pkts → 1,955 flows in 0.4s (0.05M pkts/s)
|
||||
[pcap:Mirai-greeth_flood.pcap] label=mirai-greeth_flood 83,075 pkts → 5,465 flows in 60.5s (0.00M pkts/s)
|
||||
[pcap:SqlInjection.pcap] label=sqlinjection 49,185 pkts → 6,693 flows in 1.0s (0.05M pkts/s)
|
||||
[pcap:Uploading_Attack.pcap] label=uploading_attack 10,826 pkts → 1,338 flows in 0.2s (0.05M pkts/s)
|
||||
[pcap:Recon-OSScan.pcap] label=recon-osscan 948,173 pkts → 193,983 flows in 22.2s (0.04M pkts/s)
|
||||
[pcap:XSS.pcap] label=xss 34,617 pkts → 3,209 flows in 0.7s (0.05M pkts/s)
|
||||
[pcap:Mirai-greip_flood.pcap] label=mirai-greip_flood 99,556 pkts → 10,526 flows in 51.7s (0.00M pkts/s)
|
||||
[pcap:Recon-PortScan.pcap] label=recon-portscan 794,588 pkts → 225,917 flows in 20.2s (0.04M pkts/s)
|
||||
[pcap:VulnerabilityScan.pcap] label=vulnerabilityscan 2,000,000 pkts → 290,077 flows in 42.2s (0.05M pkts/s)
|
||||
[extract_labeled_pcaps] wrote sharded store datasets/ciciot2023/processed/full_store
|
||||
@@ -0,0 +1,315 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 55918,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.34,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.4253447725,
|
||||
"auprc": 0.6336203824974163
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.3284644375,
|
||||
"auprc": 0.5703262532476359
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.474287345,
|
||||
"auprc": 0.6604585372129211
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.345611965,
|
||||
"auprc": 0.5911061098185502
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.37572706866197186
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.3659486545138889
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.3971305726872247
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.4506702391118702
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.39661989708404805
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.36687532228360953
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.4189297619047619
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.41150360685302073
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.36997108597285067
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.39456355574324325
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.4013035087719298
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.4142306954436451
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.5423447932163046
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.41400976491862573
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.41174555314533623
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.4536086931155192
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.08040000000000003
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.23766461267605635
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.2267169704861111
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.2521197356828194
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.675849146029035
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.25728048885077187
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.2260081952117864
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.33097696886446887
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.3387283137962128
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.23371393665158371
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.24820565878378376
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.2602960688758934
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.2658227817745803
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.4394152038083904
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.3338644665461121
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.3339986623282719
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.35733786464410733
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.17759999999999998
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.44823058978873237
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.43687092013888884
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.46721519823788543
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.5051565328778821
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.4637955403087478
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.4399157458563536
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.4803783882783883
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.466178088367899
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.4384012669683257
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.46593053209459456
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.46961754385964916
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.4834028776978418
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.5197994346920559
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.4705183092224231
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.47341887201735355
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.4768441073512252
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.1069
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.26284080105633806
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.25305117187500004
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.2844314537444934
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.4862088385994876
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.29204897084048026
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.25303305709023943
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.3609811813186813
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.3631856627592425
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.26044470588235297
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.2799416385135135
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.2939029564652372
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.30097362110311754
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.4718764207081226
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.3635770795660036
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.3609580621836587
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.3887544340723454
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.1442
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.npz
Normal file
Binary file not shown.
@@ -0,0 +1,315 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 55952,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.38,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.43168440249999995,
|
||||
"auprc": 0.653248355032242
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.34255154250000003,
|
||||
"auprc": 0.5775747163392775
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.4719932275,
|
||||
"auprc": 0.6770389923462667
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.3623585325,
|
||||
"auprc": 0.6097127275583607
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.4625233661593554
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.49788264248704667
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.3625894806338028
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.34427651727357605
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.3367791559000861
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.48163343321917806
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.3569629443938013
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.375805996393147
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.4901982485404504
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.34968621848739495
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.3650196053469128
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.35616412776412776
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.5789207841356343
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.39400925605536335
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.37381038230884556
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.49753053527980534
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.41180000000000005
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.33061566696508504
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.3498836787564767
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.22813639964788732
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.5780242763772176
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.2146053402239449
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.33724122431506853
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.2981407474931632
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.30952894499549144
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.3542118015012511
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.2230166806722689
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.23675684277530235
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.23488280098280095
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.4851591583409022
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.32835947231833906
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.3146345952023988
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.39591076642335765
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.1008
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.5223282452999105
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.5554525906735751
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.4286453345070423
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.431211858076564
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.3964294142980189
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.5432048801369863
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.4090533272561531
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.43014386834986473
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.5473794829024187
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.41311222689075633
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.42869287078294077
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.4212051597051597
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.5330334392976083
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.4276136678200692
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.4304715892053973
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.5127475669099757
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.4849
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.3723965085049239
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.39827927461139895
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.2534990316901409
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.4119016339869281
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.2365090439276486
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.38031429794520544
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.31528650865998176
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.328724526600541
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.3967505838198499
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.24594655462184878
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.2623697644812222
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.2580361179361179
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.5216696033908568
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.3491068339100346
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.33177863568215893
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.4339459245742092
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.13349999999999995
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.npz
Normal file
Binary file not shown.
@@ -0,0 +1,315 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 53770,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.33,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.4614600125,
|
||||
"auprc": 0.6514813502115975
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.336559935,
|
||||
"auprc": 0.5586935467559199
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.46671968499999994,
|
||||
"auprc": 0.6562102115267414
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.35462059749999997,
|
||||
"auprc": 0.5842039708602825
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.45824696699375556
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.4437374049027895
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.4331852772466539
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.4227395410414828
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.4768812669683258
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.44571107142857147
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.45909572953736655
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.4440355251544572
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.46051931719965433
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.4203869674185463
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.47621527078085646
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.4663419664268585
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.4984345669982446
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.4915897521448999
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.4453246858832225
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.4728106432748538
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.3994
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.28695820695807317
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.2802118765849535
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.25612839388145314
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.6631459841129745
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.28098542986425334
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.27987794642857144
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.34742228647686835
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.34577197705207413
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.29160108038029386
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.2567668755221386
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.27886917506297226
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.273452757793765
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.3731526477472206
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.393523832221163
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.3437822246858832
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.34739397660818716
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.5508
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.4651411239964317
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.4511319526627219
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.43998102294455066
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.47979249779346866
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.48298113122171943
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.45153102678571433
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.47963923487544485
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.46066866725507505
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.46814900605012966
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.426693567251462
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.4826334068010076
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.4743570743405276
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.472372279110591
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.4801701620591039
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.46612283813747224
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.47796064327485377
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7785
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.3187443354148082
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.3087689349112426
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.28197901529636715
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.5308290379523389
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.30983904977375565
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.3051109375
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.3719600978647687
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.3729121800529568
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.3238210025929127
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.2771423976608187
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.30957616498740553
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.30309280575539566
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.39941252194265653
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.42477235462345087
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.3666122320768662
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.3708116374269006
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.512
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.npz
Normal file
Binary file not shown.
251
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.json
Normal file
251
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.json
Normal file
@@ -0,0 +1,251 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 60260,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.53,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.723987635,
|
||||
"auprc": 0.8872871375309783
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.70639805,
|
||||
"auprc": 0.8641590977032279
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.6683948383333334,
|
||||
"auprc": 0.8659779046749252
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.7134654466666667,
|
||||
"auprc": 0.8853790719337757
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.4925913043478261
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.7028624217663422
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.6963890086206896
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.6402013784996794
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.5836448717948718
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.6075902702702703
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.6303199152542374
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.7437
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.6324439580908032
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.8726749496021221
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.5695197368421053
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.5733199999999999
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.5315
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.5696065217391304
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.7740621957579972
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.7862476293103449
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.7299238352212012
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.6455884615384615
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.7176697297297296
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.8331372881355933
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.91735
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.505422549476135
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.7242967692307694
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.8722671052631579
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.9212400000000001
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.91505
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.4132652173913043
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.5643121088317107
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.5558272629310346
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.5382328595853815
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.5202935897435899
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.528165945945946
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.5073029661016949
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.6384000000000001
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.6145343771827707
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.9034096233421751
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.49485723684210525
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.55864
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.42925
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.5577510869565218
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.7295628911682893
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.7360641163793102
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.6615358570207309
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.6040269230769231
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.6289054054054054
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.7029288135593221
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.76835
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.574446717112922
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.822038275862069
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.6556690789473686
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.57236
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.6512
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.npz
Normal file
Binary file not shown.
267
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.json
Normal file
267
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.json
Normal file
@@ -0,0 +1,267 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 59505,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.52,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6668211533333334,
|
||||
"auprc": 0.8607119185266181
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.6554650116666667,
|
||||
"auprc": 0.8277844345660434
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.634465715,
|
||||
"auprc": 0.8491222299021763
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.6561472166666666,
|
||||
"auprc": 0.8497418251887944
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.588597435897436
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.6416492323980942
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.6053505175983438
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.5833544717600931
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.5602988888888889
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.5421101796407185
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.6310906542056074
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7455999999999999
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.6006448484130744
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.8038008748814167
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.5733109289617487
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.49396666666666667
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.476
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.45262
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.5840461538461539
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.7347305540850538
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.720792132505176
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.6956247006463918
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.6544594444444445
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.6496095808383233
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.8215668224299066
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9705
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.4487339175746092
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.6489923948561189
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.8829650273224042
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.8676
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.40935
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.7960200000000001
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.5966423076923076
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.5571249250044115
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.5440619047619049
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.5237327169651372
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.5194622222222223
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.49491676646706584
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.5877266355140187
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7531000000000001
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.5841906560871625
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.8246712712132391
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.5303459016393443
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.5329333333333333
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.5776
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.47639999999999993
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.6281205128205128
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.6767522057526028
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.6634209109730849
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.6218044187771539
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.5768083333333334
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.5709479041916168
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.6819278037383177
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7867
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.5129454168640455
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.7434588858437863
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.6450237704918033
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.6121333333333333
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.4376
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.55664
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.npz
Normal file
Binary file not shown.
235
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.json
Normal file
235
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.json
Normal file
@@ -0,0 +1,235 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 60932,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.51,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.7161715483333334,
|
||||
"auprc": 0.8881672941674507
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.7170487166666666,
|
||||
"auprc": 0.8756976844169416
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.6483864266666667,
|
||||
"auprc": 0.8600219764599565
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.7141619033333333,
|
||||
"auprc": 0.8912745535785943
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.47882236842105264
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.7093726319530833
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.6815593886462883
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.6563557837206835
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.6089714285714286
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.5702905063291139
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.6069803571428571
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.5928810055223194
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.8490922833315739
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.4820925465838509
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.45777142857142855
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.1602
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.5005552631578947
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.7984934956459926
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.7867799126637556
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.7623737928472885
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.7168559523809523
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.7038743670886076
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.8151049107142857
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.47648805798435345
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.7279311886414018
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.7982416149068323
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.8218714285714286
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.33325000000000005
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.47616578947368426
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.5589718588946153
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.5580830786026201
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.5411474742650961
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.49092380952380954
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.5114113924050633
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.5382703125
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.5540403014265991
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.8657909479573525
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.4820391304347825
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.4935857142857143
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.15580000000000005
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.5257960526315789
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.7506463657366269
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.7347385371179039
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.6917450705720046
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.6745452380952381
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.5725
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.6745756696428571
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.5233837781868385
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.8083549667475985
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.5700437888198757
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.5157428571428572
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.3842
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.npz
Normal file
Binary file not shown.
315
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.json
Normal file
315
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.json
Normal file
@@ -0,0 +1,315 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 62210,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.53,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.422718281535649,
|
||||
"auprc": 0.4437923569974027
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.33296114665854154,
|
||||
"auprc": 0.39356746933635084
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.45944195612431443,
|
||||
"auprc": 0.4715220579782502
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.35375587040422507,
|
||||
"auprc": 0.40877349567832355
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3616414115646258
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3420397959183673
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.42484030612244894
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5418039115646258
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4732957482993197
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3372551020408163
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.44126096938775505
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.44336471088435375
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3339859693877551
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3942460884353742
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.49183656462585035
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.46841309523809527
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4602437925170068
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3960539115646259
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4473421768707483
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4093687925170068
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.41801986301369864
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.21539336734693879
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2121749149659864
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.28435688775510204
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7535377551020408
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3294068027210884
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.21182831632653062
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.37969600340136056
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.368281037414966
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2108563775510204
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2722716836734694
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3382204081632653
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3190501700680272
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.370610544217687
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.33109804421768707
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.37657032312925176
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30873630952380954
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.3937606164383562
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4222487244897959
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3999406462585034
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.48004974489795915
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5067812074829932
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.529956887755102
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.39267857142857143
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.46572057823129254
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4678329081632653
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.38884914965986395
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.44794132653061225
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5459670068027211
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5235412414965986
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.45838724489795923
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4241635204081633
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.47442329931972793
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.43052244897959185
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.448791894977169
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2390267006802721
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2369531462585034
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.32506156462585034
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6171353741496599
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3765078231292517
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23295960884353742
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.41379574829931975
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4043593537414966
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23030076530612245
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30233843537414967
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3889670068027211
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3703044217687075
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3973332482993197
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.35951828231292515
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4113459183673469
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3376498299319728
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.3759558219178082
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.npz
Normal file
Binary file not shown.
315
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.json
Normal file
315
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.json
Normal file
@@ -0,0 +1,315 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 61140,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.6,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.5786656865732276,
|
||||
"auprc": 0.6131257104888339
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.4778395388990453,
|
||||
"auprc": 0.49594523111163336
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5761829626244159,
|
||||
"auprc": 0.6104436039530841
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.5054213081454397,
|
||||
"auprc": 0.5640880978639875
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7842433673469388
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7843292517006804
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5089937074829931
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4897884353741497
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.439750425170068
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7818146258503402
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5504216836734694
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5729387755102041
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7924302721088435
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4896052721088435
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4496392857142857
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.42246913265306124
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5233130952380952
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6827020408163265
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5599113945578231
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5188365646258504
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.4544388127853881
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6695882653061225
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6684723639455782
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.351168962585034
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7297494047619046
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2386767006802721
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6622063775510204
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.47894753401360546
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.486478231292517
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6761340986394558
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.34190170068027215
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.25365051020408164
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23847457482993195
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4091680272108843
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.589158843537415
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4654670918367347
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.40955085034013605
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.4464783105022831
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7950007653061225
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7957119047619047
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5286866496598639
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5307670918367348
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.46200841836734696
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7926188775510205
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.515884268707483
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5247567176870749
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8047187074829932
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5102914965986395
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4737977891156463
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.442300850340136
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.49436139455782313
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5990525510204082
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5332416666666666
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5000771258503401
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.4629474885844749
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7428913265306122
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7435061224489796
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3855701530612245
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5737761054421769
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.26104804421768707
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7376933673469388
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5056143707482993
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5171555272108843
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7510192176870749
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.37658979591836733
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2760309523809524
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2579078231292517
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.43908852040816326
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.634240731292517
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4950724489795918
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4432912414965986
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.4332573059360731
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.npz
Normal file
Binary file not shown.
315
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.json
Normal file
315
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.json
Normal file
@@ -0,0 +1,315 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 59671,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.47,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.3776255789152955,
|
||||
"auprc": 0.41646982210819894
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.30400732277066833,
|
||||
"auprc": 0.3757284502898286
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.43596635181799714,
|
||||
"auprc": 0.45443162010700605
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.3205092575665245,
|
||||
"auprc": 0.38690603758534386
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2867234693877551
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30877636054421764
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.34863962585034014
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.44015178571428576
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4378643707482993
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.32090986394557824
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.40041607142857144
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4081965986394558
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.28620901360544215
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.34651632653061226
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.42906717687074836
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.455784693877551
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4428608843537415
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.34391649659863943
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.396706462585034
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3486605442176871
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.43214246575342463
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.19665042517006803
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.20129583333333334
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23523869047619048
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.6595357142857142
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3017340986394558
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.20673069727891158
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.33575365646258504
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3390980442176871
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.1903577380952381
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23556904761904762
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.29721437074829926
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3219133503401361
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.37050544217687076
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.28682568027210886
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3326247448979592
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2792321428571428
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.40313162100456623
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.36227619047619053
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.38666505102040816
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4262699829931973
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4447044217687075
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5127670068027211
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4050096088435374
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.44965484693877544
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4581757653061225
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.363368537414966
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4218748299319728
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5099818027210885
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.5303285714285714
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.43281292517006803
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.40089574829931973
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.4549595238095238
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3876406462585034
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.47365764840182645
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.2131952380952381
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.22612780612244898
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.26664659863945583
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.502262074829932
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.35126581632653064
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.23263112244897957
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.36710943877551017
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3656772108843538
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.21010391156462585
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.26292602040816326
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.341993962585034
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3713362244897959
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3910445578231293
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.30850850340136055
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.3603273809523809
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.29751003401360543
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.400362100456621
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.npz
Normal file
Binary file not shown.
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed42",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 76132,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 3.01,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.5571759146341463,
|
||||
"auprc": 0.20673369025828442
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.4999152057926829,
|
||||
"auprc": 0.12686265960277793
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5495690929878049,
|
||||
"auprc": 0.20363397238502046
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.5205511051829268,
|
||||
"auprc": 0.1880725027647867
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5571759146341463
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.4999152057926829
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5495690929878049
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5205511051829268
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.npz
Normal file
Binary file not shown.
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 77094,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 3.02,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.5718415396341463,
|
||||
"auprc": 0.2037778829445944
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.5233626143292682,
|
||||
"auprc": 0.19074944831826363
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5912113185975609,
|
||||
"auprc": 0.20822037107590194
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.5464200076219512,
|
||||
"auprc": 0.2281021363052374
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5718415396341463
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5233626143292682
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5912113185975609
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5464200076219512
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.npz
Normal file
Binary file not shown.
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 75971,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.97,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.5187383384146341,
|
||||
"auprc": 0.16885883583361713
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.4652036204268292,
|
||||
"auprc": 0.10760505657769281
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5552233231707318,
|
||||
"auprc": 0.17538497368324046
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.49095800304878046,
|
||||
"auprc": 0.15325191734494964
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5187383384146341
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.4652036204268292
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.5552233231707318
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.49095800304878046
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.npz
Normal file
Binary file not shown.
405
artifacts/baselines/kitsune_2026_04_29/master.log
Normal file
405
artifacts/baselines/kitsune_2026_04_29/master.log
Normal file
@@ -0,0 +1,405 @@
|
||||
=== protocol=iscxtor_within seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=iscxtor_within seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=1,312 D=9
|
||||
[data] train_flat packets=76,132 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/76,132 last_rmse=0.0063
|
||||
[train] {'t_train_sec': 3.01, 'n_trained_packets': 76132}
|
||||
[score] benign in 5.9s
|
||||
[score] attack in 0.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.npz
|
||||
[best agg=mean] AUROC=0.5572 AUPRC=0.2067
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.5572 AUPRC=0.2067
|
||||
median AUROC=0.5496 AUPRC=0.2036
|
||||
p90 AUROC=0.5206 AUPRC=0.1881
|
||||
max AUROC=0.4999 AUPRC=0.1269
|
||||
[done] elapsed=18s artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.json
|
||||
=== protocol=iscxtor_within seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=iscxtor_within seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=1,312 D=9
|
||||
[data] train_flat packets=77,094 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/77,094 last_rmse=0.0073
|
||||
[train] {'t_train_sec': 3.02, 'n_trained_packets': 77094}
|
||||
[score] benign in 5.9s
|
||||
[score] attack in 0.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.npz
|
||||
[best agg=median] AUROC=0.5912 AUPRC=0.2082
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.5912 AUPRC=0.2082
|
||||
mean AUROC=0.5718 AUPRC=0.2038
|
||||
p90 AUROC=0.5464 AUPRC=0.2281
|
||||
max AUROC=0.5234 AUPRC=0.1907
|
||||
[done] elapsed=17s artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.json
|
||||
=== protocol=iscxtor_within seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=iscxtor_within seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=1,312 D=9
|
||||
[data] train_flat packets=75,971 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/75,971 last_rmse=0.0220
|
||||
[train] {'t_train_sec': 2.97, 'n_trained_packets': 75971}
|
||||
[score] benign in 5.9s
|
||||
[score] attack in 0.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.npz
|
||||
[best agg=median] AUROC=0.5552 AUPRC=0.1754
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.5552 AUPRC=0.1754
|
||||
mean AUROC=0.5187 AUPRC=0.1689
|
||||
p90 AUROC=0.4910 AUPRC=0.1533
|
||||
max AUROC=0.4652 AUPRC=0.1076
|
||||
[done] elapsed=18s artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.json
|
||||
=== protocol=cicids_within seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicids_within seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=30,000 D=9
|
||||
[data] train_flat packets=60,260 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/60,260 last_rmse=0.0047
|
||||
[train] {'t_train_sec': 2.53, 'n_trained_packets': 60260}
|
||||
[score] benign in 4.9s
|
||||
[score] attack in 10.5s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.npz
|
||||
[best agg=mean] AUROC=0.7240 AUPRC=0.8873
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.7240 AUPRC=0.8873
|
||||
p90 AUROC=0.7135 AUPRC=0.8854
|
||||
max AUROC=0.7064 AUPRC=0.8642
|
||||
median AUROC=0.6684 AUPRC=0.8660
|
||||
[done] elapsed=124s artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.json
|
||||
=== protocol=cicids_within seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicids_within seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=30,000 D=9
|
||||
[data] train_flat packets=59,505 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/59,505 last_rmse=0.0066
|
||||
[train] {'t_train_sec': 2.52, 'n_trained_packets': 59505}
|
||||
[score] benign in 4.9s
|
||||
[score] attack in 10.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.npz
|
||||
[best agg=mean] AUROC=0.6668 AUPRC=0.8607
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.6668 AUPRC=0.8607
|
||||
p90 AUROC=0.6561 AUPRC=0.8497
|
||||
max AUROC=0.6555 AUPRC=0.8278
|
||||
median AUROC=0.6345 AUPRC=0.8491
|
||||
[done] elapsed=125s artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.json
|
||||
=== protocol=cicids_within seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicids_within seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=30,000 D=9
|
||||
[data] train_flat packets=60,932 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/60,932 last_rmse=0.0037
|
||||
[train] {'t_train_sec': 2.51, 'n_trained_packets': 60932}
|
||||
[score] benign in 4.7s
|
||||
[score] attack in 10.5s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.npz
|
||||
[best agg=max] AUROC=0.7170 AUPRC=0.8757
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7170 AUPRC=0.8757
|
||||
mean AUROC=0.7162 AUPRC=0.8882
|
||||
p90 AUROC=0.7142 AUPRC=0.8913
|
||||
median AUROC=0.6484 AUPRC=0.8600
|
||||
[done] elapsed=124s artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.json
|
||||
=== protocol=cicddos_within seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicddos_within seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=20,000 D=9
|
||||
[data] train_flat packets=55,918 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/55,918 last_rmse=0.0051
|
||||
[train] {'t_train_sec': 2.34, 'n_trained_packets': 55918}
|
||||
[score] benign in 4.5s
|
||||
[score] attack in 4.7s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.npz
|
||||
[best agg=median] AUROC=0.4743 AUPRC=0.6605
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4743 AUPRC=0.6605
|
||||
mean AUROC=0.4253 AUPRC=0.6336
|
||||
p90 AUROC=0.3456 AUPRC=0.5911
|
||||
max AUROC=0.3285 AUPRC=0.5703
|
||||
[done] elapsed=25s artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.json
|
||||
=== protocol=cicddos_within seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicddos_within seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=20,000 D=9
|
||||
[data] train_flat packets=55,952 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/55,952 last_rmse=0.0037
|
||||
[train] {'t_train_sec': 2.38, 'n_trained_packets': 55952}
|
||||
[score] benign in 4.6s
|
||||
[score] attack in 4.7s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.npz
|
||||
[best agg=median] AUROC=0.4720 AUPRC=0.6770
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4720 AUPRC=0.6770
|
||||
mean AUROC=0.4317 AUPRC=0.6532
|
||||
p90 AUROC=0.3624 AUPRC=0.6097
|
||||
max AUROC=0.3426 AUPRC=0.5776
|
||||
[done] elapsed=25s artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.json
|
||||
=== protocol=cicddos_within seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicddos_within seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=20,000 D=9
|
||||
[data] train_flat packets=53,770 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/53,770 last_rmse=0.0086
|
||||
[train] {'t_train_sec': 2.33, 'n_trained_packets': 53770}
|
||||
[score] benign in 4.4s
|
||||
[score] attack in 4.6s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.npz
|
||||
[best agg=median] AUROC=0.4667 AUPRC=0.6562
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4667 AUPRC=0.6562
|
||||
mean AUROC=0.4615 AUPRC=0.6515
|
||||
p90 AUROC=0.3546 AUPRC=0.5842
|
||||
max AUROC=0.3366 AUPRC=0.5587
|
||||
[done] elapsed=24s artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.json
|
||||
=== protocol=forward_cross seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=forward_cross seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=5,000 val=10,000 attack=9,846 D=9
|
||||
[data] train_flat packets=62,210 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/62,210 last_rmse=0.0155
|
||||
[train] {'t_train_sec': 2.53, 'n_trained_packets': 62210}
|
||||
[score] benign in 4.3s
|
||||
[score] attack in 2.4s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.npz
|
||||
[best agg=median] AUROC=0.4594 AUPRC=0.4715
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4594 AUPRC=0.4715
|
||||
mean AUROC=0.4227 AUPRC=0.4438
|
||||
p90 AUROC=0.3538 AUPRC=0.4088
|
||||
max AUROC=0.3330 AUPRC=0.3936
|
||||
[done] elapsed=138s artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.json
|
||||
=== protocol=forward_cross seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=forward_cross seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=5,000 val=10,000 attack=9,846 D=9
|
||||
[data] train_flat packets=61,140 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/61,140 last_rmse=0.0045
|
||||
[train] {'t_train_sec': 2.6, 'n_trained_packets': 61140}
|
||||
[score] benign in 4.5s
|
||||
[score] attack in 2.6s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.npz
|
||||
[best agg=mean] AUROC=0.5787 AUPRC=0.6131
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.5787 AUPRC=0.6131
|
||||
median AUROC=0.5762 AUPRC=0.6104
|
||||
p90 AUROC=0.5054 AUPRC=0.5641
|
||||
max AUROC=0.4778 AUPRC=0.4959
|
||||
[done] elapsed=139s artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.json
|
||||
=== protocol=forward_cross seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=forward_cross seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=5,000 val=10,000 attack=9,846 D=9
|
||||
[data] train_flat packets=59,671 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/59,671 last_rmse=0.0037
|
||||
[train] {'t_train_sec': 2.47, 'n_trained_packets': 59671}
|
||||
[score] benign in 4.5s
|
||||
[score] attack in 2.4s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.npz
|
||||
[best agg=median] AUROC=0.4360 AUPRC=0.4544
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4360 AUPRC=0.4544
|
||||
mean AUROC=0.3776 AUPRC=0.4165
|
||||
p90 AUROC=0.3205 AUPRC=0.3869
|
||||
max AUROC=0.3040 AUPRC=0.3757
|
||||
[done] elapsed=138s artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.json
|
||||
=== protocol=reverse_cross seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=reverse_cross seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=6,772 D=9
|
||||
[data] train_flat packets=54,466 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/54,466 last_rmse=0.0054
|
||||
[train] {'t_train_sec': 2.31, 'n_trained_packets': 54466}
|
||||
[score] benign in 4.8s
|
||||
[score] attack in 4.2s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.npz
|
||||
[best agg=max] AUROC=0.7549 AUPRC=0.6509
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7549 AUPRC=0.6509
|
||||
p90 AUROC=0.7233 AUPRC=0.6302
|
||||
mean AUROC=0.6985 AUPRC=0.5939
|
||||
median AUROC=0.6063 AUPRC=0.5104
|
||||
[done] elapsed=323s artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.json
|
||||
=== protocol=reverse_cross seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=reverse_cross seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=6,772 D=9
|
||||
[data] train_flat packets=54,901 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/54,901 last_rmse=0.0020
|
||||
[train] {'t_train_sec': 2.34, 'n_trained_packets': 54901}
|
||||
[score] benign in 4.8s
|
||||
[score] attack in 4.3s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.npz
|
||||
[best agg=max] AUROC=0.7325 AUPRC=0.6396
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7325 AUPRC=0.6396
|
||||
p90 AUROC=0.6841 AUPRC=0.5933
|
||||
mean AUROC=0.6346 AUPRC=0.5435
|
||||
median AUROC=0.5685 AUPRC=0.4831
|
||||
[done] elapsed=260s artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.json
|
||||
=== protocol=reverse_cross seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=reverse_cross seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=6,772 D=9
|
||||
[data] train_flat packets=57,453 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/57,453 last_rmse=0.0159
|
||||
[train] {'t_train_sec': 2.43, 'n_trained_packets': 57453}
|
||||
[score] benign in 5.0s
|
||||
[score] attack in 4.4s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.npz
|
||||
[best agg=max] AUROC=0.7573 AUPRC=0.6816
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7573 AUPRC=0.6816
|
||||
p90 AUROC=0.7113 AUPRC=0.6390
|
||||
mean AUROC=0.6710 AUPRC=0.5935
|
||||
median AUROC=0.5903 AUPRC=0.5093
|
||||
[done] elapsed=231s artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.json
|
||||
406
artifacts/baselines/kitsune_2026_04_29/orchestrator.log
Normal file
406
artifacts/baselines/kitsune_2026_04_29/orchestrator.log
Normal file
@@ -0,0 +1,406 @@
|
||||
=== protocol=iscxtor_within seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=iscxtor_within seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=1,312 D=9
|
||||
[data] train_flat packets=76,132 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/76,132 last_rmse=0.0063
|
||||
[train] {'t_train_sec': 3.01, 'n_trained_packets': 76132}
|
||||
[score] benign in 5.9s
|
||||
[score] attack in 0.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.npz
|
||||
[best agg=mean] AUROC=0.5572 AUPRC=0.2067
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.5572 AUPRC=0.2067
|
||||
median AUROC=0.5496 AUPRC=0.2036
|
||||
p90 AUROC=0.5206 AUPRC=0.1881
|
||||
max AUROC=0.4999 AUPRC=0.1269
|
||||
[done] elapsed=18s artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed42.json
|
||||
=== protocol=iscxtor_within seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=iscxtor_within seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=1,312 D=9
|
||||
[data] train_flat packets=77,094 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/77,094 last_rmse=0.0073
|
||||
[train] {'t_train_sec': 3.02, 'n_trained_packets': 77094}
|
||||
[score] benign in 5.9s
|
||||
[score] attack in 0.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.npz
|
||||
[best agg=median] AUROC=0.5912 AUPRC=0.2082
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.5912 AUPRC=0.2082
|
||||
mean AUROC=0.5718 AUPRC=0.2038
|
||||
p90 AUROC=0.5464 AUPRC=0.2281
|
||||
max AUROC=0.5234 AUPRC=0.1907
|
||||
[done] elapsed=17s artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed43.json
|
||||
=== protocol=iscxtor_within seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=iscxtor_within seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=1,312 D=9
|
||||
[data] train_flat packets=75,971 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/75,971 last_rmse=0.0220
|
||||
[train] {'t_train_sec': 2.97, 'n_trained_packets': 75971}
|
||||
[score] benign in 5.9s
|
||||
[score] attack in 0.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.npz
|
||||
[best agg=median] AUROC=0.5552 AUPRC=0.1754
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.5552 AUPRC=0.1754
|
||||
mean AUROC=0.5187 AUPRC=0.1689
|
||||
p90 AUROC=0.4910 AUPRC=0.1533
|
||||
max AUROC=0.4652 AUPRC=0.1076
|
||||
[done] elapsed=18s artifacts/baselines/kitsune_2026_04_29/iscxtor_within_seed44.json
|
||||
=== protocol=cicids_within seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicids_within seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=30,000 D=9
|
||||
[data] train_flat packets=60,260 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/60,260 last_rmse=0.0047
|
||||
[train] {'t_train_sec': 2.53, 'n_trained_packets': 60260}
|
||||
[score] benign in 4.9s
|
||||
[score] attack in 10.5s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.npz
|
||||
[best agg=mean] AUROC=0.7240 AUPRC=0.8873
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.7240 AUPRC=0.8873
|
||||
p90 AUROC=0.7135 AUPRC=0.8854
|
||||
max AUROC=0.7064 AUPRC=0.8642
|
||||
median AUROC=0.6684 AUPRC=0.8660
|
||||
[done] elapsed=124s artifacts/baselines/kitsune_2026_04_29/cicids_within_seed42.json
|
||||
=== protocol=cicids_within seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicids_within seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=30,000 D=9
|
||||
[data] train_flat packets=59,505 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/59,505 last_rmse=0.0066
|
||||
[train] {'t_train_sec': 2.52, 'n_trained_packets': 59505}
|
||||
[score] benign in 4.9s
|
||||
[score] attack in 10.8s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.npz
|
||||
[best agg=mean] AUROC=0.6668 AUPRC=0.8607
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.6668 AUPRC=0.8607
|
||||
p90 AUROC=0.6561 AUPRC=0.8497
|
||||
max AUROC=0.6555 AUPRC=0.8278
|
||||
median AUROC=0.6345 AUPRC=0.8491
|
||||
[done] elapsed=125s artifacts/baselines/kitsune_2026_04_29/cicids_within_seed43.json
|
||||
=== protocol=cicids_within seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicids_within seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train_flows=5,000 val=10,000 attack=30,000 D=9
|
||||
[data] train_flat packets=60,932 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/60,932 last_rmse=0.0037
|
||||
[train] {'t_train_sec': 2.51, 'n_trained_packets': 60932}
|
||||
[score] benign in 4.7s
|
||||
[score] attack in 10.5s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.npz
|
||||
[best agg=max] AUROC=0.7170 AUPRC=0.8757
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7170 AUPRC=0.8757
|
||||
mean AUROC=0.7162 AUPRC=0.8882
|
||||
p90 AUROC=0.7142 AUPRC=0.8913
|
||||
median AUROC=0.6484 AUPRC=0.8600
|
||||
[done] elapsed=124s artifacts/baselines/kitsune_2026_04_29/cicids_within_seed44.json
|
||||
=== protocol=cicddos_within seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicddos_within seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=20,000 D=9
|
||||
[data] train_flat packets=55,918 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/55,918 last_rmse=0.0051
|
||||
[train] {'t_train_sec': 2.34, 'n_trained_packets': 55918}
|
||||
[score] benign in 4.5s
|
||||
[score] attack in 4.7s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.npz
|
||||
[best agg=median] AUROC=0.4743 AUPRC=0.6605
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4743 AUPRC=0.6605
|
||||
mean AUROC=0.4253 AUPRC=0.6336
|
||||
p90 AUROC=0.3456 AUPRC=0.5911
|
||||
max AUROC=0.3285 AUPRC=0.5703
|
||||
[done] elapsed=25s artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed42.json
|
||||
=== protocol=cicddos_within seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicddos_within seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=20,000 D=9
|
||||
[data] train_flat packets=55,952 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/55,952 last_rmse=0.0037
|
||||
[train] {'t_train_sec': 2.38, 'n_trained_packets': 55952}
|
||||
[score] benign in 4.6s
|
||||
[score] attack in 4.7s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.npz
|
||||
[best agg=median] AUROC=0.4720 AUPRC=0.6770
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4720 AUPRC=0.6770
|
||||
mean AUROC=0.4317 AUPRC=0.6532
|
||||
p90 AUROC=0.3624 AUPRC=0.6097
|
||||
max AUROC=0.3426 AUPRC=0.5776
|
||||
[done] elapsed=25s artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed43.json
|
||||
=== protocol=cicddos_within seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=cicddos_within seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=20,000 D=9
|
||||
[data] train_flat packets=53,770 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/53,770 last_rmse=0.0086
|
||||
[train] {'t_train_sec': 2.33, 'n_trained_packets': 53770}
|
||||
[score] benign in 4.4s
|
||||
[score] attack in 4.6s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.npz
|
||||
[best agg=median] AUROC=0.4667 AUPRC=0.6562
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4667 AUPRC=0.6562
|
||||
mean AUROC=0.4615 AUPRC=0.6515
|
||||
p90 AUROC=0.3546 AUPRC=0.5842
|
||||
max AUROC=0.3366 AUPRC=0.5587
|
||||
[done] elapsed=24s artifacts/baselines/kitsune_2026_04_29/cicddos_within_seed44.json
|
||||
=== protocol=forward_cross seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=forward_cross seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=5,000 val=10,000 attack=9,846 D=9
|
||||
[data] train_flat packets=62,210 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/62,210 last_rmse=0.0155
|
||||
[train] {'t_train_sec': 2.53, 'n_trained_packets': 62210}
|
||||
[score] benign in 4.3s
|
||||
[score] attack in 2.4s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.npz
|
||||
[best agg=median] AUROC=0.4594 AUPRC=0.4715
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4594 AUPRC=0.4715
|
||||
mean AUROC=0.4227 AUPRC=0.4438
|
||||
p90 AUROC=0.3538 AUPRC=0.4088
|
||||
max AUROC=0.3330 AUPRC=0.3936
|
||||
[done] elapsed=138s artifacts/baselines/kitsune_2026_04_29/forward_cross_seed42.json
|
||||
=== protocol=forward_cross seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=forward_cross seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=5,000 val=10,000 attack=9,846 D=9
|
||||
[data] train_flat packets=61,140 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/61,140 last_rmse=0.0045
|
||||
[train] {'t_train_sec': 2.6, 'n_trained_packets': 61140}
|
||||
[score] benign in 4.5s
|
||||
[score] attack in 2.6s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.npz
|
||||
[best agg=mean] AUROC=0.5787 AUPRC=0.6131
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
mean AUROC=0.5787 AUPRC=0.6131
|
||||
median AUROC=0.5762 AUPRC=0.6104
|
||||
p90 AUROC=0.5054 AUPRC=0.5641
|
||||
max AUROC=0.4778 AUPRC=0.4959
|
||||
[done] elapsed=139s artifacts/baselines/kitsune_2026_04_29/forward_cross_seed43.json
|
||||
=== protocol=forward_cross seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=forward_cross seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=503,730 -> train=1,210,760 val=302,690
|
||||
[data] train_flows=5,000 val=10,000 attack=9,846 D=9
|
||||
[data] train_flat packets=59,671 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/59,671 last_rmse=0.0037
|
||||
[train] {'t_train_sec': 2.47, 'n_trained_packets': 59671}
|
||||
[score] benign in 4.5s
|
||||
[score] attack in 2.4s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.npz
|
||||
[best agg=median] AUROC=0.4360 AUPRC=0.4544
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
median AUROC=0.4360 AUPRC=0.4544
|
||||
mean AUROC=0.3776 AUPRC=0.4165
|
||||
p90 AUROC=0.3205 AUPRC=0.3869
|
||||
max AUROC=0.3040 AUPRC=0.3757
|
||||
[done] elapsed=138s artifacts/baselines/kitsune_2026_04_29/forward_cross_seed44.json
|
||||
=== protocol=reverse_cross seed=42 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=reverse_cross seed=42
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=6,772 D=9
|
||||
[data] train_flat packets=54,466 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/54,466 last_rmse=0.0054
|
||||
[train] {'t_train_sec': 2.31, 'n_trained_packets': 54466}
|
||||
[score] benign in 4.8s
|
||||
[score] attack in 4.2s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.npz
|
||||
[best agg=max] AUROC=0.7549 AUPRC=0.6509
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7549 AUPRC=0.6509
|
||||
p90 AUROC=0.7233 AUPRC=0.6302
|
||||
mean AUROC=0.6985 AUPRC=0.5939
|
||||
median AUROC=0.6063 AUPRC=0.5104
|
||||
[done] elapsed=323s artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.json
|
||||
=== protocol=reverse_cross seed=43 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=reverse_cross seed=43
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=6,772 D=9
|
||||
[data] train_flat packets=54,901 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/54,901 last_rmse=0.0020
|
||||
[train] {'t_train_sec': 2.34, 'n_trained_packets': 54901}
|
||||
[score] benign in 4.8s
|
||||
[score] attack in 4.3s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.npz
|
||||
[best agg=max] AUROC=0.7325 AUPRC=0.6396
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7325 AUPRC=0.6396
|
||||
p90 AUROC=0.6841 AUPRC=0.5933
|
||||
mean AUROC=0.6346 AUPRC=0.5435
|
||||
median AUROC=0.5685 AUPRC=0.4831
|
||||
[done] elapsed=260s artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.json
|
||||
=== protocol=reverse_cross seed=44 n_train_cap=5000 ===
|
||||
[run] kitsune protocol=reverse_cross seed=44
|
||||
[run] using packet stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=8,893,668 -> train=74,565 val=18,642
|
||||
[data] train_flows=5,000 val=10,000 attack=6,772 D=9
|
||||
[data] train_flat packets=57,453 FM_grace=2000 AD_grace=20000
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
The Feature-Mapper found a mapping: 9 features to 1 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[train] processed 50,000/57,453 last_rmse=0.0159
|
||||
[train] {'t_train_sec': 2.43, 'n_trained_packets': 57453}
|
||||
[score] benign in 5.0s
|
||||
[score] attack in 4.4s
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.json
|
||||
[saved] artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.npz
|
||||
[best agg=max] AUROC=0.7573 AUPRC=0.6816
|
||||
|
||||
=== overall AUROC by aggregator ===
|
||||
max AUROC=0.7573 AUPRC=0.6816
|
||||
p90 AUROC=0.7113 AUPRC=0.6390
|
||||
mean AUROC=0.6710 AUPRC=0.5935
|
||||
median AUROC=0.5903 AUPRC=0.5093
|
||||
[done] elapsed=231s artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.json
|
||||
ALL DONE
|
||||
283
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.json
Normal file
283
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.json
Normal file
@@ -0,0 +1,283 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 54466,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.31,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6984984494979326,
|
||||
"auprc": 0.5938537447029827
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.7549417528056704,
|
||||
"auprc": 0.6508896460434106
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.6062791568222091,
|
||||
"auprc": 0.5104305280266268
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.7232628987005316,
|
||||
"auprc": 0.6301781238590568
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5547075825825826
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8207093093093094
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7544175675675675
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7343030030030031
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7101981981981981
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6661138138138138
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6634243243243243
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7665
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.7007
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6582857357357358
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8860483483483483
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5690978978978979
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.49512808219178084
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5701076923076923
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.4154833333333333
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5965653153153153
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8667186936936938
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8125489489489489
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8022171171171173
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7561136636636636
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7419217717717718
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8369792042042042
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9735
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.9260857142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5376098348348348
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7660006006006005
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8304644144144144
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.7853835616438356
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.6326923076923077
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.7253611111111111
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5225315315315315
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6240541291291292
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6072938438438438
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5990743243243243
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5931954204204204
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5669638138138138
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5438021771771772
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.7924
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.5036785714285714
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6057373873873874
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9029579579579579
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5151243993993995
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.4803986301369863
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5618923076923077
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.5145055555555555
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6111415915915916
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8433192192192192
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7923900900900901
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7535391891891892
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7431529279279279
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6743316066066066
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7277819819819821
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.8218000000000001
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.7718428571428572
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5952377627627627
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.846989039039039
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6674193693693694
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.5902164383561643
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5918307692307693
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.49446666666666667
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed42.npz
Normal file
Binary file not shown.
283
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.json
Normal file
283
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.json
Normal file
@@ -0,0 +1,283 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 54901,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.34,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6345540091553455,
|
||||
"auprc": 0.5434895028964192
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.732484886296515,
|
||||
"auprc": 0.6396314263925083
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5684908298877731,
|
||||
"auprc": 0.483124225326346
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.6840660513880685,
|
||||
"auprc": 0.593328868440604
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4971394894894895
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7452524024024024
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6392565315315315
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6283921921921922
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5856407657657657
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7028736486486487
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6037015765765765
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.38680000000000003
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.4581142857142857
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.62235
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8743785285285286
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4735269519519519
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.456841095890411
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5371846153846154
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.5100166666666667
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5596978978978979
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8372762012012012
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7770656156156156
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7518904654654655
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6863843843843844
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7811388138138139
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8401141141141142
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.861
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.8005
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.49528093093093095
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7859298798798798
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8009181681681682
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.8025027397260274
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5718307692307693
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.8695999999999999
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.503235960960961
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5644545045045045
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5390427927927928
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5338515015015015
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5319112612612613
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5615001501501501
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5224403903903904
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.41400000000000003
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.4148
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5645393393393393
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8810318318318319
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4952943693693694
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.4862219178082192
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5883307692307692
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.4975888888888889
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5955102102102103
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7872588588588589
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7064437687687688
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6634027777777778
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6625460210210211
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.752024174174174
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6953288288288287
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.5695
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.6463428571428571
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5379545045045044
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8529801801801802
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6014504504504503
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.5921335616438356
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.6132384615384615
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.6022388888888889
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed43.npz
Normal file
Binary file not shown.
283
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.json
Normal file
283
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.json
Normal file
@@ -0,0 +1,283 @@
|
||||
{
|
||||
"method": "kitsune_path_b",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44",
|
||||
"n_train_flows": 5000,
|
||||
"n_train_packets": 57453,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"D": 9,
|
||||
"fm_grace": 2000,
|
||||
"ad_grace": 20000,
|
||||
"max_ae_size": 10,
|
||||
"t_train_sec": 2.43,
|
||||
"overall_by_agg": {
|
||||
"mean": {
|
||||
"auroc": 0.6710443000590667,
|
||||
"auprc": 0.5934631597417034
|
||||
},
|
||||
"max": {
|
||||
"auroc": 0.7573452894270526,
|
||||
"auprc": 0.6815579379969356
|
||||
},
|
||||
"median": {
|
||||
"auroc": 0.5903241287655051,
|
||||
"auprc": 0.5093495936339838
|
||||
},
|
||||
"p90": {
|
||||
"auroc": 0.7113272593030123,
|
||||
"auprc": 0.6390345913750659
|
||||
}
|
||||
},
|
||||
"per_class_by_agg": {
|
||||
"mean": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5135243993993993
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8236734234234235
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7263843843843844
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7083266516516517
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.630890990990991
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.647117042042042
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.600568018018018
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.6537
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.44662857142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7062888888888889
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9018552552552551
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.48096388888888886
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.5025328767123288
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5300307692307693
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.46598333333333336
|
||||
}
|
||||
},
|
||||
"max": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5894972972972973
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8936613363363363
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8184557057057058
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8016686186186187
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7029424924924925
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.736571996996997
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8200584834834835
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.8714000000000001
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.8201857142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5956076576576577
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8130694444444445
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7947412912912913
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.8261986301369864
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.6178076923076923
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.8137166666666666
|
||||
}
|
||||
},
|
||||
"median": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.4857828828828829
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6222737987987988
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5795584084084083
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5650107357357358
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5482627627627628
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5418831081081081
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5217380630630631
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.6303
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.3574857142857143
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6394728978978979
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9071283783783785
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5074786036036035
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.5025643835616438
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.5022153846153846
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.5303111111111112
|
||||
}
|
||||
},
|
||||
"p90": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6189283033033033
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8482237237237238
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7746091591591591
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7393897147147147
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7007463963963964
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6564996996996998
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6700355105105105
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.6912
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.6416571428571429
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.6458466966966967
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8758728228228227
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.5993527777777777
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.6200178082191781
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.6179384615384615
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.5767055555555556
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
BIN
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.npz
Normal file
BIN
artifacts/baselines/kitsune_2026_04_29/reverse_cross_seed44.npz
Normal file
Binary file not shown.
736
artifacts/baselines/kitsune_2026_04_29/summary.json
Normal file
736
artifacts/baselines/kitsune_2026_04_29/summary.json
Normal file
@@ -0,0 +1,736 @@
|
||||
{
|
||||
"rows": [
|
||||
{
|
||||
"protocol": "iscxtor_within",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "median",
|
||||
"auroc_mean": 0.5653345782520325,
|
||||
"auroc_std": 0.022587537447070313,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.5492519308943088,
|
||||
"auroc_std": 0.027424070333776224,
|
||||
"auprc_mean": 0.19312346967883198,
|
||||
"auprc_std": 0.021065695833526965
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.49616048018292674,
|
||||
"auroc_std": 0.029260735059793066,
|
||||
"auprc_mean": 0.14173905483291147,
|
||||
"auprc_std": 0.043522727634647734
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.5653345782520325,
|
||||
"auroc_std": 0.022587537447070313,
|
||||
"auprc_mean": 0.1957464390480543,
|
||||
"auprc_std": 0.017782033547651812
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.5193097052845528,
|
||||
"auroc_std": 0.027751834049290606,
|
||||
"auprc_mean": 0.18980885213832457,
|
||||
"auprc_std": 0.037455306719622446
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "cicids_within",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "mean",
|
||||
"auroc_mean": 0.7023267788888888,
|
||||
"auroc_std": 0.030996126971989585,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.7023267788888888,
|
||||
"auroc_std": 0.030996126971989585,
|
||||
"auprc_mean": 0.8787221167416823,
|
||||
"auprc_std": 0.015603496362533924
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.6929705927777778,
|
||||
"auroc_std": 0.032914444184471786,
|
||||
"auprc_mean": 0.855880405562071,
|
||||
"auprc_std": 0.025006447878391045
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.6504156600000001,
|
||||
"auroc_std": 0.017055342080502786,
|
||||
"auprc_mean": 0.8583740370123527,
|
||||
"auprc_std": 0.008547819682725945
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.6945915222222222,
|
||||
"auroc_std": 0.03329556629956647,
|
||||
"auprc_mean": 0.8754651502337215,
|
||||
"auprc_std": 0.022471231930655883
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "cicddos_within",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "median",
|
||||
"auroc_mean": 0.47100008583333336,
|
||||
"auroc_std": 0.003880350144826164,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.43949639583333333,
|
||||
"auroc_std": 0.01928336251654347,
|
||||
"auprc_mean": 0.646116695913752,
|
||||
"auprc_std": 0.010858128902279298
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.3358586383333333,
|
||||
"auroc_std": 0.0070696884359400785,
|
||||
"auprc_mean": 0.5688648387809444,
|
||||
"auprc_std": 0.009525042802605587
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.47100008583333336,
|
||||
"auroc_std": 0.003880350144826164,
|
||||
"auprc_mean": 0.6645692470286432,
|
||||
"auprc_std": 0.01100604284012929
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.3541970316666667,
|
||||
"auroc_std": 0.00838131473990742,
|
||||
"auprc_mean": 0.5950076027457311,
|
||||
"auprc_std": 0.013194332206143264
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "forward_cross",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "median",
|
||||
"auroc_mean": 0.49053042352224246,
|
||||
"auroc_std": 0.07510022696619303,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.45966984900805735,
|
||||
"auroc_std": 0.10549097106283167,
|
||||
"auprc_mean": 0.4911292965314786,
|
||||
"auprc_std": 0.10653156336050716
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.3716026694427517,
|
||||
"auroc_std": 0.09313584327925348,
|
||||
"auprc_mean": 0.42174705024593756,
|
||||
"auprc_std": 0.06487360929507022
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.49053042352224246,
|
||||
"auroc_std": 0.07510022696619303,
|
||||
"auprc_mean": 0.5121324273461134,
|
||||
"auprc_std": 0.0855677296125294
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.3932288120387297,
|
||||
"auroc_std": 0.09857333033475935,
|
||||
"auprc_mean": 0.45325587704255166,
|
||||
"auprc_std": 0.09660425616928427
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"protocol": "reverse_cross",
|
||||
"n_seeds": 3,
|
||||
"best_agg": "max",
|
||||
"auroc_mean": 0.748257309509746,
|
||||
"auroc_std": 0.01371208399863175,
|
||||
"all_aggs": {
|
||||
"mean": {
|
||||
"auroc_mean": 0.668032252904115,
|
||||
"auroc_std": 0.032078453574469715,
|
||||
"auprc_mean": 0.5769354691137017,
|
||||
"auprc_std": 0.028965714755702122
|
||||
},
|
||||
"max": {
|
||||
"auroc_mean": 0.748257309509746,
|
||||
"auroc_std": 0.01371208399863175,
|
||||
"auprc_mean": 0.6573596701442849,
|
||||
"auprc_std": 0.02169917055392824
|
||||
},
|
||||
"median": {
|
||||
"auroc_mean": 0.5883647051584958,
|
||||
"auroc_std": 0.01897021135305916,
|
||||
"auprc_mean": 0.5009681156623188,
|
||||
"auprc_std": 0.015462710678660731
|
||||
},
|
||||
"p90": {
|
||||
"auroc_mean": 0.7062187364638709,
|
||||
"auroc_std": 0.020091564498755668,
|
||||
"auprc_mean": 0.6208471945582422,
|
||||
"auprc_std": 0.02423949171193675
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"per_class": {
|
||||
"iscxtor_within": {
|
||||
"tor": {
|
||||
"n": 1312,
|
||||
"aurocs": [
|
||||
0.5571759146341463,
|
||||
0.5718415396341463,
|
||||
0.5187383384146341
|
||||
]
|
||||
}
|
||||
},
|
||||
"cicids_within": {
|
||||
"Botnet": {
|
||||
"n": 46,
|
||||
"aurocs": [
|
||||
0.4925913043478261,
|
||||
0.588597435897436,
|
||||
0.47882236842105264
|
||||
]
|
||||
},
|
||||
"DDoS": {
|
||||
"n": 5752,
|
||||
"aurocs": [
|
||||
0.7028624217663422,
|
||||
0.6416492323980942,
|
||||
0.7093726319530833
|
||||
]
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"n": 464,
|
||||
"aurocs": [
|
||||
0.6963890086206896,
|
||||
0.6053505175983438,
|
||||
0.6815593886462883
|
||||
]
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"n": 9358,
|
||||
"aurocs": [
|
||||
0.6402013784996794,
|
||||
0.5833544717600931,
|
||||
0.6563557837206835
|
||||
]
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"n": 78,
|
||||
"aurocs": [
|
||||
0.5836448717948718,
|
||||
0.5602988888888889,
|
||||
0.6089714285714286
|
||||
]
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"n": 185,
|
||||
"aurocs": [
|
||||
0.6075902702702703,
|
||||
0.5421101796407185,
|
||||
0.5702905063291139
|
||||
]
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"n": 236,
|
||||
"aurocs": [
|
||||
0.6303199152542374,
|
||||
0.6310906542056074,
|
||||
0.6069803571428571
|
||||
]
|
||||
},
|
||||
"Infiltration": {
|
||||
"n": 2,
|
||||
"aurocs": [
|
||||
0.7437,
|
||||
0.7455999999999999
|
||||
]
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"n": 4295,
|
||||
"aurocs": [
|
||||
0.6324439580908032,
|
||||
0.6006448484130744,
|
||||
0.5928810055223194
|
||||
]
|
||||
},
|
||||
"Portscan": {
|
||||
"n": 9425,
|
||||
"aurocs": [
|
||||
0.8726749496021221,
|
||||
0.8038008748814167,
|
||||
0.8490922833315739
|
||||
]
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"n": 152,
|
||||
"aurocs": [
|
||||
0.5695197368421053,
|
||||
0.5733109289617487,
|
||||
0.4820925465838509
|
||||
]
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"n": 5,
|
||||
"aurocs": [
|
||||
0.5733199999999999,
|
||||
0.49396666666666667,
|
||||
0.45777142857142855
|
||||
]
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"n": 2,
|
||||
"aurocs": [
|
||||
0.5315,
|
||||
0.45262
|
||||
]
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"n": 2,
|
||||
"aurocs": [
|
||||
0.476,
|
||||
0.1602
|
||||
]
|
||||
}
|
||||
},
|
||||
"cicddos_within": {
|
||||
"DrDoS_DNS": {
|
||||
"n": 1136,
|
||||
"aurocs": [
|
||||
0.37572706866197186,
|
||||
0.4625233661593554,
|
||||
0.45824696699375556
|
||||
]
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"n": 1152,
|
||||
"aurocs": [
|
||||
0.3659486545138889,
|
||||
0.49788264248704667,
|
||||
0.4437374049027895
|
||||
]
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"n": 1135,
|
||||
"aurocs": [
|
||||
0.3971305726872247,
|
||||
0.3625894806338028,
|
||||
0.4331852772466539
|
||||
]
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"n": 1171,
|
||||
"aurocs": [
|
||||
0.4506702391118702,
|
||||
0.34427651727357605,
|
||||
0.4227395410414828
|
||||
]
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"n": 1166,
|
||||
"aurocs": [
|
||||
0.39661989708404805,
|
||||
0.3367791559000861,
|
||||
0.4768812669683258
|
||||
]
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"n": 1086,
|
||||
"aurocs": [
|
||||
0.36687532228360953,
|
||||
0.48163343321917806,
|
||||
0.44571107142857147
|
||||
]
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"n": 1092,
|
||||
"aurocs": [
|
||||
0.4189297619047619,
|
||||
0.3569629443938013,
|
||||
0.45909572953736655
|
||||
]
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"n": 1109,
|
||||
"aurocs": [
|
||||
0.41150360685302073,
|
||||
0.375805996393147,
|
||||
0.4440355251544572
|
||||
]
|
||||
},
|
||||
"LDAP": {
|
||||
"n": 1105,
|
||||
"aurocs": [
|
||||
0.36997108597285067,
|
||||
0.4901982485404504,
|
||||
0.46051931719965433
|
||||
]
|
||||
},
|
||||
"MSSQL": {
|
||||
"n": 1184,
|
||||
"aurocs": [
|
||||
0.39456355574324325,
|
||||
0.34968621848739495,
|
||||
0.4203869674185463
|
||||
]
|
||||
},
|
||||
"NetBIOS": {
|
||||
"n": 1539,
|
||||
"aurocs": [
|
||||
0.4013035087719298,
|
||||
0.3650196053469128,
|
||||
0.47621527078085646
|
||||
]
|
||||
},
|
||||
"Portmap": {
|
||||
"n": 417,
|
||||
"aurocs": [
|
||||
0.4142306954436451,
|
||||
0.35616412776412776,
|
||||
0.4663419664268585
|
||||
]
|
||||
},
|
||||
"Syn": {
|
||||
"n": 3361,
|
||||
"aurocs": [
|
||||
0.5423447932163046,
|
||||
0.5789207841356343,
|
||||
0.4984345669982446
|
||||
]
|
||||
},
|
||||
"TFTP": {
|
||||
"n": 1106,
|
||||
"aurocs": [
|
||||
0.41400976491862573,
|
||||
0.39400925605536335,
|
||||
0.4915897521448999
|
||||
]
|
||||
},
|
||||
"UDP": {
|
||||
"n": 1383,
|
||||
"aurocs": [
|
||||
0.41174555314533623,
|
||||
0.37381038230884556,
|
||||
0.4453246858832225
|
||||
]
|
||||
},
|
||||
"UDPLag": {
|
||||
"n": 857,
|
||||
"aurocs": [
|
||||
0.4536086931155192,
|
||||
0.49753053527980534,
|
||||
0.4728106432748538
|
||||
]
|
||||
},
|
||||
"WebDDoS": {
|
||||
"n": 1,
|
||||
"aurocs": [
|
||||
0.08040000000000003,
|
||||
0.41180000000000005,
|
||||
0.3994
|
||||
]
|
||||
}
|
||||
},
|
||||
"forward_cross": {
|
||||
"DrDoS_DNS": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.3616414115646258,
|
||||
0.7842433673469388,
|
||||
0.2867234693877551
|
||||
]
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.3420397959183673,
|
||||
0.7843292517006804,
|
||||
0.30877636054421764
|
||||
]
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.42484030612244894,
|
||||
0.5089937074829931,
|
||||
0.34863962585034014
|
||||
]
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.5418039115646258,
|
||||
0.4897884353741497,
|
||||
0.44015178571428576
|
||||
]
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.4732957482993197,
|
||||
0.439750425170068,
|
||||
0.4378643707482993
|
||||
]
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.3372551020408163,
|
||||
0.7818146258503402,
|
||||
0.32090986394557824
|
||||
]
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.44126096938775505,
|
||||
0.5504216836734694,
|
||||
0.40041607142857144
|
||||
]
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.44336471088435375,
|
||||
0.5729387755102041,
|
||||
0.4081965986394558
|
||||
]
|
||||
},
|
||||
"LDAP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.3339859693877551,
|
||||
0.7924302721088435,
|
||||
0.28620901360544215
|
||||
]
|
||||
},
|
||||
"MSSQL": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.3942460884353742,
|
||||
0.4896052721088435,
|
||||
0.34651632653061226
|
||||
]
|
||||
},
|
||||
"NetBIOS": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.49183656462585035,
|
||||
0.4496392857142857,
|
||||
0.42906717687074836
|
||||
]
|
||||
},
|
||||
"Portmap": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.46841309523809527,
|
||||
0.42246913265306124,
|
||||
0.455784693877551
|
||||
]
|
||||
},
|
||||
"Syn": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.4602437925170068,
|
||||
0.5233130952380952,
|
||||
0.4428608843537415
|
||||
]
|
||||
},
|
||||
"TFTP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.3960539115646259,
|
||||
0.6827020408163265,
|
||||
0.34391649659863943
|
||||
]
|
||||
},
|
||||
"UDP": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.4473421768707483,
|
||||
0.5599113945578231,
|
||||
0.396706462585034
|
||||
]
|
||||
},
|
||||
"UDPLag": {
|
||||
"n": 588,
|
||||
"aurocs": [
|
||||
0.4093687925170068,
|
||||
0.5188365646258504,
|
||||
0.3486605442176871
|
||||
]
|
||||
},
|
||||
"WebDDoS": {
|
||||
"n": 438,
|
||||
"aurocs": [
|
||||
0.41801986301369864,
|
||||
0.4544388127853881,
|
||||
0.43214246575342463
|
||||
]
|
||||
}
|
||||
},
|
||||
"reverse_cross": {
|
||||
"Botnet": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.5547075825825826,
|
||||
0.4971394894894895,
|
||||
0.5135243993993993
|
||||
]
|
||||
},
|
||||
"DDoS": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8207093093093094,
|
||||
0.7452524024024024,
|
||||
0.8236734234234235
|
||||
]
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.7544175675675675,
|
||||
0.6392565315315315,
|
||||
0.7263843843843844
|
||||
]
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.7343030030030031,
|
||||
0.6283921921921922,
|
||||
0.7083266516516517
|
||||
]
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.7101981981981981,
|
||||
0.5856407657657657,
|
||||
0.630890990990991
|
||||
]
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.6661138138138138,
|
||||
0.7028736486486487,
|
||||
0.647117042042042
|
||||
]
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.6634243243243243,
|
||||
0.6037015765765765,
|
||||
0.600568018018018
|
||||
]
|
||||
},
|
||||
"Heartbleed": {
|
||||
"n": 1,
|
||||
"aurocs": [
|
||||
0.7665,
|
||||
0.38680000000000003,
|
||||
0.6537
|
||||
]
|
||||
},
|
||||
"Infiltration": {
|
||||
"n": 7,
|
||||
"aurocs": [
|
||||
0.7007,
|
||||
0.4581142857142857,
|
||||
0.44662857142857143
|
||||
]
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.6582857357357358,
|
||||
0.62235,
|
||||
0.7062888888888889
|
||||
]
|
||||
},
|
||||
"Portscan": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.8860483483483483,
|
||||
0.8743785285285286,
|
||||
0.9018552552552551
|
||||
]
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"n": 666,
|
||||
"aurocs": [
|
||||
0.5690978978978979,
|
||||
0.4735269519519519,
|
||||
0.48096388888888886
|
||||
]
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"n": 73,
|
||||
"aurocs": [
|
||||
0.49512808219178084,
|
||||
0.456841095890411,
|
||||
0.5025328767123288
|
||||
]
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"n": 13,
|
||||
"aurocs": [
|
||||
0.5701076923076923,
|
||||
0.5371846153846154,
|
||||
0.5300307692307693
|
||||
]
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"n": 18,
|
||||
"aurocs": [
|
||||
0.4154833333333333,
|
||||
0.5100166666666667,
|
||||
0.46598333333333336
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"baselines": {
|
||||
"terminal_norm": {
|
||||
"iscxtor_within": [
|
||||
0.9945,
|
||||
0.0011
|
||||
],
|
||||
"cicids_within": [
|
||||
0.9858,
|
||||
0.0021
|
||||
],
|
||||
"cicddos_within": [
|
||||
0.996,
|
||||
0.001
|
||||
],
|
||||
"forward_cross": [
|
||||
0.9109,
|
||||
0.0032
|
||||
],
|
||||
"reverse_cross": [
|
||||
0.5999,
|
||||
null
|
||||
]
|
||||
},
|
||||
"kitsune_paper": {
|
||||
"iscxtor_within": [
|
||||
0.78,
|
||||
null
|
||||
],
|
||||
"cicids_within": [
|
||||
0.85,
|
||||
null
|
||||
],
|
||||
"cicddos_within": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"forward_cross": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"reverse_cross": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
71
artifacts/baselines/kitsune_2026_04_29/summary.md
Normal file
71
artifacts/baselines/kitsune_2026_04_29/summary.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# Kitsune (Path B) Baseline — On Our 5-Protocol Layout
|
||||
|
||||
Date: 2026-04-29
|
||||
|
||||
Method: KitNET ensemble autoencoder (the ML core of Kitsune).
|
||||
**Path B**: feeds our **z-scored 9-d packet features** directly through `KitNET.process()` for the FM+AD grace, then `KitNET.execute()` per packet during eval. **AfterImage's 100-d host/session statistics are skipped** (they require sequential pcap streams which our (B,T,9) tensor abstraction discards). This keeps data usage unified with `eval_new_scores.py`.
|
||||
Train: 5000 source-benign flows → ~75-320k packets (≥ FM+AD=55k grace).
|
||||
Score: per-flow aggregate of per-packet RMSE (mean / max / median / p90).
|
||||
Sampling: same seeds & stratification as `eval_new_scores.py`.
|
||||
|
||||
## Headline AUROC (best aggregator per protocol, 3-seed mean ± std)
|
||||
|
||||
| Protocol | terminal_norm | Kitsune paper (Shafir reproduction) | **Kitsune Path B (ours)** | best agg | Δ vs paper | Δ vs terminal |
|
||||
|---|---:|---:|---:|---|---:|---:|
|
||||
| ISCXTor2016 within | 0.9945 ± 0.0011 | 0.7800 | **0.5653 ± 0.0226** | `median` | -0.2147 | -0.4292 |
|
||||
| CICIDS2017 within (σ=0.6) | 0.9858 ± 0.0021 | 0.8500 | **0.7023 ± 0.0310** | `mean` | -0.1477 | -0.2835 |
|
||||
| CICDDoS2019 within | 0.9960 ± 0.0010 | — | **0.4710 ± 0.0039** | `median` | — | -0.5250 |
|
||||
| IDS2017→DDoS2019 forward | 0.9109 ± 0.0032 | — | **0.4905 ± 0.0751** | `median` | — | -0.4204 |
|
||||
| DDoS2019→IDS2017 reverse | 0.5999 | — | **0.7483 ± 0.0137** | `max` | — | +0.1484 |
|
||||
|
||||
## All aggregators (3-seed mean ± std)
|
||||
|
||||
| Protocol | mean | max | median | p90 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| ISCXTor2016 within | 0.5493 ± 0.0274 | 0.4962 ± 0.0293 | 0.5653 ± 0.0226 | 0.5193 ± 0.0278 |
|
||||
| CICIDS2017 within (σ=0.6) | 0.7023 ± 0.0310 | 0.6930 ± 0.0329 | 0.6504 ± 0.0171 | 0.6946 ± 0.0333 |
|
||||
| CICDDoS2019 within | 0.4395 ± 0.0193 | 0.3359 ± 0.0071 | 0.4710 ± 0.0039 | 0.3542 ± 0.0084 |
|
||||
| IDS2017→DDoS2019 forward | 0.4597 ± 0.1055 | 0.3716 ± 0.0931 | 0.4905 ± 0.0751 | 0.3932 ± 0.0986 |
|
||||
| DDoS2019→IDS2017 reverse | 0.6680 ± 0.0321 | 0.7483 ± 0.0137 | 0.5884 ± 0.0190 | 0.7062 ± 0.0201 |
|
||||
|
||||
## Per-attack (forward + reverse, mean aggregator)
|
||||
|
||||
### IDS2017→DDoS2019 forward
|
||||
| attack | n | Kitsune AUROC mean ± std |
|
||||
|---|---:|---:|
|
||||
| `DrDoS_DNS` | 588 | 0.4775 ± 0.2682 |
|
||||
| `DrDoS_LDAP` | 588 | 0.4784 ± 0.2655 |
|
||||
| `DrDoS_MSSQL` | 588 | 0.4275 ± 0.0802 |
|
||||
| `DrDoS_NTP` | 588 | 0.4906 ± 0.0508 |
|
||||
| `DrDoS_NetBIOS` | 588 | 0.4503 ± 0.0199 |
|
||||
| `DrDoS_SNMP` | 588 | 0.4800 ± 0.2615 |
|
||||
| `DrDoS_SSDP` | 588 | 0.4640 ± 0.0776 |
|
||||
| `DrDoS_UDP` | 588 | 0.4748 ± 0.0868 |
|
||||
| `LDAP` | 588 | 0.4709 ± 0.2795 |
|
||||
| `MSSQL` | 588 | 0.4101 ± 0.0729 |
|
||||
| `NetBIOS` | 588 | 0.4568 ± 0.0320 |
|
||||
| `Portmap` | 588 | 0.4489 ± 0.0237 |
|
||||
| `Syn` | 588 | 0.4755 ± 0.0423 |
|
||||
| `TFTP` | 588 | 0.4742 ± 0.1824 |
|
||||
| `UDP` | 588 | 0.4680 ± 0.0835 |
|
||||
| `UDPLag` | 588 | 0.4256 ± 0.0862 |
|
||||
| `WebDDoS` | 438 | 0.4349 ± 0.0184 |
|
||||
|
||||
### DDoS2019→IDS2017 reverse
|
||||
| attack | n | Kitsune AUROC mean ± std |
|
||||
|---|---:|---:|
|
||||
| `Botnet` | 666 | 0.5218 ± 0.0297 |
|
||||
| `DDoS` | 666 | 0.7965 ± 0.0444 |
|
||||
| `DoS GoldenEye` | 666 | 0.7067 ± 0.0601 |
|
||||
| `DoS Hulk` | 666 | 0.6903 ± 0.0552 |
|
||||
| `DoS Slowhttptest` | 666 | 0.6422 ± 0.0630 |
|
||||
| `DoS Slowloris` | 666 | 0.6720 ± 0.0283 |
|
||||
| `FTP-Patator` | 666 | 0.6226 ± 0.0354 |
|
||||
| `Heartbleed` | 1 | 0.6023 ± 0.1950 |
|
||||
| `Infiltration` | 7 | 0.5351 ± 0.1435 |
|
||||
| `Infiltration - Portscan` | 666 | 0.6623 ± 0.0421 |
|
||||
| `Portscan` | 666 | 0.8874 ± 0.0138 |
|
||||
| `SSH-Patator` | 666 | 0.5079 ± 0.0532 |
|
||||
| `Web Attack - Brute Force` | 73 | 0.4848 ± 0.0245 |
|
||||
| `Web Attack - SQL Injection` | 13 | 0.5458 ± 0.0214 |
|
||||
| `Web Attack - XSS` | 18 | 0.4638 ± 0.0473 |
|
||||
@@ -0,0 +1,42 @@
|
||||
Importing Scapy Library
|
||||
[run] kitsune_path_a protocol=cicids_within seed=42
|
||||
[run] dataset=cicids2017 model_dir=/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] flows.parquet rows: 2,025,564; val=10,000 attack=30,000
|
||||
[pcap] discovered 5 pcap(s)
|
||||
/home/chy/mambafortrafficmodeling/datasets/cicids2017/raw/pcap/Friday-WorkingHours.pcap
|
||||
/home/chy/mambafortrafficmodeling/datasets/cicids2017/raw/pcap/Monday-WorkingHours.pcap
|
||||
/home/chy/mambafortrafficmodeling/datasets/cicids2017/raw/pcap/Thursday-WorkingHours.pcap
|
||||
/home/chy/mambafortrafficmodeling/datasets/cicids2017/raw/pcap/Tuesday-WorkingHours.pcap
|
||||
/home/chy/mambafortrafficmodeling/datasets/cicids2017/raw/pcap/Wednesday-workingHours.pcap
|
||||
Feature-Mapper: train-mode, Anomaly-Detector: off-mode
|
||||
[stream] Friday-WorkingHours.pcap
|
||||
Parsing with tshark...
|
||||
tshark parsing complete. File saved as: /home/chy/mambafortrafficmodeling/datasets/cicids2017/raw/pcap/Friday-WorkingHours.pcap.tsv
|
||||
counting lines in file...
|
||||
There are 9997875 Packets.
|
||||
The Feature-Mapper found a mapping: 100 features to 15 autoencoders.
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: train-mode
|
||||
Feature-Mapper: execute-mode, Anomaly-Detector: execute-mode
|
||||
[200,000] elapsed 136s (1471 pkt/s)
|
||||
[400,000] elapsed 348s (1148 pkt/s)
|
||||
[600,000] elapsed 536s (1118 pkt/s)
|
||||
[800,000] elapsed 724s (1105 pkt/s)
|
||||
[1,000,000] elapsed 904s (1106 pkt/s)
|
||||
[1,200,000] elapsed 1090s (1101 pkt/s)
|
||||
[1,400,000] elapsed 1272s (1100 pkt/s)
|
||||
[1,600,000] elapsed 1477s (1083 pkt/s)
|
||||
[1,800,000] elapsed 1665s (1081 pkt/s)
|
||||
[2,000,000] elapsed 1853s (1079 pkt/s)
|
||||
[2,200,000] elapsed 2040s (1078 pkt/s)
|
||||
[2,400,000] elapsed 2243s (1070 pkt/s)
|
||||
[2,600,000] elapsed 2531s (1027 pkt/s)
|
||||
[2,800,000] elapsed 2791s (1003 pkt/s)
|
||||
[3,000,000] elapsed 3195s (939 pkt/s)
|
||||
[3,200,000] elapsed 3524s (908 pkt/s)
|
||||
[3,400,000] elapsed 3795s (896 pkt/s)
|
||||
[3,600,000] elapsed 4656s (773 pkt/s)
|
||||
[3,800,000] elapsed 6536s (581 pkt/s)
|
||||
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 257.75,
|
||||
"t_score_sec": 7.92,
|
||||
"loss_first_last": [
|
||||
332.8706359863281,
|
||||
-13.712800025939941
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.8471867399999999,
|
||||
"auprc": 0.9237535983514962
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.9876020686619719
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1152.0,
|
||||
"auroc": 0.999830295138889
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1135.0,
|
||||
"auroc": 0.9542539207048458
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1171.0,
|
||||
"auroc": 0.9713602476515799
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1166.0,
|
||||
"auroc": 0.8149728130360205
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1086.0,
|
||||
"auroc": 0.989498802946593
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1092.0,
|
||||
"auroc": 0.943083424908425
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.9430395852119027
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.9998466063348416
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1184.0,
|
||||
"auroc": 0.9494131756756756
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1539.0,
|
||||
"auroc": 0.8166851851851852
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.8171990407673861
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3361.0,
|
||||
"auroc": 0.4932842606367153
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1106.0,
|
||||
"auroc": 0.9679500904159131
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1383.0,
|
||||
"auroc": 0.941291467823572
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 857.0,
|
||||
"auroc": 0.5736365227537923
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.46130000000000004
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 256.86,
|
||||
"t_score_sec": 8.04,
|
||||
"loss_first_last": [
|
||||
326.82415771484375,
|
||||
-14.03166389465332
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9214346274999999,
|
||||
"auprc": 0.9576402134757014
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1117.0,
|
||||
"auroc": 0.9927673679498656
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1158.0,
|
||||
"auroc": 0.9984381692573403
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1136.0,
|
||||
"auroc": 0.9898691021126761
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1071.0,
|
||||
"auroc": 0.9899109243697479
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1161.0,
|
||||
"auroc": 0.8892285099052541
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1168.0,
|
||||
"auroc": 0.9940080479452055
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1097.0,
|
||||
"auroc": 0.9895704649042845
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1109.0,
|
||||
"auroc": 0.9898962128043283
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1199.0,
|
||||
"auroc": 0.9984532110091743
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1190.0,
|
||||
"auroc": 0.9897919327731092
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1571.0,
|
||||
"auroc": 0.8936322087842139
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 407.0,
|
||||
"auroc": 0.8900004914004914
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3303.0,
|
||||
"auroc": 0.7150958371177717
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1156.0,
|
||||
"auroc": 0.9902384948096885
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1334.0,
|
||||
"auroc": 0.9899628935532233
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 822.0,
|
||||
"auroc": 0.7702770072992701
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.5650499999999999
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "cicddos_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 20000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 256.9,
|
||||
"t_score_sec": 7.91,
|
||||
"loss_first_last": [
|
||||
334.4299621582031,
|
||||
-13.083284378051758
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9023843225,
|
||||
"auprc": 0.9456418093498201
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 1121.0,
|
||||
"auroc": 0.9929867082961641
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 1183.0,
|
||||
"auroc": 0.9980066356720203
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 1046.0,
|
||||
"auroc": 0.9609366156787762
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.9897468667255076
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 1105.0,
|
||||
"auroc": 0.8955511312217195
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 1120.0,
|
||||
"auroc": 0.9927024107142858
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 1124.0,
|
||||
"auroc": 0.9676278469750889
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 1133.0,
|
||||
"auroc": 0.9686398940864961
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 1157.0,
|
||||
"auroc": 0.9983242005185826
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 1197.0,
|
||||
"auroc": 0.9607649958228907
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 1588.0,
|
||||
"auroc": 0.8999495591939547
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 417.0,
|
||||
"auroc": 0.8953611510791368
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 3418.0,
|
||||
"auroc": 0.6654265652428321
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 1049.0,
|
||||
"auroc": 0.9703411820781696
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 1353.0,
|
||||
"auroc": 0.968380561714708
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 855.0,
|
||||
"auroc": 0.7365925146198831
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9865
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 261.53,
|
||||
"t_score_sec": 8.77,
|
||||
"loss_first_last": [
|
||||
290.99407958984375,
|
||||
-7.60960054397583
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9413202350000001,
|
||||
"auprc": 0.9740946858041061
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"Botnet": {
|
||||
"_n": 46.0,
|
||||
"auroc": 0.9169891304347827
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5752.0,
|
||||
"auroc": 0.9959303894297635
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 464.0,
|
||||
"auroc": 0.9977750000000001
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9358.0,
|
||||
"auroc": 0.9656305567428938
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 78.0,
|
||||
"auroc": 0.982897435897436
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 185.0,
|
||||
"auroc": 0.9524810810810811
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 236.0,
|
||||
"auroc": 0.9858584745762712
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.99995
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4295.0,
|
||||
"auroc": 0.8910052619324794
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9425.0,
|
||||
"auroc": 0.9016478938992042
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 152.0,
|
||||
"auroc": 0.9872480263157895
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.99956
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.9998
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,82 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 261.04,
|
||||
"t_score_sec": 9.13,
|
||||
"loss_first_last": [
|
||||
298.1417236328125,
|
||||
-10.29567813873291
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9306631566666665,
|
||||
"auprc": 0.9700145777071918
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"Botnet": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.9696871794871794
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5667.0,
|
||||
"auroc": 0.9937484912652197
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 483.0,
|
||||
"auroc": 0.9948989648033126
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9437.0,
|
||||
"auroc": 0.9711513192751935
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 90.0,
|
||||
"auroc": 0.98677
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 167.0,
|
||||
"auroc": 0.9482353293413174
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 214.0,
|
||||
"auroc": 0.9937542056074766
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9996
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4222.0,
|
||||
"auroc": 0.866789199431549
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9487.0,
|
||||
"auroc": 0.8741753241277538
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 183.0,
|
||||
"auroc": 0.9920743169398907
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 3.0,
|
||||
"auroc": 0.9989333333333333
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 2.0,
|
||||
"auroc": 0.97785
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.9996
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,74 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "cicids_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 256.88,
|
||||
"t_score_sec": 8.95,
|
||||
"loss_first_last": [
|
||||
290.29083251953125,
|
||||
-9.13120174407959
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9047833383333334,
|
||||
"auprc": 0.9570561698886157
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"Botnet": {
|
||||
"_n": 38.0,
|
||||
"auroc": 0.9093065789473684
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 5627.0,
|
||||
"auroc": 0.9958099609027902
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 458.0,
|
||||
"auroc": 0.9978467248908297
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 9423.0,
|
||||
"auroc": 0.9443298684070891
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 84.0,
|
||||
"auroc": 0.9855178571428571
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 158.0,
|
||||
"auroc": 0.9215443037974684
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 224.0,
|
||||
"auroc": 0.9734799107142857
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 4346.0,
|
||||
"auroc": 0.8866304647952139
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 9473.0,
|
||||
"auroc": 0.8111246806713819
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 161.0,
|
||||
"auroc": 0.9849701863354038
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.9988
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9908000000000001
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,158 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "ciciot_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/runs/unified_cfm_ciciot2023_shafir5_2026_04_29",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 30000,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 38.87,
|
||||
"t_score_sec": 5.93,
|
||||
"loss_first_last": [
|
||||
70.3503646850586,
|
||||
-5.329672813415527
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.8996051083333333,
|
||||
"auprc": 0.9471428555703386
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"backdoor_malware": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.7523222222222222
|
||||
},
|
||||
"browserhijacking": {
|
||||
"_n": 10.0,
|
||||
"auroc": 0.78985
|
||||
},
|
||||
"commandinjection": {
|
||||
"_n": 31.0,
|
||||
"auroc": 0.6909870967741936
|
||||
},
|
||||
"ddos-ack_fragmentation": {
|
||||
"_n": 927.0,
|
||||
"auroc": 0.9660217907227617
|
||||
},
|
||||
"ddos-http_flood": {
|
||||
"_n": 2790.0,
|
||||
"auroc": 0.9405193010752688
|
||||
},
|
||||
"ddos-icmp_flood": {
|
||||
"_n": 66.0,
|
||||
"auroc": 0.5904886363636364
|
||||
},
|
||||
"ddos-icmp_fragmentation": {
|
||||
"_n": 100.0,
|
||||
"auroc": 0.554647
|
||||
},
|
||||
"ddos-pshack_flood": {
|
||||
"_n": 3118.0,
|
||||
"auroc": 0.9525152822322002
|
||||
},
|
||||
"ddos-rstfinflood": {
|
||||
"_n": 9.0,
|
||||
"auroc": 0.7346722222222222
|
||||
},
|
||||
"ddos-slowloris": {
|
||||
"_n": 960.0,
|
||||
"auroc": 0.8991201041666667
|
||||
},
|
||||
"ddos-syn_flood": {
|
||||
"_n": 3579.0,
|
||||
"auroc": 0.9506031014249791
|
||||
},
|
||||
"ddos-synonymousip_flood": {
|
||||
"_n": 499.0,
|
||||
"auroc": 0.9525009018036072
|
||||
},
|
||||
"ddos-tcp_flood": {
|
||||
"_n": 3789.0,
|
||||
"auroc": 0.9518028371602004
|
||||
},
|
||||
"ddos-udp_flood": {
|
||||
"_n": 22.0,
|
||||
"auroc": 0.7545045454545456
|
||||
},
|
||||
"ddos-udp_fragmentation": {
|
||||
"_n": 64.0,
|
||||
"auroc": 0.67508515625
|
||||
},
|
||||
"dictionarybruteforce": {
|
||||
"_n": 53.0,
|
||||
"auroc": 0.7014367924528302
|
||||
},
|
||||
"dns_spoofing": {
|
||||
"_n": 498.0,
|
||||
"auroc": 0.5668033132530121
|
||||
},
|
||||
"dos-http_flood": {
|
||||
"_n": 2145.0,
|
||||
"auroc": 0.9473534265734266
|
||||
},
|
||||
"dos-syn_flood": {
|
||||
"_n": 2656.0,
|
||||
"auroc": 0.9502796686746987
|
||||
},
|
||||
"dos-tcp_flood": {
|
||||
"_n": 3097.0,
|
||||
"auroc": 0.9512932999677108
|
||||
},
|
||||
"dos-udp_flood": {
|
||||
"_n": 539.0,
|
||||
"auroc": 0.8730077922077922
|
||||
},
|
||||
"mirai-greeth_flood": {
|
||||
"_n": 39.0,
|
||||
"auroc": 0.5726166666666667
|
||||
},
|
||||
"mirai-greip_flood": {
|
||||
"_n": 71.0,
|
||||
"auroc": 0.5358394366197183
|
||||
},
|
||||
"mirai-udpplain": {
|
||||
"_n": 33.0,
|
||||
"auroc": 0.6890560606060606
|
||||
},
|
||||
"mitm-arpspoofing": {
|
||||
"_n": 356.0,
|
||||
"auroc": 0.5290363764044945
|
||||
},
|
||||
"recon-hostdiscovery": {
|
||||
"_n": 492.0,
|
||||
"auroc": 0.7650676829268293
|
||||
},
|
||||
"recon-osscan": {
|
||||
"_n": 1063.0,
|
||||
"auroc": 0.8342711194731891
|
||||
},
|
||||
"recon-pingsweep": {
|
||||
"_n": 12.0,
|
||||
"auroc": 0.6900499999999999
|
||||
},
|
||||
"recon-portscan": {
|
||||
"_n": 1104.0,
|
||||
"auroc": 0.8466061141304349
|
||||
},
|
||||
"sqlinjection": {
|
||||
"_n": 35.0,
|
||||
"auroc": 0.5111757142857143
|
||||
},
|
||||
"uploading_attack": {
|
||||
"_n": 5.0,
|
||||
"auroc": 0.8538100000000001
|
||||
},
|
||||
"vulnerabilityscan": {
|
||||
"_n": 1802.0,
|
||||
"auroc": 0.6355951997780244
|
||||
},
|
||||
"xss": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.5890722222222222
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 260.83,
|
||||
"t_score_sec": 7.5,
|
||||
"loss_first_last": [
|
||||
286.2181091308594,
|
||||
-10.123868942260742
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.908151376193378,
|
||||
"auprc": 0.8476435665479667
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9671769557823129
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9716231292517006
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9576707482993199
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9573639455782312
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8724947278911563
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9666767006802721
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9566246598639456
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9560392857142859
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.971175850340136
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9572605442176871
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8748387755102041
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8726170068027211
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7650517857142857
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.958688775510204
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9567945578231292
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8038752551020407
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.5919326484018266
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 257.19,
|
||||
"t_score_sec": 7.23,
|
||||
"loss_first_last": [
|
||||
291.7252197265625,
|
||||
-7.248049736022949
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9279558754824294,
|
||||
"auprc": 0.8643353993931312
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9608286564625851
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.970794387755102
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9666801020408163
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9601875
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9213872448979591
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9686059523809524
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9650447278911565
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9653765306122448
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9708071428571429
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9664437074829932
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9205989795918368
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.919108163265306
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7885131802721088
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9675095238095238
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9647362244897959
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.88186981292517
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.6444300228310502
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "forward_cross",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 9846,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 257.16,
|
||||
"t_score_sec": 7.26,
|
||||
"loss_first_last": [
|
||||
286.2483825683594,
|
||||
-10.637186050415039
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9268555200081251,
|
||||
"auprc": 0.9212919110720112
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"DrDoS_DNS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9894368197278913
|
||||
},
|
||||
"DrDoS_LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9994419217687076
|
||||
},
|
||||
"DrDoS_MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9638414115646259
|
||||
},
|
||||
"DrDoS_NTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9682577380952381
|
||||
},
|
||||
"DrDoS_NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9415774659863946
|
||||
},
|
||||
"DrDoS_SNMP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9966145408163265
|
||||
},
|
||||
"DrDoS_SSDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9596625
|
||||
},
|
||||
"DrDoS_UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9592947278911564
|
||||
},
|
||||
"LDAP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9985690476190476
|
||||
},
|
||||
"MSSQL": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9632055272108845
|
||||
},
|
||||
"NetBIOS": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9394392857142857
|
||||
},
|
||||
"Portmap": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9410942176870749
|
||||
},
|
||||
"Syn": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.7449137755102041
|
||||
},
|
||||
"TFTP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9752333333333333
|
||||
},
|
||||
"UDP": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.9585020408163266
|
||||
},
|
||||
"UDPLag": {
|
||||
"_n": 588.0,
|
||||
"auroc": 0.8215611394557822
|
||||
},
|
||||
"WebDDoS": {
|
||||
"_n": 438.0,
|
||||
"auroc": 0.5362554794520548
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 261.86,
|
||||
"t_score_sec": 6.74,
|
||||
"loss_first_last": [
|
||||
266.9950256347656,
|
||||
-14.434609413146973
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.947987538109756,
|
||||
"auprc": 0.7240378342714916
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.947987538109756
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 43,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 264.57,
|
||||
"t_score_sec": 6.81,
|
||||
"loss_first_last": [
|
||||
266.81707763671875,
|
||||
-14.625688552856445
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9448752286585366,
|
||||
"auprc": 0.7116331900303116
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.9448752286585366
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "iscxtor_within",
|
||||
"seed": 44,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 1312,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 259.29,
|
||||
"t_score_sec": 6.67,
|
||||
"loss_first_last": [
|
||||
268.0535888671875,
|
||||
-16.31130027770996
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.9338064786585365,
|
||||
"auprc": 0.7069392431440469
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"tor": {
|
||||
"_n": 1312.0,
|
||||
"auroc": 0.9338064786585365
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
145
artifacts/baselines/shafir_nf_2026_04_29/master.log
Normal file
145
artifacts/baselines/shafir_nf_2026_04_29/master.log
Normal file
@@ -0,0 +1,145 @@
|
||||
[skip] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed42.json exists
|
||||
=== protocol=iscxtor_within seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=iscxtor_within seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=1,312 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed43.npz
|
||||
[result] AUROC=0.9449 AUPRC=0.7116 train=264.6s score=6.8s
|
||||
[done] elapsed=280s artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed43.json
|
||||
=== protocol=iscxtor_within seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=iscxtor_within seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=1,312 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed44.npz
|
||||
[result] AUROC=0.9338 AUPRC=0.7069 train=259.3s score=6.7s
|
||||
[done] elapsed=275s artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed44.json
|
||||
=== protocol=cicids_within seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicids_within seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=30,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed42.npz
|
||||
[result] AUROC=0.9413 AUPRC=0.9741 train=261.5s score=8.8s
|
||||
[done] elapsed=377s artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed42.json
|
||||
=== protocol=cicids_within seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicids_within seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=30,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed43.npz
|
||||
[result] AUROC=0.9307 AUPRC=0.9700 train=261.0s score=9.1s
|
||||
[done] elapsed=378s artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed43.json
|
||||
=== protocol=cicids_within seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicids_within seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=30,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed44.npz
|
||||
[result] AUROC=0.9048 AUPRC=0.9571 train=256.9s score=9.0s
|
||||
[done] elapsed=373s artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed44.json
|
||||
=== protocol=cicddos_within seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicddos_within seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train=10,000 val=10,000 attack=20,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed42.npz
|
||||
[result] AUROC=0.8472 AUPRC=0.9238 train=257.8s score=7.9s
|
||||
[done] elapsed=279s artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed42.json
|
||||
=== protocol=cicddos_within seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicddos_within seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train=10,000 val=10,000 attack=20,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed43.npz
|
||||
[result] AUROC=0.9214 AUPRC=0.9576 train=256.9s score=8.0s
|
||||
[done] elapsed=279s artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed43.json
|
||||
=== protocol=cicddos_within seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicddos_within seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train=10,000 val=10,000 attack=20,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed44.npz
|
||||
[result] AUROC=0.9024 AUPRC=0.9456 train=256.9s score=7.9s
|
||||
[done] elapsed=278s artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed44.json
|
||||
=== protocol=forward_cross seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=forward_cross seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=9,846 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed42.npz
|
||||
[result] AUROC=0.9082 AUPRC=0.8476 train=260.8s score=7.5s
|
||||
[done] elapsed=279s artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed42.json
|
||||
=== protocol=forward_cross seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=forward_cross seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=9,846 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed43.npz
|
||||
[result] AUROC=0.9280 AUPRC=0.8643 train=257.2s score=7.2s
|
||||
[done] elapsed=275s artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed43.json
|
||||
=== protocol=forward_cross seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=forward_cross seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=9,846 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed44.npz
|
||||
[result] AUROC=0.9269 AUPRC=0.9213 train=257.2s score=7.3s
|
||||
[done] elapsed=275s artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed44.json
|
||||
=== protocol=reverse_cross seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=reverse_cross seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=6,772 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed42.npz
|
||||
[result] AUROC=0.7206 AUPRC=0.6950 train=257.1s score=7.4s
|
||||
[done] elapsed=270s artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed42.json
|
||||
=== protocol=reverse_cross seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=reverse_cross seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=6,772 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed43.npz
|
||||
[result] AUROC=0.7262 AUPRC=0.7119 train=257.5s score=7.4s
|
||||
[done] elapsed=271s artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed43.json
|
||||
=== protocol=reverse_cross seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=reverse_cross seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=6,772 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed44.npz
|
||||
[result] AUROC=0.7272 AUPRC=0.6895 train=257.7s score=7.2s
|
||||
[done] elapsed=271s artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed44.json
|
||||
146
artifacts/baselines/shafir_nf_2026_04_29/orchestrator.log
Normal file
146
artifacts/baselines/shafir_nf_2026_04_29/orchestrator.log
Normal file
@@ -0,0 +1,146 @@
|
||||
[skip] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed42.json exists
|
||||
=== protocol=iscxtor_within seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=iscxtor_within seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed43/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=1,312 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed43.npz
|
||||
[result] AUROC=0.9449 AUPRC=0.7116 train=264.6s score=6.8s
|
||||
[done] elapsed=280s artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed43.json
|
||||
=== protocol=iscxtor_within seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=iscxtor_within seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/iscxtor2016_lambda0p3_seed44/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=103,079 keep len>=2: 66,189
|
||||
[data] benign=64,877 attack=1,312 -> train=51,901 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=1,312 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed44.npz
|
||||
[result] AUROC=0.9338 AUPRC=0.7069 train=259.3s score=6.7s
|
||||
[done] elapsed=275s artifacts/baselines/shafir_nf_2026_04_29/iscxtor_within_seed44.json
|
||||
=== protocol=cicids_within seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicids_within seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=30,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed42.npz
|
||||
[result] AUROC=0.9413 AUPRC=0.9741 train=261.5s score=8.8s
|
||||
[done] elapsed=377s artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed42.json
|
||||
=== protocol=cicids_within seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicids_within seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=30,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed43.npz
|
||||
[result] AUROC=0.9307 AUPRC=0.9700 train=261.0s score=9.1s
|
||||
[done] elapsed=378s artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed43.json
|
||||
=== protocol=cicids_within seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicids_within seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=2,025,564 keep len>=2: 2,017,180
|
||||
[data] benign=1,513,450 attack=30,000 -> train=1,210,760 val=10,000
|
||||
[data] train=10,000 val=10,000 attack=30,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed44.npz
|
||||
[result] AUROC=0.9048 AUPRC=0.9571 train=256.9s score=9.0s
|
||||
[done] elapsed=373s artifacts/baselines/shafir_nf_2026_04_29/cicids_within_seed44.json
|
||||
=== protocol=cicddos_within seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicddos_within seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train=10,000 val=10,000 attack=20,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed42.npz
|
||||
[result] AUROC=0.8472 AUPRC=0.9238 train=257.8s score=7.9s
|
||||
[done] elapsed=279s artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed42.json
|
||||
=== protocol=cicddos_within seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicddos_within seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train=10,000 val=10,000 attack=20,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed43.npz
|
||||
[result] AUROC=0.9214 AUPRC=0.9576 train=256.9s score=8.0s
|
||||
[done] elapsed=279s artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed43.json
|
||||
=== protocol=cicddos_within seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=cicddos_within seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt (source ckpt)
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,993,376 keep len>=2: 8,986,875
|
||||
[data] benign=93,207 attack=20,000 -> train=74,565 val=18,642
|
||||
[data] train=10,000 val=10,000 attack=20,000 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed44.npz
|
||||
[result] AUROC=0.9024 AUPRC=0.9456 train=256.9s score=7.9s
|
||||
[done] elapsed=278s artifacts/baselines/shafir_nf_2026_04_29/cicddos_within_seed44.json
|
||||
=== protocol=forward_cross seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=forward_cross seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed42/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=9,846 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed42.npz
|
||||
[result] AUROC=0.9082 AUPRC=0.8476 train=260.8s score=7.5s
|
||||
[done] elapsed=279s artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed42.json
|
||||
=== protocol=forward_cross seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=forward_cross seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed43/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=9,846 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed43.npz
|
||||
[result] AUROC=0.9280 AUPRC=0.8643 train=257.2s score=7.2s
|
||||
[done] elapsed=275s artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed43.json
|
||||
=== protocol=forward_cross seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=forward_cross seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_sigma06_multiseed_2026_04_25/cicids2017_lambda0p3_sigma0p6_seed44/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=9,846 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed44.npz
|
||||
[result] AUROC=0.9269 AUPRC=0.9213 train=257.2s score=7.3s
|
||||
[done] elapsed=275s artifacts/baselines/shafir_nf_2026_04_29/forward_cross_seed44.json
|
||||
=== protocol=reverse_cross seed=42 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=reverse_cross seed=42
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=6,772 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed42.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed42.npz
|
||||
[result] AUROC=0.7206 AUPRC=0.6950 train=257.1s score=7.4s
|
||||
[done] elapsed=270s artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed42.json
|
||||
=== protocol=reverse_cross seed=43 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=reverse_cross seed=43
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed43/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=6,772 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed43.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed43.npz
|
||||
[result] AUROC=0.7262 AUPRC=0.7119 train=257.5s score=7.4s
|
||||
[done] elapsed=271s artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed43.json
|
||||
=== protocol=reverse_cross seed=44 epochs=100 opt=sgd lr=0.001 ===
|
||||
[run] shafir_nf protocol=reverse_cross seed=44
|
||||
[run] using normalization stats from /home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed44/model.pt (source ckpt)
|
||||
[data] train=10,000 val=10,000 attack=6,772 D=20
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed44.json
|
||||
[saved] artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed44.npz
|
||||
[result] AUROC=0.7272 AUPRC=0.6895 train=257.7s score=7.2s
|
||||
[done] elapsed=271s artifacts/baselines/shafir_nf_2026_04_29/reverse_cross_seed44.json
|
||||
ALL DONE
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"method": "shafir_nf",
|
||||
"protocol": "reverse_cross",
|
||||
"seed": 42,
|
||||
"model_dir": "/home/chy/mambafortrafficmodeling/artifacts/phase25_multiseed_2026_04_25/cicddos2019_lambda0p3_seed42",
|
||||
"n_train": 10000,
|
||||
"n_val": 10000,
|
||||
"n_atk": 6772,
|
||||
"epochs": 100,
|
||||
"lr": 0.001,
|
||||
"optimizer": "sgd",
|
||||
"t_train_sec": 257.14,
|
||||
"t_score_sec": 7.35,
|
||||
"loss_first_last": [
|
||||
327.9974060058594,
|
||||
-14.816756248474121
|
||||
],
|
||||
"overall": {
|
||||
"neg_log_prob": {
|
||||
"auroc": 0.7206432811577081,
|
||||
"auprc": 0.6950180575372344
|
||||
}
|
||||
},
|
||||
"per_class": {
|
||||
"Botnet": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.564331081081081
|
||||
},
|
||||
"DDoS": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9866227477477477
|
||||
},
|
||||
"DoS GoldenEye": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9851037537537537
|
||||
},
|
||||
"DoS Hulk": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8369624624624625
|
||||
},
|
||||
"DoS Slowhttptest": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.9183935435435435
|
||||
},
|
||||
"DoS Slowloris": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8349857357357358
|
||||
},
|
||||
"FTP-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.8203914414414415
|
||||
},
|
||||
"Heartbleed": {
|
||||
"_n": 1.0,
|
||||
"auroc": 0.9994
|
||||
},
|
||||
"Infiltration": {
|
||||
"_n": 7.0,
|
||||
"auroc": 0.9531285714285714
|
||||
},
|
||||
"Infiltration - Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.19368243243243244
|
||||
},
|
||||
"Portscan": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.29466171171171174
|
||||
},
|
||||
"SSH-Patator": {
|
||||
"_n": 666.0,
|
||||
"auroc": 0.7276059309309308
|
||||
},
|
||||
"Web Attack - Brute Force": {
|
||||
"_n": 73.0,
|
||||
"auroc": 0.9990369863013698
|
||||
},
|
||||
"Web Attack - SQL Injection": {
|
||||
"_n": 13.0,
|
||||
"auroc": 0.8631230769230769
|
||||
},
|
||||
"Web Attack - XSS": {
|
||||
"_n": 18.0,
|
||||
"auroc": 0.9994055555555555
|
||||
}
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user