Initial commit: code, paper, small artifacts
This commit is contained in:
43
artifacts/route_comparison/CROSS_MATRIX.md
Normal file
43
artifacts/route_comparison/CROSS_MATRIX.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Full 4×4 Cross Matrix — A+C combo + Mahalanobis-OAS
|
||||
|
||||
3-seed mean ± std. Diagonal = within-dataset; off-diagonal = cross.
|
||||
Aggregator: Mahalanobis-OAS over 10-d A+C combo score vector,
|
||||
fit on **target-dataset benign val only** (no attack labels).
|
||||
|
||||
## Mahalanobis-OAS AUROC (4×4)
|
||||
|
||||
| Source ↓ \ Target → | iscxtor2016 | cicids2017 | cicddos2019 | ciciot2023 |
|
||||
|---|---|---|---|---|
|
||||
| iscxtor2016 | _0.9908±0.0012_ | 0.8661±0.0158 | 0.8102±0.0395 | 0.8023±0.0036 |
|
||||
| cicids2017 | 0.7786±0.0237 | _0.9845±0.0030_ | 0.9594±0.0046 | 0.8235±0.0037 |
|
||||
| cicddos2019 | 0.6908±0.0171 | 0.9300±0.0122 | _0.9913±0.0009_ | 0.8146±0.0056 |
|
||||
| ciciot2023 | 0.7504±0.0431 | 0.8983±0.0098 | 0.8944±0.0068 | _0.9594±0.0028_ |
|
||||
|
||||
(Italic diagonal = within-dataset reference)
|
||||
|
||||
## `terminal_norm` AUROC (4×4) — for comparison (selection-bias-free single fixed score)
|
||||
|
||||
| Source ↓ \ Target → | iscxtor2016 | cicids2017 | cicddos2019 | ciciot2023 |
|
||||
|---|---|---|---|---|
|
||||
| iscxtor2016 | _0.9954±0.0007_ | 0.6994±0.0190 | 0.7757±0.0064 | 0.6141±0.0096 |
|
||||
| cicids2017 | 0.4900±0.0144 | _0.9884±0.0012_ | 0.8649±0.0036 | 0.6403±0.0044 |
|
||||
| cicddos2019 | 0.6612±0.0112 | 0.5190±0.0227 | _0.9970±0.0005_ | 0.5671±0.0126 |
|
||||
| ciciot2023 | 0.4672±0.0100 | 0.7854±0.0033 | 0.8361±0.0118 | _0.9604±0.0022_ |
|
||||
|
||||
## Δ Mahalanobis − terminal_norm (where positive, Mahalanobis is better)
|
||||
|
||||
| Source ↓ \ Target → | iscxtor2016 | cicids2017 | cicddos2019 | ciciot2023 |
|
||||
|---|---|---|---|---|
|
||||
| iscxtor2016 | -0.0046 | **+0.1667** | **+0.0345** | **+0.1882** |
|
||||
| cicids2017 | **+0.2886** | -0.0039 | **+0.0945** | **+0.1831** |
|
||||
| cicddos2019 | **+0.0296** | **+0.4110** | _-0.0057_ | **+0.2475** |
|
||||
| ciciot2023 | **+0.2832** | **+0.1129** | **+0.0583** | -0.0010 |
|
||||
|
||||
## Per-source averaged cross-AUROC (Mahalanobis, off-diagonal mean)
|
||||
|
||||
| Source | mean off-diag Mahalanobis | mean off-diag terminal_norm |
|
||||
|---|---|---|
|
||||
| iscxtor2016 | 0.8262 | 0.6964 |
|
||||
| cicids2017 | 0.8538 | 0.6651 |
|
||||
| cicddos2019 | 0.8118 | 0.5824 |
|
||||
| ciciot2023 | 0.8477 | 0.6962 |
|
||||
112
artifacts/route_comparison/CROSS_RESULTS.md
Normal file
112
artifacts/route_comparison/CROSS_RESULTS.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# Cross-Dataset Eval — CICIoT2023 → {CICIDS2017, CICDDoS2019}
|
||||
|
||||
All models trained on CICIoT2023 (10K benign), evaluated on each target's
|
||||
10K benign + 10K stratified attack. Source-domain norm stats applied.
|
||||
3 seeds each. AUROC mean ± std.
|
||||
|
||||
## Primary score: `terminal_norm`
|
||||
|
||||
| Route | within-CICIoT2023 (ref) | → CICIDS2017 | → CICDDoS2019 |
|
||||
|---|---|---|---|
|
||||
| baseline | 0.9612 ± 0.0017 | 0.7700 ± 0.0133 | 0.7473 ± 0.0223 |
|
||||
| A: causal | 0.9636 ± 0.0006 | 0.7933 ± 0.0273 | 0.7754 ± 0.0214 |
|
||||
| B: spectral | 0.9619 ± 0.0013 | 0.7576 ± 0.0173 | 0.7339 ± 0.0454 |
|
||||
| C: mixed | 0.9625 ± 0.0028 | 0.7728 ± 0.0108 | 0.8371 ± 0.0117 |
|
||||
| A+C combo | 0.9587 ± 0.0017 | 0.7854 ± 0.0033 | 0.8361 ± 0.0118 |
|
||||
|
||||
## Each route's best score per target
|
||||
|
||||
### → cicids2017
|
||||
|
||||
| Route | Best score | AUROC | Δ (vs same-route's terminal_norm) |
|
||||
|---|---|---|---|
|
||||
| baseline | `terminal_flow` | 0.8814 ± 0.0296 | +0.1114 |
|
||||
| A: causal | `terminal_flow` | 0.8876 ± 0.0224 | +0.0943 |
|
||||
| B: spectral | `terminal_norm` | 0.7576 ± 0.0173 | +0.0000 |
|
||||
| C: mixed | `disc_nll_total` | 0.9121 ± 0.0046 | +0.1393 |
|
||||
| A+C combo | `disc_nll_total` | 0.9191 ± 0.0081 | +0.1337 |
|
||||
|
||||
### → cicddos2019
|
||||
|
||||
| Route | Best score | AUROC | Δ (vs same-route's terminal_norm) |
|
||||
|---|---|---|---|
|
||||
| baseline | `velocity_total` | 0.8837 ± 0.0291 | +0.1364 |
|
||||
| A: causal | `velocity_total` | 0.9027 ± 0.0039 | +0.1273 |
|
||||
| B: spectral | `curvature_packet` | 0.8802 ± 0.0385 | +0.1462 |
|
||||
| C: mixed | `disc_nll_ch7` | 0.8407 ± 0.0193 | +0.0037 |
|
||||
| A+C combo | `disc_nll_ch7` | 0.8476 ± 0.0066 | +0.0115 |
|
||||
|
||||
## All key scores → cicids2017
|
||||
|
||||
| Score | baseline | A: causal | B: spectral | C: mixed | A+C combo |
|
||||
|---|---|---|---|---|---|
|
||||
| `terminal_norm` | 0.7700 ± 0.0133 | 0.7933 ± 0.0273 | 0.7576 ± 0.0173 | 0.7728 ± 0.0108 | 0.7854 ± 0.0033 |
|
||||
| `terminal_flow` | 0.8814 ± 0.0296 | 0.8876 ± 0.0224 | 0.7395 ± 0.0024 | 0.8615 ± 0.0265 | 0.8745 ± 0.0148 |
|
||||
| `terminal_packet` | 0.7791 ± 0.0195 | 0.8254 ± 0.0209 | 0.7432 ± 0.0194 | 0.7870 ± 0.0087 | 0.8101 ± 0.0122 |
|
||||
| `flow_consistency` | 0.6391 ± 0.0187 | 0.6219 ± 0.0106 | 0.7048 ± 0.0122 | — | — |
|
||||
| `packet_consistency` | 0.7666 ± 0.0156 | 0.7658 ± 0.0064 | 0.7477 ± 0.0196 | — | — |
|
||||
| `consistency_total` | 0.6574 ± 0.0224 | 0.6292 ± 0.0060 | 0.7250 ± 0.0064 | — | — |
|
||||
| `causal_surprisal_packet_median` | 0.4515 ± 0.0807 | 0.5580 ± 0.0438 | 0.5870 ± 0.0280 | — | — |
|
||||
| `causal_surprisal_total` | 0.5523 ± 0.0104 | 0.5878 ± 0.0107 | 0.3691 ± 0.0296 | — | — |
|
||||
| `direction_drift_packet_median` | 0.3362 ± 0.1083 | 0.3378 ± 0.0157 | 0.6034 ± 0.0683 | — | — |
|
||||
| `pna_packet_median` | 0.4152 ± 0.0421 | 0.5953 ± 0.0226 | 0.3447 ± 0.0751 | — | — |
|
||||
| `kappa2_speed2norm_packet_median` | 0.4152 ± 0.0421 | 0.5953 ± 0.0226 | 0.3447 ± 0.0751 | — | — |
|
||||
| `curvature_packet` | 0.6254 ± 0.0707 | 0.8467 ± 0.0123 | 0.6734 ± 0.0512 | — | — |
|
||||
| `disc_nll_total` | — | — | — | 0.9121 ± 0.0046 | 0.9191 ± 0.0081 |
|
||||
| `disc_nll_ch3` | — | — | — | 0.8825 ± 0.0043 | 0.8752 ± 0.0075 |
|
||||
| `disc_nll_ch7` | — | — | — | 0.6554 ± 0.0404 | 0.6615 ± 0.0479 |
|
||||
|
||||
## All key scores → cicddos2019
|
||||
|
||||
| Score | baseline | A: causal | B: spectral | C: mixed | A+C combo |
|
||||
|---|---|---|---|---|---|
|
||||
| `terminal_norm` | 0.7473 ± 0.0223 | 0.7754 ± 0.0214 | 0.7339 ± 0.0454 | 0.8371 ± 0.0117 | 0.8361 ± 0.0118 |
|
||||
| `terminal_flow` | 0.7102 ± 0.0245 | 0.7123 ± 0.0153 | 0.6108 ± 0.0557 | 0.8330 ± 0.0027 | 0.8367 ± 0.0052 |
|
||||
| `terminal_packet` | 0.6789 ± 0.0538 | 0.7201 ± 0.0496 | 0.5941 ± 0.0320 | 0.6644 ± 0.0928 | 0.6871 ± 0.0731 |
|
||||
| `flow_consistency` | 0.7632 ± 0.0108 | 0.7836 ± 0.0133 | 0.5985 ± 0.0498 | — | — |
|
||||
| `packet_consistency` | 0.5124 ± 0.0569 | 0.6214 ± 0.1196 | 0.7102 ± 0.1230 | — | — |
|
||||
| `consistency_total` | 0.7601 ± 0.0094 | 0.7865 ± 0.0154 | 0.6378 ± 0.0625 | — | — |
|
||||
| `causal_surprisal_packet_median` | 0.1692 ± 0.0532 | 0.1773 ± 0.0258 | 0.2808 ± 0.0978 | — | — |
|
||||
| `causal_surprisal_total` | 0.3469 ± 0.0508 | 0.3035 ± 0.0216 | 0.1491 ± 0.0579 | — | — |
|
||||
| `direction_drift_packet_median` | 0.3153 ± 0.0540 | 0.1962 ± 0.0538 | 0.5364 ± 0.1001 | — | — |
|
||||
| `pna_packet_median` | 0.3726 ± 0.0784 | 0.4766 ± 0.1835 | 0.7535 ± 0.1415 | — | — |
|
||||
| `kappa2_speed2norm_packet_median` | 0.3726 ± 0.0784 | 0.4766 ± 0.1835 | 0.7535 ± 0.1415 | — | — |
|
||||
| `curvature_packet` | 0.6218 ± 0.0869 | 0.6687 ± 0.1187 | 0.8802 ± 0.0385 | — | — |
|
||||
| `disc_nll_total` | — | — | — | 0.3780 ± 0.0043 | 0.4222 ± 0.0506 |
|
||||
| `disc_nll_ch3` | — | — | — | 0.3297 ± 0.1172 | 0.2332 ± 0.0488 |
|
||||
| `disc_nll_ch7` | — | — | — | 0.8407 ± 0.0193 | 0.8476 ± 0.0066 |
|
||||
|
||||
## Route C ensemble (terminal_norm + disc_nll) → cicids2017
|
||||
|
||||
| α | seed42 | seed43 | seed44 | mean ± std |
|
||||
|---|---|---|---|---|
|
||||
| 0.00 | 0.9180 | 0.9069 | 0.9114 | **0.9121 ± 0.0046** |
|
||||
| 0.50 | 0.8780 | 0.8473 | 0.8719 | **0.8657 ± 0.0132** |
|
||||
| 0.70 | 0.8426 | 0.8092 | 0.8397 | **0.8305 ± 0.0151** |
|
||||
| 0.80 | 0.8182 | 0.7922 | 0.8133 | **0.8079 ± 0.0113** |
|
||||
| 0.90 | 0.8027 | 0.7741 | 0.7972 | **0.7913 ± 0.0124** |
|
||||
| 1.00 | 0.7843 | 0.7583 | 0.7757 | **0.7728 ± 0.0108** |
|
||||
|
||||
## Route C ensemble (terminal_norm + disc_nll) → cicddos2019
|
||||
|
||||
| α | seed42 | seed43 | seed44 | mean ± std |
|
||||
|---|---|---|---|---|
|
||||
| 0.00 | 0.3802 | 0.3719 | 0.3817 | **0.3780 ± 0.0043** |
|
||||
| 0.50 | 0.7318 | 0.7479 | 0.6868 | **0.7222 ± 0.0258** |
|
||||
| 0.70 | 0.7961 | 0.8147 | 0.7666 | **0.7925 ± 0.0198** |
|
||||
| 0.80 | 0.8096 | 0.8299 | 0.7887 | **0.8094 ± 0.0168** |
|
||||
| 0.90 | 0.8214 | 0.8466 | 0.8137 | **0.8272 ± 0.0141** |
|
||||
| 1.00 | 0.8318 | 0.8533 | 0.8260 | **0.8371 ± 0.0117** |
|
||||
|
||||
## Run inventory
|
||||
|
||||
- baseline → cicids2017: seeds = [42, 43, 44]
|
||||
- baseline → cicddos2019: seeds = [42, 43, 44]
|
||||
- A: causal → cicids2017: seeds = [42, 43, 44]
|
||||
- A: causal → cicddos2019: seeds = [42, 43, 44]
|
||||
- B: spectral → cicids2017: seeds = [42, 43, 44]
|
||||
- B: spectral → cicddos2019: seeds = [42, 43, 44]
|
||||
- C: mixed → cicids2017: seeds = [42, 43, 44]
|
||||
- C: mixed → cicddos2019: seeds = [42, 43, 44]
|
||||
- A+C combo → cicids2017: seeds = [42, 43, 44]
|
||||
- A+C combo → cicddos2019: seeds = [42, 43, 44]
|
||||
83
artifacts/route_comparison/PROTOCOL.md
Normal file
83
artifacts/route_comparison/PROTOCOL.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# Route Comparison Protocol
|
||||
|
||||
Goal: compare three FM-mechanism × traffic-property route variants on a unified
|
||||
training base. All routes start from the current `Unified_CFM` SOTA recipe and
|
||||
change one mechanism axis.
|
||||
|
||||
## Unified base (LOCKED)
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Dataset | CICIoT2023 |
|
||||
| Source store | `datasets/ciciot2023/processed/full_store/` |
|
||||
| Flows | `datasets/ciciot2023/processed/full_store/flows.parquet` |
|
||||
| Flow features | `datasets/ciciot2023/processed/flow_features.parquet` (canonical 20-d) |
|
||||
| Train: benign | 10,000 (Shafir within-dataset protocol) |
|
||||
| Sequence length | T = 64 |
|
||||
| Packet preprocess | `mixed_dequant` (Routes A/B); raw binaries (Route C) |
|
||||
| Benign split | 80/20, `split_seed=42` |
|
||||
| Val cap | 10,000 |
|
||||
| Attack cap | 20,000 (stratified) |
|
||||
| Multi-seed | {42, 43, 44} |
|
||||
|
||||
## Architecture base (LOCKED)
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| `d_model` | 128 |
|
||||
| `n_layers` | 4 |
|
||||
| `n_heads` | 4 |
|
||||
| `mlp_ratio` | 4.0 |
|
||||
| `time_dim` | 64 |
|
||||
| `sigma` | 0.1 |
|
||||
| `use_ot` | True |
|
||||
| `lambda_flow / lambda_packet` | 0.3 / 0.3 |
|
||||
| `packet_mask_ratio` | 0.5 |
|
||||
| Optimizer | AdamW, lr=3e-4, wd=0.01, grad_clip=1.0 |
|
||||
| Schedule | CosineAnnealingLR over total steps |
|
||||
| Epochs | 50 |
|
||||
| Batch size | 256 |
|
||||
|
||||
## Routes
|
||||
|
||||
| Route | Mechanism axis | Traffic property targeted |
|
||||
|---|---|---|
|
||||
| **Baseline** | Standard UnifiedCFM (current SOTA) | — |
|
||||
| **A: Causal** | Packet-causal attention mask | Protocol causality (TCP/HTTP handshake) |
|
||||
| **B: Spectral** | Append K=8-band DFT of (size, IAT) — 32 dims — to flow features (`flow_dim` 20→52); model architecture unchanged | Burstiness / LRD / self-similarity |
|
||||
| **C: Mixed FM** | Continuous-CFM on (size,IAT,win) + DFM on flags | Discrete-continuous mixed channels |
|
||||
|
||||
Route D (Edit Flows) is deferred until A/B/C show signal.
|
||||
|
||||
## Reporting
|
||||
|
||||
Each route × seed produces:
|
||||
|
||||
```
|
||||
artifacts/route_comparison/<route>_seed<S>/
|
||||
├── model.pt
|
||||
├── config.yaml # actual config used
|
||||
├── history.json
|
||||
├── phase1_summary.json # 34-score per-attack-class AUROC table
|
||||
└── train.log
|
||||
```
|
||||
|
||||
Final aggregate at `artifacts/route_comparison/RESULTS.md`:
|
||||
|
||||
```
|
||||
| Route | terminal_norm | route-specific score | param count | train wall |
|
||||
| baseline | 0.962 (existing) | — | 1.23M | ~2 min |
|
||||
| A | ? | causal_surprisal_packet_median | ? | ? |
|
||||
| B | ? | velocity_freq | ? | ? |
|
||||
| C | ? | nll_disc + terminal_cont | ? | ? |
|
||||
```
|
||||
|
||||
Plus per-attack-class breakdown for the top 10 attack labels by support.
|
||||
|
||||
## Baseline reference (single-seed, from existing run)
|
||||
|
||||
`artifacts/runs/unified_cfm_ciciot2023_2026_04_29/`:
|
||||
- 50 epochs, σ=0.1, λ=0.3
|
||||
- final `auroc_terminal_norm` = **0.962**
|
||||
- This is the number to compare against; we'll re-run it under multi-seed for
|
||||
fair comparison.
|
||||
93
artifacts/route_comparison/RESULTS.md
Normal file
93
artifacts/route_comparison/RESULTS.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# Route Comparison Results — CICIoT2023 (multi-seed)
|
||||
|
||||
Phase1 eval: AUROC over benign val (5k cap) vs all attacks (10k cap), 3 seeds each.
|
||||
|
||||
## Each route's best AUROC (overall)
|
||||
|
||||
| Route | Best score | AUROC | Δ vs baseline-best |
|
||||
|---|---|---|---|
|
||||
| baseline | `terminal_norm` | 0.9612 ± 0.0017 | — |
|
||||
| A: causal | `terminal_norm` | 0.9636 ± 0.0006 | +0.0024 |
|
||||
| B: spectral | `terminal_norm` | 0.9619 ± 0.0013 | +0.0007 |
|
||||
| C: mixed | `terminal_packet` | 0.9667 ± 0.0010 | +0.0056 |
|
||||
| A+C combo | `terminal_packet` | 0.9671 ± 0.0002 | +0.0059 |
|
||||
|
||||
## Primary score: `terminal_norm`
|
||||
|
||||
| Route | mean ± std | seeds |
|
||||
|---|---|---|
|
||||
| baseline | 0.9612 ± 0.0017 | [42, 43, 44] |
|
||||
| A: causal | 0.9636 ± 0.0006 | [42, 43, 44] |
|
||||
| B: spectral | 0.9619 ± 0.0013 | [42, 43, 44] |
|
||||
| C: mixed | 0.9625 ± 0.0028 | [42, 43, 44] |
|
||||
| A+C combo | 0.9604 ± 0.0022 | [42, 43, 44] |
|
||||
|
||||
## Route-specific signature scores (mean ± std, 3 seeds)
|
||||
|
||||
### Route A signature (consistency family)
|
||||
|
||||
| Score | baseline | A: causal | B: spectral | C: mixed | A+C combo |
|
||||
|---|---|---|---|---|---|
|
||||
| `flow_consistency` | 0.8862 ± 0.0301 | 0.9171 ± 0.0089 | 0.7981 ± 0.0399 | — | — |
|
||||
| `packet_consistency` | 0.8127 ± 0.0250 | 0.8526 ± 0.0128 | 0.8012 ± 0.0209 | — | — |
|
||||
| `consistency_total` | 0.9019 ± 0.0255 | 0.9310 ± 0.0089 | 0.8306 ± 0.0253 | — | — |
|
||||
| `causal_surprisal_total` | 0.5091 ± 0.0281 | 0.5669 ± 0.0301 | 0.2865 ± 0.0200 | — | — |
|
||||
| `causal_surprisal_packet_median` | 0.4075 ± 0.0767 | 0.5877 ± 0.0193 | 0.8205 ± 0.0497 | — | — |
|
||||
|
||||
### Route B signature (curvature/dynamics)
|
||||
|
||||
| Score | baseline | A: causal | B: spectral | C: mixed | A+C combo |
|
||||
|---|---|---|---|---|---|
|
||||
| `kappa2_speed2norm_packet_median` | 0.4080 ± 0.1531 | 0.4022 ± 0.1015 | 0.2354 ± 0.0562 | — | — |
|
||||
| `direction_drift_packet_median` | 0.1511 ± 0.0561 | 0.1334 ± 0.0240 | 0.4267 ± 0.0632 | — | — |
|
||||
| `pna_packet_median` | 0.4080 ± 0.1531 | 0.4022 ± 0.1015 | 0.2354 ± 0.0562 | — | — |
|
||||
| `curvature_packet` | 0.7971 ± 0.1276 | 0.8578 ± 0.0539 | 0.7965 ± 0.0978 | — | — |
|
||||
|
||||
### Route C signature (discrete NLL)
|
||||
|
||||
| Score | baseline | A: causal | B: spectral | C: mixed | A+C combo |
|
||||
|---|---|---|---|---|---|
|
||||
| `disc_nll_total` | — | — | — | 0.8853 ± 0.0073 | 0.8994 ± 0.0098 |
|
||||
| `disc_nll_ch3` | — | — | — | 0.8681 ± 0.0088 | 0.8606 ± 0.0257 |
|
||||
| `disc_nll_ch4` | — | — | — | 0.8082 ± 0.0078 | 0.8353 ± 0.0152 |
|
||||
| `disc_nll_ch5` | — | — | — | 0.7897 ± 0.0079 | 0.8474 ± 0.0208 |
|
||||
| `disc_nll_ch7` | — | — | — | 0.8818 ± 0.0227 | 0.8934 ± 0.0098 |
|
||||
|
||||
## Route C ensemble: α·terminal_norm + (1−α)·disc_nll_total (z-scored)
|
||||
|
||||
| α | seed42 | seed43 | seed44 | mean ± std |
|
||||
|---|---|---|---|---|
|
||||
| 0.00 | 0.8907 | 0.8902 | 0.8749 | **0.8853 ± 0.0073** |
|
||||
| 0.25 | 0.9479 | 0.9478 | 0.9429 | **0.9462 ± 0.0023** |
|
||||
| 0.50 | 0.9616 | 0.9605 | 0.9562 | **0.9594 ± 0.0023** |
|
||||
| 0.70 | 0.9672 | 0.9655 | 0.9610 | **0.9646 ± 0.0026** |
|
||||
| 0.80 | 0.9681 | 0.9664 | 0.9616 | **0.9654 ± 0.0028** |
|
||||
| 0.90 | 0.9674 | 0.9659 | 0.9610 | **0.9648 ± 0.0027** |
|
||||
| 1.00 | 0.9653 | 0.9634 | 0.9587 | **0.9625 ± 0.0028** |
|
||||
|
||||
(α=1.0 = terminal_norm only; α=0.0 = disc_nll only.)
|
||||
|
||||
## Per-attack-class AUROC (top 12, terminal_norm)
|
||||
|
||||
| Class | n | baseline | A: causal | B: spectral | C: mixed | A+C combo |
|
||||
|---|---|---|---|---|---|---|
|
||||
| ddos-tcp_flood | 1255 | 0.989±0.003 | 0.992±0.000 | 0.985±0.004 | 0.988±0.003 | 0.985±0.005 |
|
||||
| ddos-syn_flood | 1195 | 0.996±0.001 | 0.997±0.000 | 0.995±0.001 | 0.997±0.001 | 0.997±0.000 |
|
||||
| dos-tcp_flood | 1098 | 0.996±0.001 | 0.997±0.000 | 0.991±0.003 | 0.989±0.003 | 0.985±0.004 |
|
||||
| ddos-pshack_flood | 1030 | 0.990±0.005 | 0.994±0.001 | 0.995±0.002 | 0.985±0.003 | 0.980±0.005 |
|
||||
| ddos-http_flood | 918 | 0.990±0.001 | 0.989±0.001 | 0.985±0.001 | 0.986±0.002 | 0.984±0.002 |
|
||||
| dos-syn_flood | 844 | 0.997±0.001 | 0.998±0.001 | 0.993±0.002 | 0.993±0.002 | 0.994±0.001 |
|
||||
| dos-http_flood | 716 | 0.988±0.002 | 0.988±0.002 | 0.985±0.002 | 0.983±0.003 | 0.981±0.004 |
|
||||
| vulnerabilityscan | 568 | 0.806±0.008 | 0.803±0.008 | 0.783±0.016 | 0.806±0.001 | 0.804±0.003 |
|
||||
| recon-portscan | 370 | 0.927±0.009 | 0.927±0.008 | 0.920±0.007 | 0.935±0.013 | 0.935±0.010 |
|
||||
| recon-osscan | 368 | 0.931±0.004 | 0.932±0.004 | 0.923±0.007 | 0.941±0.003 | 0.939±0.003 |
|
||||
| ddos-ack_fragmentation | 310 | 0.987±0.003 | 0.987±0.004 | 0.989±0.002 | 0.989±0.002 | 0.988±0.003 |
|
||||
| ddos-slowloris | 304 | 0.940±0.005 | 0.952±0.010 | 0.953±0.004 | 0.939±0.008 | 0.939±0.003 |
|
||||
|
||||
## Run inventory
|
||||
|
||||
- **baseline** (`baseline_ciciot2023_seed*`): seeds = [42, 43, 44]
|
||||
- **A: causal** (`route_a_causal_ciciot2023_seed*`): seeds = [42, 43, 44]
|
||||
- **B: spectral** (`route_b_spectral_ciciot2023_seed*`): seeds = [42, 43, 44]
|
||||
- **C: mixed** (`route_c_mixed_ciciot2023_seed*`): seeds = [42, 43, 44]
|
||||
- **A+C combo** (`route_ac_combo_ciciot2023_seed*`): seeds = [42, 43, 44]
|
||||
35
artifacts/route_comparison/SCORE_ROUTER.md
Normal file
35
artifacts/route_comparison/SCORE_ROUTER.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Score-vector auto-selection: max-of-|z| / Mahalanobis vs fixed scores
|
||||
|
||||
Aggregators are fit on **benign val only** (no attack labels). All numbers
|
||||
are 3-seed mean ± std on A+C combo (Mixed_CFM + causal-packet attention).
|
||||
|
||||
Note on fairness: `auc_best_fixed` is selection-biased (picks per-dataset best
|
||||
score post-hoc on test set). `max_abs_z` and `mahalanobis` are NOT — they only
|
||||
use benign val to fit aggregator parameters.
|
||||
|
||||
## Within-dataset(A+C combo on each dataset's own benign/attack)
|
||||
|
||||
| Dataset | term_norm | best fixed | max-\|z\| (all) | mahal-OAS (all) | **mahal-OAS (term3)** | **mahal-OAS (disc7)** |
|
||||
|---|---|---|---|---|---|---|
|
||||
| iscxtor2016 | 0.9954 ± 0.0007 | 0.9955 ± 0.0005 | 0.9908 ± 0.0011 | 0.9908 ± 0.0012 | **0.9937 ± 0.0011** | **0.7705 ± 0.0528** |
|
||||
| cicids2017 | 0.9884 ± 0.0012 | 0.9932 ± 0.0013 | 0.9807 ± 0.0020 | 0.9845 ± 0.0030 | **0.9771 ± 0.0034** | **0.9840 ± 0.0047** |
|
||||
| cicddos2019 | 0.9970 ± 0.0005 | 0.9970 ± 0.0005 | 0.9883 ± 0.0012 | 0.9913 ± 0.0009 | **0.9959 ± 0.0005** | **0.7185 ± 0.0382** |
|
||||
| ciciot2023 | 0.9604 ± 0.0022 | 0.9671 ± 0.0002 | 0.9523 ± 0.0038 | 0.9594 ± 0.0028 | **0.9511 ± 0.0032** | **0.9064 ± 0.0087** |
|
||||
|
||||
## Cross-dataset(A+C combo trained on CICIoT2023 → eval on target)
|
||||
|
||||
| Target | term_norm | best fixed | max-\|z\| (all) | mahal-OAS (all) | **mahal-OAS (term3)** | **mahal-OAS (disc7)** |
|
||||
|---|---|---|---|---|---|---|
|
||||
| cicids2017 | 0.7854 ± 0.0033 | 0.9191 ± 0.0081 | 0.8750 ± 0.0137 | 0.8983 ± 0.0098 | **0.8012 ± 0.0037** | **0.8836 ± 0.0178** |
|
||||
| cicddos2019 | 0.8361 ± 0.0118 | 0.8851 ± 0.0174 | 0.6033 ± 0.0795 | 0.8944 ± 0.0068 | **0.7437 ± 0.0396** | **0.7221 ± 0.0572** |
|
||||
|
||||
## Best-fixed-score winner per setup
|
||||
|
||||
| Setup | seed42 | seed43 | seed44 |
|
||||
|---|---|---|---|
|
||||
| within iscxtor2016 | terminal_packet (0.9950) | terminal_norm (0.9963) | terminal_norm (0.9953) |
|
||||
| within cicids2017 | terminal_packet (0.9929) | terminal_packet (0.9918) | terminal_packet (0.9949) |
|
||||
| within cicddos2019 | terminal_norm (0.9966) | terminal_norm (0.9977) | terminal_norm (0.9966) |
|
||||
| within ciciot2023 | terminal_packet (0.9669) | terminal_packet (0.9674) | terminal_packet (0.9668) |
|
||||
| cross→cicids2017 | disc_nll_total (0.9194) | disc_nll_total (0.9090) | disc_nll_total (0.9288) |
|
||||
| cross→cicddos2019 | disc_nll_ch2 (0.8623) | disc_nll_ch2 (0.8884) | disc_nll_ch2 (0.9046) |
|
||||
70
artifacts/route_comparison/SOTA_COMPARISON.md
Normal file
70
artifacts/route_comparison/SOTA_COMPARISON.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# SOTA Comparison: A+C combo vs existing UnifiedCFM
|
||||
|
||||
All 4 datasets, 3 seeds each, within-dataset Shafir 10K/10K protocol.
|
||||
Existing UnifiedCFM uses Phase-2 consistency loss (λ_flow=λ_packet=0.3).
|
||||
A+C combo uses Mixed_CFM (continuous CFM + DFM) + causal-packet attention,
|
||||
**no Phase-2 consistency loss**. lambda_disc=1.0, sigma=0.1, use_ot=True.
|
||||
|
||||
## Headline: A+C combo's best score per dataset
|
||||
|
||||
| Dataset | Shafir 2026 | Existing UnifiedCFM (SOTA) | A+C combo `terminal_norm` | A+C combo `terminal_packet` | A+C combo `disc_nll_total` | A+C best | New SOTA? |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| ISCXTor2016 (NonTor → Tor) | 0.8731 | 0.9945 ± 0.0011 | 0.9954 ± 0.0007 | 0.9953 ± 0.0004 | 0.7063 ± 0.0201 | `terminal_norm` 0.9954 ± 0.0007 | ✅ +0.0009 |
|
||||
| CICIDS2017 within (Shafir 10k/10k) | 0.9303 | 0.9858 ± 0.0021 | 0.9884 ± 0.0012 | 0.9932 ± 0.0013 | 0.9839 ± 0.0005 | `terminal_packet` 0.9932 ± 0.0013 | ✅ +0.0074 |
|
||||
| CICDDoS2019 within | 0.9300 | 0.9960 ± 0.0010 | 0.9970 ± 0.0005 | 0.9909 ± 0.0010 | 0.5593 ± 0.0423 | `terminal_norm` 0.9970 ± 0.0005 | ✅ +0.0010 |
|
||||
| CICIoT2023 within (multi-seed) | — | 0.9612 ± 0.0017 | 0.9604 ± 0.0022 | 0.9671 ± 0.0002 | 0.8994 ± 0.0098 | `terminal_packet` 0.9671 ± 0.0002 | ✅ +0.0059 |
|
||||
|
||||
## Per-dataset full scoring
|
||||
|
||||
### ISCXTor2016 (NonTor → Tor)
|
||||
|
||||
| Score | mean ± std | seeds |
|
||||
|---|---|---|
|
||||
| `terminal_norm` | 0.9954 ± 0.0007 | [42, 43, 44] |
|
||||
| `terminal_flow` | 0.9283 ± 0.0096 | [42, 43, 44] |
|
||||
| `terminal_packet` | 0.9953 ± 0.0004 | [42, 43, 44] |
|
||||
| `disc_nll_total` | 0.7063 ± 0.0201 | [42, 43, 44] |
|
||||
| `disc_nll_ch3` | 0.7020 ± 0.0314 | [42, 43, 44] |
|
||||
| `disc_nll_ch4` | 0.4362 ± 0.0278 | [42, 43, 44] |
|
||||
| `disc_nll_ch5` | 0.4626 ± 0.0253 | [42, 43, 44] |
|
||||
| `disc_nll_ch7` | 0.6957 ± 0.0476 | [42, 43, 44] |
|
||||
|
||||
### CICIDS2017 within (Shafir 10k/10k)
|
||||
|
||||
| Score | mean ± std | seeds |
|
||||
|---|---|---|
|
||||
| `terminal_norm` | 0.9884 ± 0.0012 | [42, 43, 44] |
|
||||
| `terminal_flow` | 0.9628 ± 0.0024 | [42, 43, 44] |
|
||||
| `terminal_packet` | 0.9932 ± 0.0013 | [42, 43, 44] |
|
||||
| `disc_nll_total` | 0.9839 ± 0.0005 | [42, 43, 44] |
|
||||
| `disc_nll_ch3` | 0.6890 ± 0.1267 | [42, 43, 44] |
|
||||
| `disc_nll_ch4` | 0.7512 ± 0.1043 | [42, 43, 44] |
|
||||
| `disc_nll_ch5` | 0.9055 ± 0.0112 | [42, 43, 44] |
|
||||
| `disc_nll_ch7` | 0.7479 ± 0.1000 | [42, 43, 44] |
|
||||
|
||||
### CICDDoS2019 within
|
||||
|
||||
| Score | mean ± std | seeds |
|
||||
|---|---|---|
|
||||
| `terminal_norm` | 0.9970 ± 0.0005 | [42, 43, 44] |
|
||||
| `terminal_flow` | 0.9648 ± 0.0028 | [42, 43, 44] |
|
||||
| `terminal_packet` | 0.9909 ± 0.0010 | [42, 43, 44] |
|
||||
| `disc_nll_total` | 0.5593 ± 0.0423 | [42, 43, 44] |
|
||||
| `disc_nll_ch3` | 0.2648 ± 0.0263 | [42, 43, 44] |
|
||||
| `disc_nll_ch4` | 0.4641 ± 0.0579 | [42, 43, 44] |
|
||||
| `disc_nll_ch5` | 0.3949 ± 0.0518 | [42, 43, 44] |
|
||||
| `disc_nll_ch7` | 0.9280 ± 0.0253 | [42, 43, 44] |
|
||||
|
||||
### CICIoT2023 within (multi-seed)
|
||||
|
||||
| Score | mean ± std | seeds |
|
||||
|---|---|---|
|
||||
| `terminal_norm` | 0.9604 ± 0.0022 | [42, 43, 44] |
|
||||
| `terminal_flow` | 0.9186 ± 0.0031 | [42, 43, 44] |
|
||||
| `terminal_packet` | 0.9671 ± 0.0002 | [42, 43, 44] |
|
||||
| `disc_nll_total` | 0.8994 ± 0.0098 | [42, 43, 44] |
|
||||
| `disc_nll_ch3` | 0.8606 ± 0.0257 | [42, 43, 44] |
|
||||
| `disc_nll_ch4` | 0.8353 ± 0.0152 | [42, 43, 44] |
|
||||
| `disc_nll_ch5` | 0.8474 ± 0.0208 | [42, 43, 44] |
|
||||
| `disc_nll_ch7` | 0.8934 ± 0.0098 | [42, 43, 44] |
|
||||
|
||||
174
artifacts/route_comparison/aggregate_cross.py
Normal file
174
artifacts/route_comparison/aggregate_cross.py
Normal file
@@ -0,0 +1,174 @@
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from sklearn.metrics import roc_auc_score
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
CROSS_DIR = ROOT / 'cross'
|
||||
NAME_RE = re.compile('^(?P<route>.+?)_seed(?P<seed>\\d+)_to_(?P<target>cicids2017|cicddos2019)$')
|
||||
ROUTES = [('baseline', 'baseline'), ('A: causal', 'route_a_causal'), ('B: spectral', 'route_b_spectral'), ('C: mixed', 'route_c_mixed'), ('A+C combo', 'route_ac_combo')]
|
||||
TARGETS = ['cicids2017', 'cicddos2019']
|
||||
PRIMARY_SCORES = ['terminal_norm', 'terminal_flow', 'terminal_packet', 'flow_consistency', 'packet_consistency', 'consistency_total', 'causal_surprisal_packet_median', 'causal_surprisal_total', 'direction_drift_packet_median', 'pna_packet_median', 'kappa2_speed2norm_packet_median', 'curvature_packet', 'disc_nll_total', 'disc_nll_ch3', 'disc_nll_ch7']
|
||||
|
||||
def _collect() -> dict[tuple[str, str], dict[int, dict]]:
|
||||
out: dict[tuple[str, str], dict[int, dict]] = {}
|
||||
for f in sorted(CROSS_DIR.glob('*.json')):
|
||||
m = NAME_RE.match(f.stem)
|
||||
if not m:
|
||||
continue
|
||||
key = (m.group('route'), m.group('target'))
|
||||
out.setdefault(key, {})[int(m.group('seed'))] = json.loads(f.read_text())
|
||||
return out
|
||||
|
||||
def _ensemble_sweep(npz_path: Path) -> dict[float, float] | None:
|
||||
if not npz_path.exists():
|
||||
return None
|
||||
z = np.load(npz_path, allow_pickle=True)
|
||||
keys = set(z.files)
|
||||
if 'b_terminal_norm' not in keys or 'b_disc_nll_total' not in keys:
|
||||
return None
|
||||
v_tn = z['b_terminal_norm']
|
||||
a_tn = z['a_terminal_norm']
|
||||
v_dn = z['b_disc_nll_total']
|
||||
a_dn = z['a_disc_nll_total']
|
||||
|
||||
def zsc(v, a):
|
||||
(mu, sd) = (v.mean(), v.std() + 1e-09)
|
||||
return ((v - mu) / sd, (a - mu) / sd)
|
||||
(v_tn_z, a_tn_z) = zsc(v_tn, a_tn)
|
||||
(v_dn_z, a_dn_z) = zsc(v_dn, a_dn)
|
||||
out = {}
|
||||
for alpha in (0.0, 0.5, 0.7, 0.8, 0.9, 1.0):
|
||||
s_v = alpha * v_tn_z + (1.0 - alpha) * v_dn_z
|
||||
s_a = alpha * a_tn_z + (1.0 - alpha) * a_dn_z
|
||||
y = np.r_[np.zeros(len(s_v)), np.ones(len(s_a))]
|
||||
s = np.r_[s_v, s_a]
|
||||
out[alpha] = float(roc_auc_score(y, s))
|
||||
return out
|
||||
|
||||
def _mean_std(vs: list[float]) -> tuple[float, float]:
|
||||
arr = np.asarray([v for v in vs if v == v], dtype=np.float64)
|
||||
if arr.size == 0:
|
||||
return (float('nan'), float('nan'))
|
||||
return (float(arr.mean()), float(arr.std()))
|
||||
|
||||
def main() -> None:
|
||||
data = _collect()
|
||||
rows: list[str] = []
|
||||
rows.append('# Cross-Dataset Eval — CICIoT2023 → {CICIDS2017, CICDDoS2019}')
|
||||
rows.append('')
|
||||
rows.append("All models trained on CICIoT2023 (10K benign), evaluated on each target's")
|
||||
rows.append('10K benign + 10K stratified attack. Source-domain norm stats applied.')
|
||||
rows.append('3 seeds each. AUROC mean ± std.')
|
||||
rows.append('')
|
||||
rows.append('## Primary score: `terminal_norm`')
|
||||
rows.append('')
|
||||
header = '| Route | within-CICIoT2023 (ref) | → CICIDS2017 | → CICDDoS2019 |'
|
||||
rows.append(header)
|
||||
rows.append('|---|---|---|---|')
|
||||
within_fallback = {'baseline': (0.9612, 0.0017), 'A: causal': (0.9636, 0.0006), 'B: spectral': (0.9619, 0.0013), 'C: mixed': (0.9625, 0.0028), 'A+C combo': (0.9587, 0.0017)}
|
||||
within_terminal: dict[str, tuple[float, float]] = {}
|
||||
for (label, prefix) in ROUTES:
|
||||
within_seeds = sorted(ROOT.glob(f'{prefix}_seed*/phase1_summary.json'))
|
||||
vals: list[float] = []
|
||||
for f in within_seeds:
|
||||
try:
|
||||
s = json.loads(f.read_text())
|
||||
v = s.get('overall', {}).get('terminal_norm', {}).get('auroc')
|
||||
if v is not None:
|
||||
vals.append(v)
|
||||
except Exception:
|
||||
pass
|
||||
if vals:
|
||||
within_terminal[label] = _mean_std(vals)
|
||||
else:
|
||||
within_terminal[label] = within_fallback.get(label, (float('nan'), float('nan')))
|
||||
for (label, prefix) in ROUTES:
|
||||
cells = [label]
|
||||
(wm, ws) = within_terminal[label]
|
||||
cells.append(f'{wm:.4f} ± {ws:.4f}')
|
||||
for tgt in TARGETS:
|
||||
seeds = data.get((prefix, tgt), {})
|
||||
vals = [s['overall'].get('terminal_norm', {}).get('auroc', float('nan')) for s in seeds.values()]
|
||||
(m, sd) = _mean_std(vals)
|
||||
cells.append(f'{m:.4f} ± {sd:.4f}' if m == m else '—')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append("## Each route's best score per target")
|
||||
rows.append('')
|
||||
for tgt in TARGETS:
|
||||
rows.append(f'### → {tgt}')
|
||||
rows.append('')
|
||||
rows.append("| Route | Best score | AUROC | Δ (vs same-route's terminal_norm) |")
|
||||
rows.append('|---|---|---|---|')
|
||||
for (label, prefix) in ROUTES:
|
||||
seeds = data.get((prefix, tgt), {})
|
||||
if not seeds:
|
||||
rows.append(f'| {label} | — | — | — |')
|
||||
continue
|
||||
score_means: dict[str, float] = {}
|
||||
for s in seeds.values():
|
||||
for (k, v) in s.get('overall', {}).items():
|
||||
score_means.setdefault(k, []).append(v.get('auroc', float('nan')))
|
||||
mean_per_score = {k: _mean_std(v)[0] for (k, v) in score_means.items()}
|
||||
mean_per_score = {k: v for (k, v) in mean_per_score.items() if v == v}
|
||||
if not mean_per_score:
|
||||
rows.append(f'| {label} | — | — | — |')
|
||||
continue
|
||||
best = max(mean_per_score, key=mean_per_score.get)
|
||||
best_v = mean_per_score[best]
|
||||
best_sd = _mean_std(score_means[best])[1]
|
||||
tn = mean_per_score.get('terminal_norm', float('nan'))
|
||||
delta = f'{best_v - tn:+.4f}' if tn == tn else '—'
|
||||
rows.append(f'| {label} | `{best}` | {best_v:.4f} ± {best_sd:.4f} | {delta} |')
|
||||
rows.append('')
|
||||
for tgt in TARGETS:
|
||||
rows.append(f'## All key scores → {tgt}')
|
||||
rows.append('')
|
||||
header = '| Score | ' + ' | '.join((label for (label, _) in ROUTES)) + ' |'
|
||||
rows.append(header)
|
||||
rows.append('|---' * (1 + len(ROUTES)) + '|')
|
||||
for sc in PRIMARY_SCORES:
|
||||
cells = [f'`{sc}`']
|
||||
for (label, prefix) in ROUTES:
|
||||
seeds = data.get((prefix, tgt), {})
|
||||
vals = [s['overall'].get(sc, {}).get('auroc', float('nan')) for s in seeds.values()]
|
||||
(m, sd) = _mean_std(vals)
|
||||
cells.append(f'{m:.4f} ± {sd:.4f}' if m == m else '—')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
for tgt in TARGETS:
|
||||
rows.append(f'## Route C ensemble (terminal_norm + disc_nll) → {tgt}')
|
||||
rows.append('')
|
||||
c_seeds = data.get(('route_c_mixed', tgt), {})
|
||||
if c_seeds:
|
||||
alphas = (0.0, 0.5, 0.7, 0.8, 0.9, 1.0)
|
||||
rows.append('| α | ' + ' | '.join((f'seed{s}' for s in sorted(c_seeds.keys()))) + ' | mean ± std |')
|
||||
rows.append('|---' * (2 + len(c_seeds)) + '|')
|
||||
seed_sweeps = {}
|
||||
for s in c_seeds:
|
||||
npz = CROSS_DIR / f'route_c_mixed_seed{s}_to_{tgt}.npz'
|
||||
seed_sweeps[s] = _ensemble_sweep(npz) or {}
|
||||
for a in alphas:
|
||||
cells = [f'{a:.2f}']
|
||||
vals = []
|
||||
for s in sorted(c_seeds.keys()):
|
||||
v = seed_sweeps[s].get(a, float('nan'))
|
||||
cells.append(f'{v:.4f}')
|
||||
vals.append(v)
|
||||
(m, sd) = _mean_std(vals)
|
||||
cells.append(f'**{m:.4f} ± {sd:.4f}**')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Run inventory')
|
||||
rows.append('')
|
||||
for (label, prefix) in ROUTES:
|
||||
for tgt in TARGETS:
|
||||
seeds = sorted(data.get((prefix, tgt), {}).keys())
|
||||
rows.append(f"- {label} → {tgt}: seeds = {(seeds if seeds else '(none)')}")
|
||||
out = ROOT / 'CROSS_RESULTS.md'
|
||||
out.write_text('\n'.join(rows) + '\n')
|
||||
print(f'[wrote] {out}')
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
176
artifacts/route_comparison/aggregate_cross_matrix.py
Normal file
176
artifacts/route_comparison/aggregate_cross_matrix.py
Normal file
@@ -0,0 +1,176 @@
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from sklearn.covariance import OAS
|
||||
from sklearn.metrics import roc_auc_score
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
CROSS_DIR = ROOT / 'cross'
|
||||
DATASETS = ['iscxtor2016', 'cicids2017', 'cicddos2019', 'ciciot2023']
|
||||
SEEDS = [42, 43, 44]
|
||||
|
||||
def _mahal_eval(npz_path: Path, val_prefix: str, atk_prefix: str) -> float:
|
||||
if not npz_path.exists():
|
||||
return float('nan')
|
||||
z = np.load(npz_path, allow_pickle=True)
|
||||
keys = sorted([k.replace(val_prefix, '') for k in z.files if k.startswith(val_prefix) and (not k.endswith('labels'))])
|
||||
val_S = np.stack([z[f'{val_prefix}{k}'] for k in keys], axis=1)
|
||||
atk_S = np.stack([z[f'{atk_prefix}{k}'] for k in keys], axis=1)
|
||||
val_S = np.nan_to_num(val_S, nan=0.0, posinf=1000000.0, neginf=-1000000.0)
|
||||
atk_S = np.nan_to_num(atk_S, nan=0.0, posinf=1000000.0, neginf=-1000000.0)
|
||||
if len(val_S) < 50 or len(atk_S) < 50:
|
||||
return float('nan')
|
||||
y = np.r_[np.zeros(len(val_S)), np.ones(len(atk_S))]
|
||||
K = val_S.shape[1]
|
||||
try:
|
||||
oas = OAS().fit(val_S)
|
||||
inv_cov = np.linalg.inv(oas.covariance_ + 1e-09 * np.eye(K))
|
||||
except Exception:
|
||||
return float('nan')
|
||||
mu = val_S.mean(0)
|
||||
|
||||
def m(S):
|
||||
d = S - mu
|
||||
return np.einsum('ni,ij,nj->n', d, inv_cov, d)
|
||||
s = np.r_[m(val_S), m(atk_S)]
|
||||
s = np.nan_to_num(s, nan=0.0, posinf=1000000000000.0, neginf=-1000000000000.0)
|
||||
try:
|
||||
return float(roc_auc_score(y, s))
|
||||
except ValueError:
|
||||
return float('nan')
|
||||
|
||||
def _within_mahal(ds: str, seed: int) -> float:
|
||||
md = ROOT / f'route_ac_combo_{ds}_seed{seed}'
|
||||
return _mahal_eval(md / 'phase1_scores.npz', 'val_', 'atk_')
|
||||
|
||||
def _within_terminal_norm(ds: str, seed: int) -> float:
|
||||
f = ROOT / f'route_ac_combo_{ds}_seed{seed}' / 'phase1_summary.json'
|
||||
if not f.exists():
|
||||
return float('nan')
|
||||
return json.loads(f.read_text())['overall'].get('terminal_norm', {}).get('auroc', float('nan'))
|
||||
|
||||
def _src_aliases(src: str) -> list[str]:
|
||||
aliases = [src]
|
||||
if src == 'cicddos2019':
|
||||
aliases.append('ddos2019')
|
||||
return aliases
|
||||
|
||||
def _cross_mahal(src: str, tgt: str, seed: int) -> float:
|
||||
candidates = []
|
||||
for alias in _src_aliases(src):
|
||||
candidates.append(CROSS_DIR / f'route_ac_combo_seed{seed}_{alias}_to_{tgt}.npz')
|
||||
if src == 'ciciot2023':
|
||||
candidates.append(CROSS_DIR / f'route_ac_combo_seed{seed}_to_{tgt}.npz')
|
||||
for c in candidates:
|
||||
if c.exists():
|
||||
return _mahal_eval(c, 'b_', 'a_')
|
||||
return float('nan')
|
||||
|
||||
def _cross_terminal_norm(src: str, tgt: str, seed: int) -> float:
|
||||
candidates = []
|
||||
for alias in _src_aliases(src):
|
||||
candidates.append(CROSS_DIR / f'route_ac_combo_seed{seed}_{alias}_to_{tgt}.json')
|
||||
if src == 'ciciot2023':
|
||||
candidates.append(CROSS_DIR / f'route_ac_combo_seed{seed}_to_{tgt}.json')
|
||||
for c in candidates:
|
||||
if c.exists():
|
||||
d = json.loads(c.read_text())
|
||||
return d['overall'].get('terminal_norm', {}).get('auroc', float('nan'))
|
||||
return float('nan')
|
||||
|
||||
def _ms(vals: list[float]) -> str:
|
||||
arr = np.asarray([v for v in vals if not np.isnan(v)], dtype=np.float64)
|
||||
if arr.size == 0:
|
||||
return '—'
|
||||
if arr.size == 1:
|
||||
return f'{arr[0]:.4f}'
|
||||
return f'{arr.mean():.4f}±{arr.std():.4f}'
|
||||
|
||||
def main() -> None:
|
||||
rows: list[str] = []
|
||||
rows.append('# Full 4×4 Cross Matrix — A+C combo + Mahalanobis-OAS')
|
||||
rows.append('')
|
||||
rows.append('3-seed mean ± std. Diagonal = within-dataset; off-diagonal = cross.')
|
||||
rows.append('Aggregator: Mahalanobis-OAS over 10-d A+C combo score vector,')
|
||||
rows.append('fit on **target-dataset benign val only** (no attack labels).')
|
||||
rows.append('')
|
||||
rows.append('## Mahalanobis-OAS AUROC (4×4)')
|
||||
rows.append('')
|
||||
rows.append('| Source ↓ \\ Target → | ' + ' | '.join(DATASETS) + ' |')
|
||||
rows.append('|---' * (1 + len(DATASETS)) + '|')
|
||||
for src in DATASETS:
|
||||
cells = [src]
|
||||
for tgt in DATASETS:
|
||||
if src == tgt:
|
||||
vals = [_within_mahal(src, s) for s in SEEDS]
|
||||
cells.append(f'_{_ms(vals)}_')
|
||||
else:
|
||||
vals = [_cross_mahal(src, tgt, s) for s in SEEDS]
|
||||
cells.append(_ms(vals))
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('(Italic diagonal = within-dataset reference)')
|
||||
rows.append('')
|
||||
rows.append('## `terminal_norm` AUROC (4×4) — for comparison (selection-bias-free single fixed score)')
|
||||
rows.append('')
|
||||
rows.append('| Source ↓ \\ Target → | ' + ' | '.join(DATASETS) + ' |')
|
||||
rows.append('|---' * (1 + len(DATASETS)) + '|')
|
||||
for src in DATASETS:
|
||||
cells = [src]
|
||||
for tgt in DATASETS:
|
||||
if src == tgt:
|
||||
vals = [_within_terminal_norm(src, s) for s in SEEDS]
|
||||
cells.append(f'_{_ms(vals)}_')
|
||||
else:
|
||||
vals = [_cross_terminal_norm(src, tgt, s) for s in SEEDS]
|
||||
cells.append(_ms(vals))
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Δ Mahalanobis − terminal_norm (where positive, Mahalanobis is better)')
|
||||
rows.append('')
|
||||
rows.append('| Source ↓ \\ Target → | ' + ' | '.join(DATASETS) + ' |')
|
||||
rows.append('|---' * (1 + len(DATASETS)) + '|')
|
||||
for src in DATASETS:
|
||||
cells = [src]
|
||||
for tgt in DATASETS:
|
||||
if src == tgt:
|
||||
m = np.mean([v for v in [_within_mahal(src, s) for s in SEEDS] if not np.isnan(v)])
|
||||
t = np.mean([v for v in [_within_terminal_norm(src, s) for s in SEEDS] if not np.isnan(v)])
|
||||
else:
|
||||
m = np.mean([v for v in [_cross_mahal(src, tgt, s) for s in SEEDS] if not np.isnan(v)])
|
||||
t = np.mean([v for v in [_cross_terminal_norm(src, tgt, s) for s in SEEDS] if not np.isnan(v)])
|
||||
if np.isnan(m) or np.isnan(t):
|
||||
cells.append('—')
|
||||
else:
|
||||
d = m - t
|
||||
if abs(d) < 0.005:
|
||||
cells.append(f'{d:+.4f}')
|
||||
elif d > 0:
|
||||
cells.append(f'**{d:+.4f}**')
|
||||
else:
|
||||
cells.append(f'_{d:+.4f}_')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Per-source averaged cross-AUROC (Mahalanobis, off-diagonal mean)')
|
||||
rows.append('')
|
||||
rows.append('| Source | mean off-diag Mahalanobis | mean off-diag terminal_norm |')
|
||||
rows.append('|---|---|---|')
|
||||
for src in DATASETS:
|
||||
m_offs = []
|
||||
t_offs = []
|
||||
for tgt in DATASETS:
|
||||
if src == tgt:
|
||||
continue
|
||||
m_vals = [_cross_mahal(src, tgt, s) for s in SEEDS]
|
||||
t_vals = [_cross_terminal_norm(src, tgt, s) for s in SEEDS]
|
||||
m_offs.extend([v for v in m_vals if not np.isnan(v)])
|
||||
t_offs.extend([v for v in t_vals if not np.isnan(v)])
|
||||
m_mean = np.mean(m_offs) if m_offs else float('nan')
|
||||
t_mean = np.mean(t_offs) if t_offs else float('nan')
|
||||
rows.append(f'| {src} | {m_mean:.4f} | {t_mean:.4f} |')
|
||||
out = ROOT / 'CROSS_MATRIX.md'
|
||||
out.write_text('\n'.join(rows) + '\n')
|
||||
print(f'[wrote] {out}')
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
84
artifacts/route_comparison/aggregate_full_sota.py
Normal file
84
artifacts/route_comparison/aggregate_full_sota.py
Normal file
@@ -0,0 +1,84 @@
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
SEED_RE = re.compile('_seed(\\d+)$')
|
||||
EXISTING_SOTA = {'ISCXTor2016 (NonTor → Tor)': {'shafir_baseline': 0.8731, 'shafir_ref': 'Table VI', 'ours_existing': (0.9945, 0.0011), 'ours_score': 'terminal_norm', 'sigma': 0.1, 'ac_prefix': 'route_ac_combo_iscxtor2016'}, 'CICIDS2017 within (Shafir 10k/10k)': {'shafir_baseline': 0.9303, 'shafir_ref': 'Table VII', 'ours_existing': (0.9858, 0.0021), 'ours_score': 'terminal_norm', 'sigma': 0.6, 'ac_prefix': 'route_ac_combo_cicids2017'}, 'CICDDoS2019 within': {'shafir_baseline': 0.93, 'shafir_ref': 'Table IX, row 1', 'ours_existing': (0.996, 0.001), 'ours_score': 'terminal_norm', 'sigma': 0.1, 'ac_prefix': 'route_ac_combo_cicddos2019'}, 'CICIoT2023 within (multi-seed)': {'shafir_baseline': None, 'shafir_ref': None, 'ours_existing': (0.9612, 0.0017), 'ours_score': 'terminal_norm', 'sigma': 0.1, 'ac_prefix': 'route_ac_combo_ciciot2023'}}
|
||||
|
||||
def _seeds(prefix: str) -> dict[int, Path]:
|
||||
out = {}
|
||||
for d in sorted(ROOT.glob(f'{prefix}_seed*')):
|
||||
m = SEED_RE.search(d.name)
|
||||
if m and (d / 'phase1_summary.json').exists():
|
||||
out[int(m.group(1))] = d
|
||||
return out
|
||||
|
||||
def _load(d: Path) -> dict:
|
||||
return json.loads((d / 'phase1_summary.json').read_text())
|
||||
|
||||
def _mean_std(vs: list[float]) -> tuple[float, float]:
|
||||
arr = np.asarray([v for v in vs if v == v], dtype=np.float64)
|
||||
if arr.size == 0:
|
||||
return (float('nan'), float('nan'))
|
||||
return (float(arr.mean()), float(arr.std()))
|
||||
|
||||
def main() -> None:
|
||||
rows: list[str] = []
|
||||
rows.append('# SOTA Comparison: A+C combo vs existing UnifiedCFM')
|
||||
rows.append('')
|
||||
rows.append('All 4 datasets, 3 seeds each, within-dataset Shafir 10K/10K protocol.')
|
||||
rows.append('Existing UnifiedCFM uses Phase-2 consistency loss (λ_flow=λ_packet=0.3).')
|
||||
rows.append('A+C combo uses Mixed_CFM (continuous CFM + DFM) + causal-packet attention,')
|
||||
rows.append('**no Phase-2 consistency loss**. lambda_disc=1.0, sigma=0.1, use_ot=True.')
|
||||
rows.append('')
|
||||
rows.append("## Headline: A+C combo's best score per dataset")
|
||||
rows.append('')
|
||||
rows.append('| Dataset | Shafir 2026 | Existing UnifiedCFM (SOTA) | A+C combo `terminal_norm` | A+C combo `terminal_packet` | A+C combo `disc_nll_total` | A+C best | New SOTA? |')
|
||||
rows.append('|---|---|---|---|---|---|---|---|')
|
||||
for (label, meta) in EXISTING_SOTA.items():
|
||||
seeds = _seeds(meta['ac_prefix'])
|
||||
shafir_str = f"{meta['shafir_baseline']:.4f}" if meta['shafir_baseline'] else '—'
|
||||
(existing_m, existing_sd) = meta['ours_existing']
|
||||
existing_str = f'{existing_m:.4f} ± {existing_sd:.4f}'
|
||||
if not seeds:
|
||||
rows.append(f'| {label} | {shafir_str} | {existing_str} | (running) | — | — | — | — |')
|
||||
continue
|
||||
vals_term = [_load(d).get('overall', {}).get('terminal_norm', {}).get('auroc', float('nan')) for d in seeds.values()]
|
||||
vals_pkt = [_load(d).get('overall', {}).get('terminal_packet', {}).get('auroc', float('nan')) for d in seeds.values()]
|
||||
vals_disc = [_load(d).get('overall', {}).get('disc_nll_total', {}).get('auroc', float('nan')) for d in seeds.values()]
|
||||
(m_t, s_t) = _mean_std(vals_term)
|
||||
(m_p, s_p) = _mean_std(vals_pkt)
|
||||
(m_d, s_d) = _mean_std(vals_disc)
|
||||
(best_score, best_m, best_sd) = ('terminal_norm', m_t, s_t)
|
||||
if m_p > best_m:
|
||||
(best_score, best_m, best_sd) = ('terminal_packet', m_p, s_p)
|
||||
if m_d > best_m:
|
||||
(best_score, best_m, best_sd) = ('disc_nll_total', m_d, s_d)
|
||||
beats = '✅' if best_m > existing_m else '❌'
|
||||
rows.append(f'| {label} | {shafir_str} | {existing_str} | {m_t:.4f} ± {s_t:.4f} | {m_p:.4f} ± {s_p:.4f} | {m_d:.4f} ± {s_d:.4f} | `{best_score}` {best_m:.4f} ± {best_sd:.4f} | {beats} {best_m - existing_m:+.4f} |')
|
||||
rows.append('')
|
||||
rows.append('## Per-dataset full scoring')
|
||||
rows.append('')
|
||||
score_keys = ['terminal_norm', 'terminal_flow', 'terminal_packet', 'disc_nll_total', 'disc_nll_ch3', 'disc_nll_ch4', 'disc_nll_ch5', 'disc_nll_ch7']
|
||||
for (label, meta) in EXISTING_SOTA.items():
|
||||
rows.append(f'### {label}')
|
||||
rows.append('')
|
||||
seeds = _seeds(meta['ac_prefix'])
|
||||
if not seeds:
|
||||
rows.append('(not yet completed)\n')
|
||||
continue
|
||||
rows.append('| Score | mean ± std | seeds |')
|
||||
rows.append('|---|---|---|')
|
||||
for sc in score_keys:
|
||||
vals = [_load(d).get('overall', {}).get(sc, {}).get('auroc', float('nan')) for d in seeds.values()]
|
||||
(m, sd) = _mean_std(vals)
|
||||
if m == m:
|
||||
rows.append(f'| `{sc}` | {m:.4f} ± {sd:.4f} | {sorted(seeds.keys())} |')
|
||||
rows.append('')
|
||||
out = ROOT / 'SOTA_COMPARISON.md'
|
||||
out.write_text('\n'.join(rows) + '\n')
|
||||
print(f'[wrote] {out}')
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
94
artifacts/route_comparison/aggregate_results.py
Normal file
94
artifacts/route_comparison/aggregate_results.py
Normal file
@@ -0,0 +1,94 @@
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
SEED_RE = re.compile('_seed(\\d+)$')
|
||||
ROUTES = [('baseline', 'baseline_ciciot2023'), ('A: causal', 'route_a_causal_ciciot2023'), ('B: spectral', 'route_b_spectral_ciciot2023'), ('C: mixed', 'route_c_mixed_ciciot2023')]
|
||||
PRIMARY_SCORES = ['terminal_norm', 'terminal_flow', 'terminal_packet', 'causal_surprisal_packet_median', 'causal_surprisal_packet_max', 'causal_surprisal_total', 'consistency_total', 'flow_consistency', 'packet_consistency', 'kappa2_speed2norm_packet_median', 'direction_drift_packet_median', 'pna_packet_median', 'disc_nll_total', 'disc_nll_ch2', 'disc_nll_ch3', 'disc_nll_ch4', 'disc_nll_ch5', 'disc_nll_ch6', 'disc_nll_ch7']
|
||||
|
||||
def _collect(prefix: str) -> dict[int, dict]:
|
||||
out: dict[int, dict] = {}
|
||||
for d in sorted(ROOT.glob(f'{prefix}_seed*')):
|
||||
m = SEED_RE.search(d.name)
|
||||
if not m:
|
||||
continue
|
||||
f = d / 'phase1_summary.json'
|
||||
if not f.exists():
|
||||
continue
|
||||
out[int(m.group(1))] = json.loads(f.read_text())
|
||||
return out
|
||||
|
||||
def _mean_std(values: list[float]) -> tuple[float, float]:
|
||||
arr = np.asarray([v for v in values if v == v], dtype=np.float64)
|
||||
if arr.size == 0:
|
||||
return (float('nan'), float('nan'))
|
||||
return (float(arr.mean()), float(arr.std()))
|
||||
|
||||
def main() -> None:
|
||||
routes_data = {label: _collect(prefix) for (label, prefix) in ROUTES}
|
||||
rows = []
|
||||
rows.append('# Route Comparison Results — CICIoT2023')
|
||||
rows.append('')
|
||||
rows.append('All routes trained on CICIoT2023 with the protocol locked in `PROTOCOL.md`. ')
|
||||
rows.append('Numbers are AUROC over benign val (10k cap) vs all attacks (10k cap), ')
|
||||
rows.append('3 seeds each. ± std across seeds.')
|
||||
rows.append('')
|
||||
rows.append('## Overall AUROC by score')
|
||||
rows.append('')
|
||||
header = '| Score | ' + ' | '.join((label for (label, _) in ROUTES)) + ' |'
|
||||
sep = '|---' * (1 + len(ROUTES)) + '|'
|
||||
rows.append(header)
|
||||
rows.append(sep)
|
||||
for score in PRIMARY_SCORES:
|
||||
cells = [f'`{score}`']
|
||||
for (label, _) in ROUTES:
|
||||
seeds = routes_data[label]
|
||||
if not seeds:
|
||||
cells.append('—')
|
||||
continue
|
||||
vals = [summary.get('overall', {}).get(score, {}).get('auroc', float('nan')) for summary in seeds.values()]
|
||||
(mean, std) = _mean_std(vals)
|
||||
cells.append(f'{mean:.4f} ± {std:.4f}' if mean == mean else '—')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Per-attack-class `terminal_norm` AUROC (top 12 by support)')
|
||||
rows.append('')
|
||||
seed_dicts = list(routes_data['baseline'].values())
|
||||
if seed_dicts:
|
||||
all_classes: dict[str, float] = {}
|
||||
for s in seed_dicts:
|
||||
for (cls, cls_data) in s.get('per_class', {}).items():
|
||||
if cls.startswith('_'):
|
||||
continue
|
||||
n = cls_data.get('_n', 0.0)
|
||||
all_classes[cls] = max(all_classes.get(cls, 0.0), n)
|
||||
ranked = sorted(all_classes.items(), key=lambda kv: -kv[1])[:12]
|
||||
header = '| Class | n | ' + ' | '.join((label for (label, _) in ROUTES)) + ' |'
|
||||
sep = '|---' * (2 + len(ROUTES)) + '|'
|
||||
rows.append(header)
|
||||
rows.append(sep)
|
||||
for (cls, n) in ranked:
|
||||
cells = [cls, f'{int(n)}']
|
||||
for (label, _) in ROUTES:
|
||||
seeds = routes_data[label]
|
||||
if not seeds:
|
||||
cells.append('—')
|
||||
continue
|
||||
vals = [summary.get('per_class', {}).get(cls, {}).get('terminal_norm', float('nan')) for summary in seeds.values()]
|
||||
(mean, std) = _mean_std(vals)
|
||||
cells.append(f'{mean:.3f} ± {std:.3f}' if mean == mean else '—')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Run inventory')
|
||||
rows.append('')
|
||||
for (label, prefix) in ROUTES:
|
||||
seeds = sorted(routes_data[label].keys())
|
||||
rows.append(f"- **{label}** (`{prefix}_seed*`): seeds = {(seeds if seeds else '(none yet)')}")
|
||||
out = ROOT / 'RESULTS.md'
|
||||
out.write_text('\n'.join(rows) + '\n')
|
||||
print(f'[wrote] {out}')
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
180
artifacts/route_comparison/aggregate_score_router.py
Normal file
180
artifacts/route_comparison/aggregate_score_router.py
Normal file
@@ -0,0 +1,180 @@
|
||||
from __future__ import annotations
|
||||
import json
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from sklearn.covariance import LedoitWolf, OAS, GraphicalLassoCV
|
||||
from sklearn.metrics import roc_auc_score
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
CROSS_DIR = ROOT / 'cross'
|
||||
WITHIN_DATASETS = ['iscxtor2016', 'cicids2017', 'cicddos2019', 'ciciot2023']
|
||||
CROSS_TARGETS = ['cicids2017', 'cicddos2019']
|
||||
SEEDS = [42, 43, 44]
|
||||
|
||||
def _aggregators(val_S: np.ndarray, test_S_list: list[np.ndarray]) -> dict[str, list[np.ndarray]]:
|
||||
val_S = np.nan_to_num(val_S, nan=0.0, posinf=1000000.0, neginf=-1000000.0)
|
||||
test_S_list = [np.nan_to_num(t, nan=0.0, posinf=1000000.0, neginf=-1000000.0) for t in test_S_list]
|
||||
mu = val_S.mean(axis=0)
|
||||
sigma = val_S.std(axis=0) + 1e-09
|
||||
K = val_S.shape[1]
|
||||
cov_emp = np.cov(val_S, rowvar=False)
|
||||
inv_cov_plain = np.linalg.inv(cov_emp + 0.001 * np.eye(K))
|
||||
lw = LedoitWolf().fit(val_S)
|
||||
inv_cov_lw = np.linalg.inv(lw.covariance_ + 1e-09 * np.eye(K))
|
||||
oas = OAS().fit(val_S)
|
||||
inv_cov_oas = np.linalg.inv(oas.covariance_ + 1e-09 * np.eye(K))
|
||||
|
||||
def _max_abs_z(S):
|
||||
return np.abs((S - mu) / sigma).max(axis=1)
|
||||
|
||||
def _max_pos_z(S):
|
||||
return ((S - mu) / sigma).max(axis=1)
|
||||
|
||||
def _mahal_factory(inv_cov):
|
||||
|
||||
def f(S):
|
||||
d = S - mu
|
||||
return np.einsum('ni,ij,nj->n', d, inv_cov, d)
|
||||
return f
|
||||
out: dict[str, list[np.ndarray]] = {}
|
||||
for (tag, fn) in [('max_abs_z', _max_abs_z), ('max_pos_z', _max_pos_z), ('mahal_plain', _mahal_factory(inv_cov_plain)), ('mahal_lw', _mahal_factory(inv_cov_lw)), ('mahal_oas', _mahal_factory(inv_cov_oas))]:
|
||||
out[tag] = [fn(t) for t in test_S_list]
|
||||
return out
|
||||
SCORE_SUBSETS = {'all': None, 'terminal3': ['terminal_norm', 'terminal_flow', 'terminal_packet'], 'disc7': ['disc_nll_total', 'disc_nll_ch2', 'disc_nll_ch3', 'disc_nll_ch4', 'disc_nll_ch5', 'disc_nll_ch6', 'disc_nll_ch7']}
|
||||
|
||||
def _evaluate(npz: Path, val_prefix: str, atk_prefix: str) -> dict:
|
||||
z = np.load(npz, allow_pickle=True)
|
||||
all_keys = sorted([k.replace(val_prefix, '') for k in z.files if k.startswith(val_prefix) and (not k.endswith('labels'))])
|
||||
out: dict = {'n_val': None, 'n_atk': None}
|
||||
for (subset_name, subset_keys) in SCORE_SUBSETS.items():
|
||||
if subset_keys is None:
|
||||
keys = all_keys
|
||||
else:
|
||||
keys = [k for k in subset_keys if k in all_keys]
|
||||
if len(keys) < 2:
|
||||
continue
|
||||
val_S = np.stack([z[f'{val_prefix}{k}'] for k in keys], axis=1)
|
||||
atk_S = np.stack([z[f'{atk_prefix}{k}'] for k in keys], axis=1)
|
||||
(n_val, n_atk) = (val_S.shape[0], atk_S.shape[0])
|
||||
out['n_val'] = n_val
|
||||
out['n_atk'] = n_atk
|
||||
y = np.r_[np.zeros(n_val), np.ones(n_atk)]
|
||||
aggs = _aggregators(val_S, [val_S, atk_S])
|
||||
for (tag, (v_agg, a_agg)) in aggs.items():
|
||||
s = np.r_[v_agg, a_agg]
|
||||
s = np.nan_to_num(s, nan=0.0, posinf=1000000000000.0, neginf=-1000000000000.0)
|
||||
try:
|
||||
auc = float(roc_auc_score(y, s))
|
||||
except ValueError:
|
||||
auc = float('nan')
|
||||
out[f'auc_{tag}_{subset_name}'] = auc
|
||||
out['auc_max_abs_z'] = out.get('auc_max_abs_z_all')
|
||||
out['auc_max_pos_z'] = out.get('auc_max_pos_z_all')
|
||||
out['auc_mahal_plain'] = out.get('auc_mahal_plain_all')
|
||||
out['auc_mahal_lw'] = out.get('auc_mahal_lw_all')
|
||||
out['auc_mahal_oas'] = out.get('auc_mahal_oas_all')
|
||||
val_S = np.stack([z[f'{val_prefix}{k}'] for k in all_keys], axis=1)
|
||||
atk_S = np.stack([z[f'{atk_prefix}{k}'] for k in all_keys], axis=1)
|
||||
val_S = np.nan_to_num(val_S, nan=0.0, posinf=1000000000000.0, neginf=-1000000000000.0)
|
||||
atk_S = np.nan_to_num(atk_S, nan=0.0, posinf=1000000000000.0, neginf=-1000000000000.0)
|
||||
y = np.r_[np.zeros(val_S.shape[0]), np.ones(atk_S.shape[0])]
|
||||
per_score = {}
|
||||
for (i, k) in enumerate(all_keys):
|
||||
s = np.r_[val_S[:, i], atk_S[:, i]]
|
||||
s = np.nan_to_num(s, nan=0.0, posinf=1000000000000.0, neginf=-1000000000000.0)
|
||||
a1 = roc_auc_score(y, s)
|
||||
per_score[k] = max(a1, 1 - a1)
|
||||
best_score = max(per_score, key=per_score.get)
|
||||
out['auc_best_fixed'] = per_score[best_score]
|
||||
out['best_fixed_name'] = best_score
|
||||
out['auc_term_norm'] = per_score.get('terminal_norm', float('nan'))
|
||||
out['auc_term_pkt'] = per_score.get('terminal_packet', float('nan'))
|
||||
out['auc_disc_total'] = per_score.get('disc_nll_total', float('nan'))
|
||||
return out
|
||||
|
||||
def _mean_std(vs: list[float]) -> tuple[float, float]:
|
||||
arr = np.asarray([v for v in vs if v == v], dtype=np.float64)
|
||||
if arr.size == 0:
|
||||
return (float('nan'), float('nan'))
|
||||
return (float(arr.mean()), float(arr.std()))
|
||||
|
||||
def main() -> None:
|
||||
rows: list[str] = []
|
||||
rows.append('# Score-vector auto-selection: max-of-|z| / Mahalanobis vs fixed scores')
|
||||
rows.append('')
|
||||
rows.append('Aggregators are fit on **benign val only** (no attack labels). All numbers')
|
||||
rows.append('are 3-seed mean ± std on A+C combo (Mixed_CFM + causal-packet attention).')
|
||||
rows.append('')
|
||||
rows.append('Note on fairness: `auc_best_fixed` is selection-biased (picks per-dataset best')
|
||||
rows.append('score post-hoc on test set). `max_abs_z` and `mahalanobis` are NOT — they only')
|
||||
rows.append('use benign val to fit aggregator parameters.')
|
||||
rows.append('')
|
||||
rows.append("## Within-dataset(A+C combo on each dataset's own benign/attack)")
|
||||
rows.append('')
|
||||
rows.append('| Dataset | term_norm | best fixed | max-\\|z\\| (all) | mahal-OAS (all) | **mahal-OAS (term3)** | **mahal-OAS (disc7)** |')
|
||||
rows.append('|---|---|---|---|---|---|---|')
|
||||
for ds in WITHIN_DATASETS:
|
||||
rows_per_seed: list[dict] = []
|
||||
for s in SEEDS:
|
||||
md = ROOT / f'route_ac_combo_{ds}_seed{s}'
|
||||
npz = md / 'phase1_scores.npz'
|
||||
if not npz.exists():
|
||||
continue
|
||||
rows_per_seed.append(_evaluate(npz, 'val_', 'atk_'))
|
||||
if not rows_per_seed:
|
||||
rows.append(f'| {ds} | (no data) | | | | | |')
|
||||
continue
|
||||
|
||||
def col(field):
|
||||
(m, sd) = _mean_std([r[field] for r in rows_per_seed])
|
||||
return f'{m:.4f} ± {sd:.4f}'
|
||||
rows.append(f"| {ds} | {col('auc_term_norm')} | {col('auc_best_fixed')} | {col('auc_max_abs_z_all')} | {col('auc_mahal_oas_all')} | **{col('auc_mahal_oas_terminal3')}** | **{col('auc_mahal_oas_disc7')}** |")
|
||||
rows.append('')
|
||||
rows.append('## Cross-dataset(A+C combo trained on CICIoT2023 → eval on target)')
|
||||
rows.append('')
|
||||
rows.append('| Target | term_norm | best fixed | max-\\|z\\| (all) | mahal-OAS (all) | **mahal-OAS (term3)** | **mahal-OAS (disc7)** |')
|
||||
rows.append('|---|---|---|---|---|---|---|')
|
||||
for tgt in CROSS_TARGETS:
|
||||
rows_per_seed: list[dict] = []
|
||||
for s in SEEDS:
|
||||
npz = CROSS_DIR / f'route_ac_combo_seed{s}_to_{tgt}.npz'
|
||||
if not npz.exists():
|
||||
continue
|
||||
rows_per_seed.append(_evaluate(npz, 'b_', 'a_'))
|
||||
if not rows_per_seed:
|
||||
rows.append(f'| {tgt} | (no data) | | | | | |')
|
||||
continue
|
||||
|
||||
def col(field):
|
||||
(m, sd) = _mean_std([r[field] for r in rows_per_seed])
|
||||
return f'{m:.4f} ± {sd:.4f}'
|
||||
rows.append(f"| {tgt} | {col('auc_term_norm')} | {col('auc_best_fixed')} | {col('auc_max_abs_z_all')} | {col('auc_mahal_oas_all')} | **{col('auc_mahal_oas_terminal3')}** | **{col('auc_mahal_oas_disc7')}** |")
|
||||
rows.append('')
|
||||
rows.append('## Best-fixed-score winner per setup')
|
||||
rows.append('')
|
||||
rows.append('| Setup | seed42 | seed43 | seed44 |')
|
||||
rows.append('|---|---|---|---|')
|
||||
for ds in WITHIN_DATASETS:
|
||||
cells = [f'within {ds}']
|
||||
for s in SEEDS:
|
||||
npz = ROOT / f'route_ac_combo_{ds}_seed{s}/phase1_scores.npz'
|
||||
if not npz.exists():
|
||||
cells.append('—')
|
||||
continue
|
||||
r = _evaluate(npz, 'val_', 'atk_')
|
||||
cells.append(f"{r['best_fixed_name']} ({r['auc_best_fixed']:.4f})")
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
for tgt in CROSS_TARGETS:
|
||||
cells = [f'cross→{tgt}']
|
||||
for s in SEEDS:
|
||||
npz = CROSS_DIR / f'route_ac_combo_seed{s}_to_{tgt}.npz'
|
||||
if not npz.exists():
|
||||
cells.append('—')
|
||||
continue
|
||||
r = _evaluate(npz, 'b_', 'a_')
|
||||
cells.append(f"{r['best_fixed_name']} ({r['auc_best_fixed']:.4f})")
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
out = ROOT / 'SCORE_ROUTER.md'
|
||||
out.write_text('\n'.join(rows) + '\n')
|
||||
print(f'[wrote] {out}')
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
182
artifacts/route_comparison/aggregate_v2.py
Normal file
182
artifacts/route_comparison/aggregate_v2.py
Normal file
@@ -0,0 +1,182 @@
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from sklearn.metrics import roc_auc_score
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
SEED_RE = re.compile('_seed(\\d+)$')
|
||||
ROUTES = [('baseline', 'baseline_ciciot2023'), ('A: causal', 'route_a_causal_ciciot2023'), ('B: spectral', 'route_b_spectral_ciciot2023'), ('C: mixed', 'route_c_mixed_ciciot2023'), ('A+C combo', 'route_ac_combo_ciciot2023')]
|
||||
|
||||
def _seeds(prefix: str) -> dict[int, Path]:
|
||||
out = {}
|
||||
for d in sorted(ROOT.glob(f'{prefix}_seed*')):
|
||||
m = SEED_RE.search(d.name)
|
||||
if m and (d / 'phase1_summary.json').exists():
|
||||
out[int(m.group(1))] = d
|
||||
return out
|
||||
|
||||
def _load_summary(d: Path) -> dict:
|
||||
return json.loads((d / 'phase1_summary.json').read_text())
|
||||
|
||||
def _ensemble_sweep(d: Path) -> dict[float, float] | None:
|
||||
f = d / 'phase1_scores.npz'
|
||||
if not f.exists():
|
||||
return None
|
||||
z = np.load(f, allow_pickle=True)
|
||||
keys = set(z.files)
|
||||
if 'val_terminal_norm' not in keys or 'val_disc_nll_total' not in keys:
|
||||
return None
|
||||
v_tn = z['val_terminal_norm']
|
||||
a_tn = z['atk_terminal_norm']
|
||||
v_dn = z['val_disc_nll_total']
|
||||
a_dn = z['atk_disc_nll_total']
|
||||
|
||||
def zsc(v, a):
|
||||
(mu, sd) = (v.mean(), v.std() + 1e-09)
|
||||
return ((v - mu) / sd, (a - mu) / sd)
|
||||
(v_tn_z, a_tn_z) = zsc(v_tn, a_tn)
|
||||
(v_dn_z, a_dn_z) = zsc(v_dn, a_dn)
|
||||
out: dict[float, float] = {}
|
||||
for alpha in (0.0, 0.25, 0.5, 0.7, 0.8, 0.9, 1.0):
|
||||
s_v = alpha * v_tn_z + (1.0 - alpha) * v_dn_z
|
||||
s_a = alpha * a_tn_z + (1.0 - alpha) * a_dn_z
|
||||
y = np.r_[np.zeros(len(s_v)), np.ones(len(s_a))]
|
||||
s = np.r_[s_v, s_a]
|
||||
out[alpha] = float(roc_auc_score(y, s))
|
||||
return out
|
||||
|
||||
def _ensemble_score(d: Path) -> tuple[float, float] | None:
|
||||
sweep = _ensemble_sweep(d)
|
||||
if sweep is None:
|
||||
return None
|
||||
best_alpha = max(sweep, key=sweep.get)
|
||||
return (sweep[best_alpha], best_alpha)
|
||||
|
||||
def _mean_std(vals: list[float]) -> tuple[float, float]:
|
||||
arr = np.asarray([v for v in vals if v == v], dtype=np.float64)
|
||||
if arr.size == 0:
|
||||
return (float('nan'), float('nan'))
|
||||
return (float(arr.mean()), float(arr.std()))
|
||||
|
||||
def main() -> None:
|
||||
routes_data: dict[str, dict[int, dict]] = {}
|
||||
routes_dirs: dict[str, dict[int, Path]] = {}
|
||||
for (label, prefix) in ROUTES:
|
||||
seeds = _seeds(prefix)
|
||||
routes_dirs[label] = seeds
|
||||
routes_data[label] = {s: _load_summary(d) for (s, d) in seeds.items()}
|
||||
rows: list[str] = []
|
||||
rows.append('# Route Comparison Results — CICIoT2023 (multi-seed)')
|
||||
rows.append('')
|
||||
rows.append('Phase1 eval: AUROC over benign val (5k cap) vs all attacks (10k cap), 3 seeds each.')
|
||||
rows.append('')
|
||||
rows.append("## Each route's best AUROC (overall)")
|
||||
rows.append('')
|
||||
rows.append('| Route | Best score | AUROC | Δ vs baseline-best |')
|
||||
rows.append('|---|---|---|---|')
|
||||
baseline_best = None
|
||||
for (label, _) in ROUTES:
|
||||
seeds = routes_data[label]
|
||||
if not seeds:
|
||||
rows.append(f'| {label} | — | — | — |')
|
||||
continue
|
||||
all_scores: dict[str, list[float]] = {}
|
||||
for s in seeds.values():
|
||||
for (k, v) in s.get('overall', {}).items():
|
||||
all_scores.setdefault(k, []).append(v.get('auroc', float('nan')))
|
||||
score_means = {k: _mean_std(v)[0] for (k, v) in all_scores.items()}
|
||||
score_means = {k: v for (k, v) in score_means.items() if v == v}
|
||||
if not score_means:
|
||||
rows.append(f'| {label} | — | — | — |')
|
||||
continue
|
||||
best_score = max(score_means, key=score_means.get)
|
||||
best_val = score_means[best_score]
|
||||
if label == 'baseline':
|
||||
baseline_best = best_val
|
||||
delta_str = '—'
|
||||
else:
|
||||
delta_str = f'{best_val - baseline_best:+.4f}' if baseline_best else '—'
|
||||
std = _mean_std(all_scores[best_score])[1]
|
||||
rows.append(f'| {label} | `{best_score}` | {best_val:.4f} ± {std:.4f} | {delta_str} |')
|
||||
rows.append('')
|
||||
rows.append('## Primary score: `terminal_norm`')
|
||||
rows.append('')
|
||||
rows.append('| Route | mean ± std | seeds |')
|
||||
rows.append('|---|---|---|')
|
||||
for (label, _) in ROUTES:
|
||||
seeds = routes_data[label]
|
||||
if not seeds:
|
||||
rows.append(f'| {label} | — | — |')
|
||||
continue
|
||||
vals = [s['overall'].get('terminal_norm', {}).get('auroc', float('nan')) for s in seeds.values()]
|
||||
(m, sd) = _mean_std(vals)
|
||||
rows.append(f'| {label} | {m:.4f} ± {sd:.4f} | {sorted(seeds.keys())} |')
|
||||
rows.append('')
|
||||
rows.append('## Route-specific signature scores (mean ± std, 3 seeds)')
|
||||
rows.append('')
|
||||
score_groups = [('Route A signature (consistency family)', ['flow_consistency', 'packet_consistency', 'consistency_total', 'causal_surprisal_total', 'causal_surprisal_packet_median']), ('Route B signature (curvature/dynamics)', ['kappa2_speed2norm_packet_median', 'direction_drift_packet_median', 'pna_packet_median', 'curvature_packet']), ('Route C signature (discrete NLL)', ['disc_nll_total', 'disc_nll_ch3', 'disc_nll_ch4', 'disc_nll_ch5', 'disc_nll_ch7'])]
|
||||
for (grp_name, scores) in score_groups:
|
||||
rows.append(f'### {grp_name}')
|
||||
rows.append('')
|
||||
rows.append('| Score | ' + ' | '.join((label for (label, _) in ROUTES)) + ' |')
|
||||
rows.append('|---' * (1 + len(ROUTES)) + '|')
|
||||
for sc in scores:
|
||||
cells = [f'`{sc}`']
|
||||
for (label, _) in ROUTES:
|
||||
seeds = routes_data[label]
|
||||
vals = [s['overall'].get(sc, {}).get('auroc', float('nan')) for s in seeds.values()]
|
||||
(m, sd) = _mean_std(vals)
|
||||
cells.append(f'{m:.4f} ± {sd:.4f}' if m == m else '—')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Route C ensemble: α·terminal_norm + (1−α)·disc_nll_total (z-scored)')
|
||||
rows.append('')
|
||||
c_dirs = routes_dirs.get('C: mixed', {})
|
||||
if c_dirs:
|
||||
alphas = (0.0, 0.25, 0.5, 0.7, 0.8, 0.9, 1.0)
|
||||
rows.append('| α | ' + ' | '.join((f'seed{s}' for s in sorted(c_dirs.keys()))) + ' | mean ± std |')
|
||||
rows.append('|---' * (2 + len(c_dirs)) + '|')
|
||||
per_alpha: dict[float, list[float]] = {a: [] for a in alphas}
|
||||
per_seed_sweeps = {s: _ensemble_sweep(d) or {} for (s, d) in c_dirs.items()}
|
||||
for a in alphas:
|
||||
cells = [f'{a:.2f}']
|
||||
vals = []
|
||||
for s in sorted(c_dirs.keys()):
|
||||
v = per_seed_sweeps[s].get(a, float('nan'))
|
||||
cells.append(f'{v:.4f}')
|
||||
vals.append(v)
|
||||
(m, sd) = _mean_std(vals)
|
||||
cells.append(f'**{m:.4f} ± {sd:.4f}**')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('(α=1.0 = terminal_norm only; α=0.0 = disc_nll only.)')
|
||||
rows.append('')
|
||||
rows.append('## Per-attack-class AUROC (top 12, terminal_norm)')
|
||||
rows.append('')
|
||||
if routes_data['baseline']:
|
||||
any_summary = next(iter(routes_data['baseline'].values()))
|
||||
classes = sorted([(c, d.get('_n', 0)) for (c, d) in any_summary.get('per_class', {}).items() if not c.startswith('_')], key=lambda kv: -kv[1])[:12]
|
||||
header = '| Class | n | ' + ' | '.join((label for (label, _) in ROUTES)) + ' |'
|
||||
sep = '|---' * (2 + len(ROUTES)) + '|'
|
||||
rows.append(header)
|
||||
rows.append(sep)
|
||||
for (cls, n) in classes:
|
||||
cells = [cls, f'{int(n)}']
|
||||
for (label, _) in ROUTES:
|
||||
seeds = routes_data[label]
|
||||
vals = [s.get('per_class', {}).get(cls, {}).get('terminal_norm', float('nan')) for s in seeds.values()]
|
||||
(m, sd) = _mean_std(vals)
|
||||
cells.append(f'{m:.3f}±{sd:.3f}' if m == m else '—')
|
||||
rows.append('| ' + ' | '.join(cells) + ' |')
|
||||
rows.append('')
|
||||
rows.append('## Run inventory')
|
||||
rows.append('')
|
||||
for (label, prefix) in ROUTES:
|
||||
seeds = sorted(routes_data[label].keys())
|
||||
rows.append(f"- **{label}** (`{prefix}_seed*`): seeds = {(seeds if seeds else '(none yet)')}")
|
||||
out = ROOT / 'RESULTS.md'
|
||||
out.write_text('\n'.join(rows) + '\n')
|
||||
print(f'[wrote] {out}')
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
61
artifacts/route_comparison/baseline_seed42.log
Normal file
61
artifacts/route_comparison/baseline_seed42.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=42 data=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,226,261 token_dim=21 seq_len=65 sigma=0.1 use_ot=True reference_mode=None
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (2.8s) loss=2.2385 aux_flow=2.6910 aux_pkt=1.0210
|
||||
[epoch 2/50 ] (2.1s) loss=1.8185 aux_flow=2.2098 aux_pkt=0.9856
|
||||
[epoch 3/50 ] (2.2s) loss=1.5694 aux_flow=1.9865 aux_pkt=0.9600
|
||||
[epoch 4/50 ] (2.9s) loss=1.4088 aux_flow=1.8584 aux_pkt=0.9427
|
||||
[epoch 5/50 ] (6.0s) loss=1.2803 aux_flow=1.7153 aux_pkt=0.9398
|
||||
[epoch 6/50 ] (6.0s) loss=1.1939 aux_flow=1.6313 aux_pkt=0.9344
|
||||
[epoch 7/50 ] (6.0s) loss=1.1530 aux_flow=1.5987 aux_pkt=0.9302
|
||||
[epoch 8/50 ] (6.0s) loss=1.1123 aux_flow=1.5272 aux_pkt=0.9287
|
||||
[epoch 9/50 ] (6.1s) loss=1.0785 aux_flow=1.4726 aux_pkt=0.9311
|
||||
[epoch 10/50 ] (71.6s) loss=1.0583 auroc_terminal=0.953 aux_flow=1.4576 aux_pkt=0.9233
|
||||
[epoch 11/50 ] (3.1s) loss=1.0317 aux_flow=1.4195 aux_pkt=0.9233
|
||||
[epoch 12/50 ] (3.0s) loss=1.0145 aux_flow=1.3876 aux_pkt=0.9228
|
||||
[epoch 13/50 ] (3.4s) loss=1.0529 aux_flow=1.4878 aux_pkt=0.9247
|
||||
[epoch 14/50 ] (5.9s) loss=0.9897 aux_flow=1.3594 aux_pkt=0.9167
|
||||
[epoch 15/50 ] (6.0s) loss=0.9755 aux_flow=1.3333 aux_pkt=0.9196
|
||||
[epoch 16/50 ] (6.1s) loss=0.9706 aux_flow=1.3205 aux_pkt=0.9151
|
||||
[epoch 17/50 ] (5.9s) loss=0.9741 aux_flow=1.3408 aux_pkt=0.9169
|
||||
[epoch 18/50 ] (6.0s) loss=0.9865 aux_flow=1.3802 aux_pkt=0.9176
|
||||
[epoch 19/50 ] (6.0s) loss=0.9678 aux_flow=1.3466 aux_pkt=0.9225
|
||||
[epoch 20/50 ] (68.4s) loss=0.9453 auroc_terminal=0.960 aux_flow=1.2853 aux_pkt=0.9216
|
||||
[epoch 21/50 ] (3.1s) loss=0.9450 aux_flow=1.3088 aux_pkt=0.9112
|
||||
[epoch 22/50 ] (3.0s) loss=0.9600 aux_flow=1.3598 aux_pkt=0.9128
|
||||
[epoch 23/50 ] (4.7s) loss=0.9320 aux_flow=1.2747 aux_pkt=0.9135
|
||||
[epoch 24/50 ] (6.0s) loss=0.9258 aux_flow=1.2705 aux_pkt=0.9177
|
||||
[epoch 25/50 ] (6.0s) loss=0.9202 aux_flow=1.2642 aux_pkt=0.9153
|
||||
[epoch 26/50 ] (6.0s) loss=0.9248 aux_flow=1.2816 aux_pkt=0.9132
|
||||
[epoch 27/50 ] (6.0s) loss=0.9080 aux_flow=1.2399 aux_pkt=0.9179
|
||||
[epoch 28/50 ] (6.1s) loss=0.9162 aux_flow=1.2700 aux_pkt=0.9129
|
||||
[epoch 29/50 ] (5.9s) loss=0.9037 aux_flow=1.2479 aux_pkt=0.9110
|
||||
[epoch 30/50 ] (67.0s) loss=0.9134 auroc_terminal=0.959 aux_flow=1.2686 aux_pkt=0.9155
|
||||
[epoch 31/50 ] (3.0s) loss=0.9049 aux_flow=1.2512 aux_pkt=0.9138
|
||||
[epoch 32/50 ] (3.8s) loss=0.9110 aux_flow=1.2720 aux_pkt=0.9133
|
||||
[epoch 33/50 ] (4.6s) loss=0.9011 aux_flow=1.2387 aux_pkt=0.9169
|
||||
[epoch 34/50 ] (6.0s) loss=0.9061 aux_flow=1.2695 aux_pkt=0.9149
|
||||
[epoch 35/50 ] (6.1s) loss=0.8893 aux_flow=1.2278 aux_pkt=0.9084
|
||||
[epoch 36/50 ] (6.0s) loss=0.8844 aux_flow=1.2182 aux_pkt=0.9060
|
||||
[epoch 37/50 ] (6.0s) loss=0.8820 aux_flow=1.2183 aux_pkt=0.9076
|
||||
[epoch 38/50 ] (6.0s) loss=0.8884 aux_flow=1.2248 aux_pkt=0.9118
|
||||
[epoch 39/50 ] (6.0s) loss=0.8901 aux_flow=1.2342 aux_pkt=0.9173
|
||||
[epoch 40/50 ] (66.1s) loss=0.8866 auroc_terminal=0.963 aux_flow=1.2305 aux_pkt=0.9071
|
||||
[epoch 41/50 ] (3.2s) loss=0.8884 aux_flow=1.2274 aux_pkt=0.9106
|
||||
[epoch 42/50 ] (4.3s) loss=0.8857 aux_flow=1.2260 aux_pkt=0.9116
|
||||
[epoch 43/50 ] (4.5s) loss=0.8710 aux_flow=1.1892 aux_pkt=0.9070
|
||||
[epoch 44/50 ] (6.0s) loss=0.8824 aux_flow=1.2141 aux_pkt=0.9124
|
||||
[epoch 45/50 ] (6.0s) loss=0.8768 aux_flow=1.2025 aux_pkt=0.9081
|
||||
[epoch 46/50 ] (6.0s) loss=0.8775 aux_flow=1.1961 aux_pkt=0.9122
|
||||
[epoch 47/50 ] (6.0s) loss=0.8779 aux_flow=1.2098 aux_pkt=0.9116
|
||||
[epoch 48/50 ] (6.0s) loss=0.8735 aux_flow=1.1931 aux_pkt=0.9098
|
||||
[epoch 49/50 ] (6.0s) loss=0.8781 aux_flow=1.2054 aux_pkt=0.9092
|
||||
[epoch 50/50 ] (65.3s) loss=0.8739 auroc_terminal=0.964 aux_flow=1.1932 aux_pkt=0.9092
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/baseline_ciciot2023_seed42/model.pt
|
||||
61
artifacts/route_comparison/baseline_seed43.log
Normal file
61
artifacts/route_comparison/baseline_seed43.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=43 data=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,226,261 token_dim=21 seq_len=65 sigma=0.1 use_ot=True reference_mode=None
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (6.5s) loss=2.2950 aux_flow=2.8636 aux_pkt=1.0202
|
||||
[epoch 2/50 ] (6.0s) loss=1.8686 aux_flow=2.3480 aux_pkt=0.9926
|
||||
[epoch 3/50 ] (6.0s) loss=1.5990 aux_flow=2.0891 aux_pkt=0.9587
|
||||
[epoch 4/50 ] (5.9s) loss=1.4314 aux_flow=1.9244 aux_pkt=0.9541
|
||||
[epoch 5/50 ] (6.0s) loss=1.3124 aux_flow=1.7769 aux_pkt=0.9525
|
||||
[epoch 6/50 ] (6.0s) loss=1.2334 aux_flow=1.7219 aux_pkt=0.9403
|
||||
[epoch 7/50 ] (4.9s) loss=1.1754 aux_flow=1.6386 aux_pkt=0.9388
|
||||
[epoch 8/50 ] (4.3s) loss=1.1245 aux_flow=1.5572 aux_pkt=0.9406
|
||||
[epoch 9/50 ] (4.3s) loss=1.0910 aux_flow=1.5038 aux_pkt=0.9323
|
||||
[epoch 10/50 ] (61.6s) loss=1.0686 auroc_terminal=0.943 aux_flow=1.4825 aux_pkt=0.9287
|
||||
[epoch 11/50 ] (5.5s) loss=1.0531 aux_flow=1.4595 aux_pkt=0.9286
|
||||
[epoch 12/50 ] (6.0s) loss=1.0392 aux_flow=1.4418 aux_pkt=0.9294
|
||||
[epoch 13/50 ] (5.9s) loss=1.0203 aux_flow=1.4019 aux_pkt=0.9300
|
||||
[epoch 14/50 ] (6.0s) loss=1.0114 aux_flow=1.3792 aux_pkt=0.9340
|
||||
[epoch 15/50 ] (6.0s) loss=1.0096 aux_flow=1.3923 aux_pkt=0.9287
|
||||
[epoch 16/50 ] (6.0s) loss=1.0078 aux_flow=1.4262 aux_pkt=0.9241
|
||||
[epoch 17/50 ] (6.0s) loss=0.9922 aux_flow=1.3857 aux_pkt=0.9235
|
||||
[epoch 18/50 ] (4.8s) loss=0.9932 aux_flow=1.3899 aux_pkt=0.9210
|
||||
[epoch 19/50 ] (4.5s) loss=0.9798 aux_flow=1.3516 aux_pkt=0.9195
|
||||
[epoch 20/50 ] (61.0s) loss=0.9674 auroc_terminal=0.953 aux_flow=1.3313 aux_pkt=0.9260
|
||||
[epoch 21/50 ] (4.7s) loss=0.9683 aux_flow=1.3487 aux_pkt=0.9208
|
||||
[epoch 22/50 ] (6.0s) loss=0.9630 aux_flow=1.3441 aux_pkt=0.9214
|
||||
[epoch 23/50 ] (6.0s) loss=0.9523 aux_flow=1.3144 aux_pkt=0.9227
|
||||
[epoch 24/50 ] (6.0s) loss=0.9441 aux_flow=1.2960 aux_pkt=0.9210
|
||||
[epoch 25/50 ] (6.0s) loss=0.9389 aux_flow=1.2984 aux_pkt=0.9188
|
||||
[epoch 26/50 ] (6.0s) loss=0.9517 aux_flow=1.3492 aux_pkt=0.9192
|
||||
[epoch 27/50 ] (5.9s) loss=0.9351 aux_flow=1.2965 aux_pkt=0.9173
|
||||
[epoch 28/50 ] (6.0s) loss=0.9299 aux_flow=1.2959 aux_pkt=0.9192
|
||||
[epoch 29/50 ] (4.4s) loss=0.9205 aux_flow=1.2822 aux_pkt=0.9168
|
||||
[epoch 30/50 ] (61.0s) loss=0.9184 auroc_terminal=0.954 aux_flow=1.2632 aux_pkt=0.9191
|
||||
[epoch 31/50 ] (4.3s) loss=0.9260 aux_flow=1.2890 aux_pkt=0.9184
|
||||
[epoch 32/50 ] (5.3s) loss=0.9211 aux_flow=1.2825 aux_pkt=0.9203
|
||||
[epoch 33/50 ] (5.9s) loss=0.9169 aux_flow=1.2665 aux_pkt=0.9199
|
||||
[epoch 34/50 ] (6.0s) loss=0.9252 aux_flow=1.2949 aux_pkt=0.9254
|
||||
[epoch 35/50 ] (6.0s) loss=0.9108 aux_flow=1.2644 aux_pkt=0.9169
|
||||
[epoch 36/50 ] (5.9s) loss=0.9040 aux_flow=1.2475 aux_pkt=0.9174
|
||||
[epoch 37/50 ] (6.0s) loss=0.9060 aux_flow=1.2480 aux_pkt=0.9182
|
||||
[epoch 38/50 ] (6.0s) loss=0.9034 aux_flow=1.2471 aux_pkt=0.9143
|
||||
[epoch 39/50 ] (5.4s) loss=0.9016 aux_flow=1.2337 aux_pkt=0.9205
|
||||
[epoch 40/50 ] (61.0s) loss=0.8977 auroc_terminal=0.958 aux_flow=1.2398 aux_pkt=0.9147
|
||||
[epoch 41/50 ] (4.3s) loss=0.8944 aux_flow=1.2394 aux_pkt=0.9147
|
||||
[epoch 42/50 ] (4.3s) loss=0.8916 aux_flow=1.2291 aux_pkt=0.9174
|
||||
[epoch 43/50 ] (5.7s) loss=0.8987 aux_flow=1.2367 aux_pkt=0.9171
|
||||
[epoch 44/50 ] (6.0s) loss=0.8854 aux_flow=1.2158 aux_pkt=0.9101
|
||||
[epoch 45/50 ] (6.0s) loss=0.8927 aux_flow=1.2282 aux_pkt=0.9127
|
||||
[epoch 46/50 ] (6.0s) loss=0.8910 aux_flow=1.2225 aux_pkt=0.9180
|
||||
[epoch 47/50 ] (6.0s) loss=0.8925 aux_flow=1.2352 aux_pkt=0.9123
|
||||
[epoch 48/50 ] (5.9s) loss=0.8876 aux_flow=1.2166 aux_pkt=0.9132
|
||||
[epoch 49/50 ] (6.0s) loss=0.8876 aux_flow=1.2258 aux_pkt=0.9081
|
||||
[epoch 50/50 ] (61.1s) loss=0.8882 auroc_terminal=0.960 aux_flow=1.2200 aux_pkt=0.9186
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/baseline_ciciot2023_seed43/model.pt
|
||||
61
artifacts/route_comparison/baseline_seed44.log
Normal file
61
artifacts/route_comparison/baseline_seed44.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=44 data=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,226,261 token_dim=21 seq_len=65 sigma=0.1 use_ot=True reference_mode=None
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (4.6s) loss=2.3754 aux_flow=3.0671 aux_pkt=1.0371
|
||||
[epoch 2/50 ] (3.9s) loss=1.9514 aux_flow=2.5465 aux_pkt=0.9975
|
||||
[epoch 3/50 ] (4.1s) loss=1.6782 aux_flow=2.2742 aux_pkt=0.9602
|
||||
[epoch 4/50 ] (4.0s) loss=1.5021 aux_flow=2.0847 aux_pkt=0.9582
|
||||
[epoch 5/50 ] (4.1s) loss=1.3728 aux_flow=1.9244 aux_pkt=0.9510
|
||||
[epoch 6/50 ] (4.0s) loss=1.2696 aux_flow=1.8004 aux_pkt=0.9442
|
||||
[epoch 7/50 ] (4.0s) loss=1.2203 aux_flow=1.7529 aux_pkt=0.9420
|
||||
[epoch 8/50 ] (3.9s) loss=1.1492 aux_flow=1.6252 aux_pkt=0.9396
|
||||
[epoch 9/50 ] (4.0s) loss=1.1424 aux_flow=1.6307 aux_pkt=0.9396
|
||||
[epoch 10/50 ] (39.7s) loss=1.0963 auroc_terminal=0.952 aux_flow=1.5450 aux_pkt=0.9282
|
||||
[epoch 11/50 ] (3.8s) loss=1.0864 aux_flow=1.5387 aux_pkt=0.9311
|
||||
[epoch 12/50 ] (3.9s) loss=1.0739 aux_flow=1.5157 aux_pkt=0.9392
|
||||
[epoch 13/50 ] (4.0s) loss=1.0486 aux_flow=1.4622 aux_pkt=0.9310
|
||||
[epoch 14/50 ] (3.9s) loss=1.0427 aux_flow=1.4598 aux_pkt=0.9299
|
||||
[epoch 15/50 ] (3.9s) loss=1.0284 aux_flow=1.4427 aux_pkt=0.9324
|
||||
[epoch 16/50 ] (3.9s) loss=1.0063 aux_flow=1.3971 aux_pkt=0.9304
|
||||
[epoch 17/50 ] (3.9s) loss=1.0274 aux_flow=1.4575 aux_pkt=0.9361
|
||||
[epoch 18/50 ] (4.0s) loss=0.9907 aux_flow=1.3769 aux_pkt=0.9310
|
||||
[epoch 19/50 ] (3.9s) loss=0.9896 aux_flow=1.3886 aux_pkt=0.9214
|
||||
[epoch 20/50 ] (39.5s) loss=0.9752 auroc_terminal=0.950 aux_flow=1.3546 aux_pkt=0.9228
|
||||
[epoch 21/50 ] (3.9s) loss=0.9695 aux_flow=1.3517 aux_pkt=0.9208
|
||||
[epoch 22/50 ] (4.1s) loss=0.9612 aux_flow=1.3437 aux_pkt=0.9179
|
||||
[epoch 23/50 ] (3.9s) loss=0.9662 aux_flow=1.3626 aux_pkt=0.9198
|
||||
[epoch 24/50 ] (3.9s) loss=0.9572 aux_flow=1.3308 aux_pkt=0.9202
|
||||
[epoch 25/50 ] (3.9s) loss=0.9416 aux_flow=1.3112 aux_pkt=0.9232
|
||||
[epoch 26/50 ] (3.9s) loss=0.9377 aux_flow=1.2940 aux_pkt=0.9218
|
||||
[epoch 27/50 ] (3.9s) loss=0.9386 aux_flow=1.2995 aux_pkt=0.9210
|
||||
[epoch 28/50 ] (3.9s) loss=0.9421 aux_flow=1.3222 aux_pkt=0.9225
|
||||
[epoch 29/50 ] (3.9s) loss=0.9332 aux_flow=1.2946 aux_pkt=0.9210
|
||||
[epoch 30/50 ] (39.5s) loss=0.9247 auroc_terminal=0.955 aux_flow=1.2880 aux_pkt=0.9146
|
||||
[epoch 31/50 ] (3.9s) loss=0.9301 aux_flow=1.2971 aux_pkt=0.9214
|
||||
[epoch 32/50 ] (4.0s) loss=0.9165 aux_flow=1.2809 aux_pkt=0.9139
|
||||
[epoch 33/50 ] (4.0s) loss=0.9202 aux_flow=1.2862 aux_pkt=0.9185
|
||||
[epoch 34/50 ] (4.1s) loss=0.9154 aux_flow=1.2710 aux_pkt=0.9152
|
||||
[epoch 35/50 ] (4.0s) loss=0.9058 aux_flow=1.2471 aux_pkt=0.9188
|
||||
[epoch 36/50 ] (4.0s) loss=0.9158 aux_flow=1.2780 aux_pkt=0.9210
|
||||
[epoch 37/50 ] (4.1s) loss=0.9052 aux_flow=1.2524 aux_pkt=0.9231
|
||||
[epoch 38/50 ] (4.0s) loss=0.9068 aux_flow=1.2526 aux_pkt=0.9238
|
||||
[epoch 39/50 ] (4.0s) loss=0.9058 aux_flow=1.2551 aux_pkt=0.9143
|
||||
[epoch 40/50 ] (39.8s) loss=0.9015 auroc_terminal=0.956 aux_flow=1.2396 aux_pkt=0.9243
|
||||
[epoch 41/50 ] (3.4s) loss=0.8986 aux_flow=1.2317 aux_pkt=0.9171
|
||||
[epoch 42/50 ] (4.0s) loss=0.8967 aux_flow=1.2348 aux_pkt=0.9194
|
||||
[epoch 43/50 ] (3.9s) loss=0.8985 aux_flow=1.2380 aux_pkt=0.9183
|
||||
[epoch 44/50 ] (3.9s) loss=0.8979 aux_flow=1.2323 aux_pkt=0.9215
|
||||
[epoch 45/50 ] (4.0s) loss=0.9030 aux_flow=1.2545 aux_pkt=0.9187
|
||||
[epoch 46/50 ] (3.9s) loss=0.9023 aux_flow=1.2471 aux_pkt=0.9242
|
||||
[epoch 47/50 ] (3.9s) loss=0.8925 aux_flow=1.2208 aux_pkt=0.9262
|
||||
[epoch 48/50 ] (4.0s) loss=0.8903 aux_flow=1.2234 aux_pkt=0.9141
|
||||
[epoch 49/50 ] (4.0s) loss=0.8955 aux_flow=1.2256 aux_pkt=0.9165
|
||||
[epoch 50/50 ] (40.0s) loss=0.9002 auroc_terminal=0.960 aux_flow=1.2373 aux_pkt=0.9226
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/baseline_ciciot2023_seed44/model.pt
|
||||
61
artifacts/route_comparison/route_a_causal_seed42.log
Normal file
61
artifacts/route_comparison/route_a_causal_seed42.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=42 data=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,226,261 token_dim=21 seq_len=65 sigma=0.1 use_ot=True reference_mode=causal_packets
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (2.6s) loss=2.2380 aux_flow=2.6923 aux_pkt=1.0203
|
||||
[epoch 2/50 ] (2.1s) loss=1.8172 aux_flow=2.2086 aux_pkt=0.9828
|
||||
[epoch 3/50 ] (2.2s) loss=1.5662 aux_flow=1.9835 aux_pkt=0.9600
|
||||
[epoch 4/50 ] (2.2s) loss=1.4073 aux_flow=1.8576 aux_pkt=0.9430
|
||||
[epoch 5/50 ] (2.2s) loss=1.2753 aux_flow=1.7117 aux_pkt=0.9369
|
||||
[epoch 6/50 ] (2.2s) loss=1.1937 aux_flow=1.6339 aux_pkt=0.9321
|
||||
[epoch 7/50 ] (2.1s) loss=1.1450 aux_flow=1.5880 aux_pkt=0.9273
|
||||
[epoch 8/50 ] (2.2s) loss=1.1089 aux_flow=1.5219 aux_pkt=0.9279
|
||||
[epoch 9/50 ] (2.1s) loss=1.0736 aux_flow=1.4709 aux_pkt=0.9299
|
||||
[epoch 10/50 ] (16.5s) loss=1.0578 auroc_terminal=0.955 aux_flow=1.4602 aux_pkt=0.9224
|
||||
[epoch 11/50 ] (2.1s) loss=1.0278 aux_flow=1.4116 aux_pkt=0.9229
|
||||
[epoch 12/50 ] (2.1s) loss=1.0097 aux_flow=1.3753 aux_pkt=0.9225
|
||||
[epoch 13/50 ] (2.1s) loss=1.0283 aux_flow=1.4326 aux_pkt=0.9229
|
||||
[epoch 14/50 ] (2.1s) loss=0.9791 aux_flow=1.3302 aux_pkt=0.9169
|
||||
[epoch 15/50 ] (2.1s) loss=0.9650 aux_flow=1.3061 aux_pkt=0.9191
|
||||
[epoch 16/50 ] (2.1s) loss=0.9617 aux_flow=1.3056 aux_pkt=0.9150
|
||||
[epoch 17/50 ] (2.1s) loss=0.9696 aux_flow=1.3364 aux_pkt=0.9165
|
||||
[epoch 18/50 ] (2.1s) loss=0.9715 aux_flow=1.3450 aux_pkt=0.9175
|
||||
[epoch 19/50 ] (2.1s) loss=0.9673 aux_flow=1.3524 aux_pkt=0.9220
|
||||
[epoch 20/50 ] (16.3s) loss=0.9409 auroc_terminal=0.959 aux_flow=1.2745 aux_pkt=0.9217
|
||||
[epoch 21/50 ] (2.1s) loss=0.9365 aux_flow=1.2878 aux_pkt=0.9114
|
||||
[epoch 22/50 ] (2.1s) loss=0.9301 aux_flow=1.2805 aux_pkt=0.9120
|
||||
[epoch 23/50 ] (2.1s) loss=0.9262 aux_flow=1.2736 aux_pkt=0.9136
|
||||
[epoch 24/50 ] (2.1s) loss=0.9245 aux_flow=1.2689 aux_pkt=0.9180
|
||||
[epoch 25/50 ] (2.1s) loss=0.9169 aux_flow=1.2569 aux_pkt=0.9155
|
||||
[epoch 26/50 ] (2.1s) loss=0.9217 aux_flow=1.2750 aux_pkt=0.9138
|
||||
[epoch 27/50 ] (2.1s) loss=0.9034 aux_flow=1.2295 aux_pkt=0.9182
|
||||
[epoch 28/50 ] (2.1s) loss=0.9068 aux_flow=1.2465 aux_pkt=0.9133
|
||||
[epoch 29/50 ] (2.1s) loss=0.9019 aux_flow=1.2457 aux_pkt=0.9116
|
||||
[epoch 30/50 ] (16.3s) loss=0.9012 auroc_terminal=0.951 aux_flow=1.2319 aux_pkt=0.9162
|
||||
[epoch 31/50 ] (2.1s) loss=0.9013 aux_flow=1.2417 aux_pkt=0.9145
|
||||
[epoch 32/50 ] (2.1s) loss=0.9059 aux_flow=1.2610 aux_pkt=0.9132
|
||||
[epoch 33/50 ] (2.1s) loss=0.8978 aux_flow=1.2296 aux_pkt=0.9174
|
||||
[epoch 34/50 ] (2.1s) loss=0.8993 aux_flow=1.2505 aux_pkt=0.9153
|
||||
[epoch 35/50 ] (2.1s) loss=0.8850 aux_flow=1.2160 aux_pkt=0.9091
|
||||
[epoch 36/50 ] (2.1s) loss=0.8795 aux_flow=1.2042 aux_pkt=0.9063
|
||||
[epoch 37/50 ] (2.1s) loss=0.8763 aux_flow=1.2019 aux_pkt=0.9084
|
||||
[epoch 38/50 ] (2.1s) loss=0.8860 aux_flow=1.2183 aux_pkt=0.9124
|
||||
[epoch 39/50 ] (2.1s) loss=0.8856 aux_flow=1.2206 aux_pkt=0.9178
|
||||
[epoch 40/50 ] (16.4s) loss=0.8829 auroc_terminal=0.966 aux_flow=1.2216 aux_pkt=0.9077
|
||||
[epoch 41/50 ] (2.2s) loss=0.8864 aux_flow=1.2224 aux_pkt=0.9110
|
||||
[epoch 42/50 ] (2.2s) loss=0.8806 aux_flow=1.2121 aux_pkt=0.9121
|
||||
[epoch 43/50 ] (2.2s) loss=0.8681 aux_flow=1.1815 aux_pkt=0.9076
|
||||
[epoch 44/50 ] (2.2s) loss=0.8800 aux_flow=1.2063 aux_pkt=0.9135
|
||||
[epoch 45/50 ] (2.2s) loss=0.8749 aux_flow=1.1974 aux_pkt=0.9087
|
||||
[epoch 46/50 ] (2.2s) loss=0.8747 aux_flow=1.1876 aux_pkt=0.9129
|
||||
[epoch 47/50 ] (2.1s) loss=0.8759 aux_flow=1.2051 aux_pkt=0.9121
|
||||
[epoch 48/50 ] (2.1s) loss=0.8707 aux_flow=1.1847 aux_pkt=0.9106
|
||||
[epoch 49/50 ] (2.1s) loss=0.8762 aux_flow=1.1999 aux_pkt=0.9097
|
||||
[epoch 50/50 ] (16.3s) loss=0.8713 auroc_terminal=0.965 aux_flow=1.1869 aux_pkt=0.9094
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_a_causal_ciciot2023_seed42/model.pt
|
||||
61
artifacts/route_comparison/route_a_causal_seed43.log
Normal file
61
artifacts/route_comparison/route_a_causal_seed43.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=43 data=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,226,261 token_dim=21 seq_len=65 sigma=0.1 use_ot=True reference_mode=causal_packets
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (6.6s) loss=2.2959 aux_flow=2.8675 aux_pkt=1.0198
|
||||
[epoch 2/50 ] (6.0s) loss=1.8685 aux_flow=2.3522 aux_pkt=0.9891
|
||||
[epoch 3/50 ] (6.2s) loss=1.5978 aux_flow=2.0900 aux_pkt=0.9588
|
||||
[epoch 4/50 ] (6.1s) loss=1.4301 aux_flow=1.9213 aux_pkt=0.9547
|
||||
[epoch 5/50 ] (6.1s) loss=1.3076 aux_flow=1.7716 aux_pkt=0.9504
|
||||
[epoch 6/50 ] (6.0s) loss=1.2276 aux_flow=1.7145 aux_pkt=0.9371
|
||||
[epoch 7/50 ] (4.7s) loss=1.1703 aux_flow=1.6318 aux_pkt=0.9364
|
||||
[epoch 8/50 ] (4.5s) loss=1.1194 aux_flow=1.5476 aux_pkt=0.9386
|
||||
[epoch 9/50 ] (4.5s) loss=1.0877 aux_flow=1.4994 aux_pkt=0.9305
|
||||
[epoch 10/50 ] (62.0s) loss=1.0637 auroc_terminal=0.947 aux_flow=1.4755 aux_pkt=0.9272
|
||||
[epoch 11/50 ] (6.2s) loss=1.0444 aux_flow=1.4389 aux_pkt=0.9275
|
||||
[epoch 12/50 ] (6.1s) loss=1.0405 aux_flow=1.4481 aux_pkt=0.9283
|
||||
[epoch 13/50 ] (6.0s) loss=1.0182 aux_flow=1.3953 aux_pkt=0.9292
|
||||
[epoch 14/50 ] (6.1s) loss=1.0272 aux_flow=1.4086 aux_pkt=0.9345
|
||||
[epoch 15/50 ] (6.0s) loss=1.0115 aux_flow=1.3953 aux_pkt=0.9279
|
||||
[epoch 16/50 ] (6.0s) loss=0.9934 aux_flow=1.3773 aux_pkt=0.9232
|
||||
[epoch 17/50 ] (5.9s) loss=0.9976 aux_flow=1.3996 aux_pkt=0.9226
|
||||
[epoch 18/50 ] (4.4s) loss=0.9984 aux_flow=1.4081 aux_pkt=0.9210
|
||||
[epoch 19/50 ] (4.4s) loss=0.9801 aux_flow=1.3475 aux_pkt=0.9192
|
||||
[epoch 20/50 ] (62.5s) loss=0.9614 auroc_terminal=0.958 aux_flow=1.3169 aux_pkt=0.9251
|
||||
[epoch 21/50 ] (6.0s) loss=0.9617 aux_flow=1.3308 aux_pkt=0.9201
|
||||
[epoch 22/50 ] (6.2s) loss=0.9594 aux_flow=1.3368 aux_pkt=0.9207
|
||||
[epoch 23/50 ] (6.2s) loss=0.9529 aux_flow=1.3144 aux_pkt=0.9228
|
||||
[epoch 24/50 ] (6.1s) loss=0.9419 aux_flow=1.2857 aux_pkt=0.9203
|
||||
[epoch 25/50 ] (6.0s) loss=0.9423 aux_flow=1.3042 aux_pkt=0.9192
|
||||
[epoch 26/50 ] (6.1s) loss=0.9366 aux_flow=1.3046 aux_pkt=0.9189
|
||||
[epoch 27/50 ] (6.2s) loss=0.9281 aux_flow=1.2809 aux_pkt=0.9168
|
||||
[epoch 28/50 ] (4.8s) loss=0.9284 aux_flow=1.2921 aux_pkt=0.9189
|
||||
[epoch 29/50 ] (4.4s) loss=0.9183 aux_flow=1.2753 aux_pkt=0.9168
|
||||
[epoch 30/50 ] (63.0s) loss=0.9140 auroc_terminal=0.949 aux_flow=1.2496 aux_pkt=0.9194
|
||||
[epoch 31/50 ] (6.2s) loss=0.9256 aux_flow=1.2875 aux_pkt=0.9186
|
||||
[epoch 32/50 ] (6.0s) loss=0.9190 aux_flow=1.2749 aux_pkt=0.9204
|
||||
[epoch 33/50 ] (6.0s) loss=0.9111 aux_flow=1.2474 aux_pkt=0.9200
|
||||
[epoch 34/50 ] (6.0s) loss=0.9224 aux_flow=1.2890 aux_pkt=0.9254
|
||||
[epoch 35/50 ] (6.0s) loss=0.9126 aux_flow=1.2701 aux_pkt=0.9168
|
||||
[epoch 36/50 ] (6.0s) loss=0.9046 aux_flow=1.2480 aux_pkt=0.9178
|
||||
[epoch 37/50 ] (6.0s) loss=0.9037 aux_flow=1.2394 aux_pkt=0.9186
|
||||
[epoch 38/50 ] (4.6s) loss=0.9031 aux_flow=1.2446 aux_pkt=0.9147
|
||||
[epoch 39/50 ] (3.7s) loss=0.9007 aux_flow=1.2302 aux_pkt=0.9203
|
||||
[epoch 40/50 ] (64.2s) loss=0.8960 auroc_terminal=0.963 aux_flow=1.2332 aux_pkt=0.9150
|
||||
[epoch 41/50 ] (6.1s) loss=0.8942 aux_flow=1.2376 aux_pkt=0.9147
|
||||
[epoch 42/50 ] (6.0s) loss=0.8919 aux_flow=1.2276 aux_pkt=0.9181
|
||||
[epoch 43/50 ] (6.0s) loss=0.8982 aux_flow=1.2339 aux_pkt=0.9173
|
||||
[epoch 44/50 ] (6.0s) loss=0.8850 aux_flow=1.2133 aux_pkt=0.9105
|
||||
[epoch 45/50 ] (6.0s) loss=0.8926 aux_flow=1.2262 aux_pkt=0.9129
|
||||
[epoch 46/50 ] (6.1s) loss=0.8909 aux_flow=1.2210 aux_pkt=0.9181
|
||||
[epoch 47/50 ] (6.0s) loss=0.8919 aux_flow=1.2334 aux_pkt=0.9124
|
||||
[epoch 48/50 ] (4.3s) loss=0.8868 aux_flow=1.2138 aux_pkt=0.9134
|
||||
[epoch 49/50 ] (3.0s) loss=0.8867 aux_flow=1.2226 aux_pkt=0.9084
|
||||
[epoch 50/50 ] (53.7s) loss=0.8878 auroc_terminal=0.963 aux_flow=1.2178 aux_pkt=0.9188
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_a_causal_ciciot2023_seed43/model.pt
|
||||
61
artifacts/route_comparison/route_a_causal_seed44.log
Normal file
61
artifacts/route_comparison/route_a_causal_seed44.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=44 data=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=20
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,226,261 token_dim=21 seq_len=65 sigma=0.1 use_ot=True reference_mode=causal_packets
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (4.7s) loss=2.3750 aux_flow=3.0676 aux_pkt=1.0370
|
||||
[epoch 2/50 ] (4.0s) loss=1.9489 aux_flow=2.5473 aux_pkt=0.9941
|
||||
[epoch 3/50 ] (4.2s) loss=1.6761 aux_flow=2.2727 aux_pkt=0.9608
|
||||
[epoch 4/50 ] (4.2s) loss=1.5044 aux_flow=2.0865 aux_pkt=0.9600
|
||||
[epoch 5/50 ] (4.0s) loss=1.3762 aux_flow=1.9236 aux_pkt=0.9518
|
||||
[epoch 6/50 ] (4.2s) loss=1.2737 aux_flow=1.8012 aux_pkt=0.9448
|
||||
[epoch 7/50 ] (4.1s) loss=1.2216 aux_flow=1.7500 aux_pkt=0.9405
|
||||
[epoch 8/50 ] (4.1s) loss=1.1493 aux_flow=1.6228 aux_pkt=0.9387
|
||||
[epoch 9/50 ] (4.1s) loss=1.1219 aux_flow=1.5798 aux_pkt=0.9383
|
||||
[epoch 10/50 ] (39.1s) loss=1.0879 auroc_terminal=0.947 aux_flow=1.5309 aux_pkt=0.9272
|
||||
[epoch 11/50 ] (4.0s) loss=1.0890 aux_flow=1.5261 aux_pkt=0.9321
|
||||
[epoch 12/50 ] (4.0s) loss=1.0708 aux_flow=1.5040 aux_pkt=0.9397
|
||||
[epoch 13/50 ] (4.0s) loss=1.0345 aux_flow=1.4267 aux_pkt=0.9298
|
||||
[epoch 14/50 ] (3.9s) loss=1.0338 aux_flow=1.4521 aux_pkt=0.9289
|
||||
[epoch 15/50 ] (3.9s) loss=1.0246 aux_flow=1.4387 aux_pkt=0.9316
|
||||
[epoch 16/50 ] (3.9s) loss=1.0059 aux_flow=1.3998 aux_pkt=0.9301
|
||||
[epoch 17/50 ] (4.0s) loss=1.0186 aux_flow=1.4382 aux_pkt=0.9363
|
||||
[epoch 18/50 ] (4.1s) loss=0.9842 aux_flow=1.3565 aux_pkt=0.9313
|
||||
[epoch 19/50 ] (4.0s) loss=0.9716 aux_flow=1.3373 aux_pkt=0.9212
|
||||
[epoch 20/50 ] (38.9s) loss=0.9684 auroc_terminal=0.954 aux_flow=1.3399 aux_pkt=0.9230
|
||||
[epoch 21/50 ] (4.0s) loss=0.9674 aux_flow=1.3455 aux_pkt=0.9214
|
||||
[epoch 22/50 ] (4.2s) loss=0.9620 aux_flow=1.3472 aux_pkt=0.9187
|
||||
[epoch 23/50 ] (4.0s) loss=0.9594 aux_flow=1.3501 aux_pkt=0.9199
|
||||
[epoch 24/50 ] (3.9s) loss=0.9626 aux_flow=1.3495 aux_pkt=0.9213
|
||||
[epoch 25/50 ] (3.9s) loss=0.9356 aux_flow=1.2959 aux_pkt=0.9235
|
||||
[epoch 26/50 ] (4.0s) loss=0.9334 aux_flow=1.2857 aux_pkt=0.9222
|
||||
[epoch 27/50 ] (4.0s) loss=0.9385 aux_flow=1.2991 aux_pkt=0.9215
|
||||
[epoch 28/50 ] (4.0s) loss=0.9381 aux_flow=1.3131 aux_pkt=0.9231
|
||||
[epoch 29/50 ] (3.9s) loss=0.9329 aux_flow=1.2921 aux_pkt=0.9215
|
||||
[epoch 30/50 ] (38.9s) loss=0.9211 auroc_terminal=0.960 aux_flow=1.2794 aux_pkt=0.9148
|
||||
[epoch 31/50 ] (4.0s) loss=0.9249 aux_flow=1.2804 aux_pkt=0.9218
|
||||
[epoch 32/50 ] (4.2s) loss=0.9136 aux_flow=1.2724 aux_pkt=0.9143
|
||||
[epoch 33/50 ] (4.1s) loss=0.9095 aux_flow=1.2530 aux_pkt=0.9194
|
||||
[epoch 34/50 ] (4.1s) loss=0.9063 aux_flow=1.2465 aux_pkt=0.9158
|
||||
[epoch 35/50 ] (4.1s) loss=0.9018 aux_flow=1.2373 aux_pkt=0.9193
|
||||
[epoch 36/50 ] (4.2s) loss=0.9110 aux_flow=1.2656 aux_pkt=0.9214
|
||||
[epoch 37/50 ] (4.2s) loss=0.9033 aux_flow=1.2478 aux_pkt=0.9238
|
||||
[epoch 38/50 ] (4.1s) loss=0.9051 aux_flow=1.2477 aux_pkt=0.9244
|
||||
[epoch 39/50 ] (4.1s) loss=0.9041 aux_flow=1.2505 aux_pkt=0.9149
|
||||
[epoch 40/50 ] (39.4s) loss=0.8997 auroc_terminal=0.961 aux_flow=1.2348 aux_pkt=0.9246
|
||||
[epoch 41/50 ] (4.0s) loss=0.8969 aux_flow=1.2279 aux_pkt=0.9180
|
||||
[epoch 42/50 ] (3.9s) loss=0.8951 aux_flow=1.2318 aux_pkt=0.9196
|
||||
[epoch 43/50 ] (3.9s) loss=0.8981 aux_flow=1.2364 aux_pkt=0.9190
|
||||
[epoch 44/50 ] (4.1s) loss=0.8958 aux_flow=1.2263 aux_pkt=0.9220
|
||||
[epoch 45/50 ] (3.9s) loss=0.9015 aux_flow=1.2515 aux_pkt=0.9192
|
||||
[epoch 46/50 ] (3.9s) loss=0.9003 aux_flow=1.2421 aux_pkt=0.9244
|
||||
[epoch 47/50 ] (4.0s) loss=0.8903 aux_flow=1.2153 aux_pkt=0.9264
|
||||
[epoch 48/50 ] (4.1s) loss=0.8877 aux_flow=1.2175 aux_pkt=0.9142
|
||||
[epoch 49/50 ] (4.2s) loss=0.8928 aux_flow=1.2186 aux_pkt=0.9169
|
||||
[epoch 50/50 ] (38.3s) loss=0.8974 auroc_terminal=0.964 aux_flow=1.2308 aux_pkt=0.9225
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_a_causal_ciciot2023_seed44/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:42/data:42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] kept 8,986,875 of 8,993,376 (min_len=2)
|
||||
[data] train=74,565 val=18,642 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=74,565 val=18,642 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.2s) loss=1.2867 L_disc=0.4776
|
||||
[epoch 2/50 ] (1.6s) loss=0.9260 L_disc=0.3648
|
||||
[epoch 3/50 ] (1.6s) loss=0.7255 L_disc=0.3391
|
||||
[epoch 4/50 ] (1.6s) loss=0.6166 L_disc=0.3174
|
||||
[epoch 5/50 ] (1.6s) loss=0.5367 L_disc=0.2993
|
||||
[epoch 6/50 ] (1.4s) loss=0.4895 L_disc=0.2824
|
||||
[epoch 7/50 ] (1.5s) loss=0.4529 L_disc=0.2657
|
||||
[epoch 8/50 ] (1.7s) loss=0.4244 L_disc=0.2501
|
||||
[epoch 9/50 ] (1.6s) loss=0.3960 L_disc=0.2316
|
||||
[epoch 10/50 ] (5.3s) loss=0.3785 auroc_term=0.989 auroc_disc=0.615 L_disc=0.2210
|
||||
[epoch 11/50 ] (1.7s) loss=0.3633 L_disc=0.2144
|
||||
[epoch 12/50 ] (1.7s) loss=0.3543 L_disc=0.2076
|
||||
[epoch 13/50 ] (1.6s) loss=0.3473 L_disc=0.2045
|
||||
[epoch 14/50 ] (1.7s) loss=0.3343 L_disc=0.1972
|
||||
[epoch 15/50 ] (1.5s) loss=0.3288 L_disc=0.1944
|
||||
[epoch 16/50 ] (1.4s) loss=0.3252 L_disc=0.1947
|
||||
[epoch 17/50 ] (1.5s) loss=0.3161 L_disc=0.1869
|
||||
[epoch 18/50 ] (1.7s) loss=0.3169 L_disc=0.1886
|
||||
[epoch 19/50 ] (1.7s) loss=0.3122 L_disc=0.1870
|
||||
[epoch 20/50 ] (5.2s) loss=0.3064 auroc_term=0.995 auroc_disc=0.537 L_disc=0.1839
|
||||
[epoch 21/50 ] (1.7s) loss=0.3052 L_disc=0.1816
|
||||
[epoch 22/50 ] (1.6s) loss=0.3031 L_disc=0.1823
|
||||
[epoch 23/50 ] (1.6s) loss=0.2957 L_disc=0.1774
|
||||
[epoch 24/50 ] (1.6s) loss=0.2939 L_disc=0.1758
|
||||
[epoch 25/50 ] (1.5s) loss=0.2907 L_disc=0.1753
|
||||
[epoch 26/50 ] (1.5s) loss=0.2924 L_disc=0.1752
|
||||
[epoch 27/50 ] (1.7s) loss=0.2846 L_disc=0.1712
|
||||
[epoch 28/50 ] (1.6s) loss=0.2877 L_disc=0.1729
|
||||
[epoch 29/50 ] (1.7s) loss=0.2821 L_disc=0.1711
|
||||
[epoch 30/50 ] (5.3s) loss=0.2816 auroc_term=0.996 auroc_disc=0.530 L_disc=0.1708
|
||||
[epoch 31/50 ] (1.6s) loss=0.2805 L_disc=0.1691
|
||||
[epoch 32/50 ] (1.7s) loss=0.2784 L_disc=0.1689
|
||||
[epoch 33/50 ] (1.6s) loss=0.2756 L_disc=0.1673
|
||||
[epoch 34/50 ] (1.4s) loss=0.2718 L_disc=0.1650
|
||||
[epoch 35/50 ] (1.5s) loss=0.2772 L_disc=0.1674
|
||||
[epoch 36/50 ] (1.6s) loss=0.2718 L_disc=0.1652
|
||||
[epoch 37/50 ] (1.7s) loss=0.2726 L_disc=0.1653
|
||||
[epoch 38/50 ] (1.6s) loss=0.2738 L_disc=0.1654
|
||||
[epoch 39/50 ] (1.6s) loss=0.2707 L_disc=0.1646
|
||||
[epoch 40/50 ] (5.2s) loss=0.2712 auroc_term=0.996 auroc_disc=0.531 L_disc=0.1644
|
||||
[epoch 41/50 ] (1.6s) loss=0.2676 L_disc=0.1622
|
||||
[epoch 42/50 ] (1.5s) loss=0.2703 L_disc=0.1658
|
||||
[epoch 43/50 ] (1.5s) loss=0.2672 L_disc=0.1621
|
||||
[epoch 44/50 ] (1.5s) loss=0.2665 L_disc=0.1633
|
||||
[epoch 45/50 ] (1.4s) loss=0.2674 L_disc=0.1640
|
||||
[epoch 46/50 ] (1.4s) loss=0.2681 L_disc=0.1633
|
||||
[epoch 47/50 ] (1.3s) loss=0.2658 L_disc=0.1613
|
||||
[epoch 48/50 ] (1.4s) loss=0.2658 L_disc=0.1619
|
||||
[epoch 49/50 ] (1.4s) loss=0.2669 L_disc=0.1626
|
||||
[epoch 50/50 ] (5.0s) loss=0.2660 auroc_term=0.997 auroc_disc=0.527 L_disc=0.1611
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_cicddos2019_seed42/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:43/data:43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] kept 8,986,875 of 8,993,376 (min_len=2)
|
||||
[data] train=74,565 val=18,642 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=74,565 val=18,642 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.2s) loss=1.2920 L_disc=0.4804
|
||||
[epoch 2/50 ] (1.6s) loss=0.9200 L_disc=0.3650
|
||||
[epoch 3/50 ] (1.6s) loss=0.7199 L_disc=0.3403
|
||||
[epoch 4/50 ] (1.6s) loss=0.6052 L_disc=0.3151
|
||||
[epoch 5/50 ] (1.6s) loss=0.5306 L_disc=0.2963
|
||||
[epoch 6/50 ] (1.6s) loss=0.4815 L_disc=0.2798
|
||||
[epoch 7/50 ] (1.6s) loss=0.4433 L_disc=0.2602
|
||||
[epoch 8/50 ] (1.7s) loss=0.4084 L_disc=0.2369
|
||||
[epoch 9/50 ] (1.6s) loss=0.3860 L_disc=0.2224
|
||||
[epoch 10/50 ] (5.1s) loss=0.3686 auroc_term=0.995 auroc_disc=0.758 L_disc=0.2134
|
||||
[epoch 11/50 ] (1.5s) loss=0.3569 L_disc=0.2050
|
||||
[epoch 12/50 ] (1.6s) loss=0.3445 L_disc=0.1977
|
||||
[epoch 13/50 ] (1.5s) loss=0.3348 L_disc=0.1928
|
||||
[epoch 14/50 ] (1.5s) loss=0.3227 L_disc=0.1892
|
||||
[epoch 15/50 ] (1.6s) loss=0.3224 L_disc=0.1885
|
||||
[epoch 16/50 ] (1.5s) loss=0.3217 L_disc=0.1881
|
||||
[epoch 17/50 ] (1.7s) loss=0.3131 L_disc=0.1827
|
||||
[epoch 18/50 ] (1.6s) loss=0.3080 L_disc=0.1806
|
||||
[epoch 19/50 ] (1.6s) loss=0.3051 L_disc=0.1799
|
||||
[epoch 20/50 ] (5.1s) loss=0.3001 auroc_term=0.996 auroc_disc=0.658 L_disc=0.1755
|
||||
[epoch 21/50 ] (1.5s) loss=0.2930 L_disc=0.1721
|
||||
[epoch 22/50 ] (1.7s) loss=0.2942 L_disc=0.1731
|
||||
[epoch 23/50 ] (1.7s) loss=0.2915 L_disc=0.1727
|
||||
[epoch 24/50 ] (1.6s) loss=0.2849 L_disc=0.1684
|
||||
[epoch 25/50 ] (1.6s) loss=0.2844 L_disc=0.1689
|
||||
[epoch 26/50 ] (1.6s) loss=0.2818 L_disc=0.1664
|
||||
[epoch 27/50 ] (1.7s) loss=0.2816 L_disc=0.1683
|
||||
[epoch 28/50 ] (1.6s) loss=0.2796 L_disc=0.1669
|
||||
[epoch 29/50 ] (1.5s) loss=0.2764 L_disc=0.1649
|
||||
[epoch 30/50 ] (5.1s) loss=0.2758 auroc_term=0.997 auroc_disc=0.643 L_disc=0.1656
|
||||
[epoch 31/50 ] (1.5s) loss=0.2749 L_disc=0.1639
|
||||
[epoch 32/50 ] (1.6s) loss=0.2750 L_disc=0.1658
|
||||
[epoch 33/50 ] (1.5s) loss=0.2738 L_disc=0.1632
|
||||
[epoch 34/50 ] (1.6s) loss=0.2719 L_disc=0.1620
|
||||
[epoch 35/50 ] (1.6s) loss=0.2716 L_disc=0.1621
|
||||
[epoch 36/50 ] (1.6s) loss=0.2685 L_disc=0.1619
|
||||
[epoch 37/50 ] (1.5s) loss=0.2675 L_disc=0.1607
|
||||
[epoch 38/50 ] (1.6s) loss=0.2636 L_disc=0.1583
|
||||
[epoch 39/50 ] (1.5s) loss=0.2641 L_disc=0.1592
|
||||
[epoch 40/50 ] (5.1s) loss=0.2653 auroc_term=0.998 auroc_disc=0.609 L_disc=0.1611
|
||||
[epoch 41/50 ] (1.4s) loss=0.2640 L_disc=0.1588
|
||||
[epoch 42/50 ] (1.4s) loss=0.2637 L_disc=0.1590
|
||||
[epoch 43/50 ] (1.4s) loss=0.2632 L_disc=0.1592
|
||||
[epoch 44/50 ] (1.4s) loss=0.2650 L_disc=0.1604
|
||||
[epoch 45/50 ] (1.5s) loss=0.2628 L_disc=0.1589
|
||||
[epoch 46/50 ] (1.6s) loss=0.2620 L_disc=0.1583
|
||||
[epoch 47/50 ] (1.6s) loss=0.2628 L_disc=0.1601
|
||||
[epoch 48/50 ] (1.7s) loss=0.2652 L_disc=0.1603
|
||||
[epoch 49/50 ] (1.6s) loss=0.2632 L_disc=0.1587
|
||||
[epoch 50/50 ] (5.2s) loss=0.2614 auroc_term=0.998 auroc_disc=0.620 L_disc=0.1572
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_cicddos2019_seed43/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:44/data:44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/cicddos2019/processed/full_store
|
||||
[data] kept 8,986,875 of 8,993,376 (min_len=2)
|
||||
[data] train=74,565 val=18,642 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=74,565 val=18,642 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.3s) loss=1.2951 L_disc=0.4836
|
||||
[epoch 2/50 ] (1.6s) loss=0.9292 L_disc=0.3686
|
||||
[epoch 3/50 ] (1.6s) loss=0.7342 L_disc=0.3448
|
||||
[epoch 4/50 ] (1.5s) loss=0.6189 L_disc=0.3213
|
||||
[epoch 5/50 ] (1.4s) loss=0.5469 L_disc=0.3075
|
||||
[epoch 6/50 ] (1.6s) loss=0.5009 L_disc=0.2914
|
||||
[epoch 7/50 ] (1.6s) loss=0.4686 L_disc=0.2754
|
||||
[epoch 8/50 ] (1.5s) loss=0.4422 L_disc=0.2644
|
||||
[epoch 9/50 ] (1.6s) loss=0.4251 L_disc=0.2514
|
||||
[epoch 10/50 ] (5.3s) loss=0.4005 auroc_term=0.983 auroc_disc=0.368 L_disc=0.2362
|
||||
[epoch 11/50 ] (1.6s) loss=0.3802 L_disc=0.2219
|
||||
[epoch 12/50 ] (1.6s) loss=0.3649 L_disc=0.2145
|
||||
[epoch 13/50 ] (1.5s) loss=0.3584 L_disc=0.2113
|
||||
[epoch 14/50 ] (1.4s) loss=0.3491 L_disc=0.2053
|
||||
[epoch 15/50 ] (1.6s) loss=0.3367 L_disc=0.2004
|
||||
[epoch 16/50 ] (1.6s) loss=0.3318 L_disc=0.1953
|
||||
[epoch 17/50 ] (1.6s) loss=0.3295 L_disc=0.1955
|
||||
[epoch 18/50 ] (1.6s) loss=0.3238 L_disc=0.1926
|
||||
[epoch 19/50 ] (1.5s) loss=0.3208 L_disc=0.1914
|
||||
[epoch 20/50 ] (5.3s) loss=0.3097 auroc_term=0.988 auroc_disc=0.518 L_disc=0.1835
|
||||
[epoch 21/50 ] (1.6s) loss=0.3120 L_disc=0.1854
|
||||
[epoch 22/50 ] (1.7s) loss=0.3069 L_disc=0.1825
|
||||
[epoch 23/50 ] (1.5s) loss=0.3065 L_disc=0.1829
|
||||
[epoch 24/50 ] (1.6s) loss=0.2998 L_disc=0.1775
|
||||
[epoch 25/50 ] (1.8s) loss=0.2991 L_disc=0.1790
|
||||
[epoch 26/50 ] (1.8s) loss=0.2959 L_disc=0.1784
|
||||
[epoch 27/50 ] (1.8s) loss=0.2926 L_disc=0.1751
|
||||
[epoch 28/50 ] (1.6s) loss=0.2944 L_disc=0.1762
|
||||
[epoch 29/50 ] (1.6s) loss=0.2895 L_disc=0.1742
|
||||
[epoch 30/50 ] (5.3s) loss=0.2874 auroc_term=0.994 auroc_disc=0.553 L_disc=0.1723
|
||||
[epoch 31/50 ] (1.6s) loss=0.2861 L_disc=0.1711
|
||||
[epoch 32/50 ] (1.5s) loss=0.2854 L_disc=0.1717
|
||||
[epoch 33/50 ] (1.4s) loss=0.2822 L_disc=0.1699
|
||||
[epoch 34/50 ] (1.4s) loss=0.2818 L_disc=0.1710
|
||||
[epoch 35/50 ] (1.4s) loss=0.2818 L_disc=0.1710
|
||||
[epoch 36/50 ] (1.4s) loss=0.2824 L_disc=0.1708
|
||||
[epoch 37/50 ] (1.4s) loss=0.2779 L_disc=0.1684
|
||||
[epoch 38/50 ] (1.4s) loss=0.2777 L_disc=0.1681
|
||||
[epoch 39/50 ] (1.4s) loss=0.2749 L_disc=0.1653
|
||||
[epoch 40/50 ] (4.9s) loss=0.2787 auroc_term=0.995 auroc_disc=0.515 L_disc=0.1683
|
||||
[epoch 41/50 ] (1.4s) loss=0.2740 L_disc=0.1655
|
||||
[epoch 42/50 ] (1.4s) loss=0.2731 L_disc=0.1675
|
||||
[epoch 43/50 ] (1.4s) loss=0.2747 L_disc=0.1665
|
||||
[epoch 44/50 ] (1.4s) loss=0.2724 L_disc=0.1646
|
||||
[epoch 45/50 ] (1.3s) loss=0.2724 L_disc=0.1652
|
||||
[epoch 46/50 ] (1.4s) loss=0.2739 L_disc=0.1663
|
||||
[epoch 47/50 ] (1.3s) loss=0.2764 L_disc=0.1668
|
||||
[epoch 48/50 ] (1.4s) loss=0.2719 L_disc=0.1652
|
||||
[epoch 49/50 ] (1.4s) loss=0.2739 L_disc=0.1667
|
||||
[epoch 50/50 ] (5.0s) loss=0.2721 auroc_term=0.996 auroc_disc=0.523 L_disc=0.1643
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_cicddos2019_seed44/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:42/data:42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] kept 2,017,180 of 2,025,564 (min_len=2)
|
||||
[data] train=1,210,760 val=302,690 attack=503,730
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=1,210,760 val=302,690 attack=503,730
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.1s) loss=1.2840 L_disc=0.4660
|
||||
[epoch 2/50 ] (1.4s) loss=0.9156 L_disc=0.3417
|
||||
[epoch 3/50 ] (1.4s) loss=0.7043 L_disc=0.3158
|
||||
[epoch 4/50 ] (1.3s) loss=0.5913 L_disc=0.2964
|
||||
[epoch 5/50 ] (1.0s) loss=0.5248 L_disc=0.2786
|
||||
[epoch 6/50 ] (1.1s) loss=0.4805 L_disc=0.2646
|
||||
[epoch 7/50 ] (1.0s) loss=0.4503 L_disc=0.2524
|
||||
[epoch 8/50 ] (1.2s) loss=0.4252 L_disc=0.2396
|
||||
[epoch 9/50 ] (1.3s) loss=0.4064 L_disc=0.2290
|
||||
[epoch 10/50 ] (8.5s) loss=0.3940 auroc_term=0.966 auroc_disc=0.963 L_disc=0.2189
|
||||
[epoch 11/50 ] (1.4s) loss=0.3775 L_disc=0.2118
|
||||
[epoch 12/50 ] (1.3s) loss=0.3669 L_disc=0.2051
|
||||
[epoch 13/50 ] (1.3s) loss=0.3600 L_disc=0.2020
|
||||
[epoch 14/50 ] (1.3s) loss=0.3533 L_disc=0.1978
|
||||
[epoch 15/50 ] (1.4s) loss=0.3465 L_disc=0.1963
|
||||
[epoch 16/50 ] (1.3s) loss=0.3442 L_disc=0.1956
|
||||
[epoch 17/50 ] (1.4s) loss=0.3384 L_disc=0.1910
|
||||
[epoch 18/50 ] (1.4s) loss=0.3324 L_disc=0.1894
|
||||
[epoch 19/50 ] (1.4s) loss=0.3266 L_disc=0.1868
|
||||
[epoch 20/50 ] (8.5s) loss=0.3227 auroc_term=0.970 auroc_disc=0.982 L_disc=0.1839
|
||||
[epoch 21/50 ] (1.4s) loss=0.3232 L_disc=0.1853
|
||||
[epoch 22/50 ] (1.4s) loss=0.3175 L_disc=0.1824
|
||||
[epoch 23/50 ] (1.4s) loss=0.3149 L_disc=0.1809
|
||||
[epoch 24/50 ] (1.4s) loss=0.3087 L_disc=0.1761
|
||||
[epoch 25/50 ] (1.4s) loss=0.3071 L_disc=0.1761
|
||||
[epoch 26/50 ] (1.3s) loss=0.3058 L_disc=0.1736
|
||||
[epoch 27/50 ] (1.3s) loss=0.2993 L_disc=0.1708
|
||||
[epoch 28/50 ] (1.3s) loss=0.3011 L_disc=0.1725
|
||||
[epoch 29/50 ] (1.3s) loss=0.3013 L_disc=0.1726
|
||||
[epoch 30/50 ] (8.4s) loss=0.2981 auroc_term=0.983 auroc_disc=0.983 L_disc=0.1702
|
||||
[epoch 31/50 ] (1.4s) loss=0.2978 L_disc=0.1703
|
||||
[epoch 32/50 ] (1.4s) loss=0.2943 L_disc=0.1691
|
||||
[epoch 33/50 ] (1.4s) loss=0.2940 L_disc=0.1691
|
||||
[epoch 34/50 ] (1.4s) loss=0.2892 L_disc=0.1659
|
||||
[epoch 35/50 ] (1.4s) loss=0.2921 L_disc=0.1694
|
||||
[epoch 36/50 ] (1.4s) loss=0.2902 L_disc=0.1677
|
||||
[epoch 37/50 ] (1.4s) loss=0.2890 L_disc=0.1669
|
||||
[epoch 38/50 ] (1.4s) loss=0.2916 L_disc=0.1669
|
||||
[epoch 39/50 ] (1.4s) loss=0.2838 L_disc=0.1629
|
||||
[epoch 40/50 ] (8.5s) loss=0.2891 auroc_term=0.984 auroc_disc=0.983 L_disc=0.1666
|
||||
[epoch 41/50 ] (1.4s) loss=0.2848 L_disc=0.1637
|
||||
[epoch 42/50 ] (1.3s) loss=0.2846 L_disc=0.1646
|
||||
[epoch 43/50 ] (1.4s) loss=0.2851 L_disc=0.1645
|
||||
[epoch 44/50 ] (1.4s) loss=0.2827 L_disc=0.1632
|
||||
[epoch 45/50 ] (1.4s) loss=0.2839 L_disc=0.1646
|
||||
[epoch 46/50 ] (1.4s) loss=0.2827 L_disc=0.1629
|
||||
[epoch 47/50 ] (1.4s) loss=0.2830 L_disc=0.1629
|
||||
[epoch 48/50 ] (1.3s) loss=0.2850 L_disc=0.1645
|
||||
[epoch 49/50 ] (1.3s) loss=0.2828 L_disc=0.1631
|
||||
[epoch 50/50 ] (8.5s) loss=0.2831 auroc_term=0.986 auroc_disc=0.983 L_disc=0.1633
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_cicids2017_seed42/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:43/data:43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] kept 2,017,180 of 2,025,564 (min_len=2)
|
||||
[data] train=1,210,760 val=302,690 attack=503,730
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=1,210,760 val=302,690 attack=503,730
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.1s) loss=1.2874 L_disc=0.4639
|
||||
[epoch 2/50 ] (1.5s) loss=0.9137 L_disc=0.3415
|
||||
[epoch 3/50 ] (1.5s) loss=0.7024 L_disc=0.3152
|
||||
[epoch 4/50 ] (1.4s) loss=0.5849 L_disc=0.2928
|
||||
[epoch 5/50 ] (1.4s) loss=0.5184 L_disc=0.2772
|
||||
[epoch 6/50 ] (1.4s) loss=0.4747 L_disc=0.2611
|
||||
[epoch 7/50 ] (1.4s) loss=0.4455 L_disc=0.2492
|
||||
[epoch 8/50 ] (1.4s) loss=0.4191 L_disc=0.2335
|
||||
[epoch 9/50 ] (1.4s) loss=0.3992 L_disc=0.2227
|
||||
[epoch 10/50 ] (8.4s) loss=0.3849 auroc_term=0.981 auroc_disc=0.953 L_disc=0.2147
|
||||
[epoch 11/50 ] (1.4s) loss=0.3738 L_disc=0.2083
|
||||
[epoch 12/50 ] (1.4s) loss=0.3642 L_disc=0.2025
|
||||
[epoch 13/50 ] (1.4s) loss=0.3566 L_disc=0.1983
|
||||
[epoch 14/50 ] (1.4s) loss=0.3498 L_disc=0.1971
|
||||
[epoch 15/50 ] (1.4s) loss=0.3385 L_disc=0.1900
|
||||
[epoch 16/50 ] (1.4s) loss=0.3364 L_disc=0.1894
|
||||
[epoch 17/50 ] (1.4s) loss=0.3330 L_disc=0.1871
|
||||
[epoch 18/50 ] (1.4s) loss=0.3278 L_disc=0.1855
|
||||
[epoch 19/50 ] (1.4s) loss=0.3268 L_disc=0.1841
|
||||
[epoch 20/50 ] (8.5s) loss=0.3200 auroc_term=0.979 auroc_disc=0.982 L_disc=0.1817
|
||||
[epoch 21/50 ] (1.4s) loss=0.3171 L_disc=0.1797
|
||||
[epoch 22/50 ] (1.4s) loss=0.3147 L_disc=0.1788
|
||||
[epoch 23/50 ] (1.4s) loss=0.3111 L_disc=0.1772
|
||||
[epoch 24/50 ] (1.4s) loss=0.3071 L_disc=0.1755
|
||||
[epoch 25/50 ] (1.4s) loss=0.3070 L_disc=0.1756
|
||||
[epoch 26/50 ] (1.3s) loss=0.3058 L_disc=0.1732
|
||||
[epoch 27/50 ] (1.4s) loss=0.3013 L_disc=0.1725
|
||||
[epoch 28/50 ] (1.4s) loss=0.3001 L_disc=0.1713
|
||||
[epoch 29/50 ] (1.4s) loss=0.2986 L_disc=0.1706
|
||||
[epoch 30/50 ] (8.5s) loss=0.2953 auroc_term=0.976 auroc_disc=0.981 L_disc=0.1693
|
||||
[epoch 31/50 ] (1.4s) loss=0.2936 L_disc=0.1672
|
||||
[epoch 32/50 ] (1.4s) loss=0.2931 L_disc=0.1686
|
||||
[epoch 33/50 ] (1.4s) loss=0.2927 L_disc=0.1674
|
||||
[epoch 34/50 ] (1.4s) loss=0.2896 L_disc=0.1642
|
||||
[epoch 35/50 ] (1.4s) loss=0.2885 L_disc=0.1650
|
||||
[epoch 36/50 ] (1.4s) loss=0.2869 L_disc=0.1653
|
||||
[epoch 37/50 ] (1.4s) loss=0.2869 L_disc=0.1650
|
||||
[epoch 38/50 ] (1.4s) loss=0.2849 L_disc=0.1626
|
||||
[epoch 39/50 ] (1.4s) loss=0.2895 L_disc=0.1678
|
||||
[epoch 40/50 ] (8.5s) loss=0.2858 auroc_term=0.984 auroc_disc=0.983 L_disc=0.1661
|
||||
[epoch 41/50 ] (1.4s) loss=0.2848 L_disc=0.1628
|
||||
[epoch 42/50 ] (1.4s) loss=0.2844 L_disc=0.1637
|
||||
[epoch 43/50 ] (1.4s) loss=0.2844 L_disc=0.1627
|
||||
[epoch 44/50 ] (1.4s) loss=0.2824 L_disc=0.1630
|
||||
[epoch 45/50 ] (1.4s) loss=0.2812 L_disc=0.1615
|
||||
[epoch 46/50 ] (1.4s) loss=0.2827 L_disc=0.1634
|
||||
[epoch 47/50 ] (1.4s) loss=0.2805 L_disc=0.1621
|
||||
[epoch 48/50 ] (1.4s) loss=0.2835 L_disc=0.1639
|
||||
[epoch 49/50 ] (1.4s) loss=0.2845 L_disc=0.1642
|
||||
[epoch 50/50 ] (8.6s) loss=0.2812 auroc_term=0.986 auroc_disc=0.983 L_disc=0.1620
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_cicids2017_seed43/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:44/data:44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/cicids2017/processed/packets.npz
|
||||
[data] kept 2,017,180 of 2,025,564 (min_len=2)
|
||||
[data] train=1,210,760 val=302,690 attack=503,730
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=1,210,760 val=302,690 attack=503,730
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.0s) loss=1.2839 L_disc=0.4620
|
||||
[epoch 2/50 ] (1.5s) loss=0.9119 L_disc=0.3367
|
||||
[epoch 3/50 ] (1.4s) loss=0.7023 L_disc=0.3103
|
||||
[epoch 4/50 ] (1.4s) loss=0.5846 L_disc=0.2888
|
||||
[epoch 5/50 ] (1.4s) loss=0.5216 L_disc=0.2747
|
||||
[epoch 6/50 ] (1.4s) loss=0.4807 L_disc=0.2620
|
||||
[epoch 7/50 ] (1.4s) loss=0.4525 L_disc=0.2490
|
||||
[epoch 8/50 ] (1.4s) loss=0.4253 L_disc=0.2373
|
||||
[epoch 9/50 ] (1.4s) loss=0.4102 L_disc=0.2275
|
||||
[epoch 10/50 ] (8.5s) loss=0.3861 auroc_term=0.898 auroc_disc=0.955 L_disc=0.2143
|
||||
[epoch 11/50 ] (1.4s) loss=0.3707 L_disc=0.2056
|
||||
[epoch 12/50 ] (1.4s) loss=0.3645 L_disc=0.2043
|
||||
[epoch 13/50 ] (1.4s) loss=0.3519 L_disc=0.1962
|
||||
[epoch 14/50 ] (1.4s) loss=0.3469 L_disc=0.1932
|
||||
[epoch 15/50 ] (1.3s) loss=0.3366 L_disc=0.1879
|
||||
[epoch 16/50 ] (1.3s) loss=0.3355 L_disc=0.1874
|
||||
[epoch 17/50 ] (1.4s) loss=0.3300 L_disc=0.1841
|
||||
[epoch 18/50 ] (1.4s) loss=0.3254 L_disc=0.1831
|
||||
[epoch 19/50 ] (1.4s) loss=0.3216 L_disc=0.1812
|
||||
[epoch 20/50 ] (8.5s) loss=0.3170 auroc_term=0.969 auroc_disc=0.976 L_disc=0.1795
|
||||
[epoch 21/50 ] (1.4s) loss=0.3158 L_disc=0.1778
|
||||
[epoch 22/50 ] (1.4s) loss=0.3138 L_disc=0.1785
|
||||
[epoch 23/50 ] (1.4s) loss=0.3140 L_disc=0.1780
|
||||
[epoch 24/50 ] (1.4s) loss=0.3077 L_disc=0.1732
|
||||
[epoch 25/50 ] (1.4s) loss=0.3016 L_disc=0.1702
|
||||
[epoch 26/50 ] (1.4s) loss=0.3023 L_disc=0.1712
|
||||
[epoch 27/50 ] (1.4s) loss=0.2995 L_disc=0.1695
|
||||
[epoch 28/50 ] (1.4s) loss=0.2985 L_disc=0.1686
|
||||
[epoch 29/50 ] (1.4s) loss=0.2936 L_disc=0.1662
|
||||
[epoch 30/50 ] (8.4s) loss=0.2926 auroc_term=0.981 auroc_disc=0.987 L_disc=0.1666
|
||||
[epoch 31/50 ] (1.4s) loss=0.2948 L_disc=0.1666
|
||||
[epoch 32/50 ] (1.4s) loss=0.2898 L_disc=0.1646
|
||||
[epoch 33/50 ] (1.3s) loss=0.2892 L_disc=0.1650
|
||||
[epoch 34/50 ] (1.4s) loss=0.2872 L_disc=0.1636
|
||||
[epoch 35/50 ] (1.4s) loss=0.2839 L_disc=0.1617
|
||||
[epoch 36/50 ] (1.4s) loss=0.2871 L_disc=0.1628
|
||||
[epoch 37/50 ] (1.4s) loss=0.2843 L_disc=0.1626
|
||||
[epoch 38/50 ] (1.4s) loss=0.2840 L_disc=0.1627
|
||||
[epoch 39/50 ] (1.4s) loss=0.2833 L_disc=0.1615
|
||||
[epoch 40/50 ] (8.5s) loss=0.2839 auroc_term=0.988 auroc_disc=0.987 L_disc=0.1618
|
||||
[epoch 41/50 ] (1.4s) loss=0.2826 L_disc=0.1624
|
||||
[epoch 42/50 ] (1.4s) loss=0.2779 L_disc=0.1589
|
||||
[epoch 43/50 ] (1.4s) loss=0.2800 L_disc=0.1590
|
||||
[epoch 44/50 ] (1.4s) loss=0.2821 L_disc=0.1622
|
||||
[epoch 45/50 ] (1.4s) loss=0.2813 L_disc=0.1600
|
||||
[epoch 46/50 ] (1.4s) loss=0.2798 L_disc=0.1596
|
||||
[epoch 47/50 ] (1.4s) loss=0.2821 L_disc=0.1610
|
||||
[epoch 48/50 ] (1.4s) loss=0.2813 L_disc=0.1614
|
||||
[epoch 49/50 ] (1.4s) loss=0.2805 L_disc=0.1596
|
||||
[epoch 50/50 ] (8.5s) loss=0.2803 auroc_term=0.989 auroc_disc=0.985 L_disc=0.1597
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_cicids2017_seed44/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:42/data:42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] kept 66,189 of 103,079 (min_len=2)
|
||||
[data] train=51,901 val=12,976 attack=1,312
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=51,901 val=12,976 attack=1,312
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.0s) loss=1.3093 L_disc=0.4591
|
||||
[epoch 2/50 ] (1.4s) loss=0.9363 L_disc=0.3326
|
||||
[epoch 3/50 ] (1.4s) loss=0.7199 L_disc=0.3112
|
||||
[epoch 4/50 ] (1.4s) loss=0.6218 L_disc=0.2963
|
||||
[epoch 5/50 ] (1.4s) loss=0.5451 L_disc=0.2822
|
||||
[epoch 6/50 ] (1.5s) loss=0.4874 L_disc=0.2658
|
||||
[epoch 7/50 ] (1.6s) loss=0.4532 L_disc=0.2530
|
||||
[epoch 8/50 ] (1.7s) loss=0.4271 L_disc=0.2401
|
||||
[epoch 9/50 ] (1.7s) loss=0.3999 L_disc=0.2297
|
||||
[epoch 10/50 ] (4.3s) loss=0.3917 auroc_term=0.943 auroc_disc=0.574 L_disc=0.2261
|
||||
[epoch 11/50 ] (1.6s) loss=0.3770 L_disc=0.2223
|
||||
[epoch 12/50 ] (1.7s) loss=0.3680 L_disc=0.2170
|
||||
[epoch 13/50 ] (1.7s) loss=0.3591 L_disc=0.2115
|
||||
[epoch 14/50 ] (1.5s) loss=0.3536 L_disc=0.2094
|
||||
[epoch 15/50 ] (1.5s) loss=0.3421 L_disc=0.2028
|
||||
[epoch 16/50 ] (1.4s) loss=0.3306 L_disc=0.1964
|
||||
[epoch 17/50 ] (1.8s) loss=0.3257 L_disc=0.1906
|
||||
[epoch 18/50 ] (1.7s) loss=0.3177 L_disc=0.1893
|
||||
[epoch 19/50 ] (1.6s) loss=0.3141 L_disc=0.1869
|
||||
[epoch 20/50 ] (4.3s) loss=0.3126 auroc_term=0.989 auroc_disc=0.554 L_disc=0.1860
|
||||
[epoch 21/50 ] (1.7s) loss=0.3086 L_disc=0.1840
|
||||
[epoch 22/50 ] (1.7s) loss=0.3031 L_disc=0.1802
|
||||
[epoch 23/50 ] (1.6s) loss=0.3005 L_disc=0.1805
|
||||
[epoch 24/50 ] (1.7s) loss=0.3003 L_disc=0.1793
|
||||
[epoch 25/50 ] (1.4s) loss=0.2926 L_disc=0.1753
|
||||
[epoch 26/50 ] (1.4s) loss=0.2918 L_disc=0.1738
|
||||
[epoch 27/50 ] (1.6s) loss=0.2914 L_disc=0.1754
|
||||
[epoch 28/50 ] (1.6s) loss=0.2879 L_disc=0.1721
|
||||
[epoch 29/50 ] (1.6s) loss=0.2866 L_disc=0.1733
|
||||
[epoch 30/50 ] (4.2s) loss=0.2815 auroc_term=0.994 auroc_disc=0.675 L_disc=0.1697
|
||||
[epoch 31/50 ] (1.6s) loss=0.2784 L_disc=0.1665
|
||||
[epoch 32/50 ] (1.6s) loss=0.2781 L_disc=0.1669
|
||||
[epoch 33/50 ] (1.5s) loss=0.2776 L_disc=0.1677
|
||||
[epoch 34/50 ] (1.6s) loss=0.2750 L_disc=0.1662
|
||||
[epoch 35/50 ] (1.5s) loss=0.2752 L_disc=0.1665
|
||||
[epoch 36/50 ] (1.4s) loss=0.2715 L_disc=0.1645
|
||||
[epoch 37/50 ] (1.4s) loss=0.2727 L_disc=0.1652
|
||||
[epoch 38/50 ] (1.6s) loss=0.2721 L_disc=0.1634
|
||||
[epoch 39/50 ] (1.6s) loss=0.2712 L_disc=0.1646
|
||||
[epoch 40/50 ] (4.3s) loss=0.2721 auroc_term=0.994 auroc_disc=0.636 L_disc=0.1640
|
||||
[epoch 41/50 ] (1.5s) loss=0.2694 L_disc=0.1625
|
||||
[epoch 42/50 ] (1.7s) loss=0.2693 L_disc=0.1642
|
||||
[epoch 43/50 ] (1.6s) loss=0.2700 L_disc=0.1642
|
||||
[epoch 44/50 ] (1.7s) loss=0.2657 L_disc=0.1610
|
||||
[epoch 45/50 ] (1.6s) loss=0.2632 L_disc=0.1597
|
||||
[epoch 46/50 ] (1.5s) loss=0.2672 L_disc=0.1624
|
||||
[epoch 47/50 ] (1.5s) loss=0.2695 L_disc=0.1639
|
||||
[epoch 48/50 ] (1.5s) loss=0.2683 L_disc=0.1628
|
||||
[epoch 49/50 ] (1.5s) loss=0.2660 L_disc=0.1610
|
||||
[epoch 50/50 ] (4.3s) loss=0.2662 auroc_term=0.995 auroc_disc=0.680 L_disc=0.1615
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_iscxtor2016_seed42/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:43/data:43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] kept 66,189 of 103,079 (min_len=2)
|
||||
[data] train=51,901 val=12,976 attack=1,312
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=51,901 val=12,976 attack=1,312
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.1s) loss=1.3076 L_disc=0.4549
|
||||
[epoch 2/50 ] (1.4s) loss=0.9314 L_disc=0.3308
|
||||
[epoch 3/50 ] (1.4s) loss=0.7183 L_disc=0.3095
|
||||
[epoch 4/50 ] (1.4s) loss=0.6044 L_disc=0.2891
|
||||
[epoch 5/50 ] (1.4s) loss=0.5339 L_disc=0.2786
|
||||
[epoch 6/50 ] (1.4s) loss=0.4813 L_disc=0.2643
|
||||
[epoch 7/50 ] (1.3s) loss=0.4508 L_disc=0.2532
|
||||
[epoch 8/50 ] (1.3s) loss=0.4191 L_disc=0.2387
|
||||
[epoch 9/50 ] (1.3s) loss=0.4026 L_disc=0.2304
|
||||
[epoch 10/50 ] (3.9s) loss=0.3872 auroc_term=0.981 auroc_disc=0.724 L_disc=0.2259
|
||||
[epoch 11/50 ] (1.6s) loss=0.3806 L_disc=0.2222
|
||||
[epoch 12/50 ] (1.7s) loss=0.3655 L_disc=0.2160
|
||||
[epoch 13/50 ] (1.6s) loss=0.3576 L_disc=0.2128
|
||||
[epoch 14/50 ] (1.6s) loss=0.3509 L_disc=0.2083
|
||||
[epoch 15/50 ] (1.6s) loss=0.3391 L_disc=0.2016
|
||||
[epoch 16/50 ] (1.7s) loss=0.3316 L_disc=0.1961
|
||||
[epoch 17/50 ] (1.5s) loss=0.3235 L_disc=0.1911
|
||||
[epoch 18/50 ] (1.6s) loss=0.3195 L_disc=0.1887
|
||||
[epoch 19/50 ] (1.5s) loss=0.3225 L_disc=0.1894
|
||||
[epoch 20/50 ] (4.0s) loss=0.3111 auroc_term=0.977 auroc_disc=0.657 L_disc=0.1855
|
||||
[epoch 21/50 ] (1.4s) loss=0.3063 L_disc=0.1822
|
||||
[epoch 22/50 ] (1.6s) loss=0.3012 L_disc=0.1783
|
||||
[epoch 23/50 ] (1.5s) loss=0.2992 L_disc=0.1790
|
||||
[epoch 24/50 ] (1.6s) loss=0.2934 L_disc=0.1749
|
||||
[epoch 25/50 ] (1.5s) loss=0.2903 L_disc=0.1734
|
||||
[epoch 26/50 ] (1.6s) loss=0.2881 L_disc=0.1700
|
||||
[epoch 27/50 ] (1.6s) loss=0.2846 L_disc=0.1699
|
||||
[epoch 28/50 ] (1.7s) loss=0.2823 L_disc=0.1686
|
||||
[epoch 29/50 ] (1.6s) loss=0.2834 L_disc=0.1701
|
||||
[epoch 30/50 ] (4.2s) loss=0.2815 auroc_term=0.993 auroc_disc=0.672 L_disc=0.1688
|
||||
[epoch 31/50 ] (1.4s) loss=0.2765 L_disc=0.1652
|
||||
[epoch 32/50 ] (1.5s) loss=0.2752 L_disc=0.1656
|
||||
[epoch 33/50 ] (1.6s) loss=0.2800 L_disc=0.1677
|
||||
[epoch 34/50 ] (1.7s) loss=0.2748 L_disc=0.1647
|
||||
[epoch 35/50 ] (1.5s) loss=0.2779 L_disc=0.1664
|
||||
[epoch 36/50 ] (1.6s) loss=0.2712 L_disc=0.1634
|
||||
[epoch 37/50 ] (1.7s) loss=0.2705 L_disc=0.1630
|
||||
[epoch 38/50 ] (1.7s) loss=0.2712 L_disc=0.1638
|
||||
[epoch 39/50 ] (1.6s) loss=0.2708 L_disc=0.1648
|
||||
[epoch 40/50 ] (4.3s) loss=0.2663 auroc_term=0.995 auroc_disc=0.704 L_disc=0.1610
|
||||
[epoch 41/50 ] (1.5s) loss=0.2672 L_disc=0.1605
|
||||
[epoch 42/50 ] (1.4s) loss=0.2668 L_disc=0.1612
|
||||
[epoch 43/50 ] (1.6s) loss=0.2668 L_disc=0.1614
|
||||
[epoch 44/50 ] (1.6s) loss=0.2647 L_disc=0.1609
|
||||
[epoch 45/50 ] (1.5s) loss=0.2656 L_disc=0.1599
|
||||
[epoch 46/50 ] (1.6s) loss=0.2675 L_disc=0.1617
|
||||
[epoch 47/50 ] (1.6s) loss=0.2650 L_disc=0.1609
|
||||
[epoch 48/50 ] (1.6s) loss=0.2685 L_disc=0.1620
|
||||
[epoch 49/50 ] (1.7s) loss=0.2645 L_disc=0.1595
|
||||
[epoch 50/50 ] (4.3s) loss=0.2637 auroc_term=0.995 auroc_disc=0.711 L_disc=0.1591
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_iscxtor2016_seed43/model.pt
|
||||
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:44/data:44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/iscxtor2016/processed/packets.npz
|
||||
[data] kept 66,189 of 103,079 (min_len=2)
|
||||
[data] train=51,901 val=12,976 attack=1,312
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=51,901 val=12,976 attack=1,312
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.3s) loss=1.3142 L_disc=0.4635
|
||||
[epoch 2/50 ] (1.8s) loss=0.9383 L_disc=0.3372
|
||||
[epoch 3/50 ] (1.6s) loss=0.7307 L_disc=0.3152
|
||||
[epoch 4/50 ] (1.7s) loss=0.6097 L_disc=0.2950
|
||||
[epoch 5/50 ] (1.6s) loss=0.5325 L_disc=0.2807
|
||||
[epoch 6/50 ] (1.5s) loss=0.4878 L_disc=0.2681
|
||||
[epoch 7/50 ] (1.4s) loss=0.4522 L_disc=0.2530
|
||||
[epoch 8/50 ] (1.4s) loss=0.4242 L_disc=0.2416
|
||||
[epoch 9/50 ] (1.4s) loss=0.4055 L_disc=0.2331
|
||||
[epoch 10/50 ] (4.0s) loss=0.3890 auroc_term=0.971 auroc_disc=0.707 L_disc=0.2272
|
||||
[epoch 11/50 ] (1.4s) loss=0.3812 L_disc=0.2257
|
||||
[epoch 12/50 ] (1.4s) loss=0.3717 L_disc=0.2220
|
||||
[epoch 13/50 ] (1.3s) loss=0.3673 L_disc=0.2204
|
||||
[epoch 14/50 ] (1.4s) loss=0.3583 L_disc=0.2168
|
||||
[epoch 15/50 ] (1.3s) loss=0.3469 L_disc=0.2117
|
||||
[epoch 16/50 ] (1.4s) loss=0.3403 L_disc=0.2052
|
||||
[epoch 17/50 ] (1.4s) loss=0.3404 L_disc=0.2045
|
||||
[epoch 18/50 ] (1.6s) loss=0.3251 L_disc=0.1952
|
||||
[epoch 19/50 ] (1.5s) loss=0.3150 L_disc=0.1887
|
||||
[epoch 20/50 ] (4.2s) loss=0.3099 auroc_term=0.989 auroc_disc=0.632 L_disc=0.1861
|
||||
[epoch 21/50 ] (1.6s) loss=0.3074 L_disc=0.1849
|
||||
[epoch 22/50 ] (1.7s) loss=0.3036 L_disc=0.1830
|
||||
[epoch 23/50 ] (1.7s) loss=0.3009 L_disc=0.1808
|
||||
[epoch 24/50 ] (1.6s) loss=0.2952 L_disc=0.1772
|
||||
[epoch 25/50 ] (1.7s) loss=0.2957 L_disc=0.1780
|
||||
[epoch 26/50 ] (1.4s) loss=0.2919 L_disc=0.1757
|
||||
[epoch 27/50 ] (1.4s) loss=0.2887 L_disc=0.1746
|
||||
[epoch 28/50 ] (1.6s) loss=0.2880 L_disc=0.1734
|
||||
[epoch 29/50 ] (1.6s) loss=0.2872 L_disc=0.1740
|
||||
[epoch 30/50 ] (4.2s) loss=0.2805 auroc_term=0.995 auroc_disc=0.657 L_disc=0.1691
|
||||
[epoch 31/50 ] (1.6s) loss=0.2818 L_disc=0.1691
|
||||
[epoch 32/50 ] (1.6s) loss=0.2783 L_disc=0.1678
|
||||
[epoch 33/50 ] (1.6s) loss=0.2778 L_disc=0.1675
|
||||
[epoch 34/50 ] (1.6s) loss=0.2763 L_disc=0.1666
|
||||
[epoch 35/50 ] (1.6s) loss=0.2775 L_disc=0.1688
|
||||
[epoch 36/50 ] (1.6s) loss=0.2731 L_disc=0.1653
|
||||
[epoch 37/50 ] (1.4s) loss=0.2723 L_disc=0.1641
|
||||
[epoch 38/50 ] (1.5s) loss=0.2693 L_disc=0.1628
|
||||
[epoch 39/50 ] (1.7s) loss=0.2719 L_disc=0.1646
|
||||
[epoch 40/50 ] (4.4s) loss=0.2703 auroc_term=0.995 auroc_disc=0.719 L_disc=0.1626
|
||||
[epoch 41/50 ] (1.8s) loss=0.2685 L_disc=0.1628
|
||||
[epoch 42/50 ] (1.9s) loss=0.2651 L_disc=0.1618
|
||||
[epoch 43/50 ] (1.7s) loss=0.2697 L_disc=0.1637
|
||||
[epoch 44/50 ] (1.6s) loss=0.2664 L_disc=0.1609
|
||||
[epoch 45/50 ] (1.5s) loss=0.2664 L_disc=0.1615
|
||||
[epoch 46/50 ] (1.7s) loss=0.2684 L_disc=0.1628
|
||||
[epoch 47/50 ] (1.5s) loss=0.2679 L_disc=0.1615
|
||||
[epoch 48/50 ] (1.4s) loss=0.2685 L_disc=0.1628
|
||||
[epoch 49/50 ] (1.5s) loss=0.2653 L_disc=0.1602
|
||||
[epoch 50/50 ] (4.2s) loss=0.2685 auroc_term=0.996 auroc_disc=0.728 L_disc=0.1622
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_iscxtor2016_seed44/model.pt
|
||||
58
artifacts/route_comparison/route_ac_combo_seed42.log
Normal file
58
artifacts/route_comparison/route_ac_combo_seed42.log
Normal file
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:42/data:42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] kept 3,797,530 of 8,193,621 (min_len=2)
|
||||
[data] train=77,636 val=10,000 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.0s) loss=1.3451 L_disc=0.5037
|
||||
[epoch 2/50 ] (1.7s) loss=1.0190 L_disc=0.4033
|
||||
[epoch 3/50 ] (2.3s) loss=0.8192 L_disc=0.3748
|
||||
[epoch 4/50 ] (2.5s) loss=0.6682 L_disc=0.3369
|
||||
[epoch 5/50 ] (2.5s) loss=0.5787 L_disc=0.3063
|
||||
[epoch 6/50 ] (2.5s) loss=0.5261 L_disc=0.2864
|
||||
[epoch 7/50 ] (2.4s) loss=0.4914 L_disc=0.2750
|
||||
[epoch 8/50 ] (2.8s) loss=0.4579 L_disc=0.2578
|
||||
[epoch 9/50 ] (2.6s) loss=0.4391 L_disc=0.2489
|
||||
[epoch 10/50 ] (13.2s) loss=0.4235 auroc_term=0.938 auroc_disc=0.811 L_disc=0.2427
|
||||
[epoch 11/50 ] (2.6s) loss=0.4009 L_disc=0.2303
|
||||
[epoch 12/50 ] (2.8s) loss=0.3990 L_disc=0.2305
|
||||
[epoch 13/50 ] (2.5s) loss=0.3869 L_disc=0.2205
|
||||
[epoch 14/50 ] (2.4s) loss=0.3739 L_disc=0.2157
|
||||
[epoch 15/50 ] (2.5s) loss=0.3571 L_disc=0.2069
|
||||
[epoch 16/50 ] (2.5s) loss=0.3455 L_disc=0.1988
|
||||
[epoch 17/50 ] (2.3s) loss=0.3432 L_disc=0.1970
|
||||
[epoch 18/50 ] (2.2s) loss=0.3434 L_disc=0.1999
|
||||
[epoch 19/50 ] (2.3s) loss=0.3390 L_disc=0.2005
|
||||
[epoch 20/50 ] (13.2s) loss=0.3335 auroc_term=0.950 auroc_disc=0.876 L_disc=0.1945
|
||||
[epoch 21/50 ] (2.6s) loss=0.3291 L_disc=0.1914
|
||||
[epoch 22/50 ] (2.4s) loss=0.3173 L_disc=0.1845
|
||||
[epoch 23/50 ] (2.0s) loss=0.3131 L_disc=0.1830
|
||||
[epoch 24/50 ] (2.3s) loss=0.3120 L_disc=0.1823
|
||||
[epoch 25/50 ] (2.6s) loss=0.3085 L_disc=0.1809
|
||||
[epoch 26/50 ] (2.6s) loss=0.3031 L_disc=0.1764
|
||||
[epoch 27/50 ] (2.6s) loss=0.3081 L_disc=0.1815
|
||||
[epoch 28/50 ] (2.5s) loss=0.3035 L_disc=0.1770
|
||||
[epoch 29/50 ] (2.5s) loss=0.2966 L_disc=0.1741
|
||||
[epoch 30/50 ] (13.5s) loss=0.2969 auroc_term=0.957 auroc_disc=0.908 L_disc=0.1747
|
||||
[epoch 31/50 ] (2.5s) loss=0.2938 L_disc=0.1712
|
||||
[epoch 32/50 ] (2.5s) loss=0.2877 L_disc=0.1672
|
||||
[epoch 33/50 ] (2.3s) loss=0.2951 L_disc=0.1749
|
||||
[epoch 34/50 ] (2.3s) loss=0.2890 L_disc=0.1693
|
||||
[epoch 35/50 ] (2.4s) loss=0.2911 L_disc=0.1720
|
||||
[epoch 36/50 ] (2.4s) loss=0.2823 L_disc=0.1660
|
||||
[epoch 37/50 ] (2.4s) loss=0.2804 L_disc=0.1642
|
||||
[epoch 38/50 ] (2.4s) loss=0.2808 L_disc=0.1623
|
||||
[epoch 39/50 ] (2.3s) loss=0.2834 L_disc=0.1685
|
||||
[epoch 40/50 ] (13.3s) loss=0.2775 auroc_term=0.961 auroc_disc=0.912 L_disc=0.1621
|
||||
[epoch 41/50 ] (2.1s) loss=0.2806 L_disc=0.1651
|
||||
[epoch 42/50 ] (2.1s) loss=0.2768 L_disc=0.1640
|
||||
[epoch 43/50 ] (2.1s) loss=0.2789 L_disc=0.1655
|
||||
[epoch 44/50 ] (2.0s) loss=0.2731 L_disc=0.1607
|
||||
[epoch 45/50 ] (2.0s) loss=0.2733 L_disc=0.1604
|
||||
[epoch 46/50 ] (2.1s) loss=0.2753 L_disc=0.1624
|
||||
[epoch 47/50 ] (2.0s) loss=0.2779 L_disc=0.1657
|
||||
[epoch 48/50 ] (2.2s) loss=0.2782 L_disc=0.1654
|
||||
[epoch 49/50 ] (2.1s) loss=0.2804 L_disc=0.1686
|
||||
[epoch 50/50 ] (13.3s) loss=0.2731 auroc_term=0.961 auroc_disc=0.914 L_disc=0.1614
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_ciciot2023_seed42/model.pt
|
||||
58
artifacts/route_comparison/route_ac_combo_seed43.log
Normal file
58
artifacts/route_comparison/route_ac_combo_seed43.log
Normal file
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:43/data:43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] kept 3,797,530 of 8,193,621 (min_len=2)
|
||||
[data] train=77,636 val=10,000 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.8s) loss=1.3526 L_disc=0.5016
|
||||
[epoch 2/50 ] (2.5s) loss=1.0229 L_disc=0.4022
|
||||
[epoch 3/50 ] (2.6s) loss=0.8147 L_disc=0.3781
|
||||
[epoch 4/50 ] (2.4s) loss=0.6833 L_disc=0.3436
|
||||
[epoch 5/50 ] (2.3s) loss=0.5983 L_disc=0.3159
|
||||
[epoch 6/50 ] (2.6s) loss=0.5383 L_disc=0.2944
|
||||
[epoch 7/50 ] (2.4s) loss=0.5056 L_disc=0.2836
|
||||
[epoch 8/50 ] (2.7s) loss=0.4754 L_disc=0.2685
|
||||
[epoch 9/50 ] (2.5s) loss=0.4531 L_disc=0.2576
|
||||
[epoch 10/50 ] (12.9s) loss=0.4327 auroc_term=0.947 auroc_disc=0.861 L_disc=0.2499
|
||||
[epoch 11/50 ] (2.7s) loss=0.4261 L_disc=0.2436
|
||||
[epoch 12/50 ] (2.7s) loss=0.3999 L_disc=0.2284
|
||||
[epoch 13/50 ] (2.3s) loss=0.3958 L_disc=0.2303
|
||||
[epoch 14/50 ] (2.6s) loss=0.3744 L_disc=0.2158
|
||||
[epoch 15/50 ] (2.4s) loss=0.3720 L_disc=0.2130
|
||||
[epoch 16/50 ] (2.4s) loss=0.3664 L_disc=0.2117
|
||||
[epoch 17/50 ] (2.1s) loss=0.3570 L_disc=0.2073
|
||||
[epoch 18/50 ] (2.3s) loss=0.3501 L_disc=0.2032
|
||||
[epoch 19/50 ] (2.4s) loss=0.3436 L_disc=0.2000
|
||||
[epoch 20/50 ] (13.3s) loss=0.3378 auroc_term=0.956 auroc_disc=0.875 L_disc=0.1949
|
||||
[epoch 21/50 ] (2.4s) loss=0.3301 L_disc=0.1905
|
||||
[epoch 22/50 ] (2.3s) loss=0.3305 L_disc=0.1924
|
||||
[epoch 23/50 ] (2.2s) loss=0.3250 L_disc=0.1896
|
||||
[epoch 24/50 ] (2.6s) loss=0.3157 L_disc=0.1827
|
||||
[epoch 25/50 ] (2.4s) loss=0.3155 L_disc=0.1833
|
||||
[epoch 26/50 ] (2.6s) loss=0.3173 L_disc=0.1854
|
||||
[epoch 27/50 ] (2.4s) loss=0.3134 L_disc=0.1846
|
||||
[epoch 28/50 ] (2.4s) loss=0.3018 L_disc=0.1744
|
||||
[epoch 29/50 ] (2.6s) loss=0.3007 L_disc=0.1744
|
||||
[epoch 30/50 ] (13.3s) loss=0.2973 auroc_term=0.953 auroc_disc=0.891 L_disc=0.1725
|
||||
[epoch 31/50 ] (2.5s) loss=0.2938 L_disc=0.1690
|
||||
[epoch 32/50 ] (2.3s) loss=0.2986 L_disc=0.1741
|
||||
[epoch 33/50 ] (2.3s) loss=0.2964 L_disc=0.1737
|
||||
[epoch 34/50 ] (2.4s) loss=0.2916 L_disc=0.1690
|
||||
[epoch 35/50 ] (2.5s) loss=0.2920 L_disc=0.1695
|
||||
[epoch 36/50 ] (2.6s) loss=0.2917 L_disc=0.1725
|
||||
[epoch 37/50 ] (2.5s) loss=0.2889 L_disc=0.1704
|
||||
[epoch 38/50 ] (2.3s) loss=0.2858 L_disc=0.1666
|
||||
[epoch 39/50 ] (2.2s) loss=0.2864 L_disc=0.1684
|
||||
[epoch 40/50 ] (13.2s) loss=0.2826 auroc_term=0.957 auroc_disc=0.902 L_disc=0.1667
|
||||
[epoch 41/50 ] (2.1s) loss=0.2873 L_disc=0.1675
|
||||
[epoch 42/50 ] (2.1s) loss=0.2897 L_disc=0.1713
|
||||
[epoch 43/50 ] (2.0s) loss=0.2848 L_disc=0.1680
|
||||
[epoch 44/50 ] (2.2s) loss=0.2852 L_disc=0.1693
|
||||
[epoch 45/50 ] (2.0s) loss=0.2839 L_disc=0.1664
|
||||
[epoch 46/50 ] (2.0s) loss=0.2832 L_disc=0.1668
|
||||
[epoch 47/50 ] (2.2s) loss=0.2778 L_disc=0.1644
|
||||
[epoch 48/50 ] (2.1s) loss=0.2858 L_disc=0.1681
|
||||
[epoch 49/50 ] (2.2s) loss=0.2841 L_disc=0.1665
|
||||
[epoch 50/50 ] (12.2s) loss=0.2803 auroc_term=0.958 auroc_disc=0.896 L_disc=0.1645
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_ciciot2023_seed43/model.pt
|
||||
58
artifacts/route_comparison/route_ac_combo_seed44.log
Normal file
58
artifacts/route_comparison/route_ac_combo_seed44.log
Normal file
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:44/data:44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] kept 3,797,530 of 8,193,621 (min_len=2)
|
||||
[data] train=77,636 val=10,000 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.8s) loss=1.3564 L_disc=0.5045
|
||||
[epoch 2/50 ] (2.3s) loss=1.0204 L_disc=0.4016
|
||||
[epoch 3/50 ] (2.3s) loss=0.8221 L_disc=0.3784
|
||||
[epoch 4/50 ] (2.2s) loss=0.6901 L_disc=0.3475
|
||||
[epoch 5/50 ] (2.2s) loss=0.5952 L_disc=0.3093
|
||||
[epoch 6/50 ] (2.3s) loss=0.5358 L_disc=0.2867
|
||||
[epoch 7/50 ] (2.2s) loss=0.5014 L_disc=0.2729
|
||||
[epoch 8/50 ] (2.1s) loss=0.4773 L_disc=0.2672
|
||||
[epoch 9/50 ] (1.9s) loss=0.4561 L_disc=0.2530
|
||||
[epoch 10/50 ] (7.1s) loss=0.4294 auroc_term=0.942 auroc_disc=0.847 L_disc=0.2410
|
||||
[epoch 11/50 ] (1.5s) loss=0.4165 L_disc=0.2326
|
||||
[epoch 12/50 ] (1.6s) loss=0.3969 L_disc=0.2230
|
||||
[epoch 13/50 ] (1.7s) loss=0.3970 L_disc=0.2259
|
||||
[epoch 14/50 ] (2.0s) loss=0.3888 L_disc=0.2208
|
||||
[epoch 15/50 ] (2.1s) loss=0.3712 L_disc=0.2116
|
||||
[epoch 16/50 ] (2.2s) loss=0.3681 L_disc=0.2095
|
||||
[epoch 17/50 ] (2.2s) loss=0.3562 L_disc=0.2044
|
||||
[epoch 18/50 ] (2.2s) loss=0.3501 L_disc=0.2004
|
||||
[epoch 19/50 ] (2.2s) loss=0.3467 L_disc=0.1997
|
||||
[epoch 20/50 ] (7.8s) loss=0.3345 auroc_term=0.952 auroc_disc=0.879 L_disc=0.1913
|
||||
[epoch 21/50 ] (1.9s) loss=0.3390 L_disc=0.1943
|
||||
[epoch 22/50 ] (1.7s) loss=0.3296 L_disc=0.1878
|
||||
[epoch 23/50 ] (1.6s) loss=0.3245 L_disc=0.1855
|
||||
[epoch 24/50 ] (1.4s) loss=0.3262 L_disc=0.1877
|
||||
[epoch 25/50 ] (1.5s) loss=0.3189 L_disc=0.1840
|
||||
[epoch 26/50 ] (1.5s) loss=0.3114 L_disc=0.1780
|
||||
[epoch 27/50 ] (1.5s) loss=0.3097 L_disc=0.1778
|
||||
[epoch 28/50 ] (1.6s) loss=0.3124 L_disc=0.1790
|
||||
[epoch 29/50 ] (1.7s) loss=0.2988 L_disc=0.1711
|
||||
[epoch 30/50 ] (7.7s) loss=0.3018 auroc_term=0.956 auroc_disc=0.873 L_disc=0.1736
|
||||
[epoch 31/50 ] (2.0s) loss=0.3028 L_disc=0.1730
|
||||
[epoch 32/50 ] (2.2s) loss=0.3024 L_disc=0.1756
|
||||
[epoch 33/50 ] (2.2s) loss=0.2964 L_disc=0.1710
|
||||
[epoch 34/50 ] (2.2s) loss=0.2962 L_disc=0.1715
|
||||
[epoch 35/50 ] (2.1s) loss=0.2889 L_disc=0.1662
|
||||
[epoch 36/50 ] (2.1s) loss=0.2888 L_disc=0.1664
|
||||
[epoch 37/50 ] (2.1s) loss=0.2921 L_disc=0.1696
|
||||
[epoch 38/50 ] (1.8s) loss=0.2934 L_disc=0.1721
|
||||
[epoch 39/50 ] (1.6s) loss=0.2890 L_disc=0.1659
|
||||
[epoch 40/50 ] (6.9s) loss=0.2872 auroc_term=0.956 auroc_disc=0.887 L_disc=0.1651
|
||||
[epoch 41/50 ] (1.6s) loss=0.2825 L_disc=0.1634
|
||||
[epoch 42/50 ] (1.8s) loss=0.2820 L_disc=0.1642
|
||||
[epoch 43/50 ] (2.1s) loss=0.2806 L_disc=0.1628
|
||||
[epoch 44/50 ] (2.2s) loss=0.2877 L_disc=0.1677
|
||||
[epoch 45/50 ] (2.3s) loss=0.2879 L_disc=0.1668
|
||||
[epoch 46/50 ] (2.2s) loss=0.2792 L_disc=0.1614
|
||||
[epoch 47/50 ] (2.2s) loss=0.2815 L_disc=0.1619
|
||||
[epoch 48/50 ] (2.3s) loss=0.2847 L_disc=0.1660
|
||||
[epoch 49/50 ] (2.2s) loss=0.2855 L_disc=0.1661
|
||||
[epoch 50/50 ] (7.8s) loss=0.2830 auroc_term=0.957 auroc_disc=0.892 L_disc=0.1640
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_ac_combo_ciciot2023_seed44/model.pt
|
||||
61
artifacts/route_comparison/route_b_spectral_seed42.log
Normal file
61
artifacts/route_comparison/route_b_spectral_seed42.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=42 data=42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=52
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=52 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,234,485 token_dim=53 seq_len=65 sigma=0.1 use_ot=True reference_mode=None
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (2.8s) loss=2.3544 aux_flow=3.1786 aux_pkt=0.9661
|
||||
[epoch 2/50 ] (4.0s) loss=2.0830 aux_flow=2.7865 aux_pkt=0.9686
|
||||
[epoch 3/50 ] (4.0s) loss=1.8203 aux_flow=2.4385 aux_pkt=0.9612
|
||||
[epoch 4/50 ] (4.0s) loss=1.6441 aux_flow=2.2780 aux_pkt=0.9528
|
||||
[epoch 5/50 ] (4.0s) loss=1.5232 aux_flow=2.1756 aux_pkt=0.9528
|
||||
[epoch 6/50 ] (4.0s) loss=1.4129 aux_flow=2.0367 aux_pkt=0.9481
|
||||
[epoch 7/50 ] (4.0s) loss=1.3444 aux_flow=1.9742 aux_pkt=0.9498
|
||||
[epoch 8/50 ] (4.0s) loss=1.2819 aux_flow=1.8964 aux_pkt=0.9474
|
||||
[epoch 9/50 ] (3.9s) loss=1.2370 aux_flow=1.8439 aux_pkt=0.9488
|
||||
[epoch 10/50 ] (40.6s) loss=1.2030 auroc_terminal=0.933 aux_flow=1.8201 aux_pkt=0.9431
|
||||
[epoch 11/50 ] (3.0s) loss=1.1606 aux_flow=1.7593 aux_pkt=0.9459
|
||||
[epoch 12/50 ] (4.0s) loss=1.1252 aux_flow=1.6987 aux_pkt=0.9423
|
||||
[epoch 13/50 ] (4.0s) loss=1.1246 aux_flow=1.7348 aux_pkt=0.9390
|
||||
[epoch 14/50 ] (4.0s) loss=1.0960 aux_flow=1.6805 aux_pkt=0.9436
|
||||
[epoch 15/50 ] (4.0s) loss=1.0787 aux_flow=1.6671 aux_pkt=0.9424
|
||||
[epoch 16/50 ] (4.1s) loss=1.0653 aux_flow=1.6495 aux_pkt=0.9405
|
||||
[epoch 17/50 ] (4.0s) loss=1.0593 aux_flow=1.6453 aux_pkt=0.9375
|
||||
[epoch 18/50 ] (4.0s) loss=1.0483 aux_flow=1.6232 aux_pkt=0.9380
|
||||
[epoch 19/50 ] (3.9s) loss=1.0290 aux_flow=1.6002 aux_pkt=0.9406
|
||||
[epoch 20/50 ] (40.5s) loss=1.0214 auroc_terminal=0.954 aux_flow=1.5838 aux_pkt=0.9412
|
||||
[epoch 21/50 ] (3.3s) loss=1.0239 aux_flow=1.6022 aux_pkt=0.9398
|
||||
[epoch 22/50 ] (4.0s) loss=1.0107 aux_flow=1.5874 aux_pkt=0.9362
|
||||
[epoch 23/50 ] (4.0s) loss=0.9994 aux_flow=1.5565 aux_pkt=0.9413
|
||||
[epoch 24/50 ] (4.0s) loss=0.9897 aux_flow=1.5394 aux_pkt=0.9374
|
||||
[epoch 25/50 ] (4.0s) loss=0.9888 aux_flow=1.5502 aux_pkt=0.9348
|
||||
[epoch 26/50 ] (3.9s) loss=0.9864 aux_flow=1.5475 aux_pkt=0.9373
|
||||
[epoch 27/50 ] (4.0s) loss=0.9821 aux_flow=1.5463 aux_pkt=0.9396
|
||||
[epoch 28/50 ] (3.9s) loss=0.9691 aux_flow=1.5212 aux_pkt=0.9353
|
||||
[epoch 29/50 ] (4.0s) loss=0.9669 aux_flow=1.5096 aux_pkt=0.9401
|
||||
[epoch 30/50 ] (40.2s) loss=0.9735 auroc_terminal=0.963 aux_flow=1.5328 aux_pkt=0.9407
|
||||
[epoch 31/50 ] (3.3s) loss=0.9612 aux_flow=1.5108 aux_pkt=0.9326
|
||||
[epoch 32/50 ] (4.0s) loss=0.9627 aux_flow=1.5228 aux_pkt=0.9332
|
||||
[epoch 33/50 ] (4.0s) loss=0.9654 aux_flow=1.5278 aux_pkt=0.9370
|
||||
[epoch 34/50 ] (3.9s) loss=0.9489 aux_flow=1.4877 aux_pkt=0.9357
|
||||
[epoch 35/50 ] (4.0s) loss=0.9475 aux_flow=1.4857 aux_pkt=0.9359
|
||||
[epoch 36/50 ] (3.9s) loss=0.9579 aux_flow=1.5192 aux_pkt=0.9345
|
||||
[epoch 37/50 ] (4.0s) loss=0.9397 aux_flow=1.4714 aux_pkt=0.9379
|
||||
[epoch 38/50 ] (4.1s) loss=0.9407 aux_flow=1.4776 aux_pkt=0.9323
|
||||
[epoch 39/50 ] (3.9s) loss=0.9388 aux_flow=1.4773 aux_pkt=0.9347
|
||||
[epoch 40/50 ] (40.1s) loss=0.9401 auroc_terminal=0.965 aux_flow=1.4798 aux_pkt=0.9325
|
||||
[epoch 41/50 ] (3.7s) loss=0.9425 aux_flow=1.4853 aux_pkt=0.9376
|
||||
[epoch 42/50 ] (3.9s) loss=0.9361 aux_flow=1.4683 aux_pkt=0.9348
|
||||
[epoch 43/50 ] (4.0s) loss=0.9382 aux_flow=1.4798 aux_pkt=0.9327
|
||||
[epoch 44/50 ] (3.9s) loss=0.9292 aux_flow=1.4476 aux_pkt=0.9336
|
||||
[epoch 45/50 ] (3.9s) loss=0.9373 aux_flow=1.4664 aux_pkt=0.9363
|
||||
[epoch 46/50 ] (3.9s) loss=0.9421 aux_flow=1.4845 aux_pkt=0.9368
|
||||
[epoch 47/50 ] (3.9s) loss=0.9305 aux_flow=1.4641 aux_pkt=0.9332
|
||||
[epoch 48/50 ] (3.9s) loss=0.9369 aux_flow=1.4747 aux_pkt=0.9344
|
||||
[epoch 49/50 ] (4.0s) loss=0.9330 aux_flow=1.4635 aux_pkt=0.9330
|
||||
[epoch 50/50 ] (40.0s) loss=0.9413 auroc_terminal=0.965 aux_flow=1.4829 aux_pkt=0.9368
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_b_spectral_ciciot2023_seed42/model.pt
|
||||
61
artifacts/route_comparison/route_b_spectral_seed43.log
Normal file
61
artifacts/route_comparison/route_b_spectral_seed43.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=43 data=43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=52
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=52 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,234,485 token_dim=53 seq_len=65 sigma=0.1 use_ot=True reference_mode=None
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (4.5s) loss=2.4201 aux_flow=3.3744 aux_pkt=0.9734
|
||||
[epoch 2/50 ] (4.0s) loss=2.0997 aux_flow=2.8514 aux_pkt=0.9646
|
||||
[epoch 3/50 ] (4.0s) loss=1.8899 aux_flow=2.6628 aux_pkt=0.9609
|
||||
[epoch 4/50 ] (4.0s) loss=1.6781 aux_flow=2.3786 aux_pkt=0.9600
|
||||
[epoch 5/50 ] (4.0s) loss=1.5686 aux_flow=2.2909 aux_pkt=0.9661
|
||||
[epoch 6/50 ] (4.0s) loss=1.4405 aux_flow=2.1185 aux_pkt=0.9573
|
||||
[epoch 7/50 ] (4.0s) loss=1.3880 aux_flow=2.0996 aux_pkt=0.9595
|
||||
[epoch 8/50 ] (3.9s) loss=1.3128 aux_flow=1.9850 aux_pkt=0.9541
|
||||
[epoch 9/50 ] (3.9s) loss=1.2755 aux_flow=1.9430 aux_pkt=0.9513
|
||||
[epoch 10/50 ] (39.1s) loss=1.2268 auroc_terminal=0.920 aux_flow=1.8673 aux_pkt=0.9498
|
||||
[epoch 11/50 ] (4.0s) loss=1.2071 aux_flow=1.8713 aux_pkt=0.9489
|
||||
[epoch 12/50 ] (4.0s) loss=1.1613 aux_flow=1.7944 aux_pkt=0.9456
|
||||
[epoch 13/50 ] (4.0s) loss=1.1536 aux_flow=1.7997 aux_pkt=0.9485
|
||||
[epoch 14/50 ] (4.0s) loss=1.1315 aux_flow=1.7690 aux_pkt=0.9453
|
||||
[epoch 15/50 ] (4.1s) loss=1.1274 aux_flow=1.7830 aux_pkt=0.9437
|
||||
[epoch 16/50 ] (4.0s) loss=1.1027 aux_flow=1.7480 aux_pkt=0.9458
|
||||
[epoch 17/50 ] (3.9s) loss=1.0860 aux_flow=1.7196 aux_pkt=0.9463
|
||||
[epoch 18/50 ] (3.9s) loss=1.0883 aux_flow=1.7458 aux_pkt=0.9437
|
||||
[epoch 19/50 ] (3.9s) loss=1.0673 aux_flow=1.6918 aux_pkt=0.9388
|
||||
[epoch 20/50 ] (38.8s) loss=1.0633 auroc_terminal=0.952 aux_flow=1.6942 aux_pkt=0.9414
|
||||
[epoch 21/50 ] (4.0s) loss=1.0518 aux_flow=1.6818 aux_pkt=0.9417
|
||||
[epoch 22/50 ] (4.0s) loss=1.0546 aux_flow=1.7053 aux_pkt=0.9434
|
||||
[epoch 23/50 ] (4.1s) loss=1.0507 aux_flow=1.6968 aux_pkt=0.9414
|
||||
[epoch 24/50 ] (4.0s) loss=1.0281 aux_flow=1.6344 aux_pkt=0.9364
|
||||
[epoch 25/50 ] (4.0s) loss=1.0245 aux_flow=1.6365 aux_pkt=0.9423
|
||||
[epoch 26/50 ] (4.0s) loss=1.0075 aux_flow=1.6021 aux_pkt=0.9414
|
||||
[epoch 27/50 ] (3.9s) loss=1.0144 aux_flow=1.6296 aux_pkt=0.9389
|
||||
[epoch 28/50 ] (3.9s) loss=1.0041 aux_flow=1.5959 aux_pkt=0.9443
|
||||
[epoch 29/50 ] (4.0s) loss=1.0105 aux_flow=1.6301 aux_pkt=0.9431
|
||||
[epoch 30/50 ] (39.3s) loss=0.9935 auroc_terminal=0.958 aux_flow=1.5859 aux_pkt=0.9375
|
||||
[epoch 31/50 ] (4.0s) loss=1.0036 aux_flow=1.6218 aux_pkt=0.9375
|
||||
[epoch 32/50 ] (4.0s) loss=0.9917 aux_flow=1.5974 aux_pkt=0.9335
|
||||
[epoch 33/50 ] (3.9s) loss=0.9823 aux_flow=1.5637 aux_pkt=0.9417
|
||||
[epoch 34/50 ] (4.0s) loss=0.9800 aux_flow=1.5719 aux_pkt=0.9344
|
||||
[epoch 35/50 ] (3.9s) loss=0.9890 aux_flow=1.6022 aux_pkt=0.9372
|
||||
[epoch 36/50 ] (3.9s) loss=0.9704 aux_flow=1.5508 aux_pkt=0.9359
|
||||
[epoch 37/50 ] (4.0s) loss=0.9831 aux_flow=1.5802 aux_pkt=0.9383
|
||||
[epoch 38/50 ] (4.0s) loss=0.9879 aux_flow=1.6067 aux_pkt=0.9354
|
||||
[epoch 39/50 ] (3.9s) loss=0.9685 aux_flow=1.5418 aux_pkt=0.9410
|
||||
[epoch 40/50 ] (38.8s) loss=0.9719 auroc_terminal=0.962 aux_flow=1.5721 aux_pkt=0.9330
|
||||
[epoch 41/50 ] (3.9s) loss=0.9679 aux_flow=1.5551 aux_pkt=0.9360
|
||||
[epoch 42/50 ] (3.9s) loss=0.9539 aux_flow=1.5209 aux_pkt=0.9356
|
||||
[epoch 43/50 ] (4.0s) loss=0.9706 aux_flow=1.5551 aux_pkt=0.9389
|
||||
[epoch 44/50 ] (3.9s) loss=0.9644 aux_flow=1.5438 aux_pkt=0.9385
|
||||
[epoch 45/50 ] (3.9s) loss=0.9691 aux_flow=1.5552 aux_pkt=0.9378
|
||||
[epoch 46/50 ] (3.9s) loss=0.9614 aux_flow=1.5372 aux_pkt=0.9367
|
||||
[epoch 47/50 ] (4.0s) loss=0.9625 aux_flow=1.5373 aux_pkt=0.9377
|
||||
[epoch 48/50 ] (3.9s) loss=0.9547 aux_flow=1.5161 aux_pkt=0.9365
|
||||
[epoch 49/50 ] (3.9s) loss=0.9616 aux_flow=1.5415 aux_pkt=0.9370
|
||||
[epoch 50/50 ] (39.3s) loss=0.9600 auroc_terminal=0.963 aux_flow=1.5274 aux_pkt=0.9456
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_b_spectral_ciciot2023_seed43/model.pt
|
||||
61
artifacts/route_comparison/route_b_spectral_seed44.log
Normal file
61
artifacts/route_comparison/route_b_spectral_seed44.log
Normal file
@@ -0,0 +1,61 @@
|
||||
Device: cuda
|
||||
[seed] model=44 data=44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets_source=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] using external flow features D=52
|
||||
[data] rows total=8,193,621 keep len>=2: 3,797,530
|
||||
[data] benign=97,045 attack=20,000 -> train=77,636 val=10,000
|
||||
[data] T=64 packet_D=9 flow_D=52 train=77,636 val=10,000 attack=20,000
|
||||
[data] using 10,000 benign training flows
|
||||
[model] params=1,234,485 token_dim=53 seq_len=65 sigma=0.1 use_ot=True reference_mode=None
|
||||
[loss] λ_flow=0.3 λ_packet=0.3 packet_mask_ratio=0.5
|
||||
[epoch 1/50 ] (2.8s) loss=2.4962 aux_flow=3.6093 aux_pkt=0.9787
|
||||
[epoch 2/50 ] (2.3s) loss=2.1768 aux_flow=3.0675 aux_pkt=0.9691
|
||||
[epoch 3/50 ] (2.3s) loss=1.9341 aux_flow=2.7710 aux_pkt=0.9656
|
||||
[epoch 4/50 ] (2.2s) loss=1.7438 aux_flow=2.5494 aux_pkt=0.9627
|
||||
[epoch 5/50 ] (2.3s) loss=1.5846 aux_flow=2.3368 aux_pkt=0.9598
|
||||
[epoch 6/50 ] (2.3s) loss=1.5026 aux_flow=2.2796 aux_pkt=0.9618
|
||||
[epoch 7/50 ] (2.3s) loss=1.3985 aux_flow=2.1222 aux_pkt=0.9540
|
||||
[epoch 8/50 ] (2.3s) loss=1.3477 aux_flow=2.0648 aux_pkt=0.9608
|
||||
[epoch 9/50 ] (2.3s) loss=1.3081 aux_flow=2.0368 aux_pkt=0.9555
|
||||
[epoch 10/50 ] (17.0s) loss=1.2861 auroc_terminal=0.922 aux_flow=2.0402 aux_pkt=0.9597
|
||||
[epoch 11/50 ] (2.2s) loss=1.2449 aux_flow=1.9686 aux_pkt=0.9484
|
||||
[epoch 12/50 ] (2.2s) loss=1.2067 aux_flow=1.8993 aux_pkt=0.9454
|
||||
[epoch 13/50 ] (2.2s) loss=1.1773 aux_flow=1.8377 aux_pkt=0.9521
|
||||
[epoch 14/50 ] (2.2s) loss=1.1562 aux_flow=1.8204 aux_pkt=0.9454
|
||||
[epoch 15/50 ] (2.2s) loss=1.1414 aux_flow=1.8006 aux_pkt=0.9505
|
||||
[epoch 16/50 ] (2.2s) loss=1.1194 aux_flow=1.7702 aux_pkt=0.9474
|
||||
[epoch 17/50 ] (2.2s) loss=1.1146 aux_flow=1.7713 aux_pkt=0.9532
|
||||
[epoch 18/50 ] (2.1s) loss=1.1101 aux_flow=1.7801 aux_pkt=0.9501
|
||||
[epoch 19/50 ] (2.2s) loss=1.0890 aux_flow=1.7346 aux_pkt=0.9440
|
||||
[epoch 20/50 ] (16.4s) loss=1.0941 auroc_terminal=0.953 aux_flow=1.7565 aux_pkt=0.9523
|
||||
[epoch 21/50 ] (2.2s) loss=1.0880 aux_flow=1.7606 aux_pkt=0.9453
|
||||
[epoch 22/50 ] (2.3s) loss=1.0633 aux_flow=1.7103 aux_pkt=0.9435
|
||||
[epoch 23/50 ] (2.3s) loss=1.0621 aux_flow=1.7152 aux_pkt=0.9426
|
||||
[epoch 24/50 ] (2.2s) loss=1.0594 aux_flow=1.7000 aux_pkt=0.9493
|
||||
[epoch 25/50 ] (2.2s) loss=1.0447 aux_flow=1.6813 aux_pkt=0.9492
|
||||
[epoch 26/50 ] (2.2s) loss=1.0395 aux_flow=1.6704 aux_pkt=0.9471
|
||||
[epoch 27/50 ] (2.3s) loss=1.0303 aux_flow=1.6538 aux_pkt=0.9418
|
||||
[epoch 28/50 ] (2.3s) loss=1.0136 aux_flow=1.6232 aux_pkt=0.9427
|
||||
[epoch 29/50 ] (2.3s) loss=1.0252 aux_flow=1.6551 aux_pkt=0.9450
|
||||
[epoch 30/50 ] (17.0s) loss=1.0092 auroc_terminal=0.955 aux_flow=1.6198 aux_pkt=0.9407
|
||||
[epoch 31/50 ] (2.2s) loss=1.0127 aux_flow=1.6250 aux_pkt=0.9485
|
||||
[epoch 32/50 ] (2.1s) loss=0.9991 aux_flow=1.6026 aux_pkt=0.9422
|
||||
[epoch 33/50 ] (2.2s) loss=0.9913 aux_flow=1.5832 aux_pkt=0.9401
|
||||
[epoch 34/50 ] (2.2s) loss=1.0053 aux_flow=1.6317 aux_pkt=0.9361
|
||||
[epoch 35/50 ] (2.2s) loss=0.9960 aux_flow=1.6005 aux_pkt=0.9456
|
||||
[epoch 36/50 ] (2.2s) loss=0.9959 aux_flow=1.6043 aux_pkt=0.9472
|
||||
[epoch 37/50 ] (2.2s) loss=0.9913 aux_flow=1.6011 aux_pkt=0.9417
|
||||
[epoch 38/50 ] (2.2s) loss=0.9909 aux_flow=1.6016 aux_pkt=0.9438
|
||||
[epoch 39/50 ] (2.2s) loss=0.9861 aux_flow=1.5866 aux_pkt=0.9386
|
||||
[epoch 40/50 ] (16.3s) loss=0.9789 auroc_terminal=0.961 aux_flow=1.5679 aux_pkt=0.9428
|
||||
[epoch 41/50 ] (2.3s) loss=0.9755 aux_flow=1.5589 aux_pkt=0.9385
|
||||
[epoch 42/50 ] (2.3s) loss=0.9816 aux_flow=1.5740 aux_pkt=0.9446
|
||||
[epoch 43/50 ] (2.3s) loss=0.9773 aux_flow=1.5663 aux_pkt=0.9411
|
||||
[epoch 44/50 ] (2.2s) loss=0.9759 aux_flow=1.5609 aux_pkt=0.9435
|
||||
[epoch 45/50 ] (2.3s) loss=0.9732 aux_flow=1.5489 aux_pkt=0.9431
|
||||
[epoch 46/50 ] (2.3s) loss=0.9797 aux_flow=1.5711 aux_pkt=0.9430
|
||||
[epoch 47/50 ] (2.3s) loss=0.9768 aux_flow=1.5666 aux_pkt=0.9455
|
||||
[epoch 48/50 ] (2.3s) loss=0.9764 aux_flow=1.5646 aux_pkt=0.9406
|
||||
[epoch 49/50 ] (2.3s) loss=0.9781 aux_flow=1.5619 aux_pkt=0.9415
|
||||
[epoch 50/50 ] (16.9s) loss=0.9714 auroc_terminal=0.961 aux_flow=1.5477 aux_pkt=0.9438
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_b_spectral_ciciot2023_seed44/model.pt
|
||||
58
artifacts/route_comparison/route_c_mixed_seed42.log
Normal file
58
artifacts/route_comparison/route_c_mixed_seed42.log
Normal file
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:42/data:42
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] kept 3,797,530 of 8,193,621 (min_len=2)
|
||||
[data] train=77,636 val=10,000 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (1.9s) loss=1.3454 L_disc=0.5034
|
||||
[epoch 2/50 ] (1.3s) loss=1.0186 L_disc=0.4025
|
||||
[epoch 3/50 ] (1.4s) loss=0.8178 L_disc=0.3743
|
||||
[epoch 4/50 ] (1.4s) loss=0.6716 L_disc=0.3415
|
||||
[epoch 5/50 ] (1.4s) loss=0.5847 L_disc=0.3137
|
||||
[epoch 6/50 ] (1.4s) loss=0.5295 L_disc=0.2908
|
||||
[epoch 7/50 ] (1.3s) loss=0.4960 L_disc=0.2801
|
||||
[epoch 8/50 ] (1.4s) loss=0.4646 L_disc=0.2629
|
||||
[epoch 9/50 ] (1.3s) loss=0.4429 L_disc=0.2522
|
||||
[epoch 10/50 ] (6.5s) loss=0.4298 auroc_term=0.952 auroc_disc=0.811 L_disc=0.2487
|
||||
[epoch 11/50 ] (1.4s) loss=0.4070 L_disc=0.2370
|
||||
[epoch 12/50 ] (1.4s) loss=0.4080 L_disc=0.2383
|
||||
[epoch 13/50 ] (1.4s) loss=0.3905 L_disc=0.2235
|
||||
[epoch 14/50 ] (1.3s) loss=0.3782 L_disc=0.2195
|
||||
[epoch 15/50 ] (1.4s) loss=0.3606 L_disc=0.2103
|
||||
[epoch 16/50 ] (1.4s) loss=0.3479 L_disc=0.2016
|
||||
[epoch 17/50 ] (1.4s) loss=0.3460 L_disc=0.1987
|
||||
[epoch 18/50 ] (1.4s) loss=0.3456 L_disc=0.2029
|
||||
[epoch 19/50 ] (1.4s) loss=0.3400 L_disc=0.2020
|
||||
[epoch 20/50 ] (6.5s) loss=0.3351 auroc_term=0.960 auroc_disc=0.851 L_disc=0.1967
|
||||
[epoch 21/50 ] (1.4s) loss=0.3316 L_disc=0.1935
|
||||
[epoch 22/50 ] (1.4s) loss=0.3199 L_disc=0.1871
|
||||
[epoch 23/50 ] (1.4s) loss=0.3147 L_disc=0.1848
|
||||
[epoch 24/50 ] (1.4s) loss=0.3142 L_disc=0.1848
|
||||
[epoch 25/50 ] (1.3s) loss=0.3097 L_disc=0.1821
|
||||
[epoch 26/50 ] (1.3s) loss=0.3056 L_disc=0.1787
|
||||
[epoch 27/50 ] (1.3s) loss=0.3059 L_disc=0.1800
|
||||
[epoch 28/50 ] (1.4s) loss=0.3039 L_disc=0.1772
|
||||
[epoch 29/50 ] (1.4s) loss=0.2965 L_disc=0.1738
|
||||
[epoch 30/50 ] (6.7s) loss=0.2975 auroc_term=0.960 auroc_disc=0.888 L_disc=0.1754
|
||||
[epoch 31/50 ] (1.4s) loss=0.2940 L_disc=0.1715
|
||||
[epoch 32/50 ] (1.4s) loss=0.2899 L_disc=0.1690
|
||||
[epoch 33/50 ] (1.4s) loss=0.2944 L_disc=0.1745
|
||||
[epoch 34/50 ] (1.4s) loss=0.2886 L_disc=0.1690
|
||||
[epoch 35/50 ] (1.7s) loss=0.2932 L_disc=0.1739
|
||||
[epoch 36/50 ] (1.9s) loss=0.2839 L_disc=0.1674
|
||||
[epoch 37/50 ] (2.1s) loss=0.2814 L_disc=0.1650
|
||||
[epoch 38/50 ] (2.3s) loss=0.2816 L_disc=0.1629
|
||||
[epoch 39/50 ] (2.2s) loss=0.2848 L_disc=0.1696
|
||||
[epoch 40/50 ] (7.8s) loss=0.2791 auroc_term=0.963 auroc_disc=0.895 L_disc=0.1636
|
||||
[epoch 41/50 ] (2.2s) loss=0.2798 L_disc=0.1644
|
||||
[epoch 42/50 ] (2.3s) loss=0.2775 L_disc=0.1644
|
||||
[epoch 43/50 ] (2.4s) loss=0.2810 L_disc=0.1671
|
||||
[epoch 44/50 ] (1.9s) loss=0.2745 L_disc=0.1616
|
||||
[epoch 45/50 ] (1.7s) loss=0.2741 L_disc=0.1610
|
||||
[epoch 46/50 ] (1.5s) loss=0.2773 L_disc=0.1642
|
||||
[epoch 47/50 ] (1.5s) loss=0.2790 L_disc=0.1667
|
||||
[epoch 48/50 ] (1.5s) loss=0.2795 L_disc=0.1664
|
||||
[epoch 49/50 ] (1.5s) loss=0.2816 L_disc=0.1698
|
||||
[epoch 50/50 ] (7.0s) loss=0.2739 auroc_term=0.964 auroc_disc=0.893 L_disc=0.1620
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_c_mixed_ciciot2023_seed42/model.pt
|
||||
58
artifacts/route_comparison/route_c_mixed_seed43.log
Normal file
58
artifacts/route_comparison/route_c_mixed_seed43.log
Normal file
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:43/data:43
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] kept 3,797,530 of 8,193,621 (min_len=2)
|
||||
[data] train=77,636 val=10,000 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (2.4s) loss=1.3530 L_disc=0.5012
|
||||
[epoch 2/50 ] (2.3s) loss=1.0244 L_disc=0.4020
|
||||
[epoch 3/50 ] (2.7s) loss=0.8157 L_disc=0.3793
|
||||
[epoch 4/50 ] (2.7s) loss=0.6868 L_disc=0.3463
|
||||
[epoch 5/50 ] (2.5s) loss=0.6018 L_disc=0.3205
|
||||
[epoch 6/50 ] (2.3s) loss=0.5437 L_disc=0.3006
|
||||
[epoch 7/50 ] (2.4s) loss=0.5085 L_disc=0.2877
|
||||
[epoch 8/50 ] (2.6s) loss=0.4799 L_disc=0.2731
|
||||
[epoch 9/50 ] (2.6s) loss=0.4592 L_disc=0.2636
|
||||
[epoch 10/50 ] (13.2s) loss=0.4371 auroc_term=0.937 auroc_disc=0.848 L_disc=0.2542
|
||||
[epoch 11/50 ] (2.4s) loss=0.4260 L_disc=0.2465
|
||||
[epoch 12/50 ] (2.2s) loss=0.4026 L_disc=0.2331
|
||||
[epoch 13/50 ] (2.2s) loss=0.3967 L_disc=0.2342
|
||||
[epoch 14/50 ] (2.1s) loss=0.3777 L_disc=0.2210
|
||||
[epoch 15/50 ] (2.0s) loss=0.3755 L_disc=0.2185
|
||||
[epoch 16/50 ] (2.1s) loss=0.3704 L_disc=0.2166
|
||||
[epoch 17/50 ] (2.0s) loss=0.3595 L_disc=0.2117
|
||||
[epoch 18/50 ] (2.0s) loss=0.3543 L_disc=0.2095
|
||||
[epoch 19/50 ] (2.1s) loss=0.3463 L_disc=0.2039
|
||||
[epoch 20/50 ] (12.8s) loss=0.3419 auroc_term=0.955 auroc_disc=0.875 L_disc=0.2006
|
||||
[epoch 21/50 ] (2.1s) loss=0.3340 L_disc=0.1968
|
||||
[epoch 22/50 ] (2.2s) loss=0.3346 L_disc=0.1976
|
||||
[epoch 23/50 ] (2.1s) loss=0.3291 L_disc=0.1950
|
||||
[epoch 24/50 ] (2.1s) loss=0.3203 L_disc=0.1883
|
||||
[epoch 25/50 ] (2.1s) loss=0.3198 L_disc=0.1882
|
||||
[epoch 26/50 ] (2.1s) loss=0.3235 L_disc=0.1922
|
||||
[epoch 27/50 ] (2.1s) loss=0.3164 L_disc=0.1893
|
||||
[epoch 28/50 ] (2.2s) loss=0.3063 L_disc=0.1807
|
||||
[epoch 29/50 ] (2.1s) loss=0.3059 L_disc=0.1808
|
||||
[epoch 30/50 ] (12.8s) loss=0.3002 auroc_term=0.949 auroc_disc=0.885 L_disc=0.1769
|
||||
[epoch 31/50 ] (2.2s) loss=0.2969 L_disc=0.1732
|
||||
[epoch 32/50 ] (2.2s) loss=0.3044 L_disc=0.1807
|
||||
[epoch 33/50 ] (2.2s) loss=0.2989 L_disc=0.1771
|
||||
[epoch 34/50 ] (2.0s) loss=0.2943 L_disc=0.1729
|
||||
[epoch 35/50 ] (2.3s) loss=0.2950 L_disc=0.1739
|
||||
[epoch 36/50 ] (2.1s) loss=0.2955 L_disc=0.1773
|
||||
[epoch 37/50 ] (2.1s) loss=0.2922 L_disc=0.1746
|
||||
[epoch 38/50 ] (2.1s) loss=0.2899 L_disc=0.1720
|
||||
[epoch 39/50 ] (2.2s) loss=0.2901 L_disc=0.1731
|
||||
[epoch 40/50 ] (12.7s) loss=0.2859 auroc_term=0.959 auroc_disc=0.896 L_disc=0.1710
|
||||
[epoch 41/50 ] (2.2s) loss=0.2915 L_disc=0.1727
|
||||
[epoch 42/50 ] (2.1s) loss=0.2935 L_disc=0.1761
|
||||
[epoch 43/50 ] (2.0s) loss=0.2883 L_disc=0.1724
|
||||
[epoch 44/50 ] (2.2s) loss=0.2895 L_disc=0.1742
|
||||
[epoch 45/50 ] (2.3s) loss=0.2882 L_disc=0.1716
|
||||
[epoch 46/50 ] (2.2s) loss=0.2870 L_disc=0.1712
|
||||
[epoch 47/50 ] (2.2s) loss=0.2812 L_disc=0.1686
|
||||
[epoch 48/50 ] (2.3s) loss=0.2886 L_disc=0.1717
|
||||
[epoch 49/50 ] (2.1s) loss=0.2881 L_disc=0.1711
|
||||
[epoch 50/50 ] (12.6s) loss=0.2832 auroc_term=0.960 auroc_disc=0.890 L_disc=0.1684
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_c_mixed_ciciot2023_seed43/model.pt
|
||||
58
artifacts/route_comparison/route_c_mixed_seed44.log
Normal file
58
artifacts/route_comparison/route_c_mixed_seed44.log
Normal file
@@ -0,0 +1,58 @@
|
||||
Device: cuda seed=model:44/data:44
|
||||
[data] flows=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store/flows.parquet packets=/home/chy/mambafortrafficmodeling/datasets/ciciot2023/processed/full_store
|
||||
[data] kept 3,797,530 of 8,193,621 (min_len=2)
|
||||
[data] train=77,636 val=10,000 attack=20,000
|
||||
[data] T=64 cont=3 disc=6 flow=20 train=77,636 val=10,000 attack=20,000
|
||||
[data] training on 10,000 flows
|
||||
[model] params=1,227,809 token_dim=21 sigma=0.1 use_ot=True lambda_disc=1.0
|
||||
[epoch 1/50 ] (3.1s) loss=1.3571 L_disc=0.5042
|
||||
[epoch 2/50 ] (2.7s) loss=1.0217 L_disc=0.4013
|
||||
[epoch 3/50 ] (2.5s) loss=0.8214 L_disc=0.3772
|
||||
[epoch 4/50 ] (2.3s) loss=0.6900 L_disc=0.3473
|
||||
[epoch 5/50 ] (2.4s) loss=0.5990 L_disc=0.3139
|
||||
[epoch 6/50 ] (2.4s) loss=0.5406 L_disc=0.2919
|
||||
[epoch 7/50 ] (2.5s) loss=0.5073 L_disc=0.2785
|
||||
[epoch 8/50 ] (2.7s) loss=0.4799 L_disc=0.2709
|
||||
[epoch 9/50 ] (2.6s) loss=0.4590 L_disc=0.2579
|
||||
[epoch 10/50 ] (13.0s) loss=0.4341 auroc_term=0.936 auroc_disc=0.845 L_disc=0.2474
|
||||
[epoch 11/50 ] (2.3s) loss=0.4240 L_disc=0.2406
|
||||
[epoch 12/50 ] (2.1s) loss=0.4045 L_disc=0.2314
|
||||
[epoch 13/50 ] (2.0s) loss=0.4008 L_disc=0.2317
|
||||
[epoch 14/50 ] (1.9s) loss=0.3907 L_disc=0.2252
|
||||
[epoch 15/50 ] (2.1s) loss=0.3753 L_disc=0.2172
|
||||
[epoch 16/50 ] (2.0s) loss=0.3732 L_disc=0.2155
|
||||
[epoch 17/50 ] (2.1s) loss=0.3614 L_disc=0.2103
|
||||
[epoch 18/50 ] (2.0s) loss=0.3560 L_disc=0.2067
|
||||
[epoch 19/50 ] (2.1s) loss=0.3498 L_disc=0.2043
|
||||
[epoch 20/50 ] (12.8s) loss=0.3383 auroc_term=0.953 auroc_disc=0.847 L_disc=0.1962
|
||||
[epoch 21/50 ] (2.1s) loss=0.3404 L_disc=0.1973
|
||||
[epoch 22/50 ] (2.2s) loss=0.3330 L_disc=0.1927
|
||||
[epoch 23/50 ] (2.1s) loss=0.3297 L_disc=0.1917
|
||||
[epoch 24/50 ] (2.2s) loss=0.3291 L_disc=0.1922
|
||||
[epoch 25/50 ] (2.0s) loss=0.3233 L_disc=0.1891
|
||||
[epoch 26/50 ] (2.1s) loss=0.3160 L_disc=0.1832
|
||||
[epoch 27/50 ] (2.0s) loss=0.3147 L_disc=0.1835
|
||||
[epoch 28/50 ] (2.2s) loss=0.3153 L_disc=0.1826
|
||||
[epoch 29/50 ] (2.4s) loss=0.3031 L_disc=0.1758
|
||||
[epoch 30/50 ] (12.6s) loss=0.3036 auroc_term=0.954 auroc_disc=0.861 L_disc=0.1768
|
||||
[epoch 31/50 ] (2.2s) loss=0.3066 L_disc=0.1773
|
||||
[epoch 32/50 ] (2.2s) loss=0.3070 L_disc=0.1807
|
||||
[epoch 33/50 ] (2.0s) loss=0.2985 L_disc=0.1739
|
||||
[epoch 34/50 ] (2.2s) loss=0.2989 L_disc=0.1747
|
||||
[epoch 35/50 ] (2.2s) loss=0.2913 L_disc=0.1692
|
||||
[epoch 36/50 ] (2.1s) loss=0.2923 L_disc=0.1704
|
||||
[epoch 37/50 ] (2.2s) loss=0.2951 L_disc=0.1732
|
||||
[epoch 38/50 ] (2.2s) loss=0.2955 L_disc=0.1750
|
||||
[epoch 39/50 ] (2.2s) loss=0.2910 L_disc=0.1687
|
||||
[epoch 40/50 ] (12.6s) loss=0.2907 auroc_term=0.958 auroc_disc=0.874 L_disc=0.1691
|
||||
[epoch 41/50 ] (2.1s) loss=0.2848 L_disc=0.1664
|
||||
[epoch 42/50 ] (2.1s) loss=0.2851 L_disc=0.1680
|
||||
[epoch 43/50 ] (2.3s) loss=0.2837 L_disc=0.1663
|
||||
[epoch 44/50 ] (2.2s) loss=0.2910 L_disc=0.1713
|
||||
[epoch 45/50 ] (2.3s) loss=0.2914 L_disc=0.1709
|
||||
[epoch 46/50 ] (2.2s) loss=0.2829 L_disc=0.1659
|
||||
[epoch 47/50 ] (2.2s) loss=0.2857 L_disc=0.1665
|
||||
[epoch 48/50 ] (2.2s) loss=0.2878 L_disc=0.1698
|
||||
[epoch 49/50 ] (2.2s) loss=0.2883 L_disc=0.1694
|
||||
[epoch 50/50 ] (11.6s) loss=0.2866 auroc_term=0.958 auroc_disc=0.877 L_disc=0.1681
|
||||
[saved] /home/chy/mambafortrafficmodeling/artifacts/route_comparison/route_c_mixed_ciciot2023_seed44/model.pt
|
||||
70
artifacts/route_comparison/run_ac_combo_evals.sh
Executable file
70
artifacts/route_comparison/run_ac_combo_evals.sh
Executable file
@@ -0,0 +1,70 @@
|
||||
#!/bin/bash
|
||||
# Phase1 + cross eval for the 3 A+C combo seeds.
|
||||
set -e
|
||||
ROOT=/home/chy/mambafortrafficmodeling
|
||||
MIXED_PHASE1=${ROOT}/Mixed_CFM/eval_phase1.py
|
||||
MIXED_CROSS=${ROOT}/Mixed_CFM/eval_cross.py
|
||||
CROSS_DIR=${ROOT}/artifacts/route_comparison/cross
|
||||
mkdir -p ${CROSS_DIR}
|
||||
|
||||
# GPU 0: phase1 + cross→IDS2017 for all 3 seeds
|
||||
{
|
||||
for seed in 42 43 44; do
|
||||
md=${ROOT}/artifacts/route_comparison/route_ac_combo_ciciot2023_seed${seed}
|
||||
[ -f "${md}/model.pt" ] || { echo "[wait] seed${seed} model.pt not yet"; continue; }
|
||||
|
||||
if [ ! -f "${md}/phase1_summary.json" ]; then
|
||||
echo "[gpu0 phase1] seed${seed}"
|
||||
cd ${ROOT}/Mixed_CFM
|
||||
CUDA_VISIBLE_DEVICES=0 stdbuf -oL uv run --no-sync python -u ${MIXED_PHASE1} \
|
||||
--model-dir ${md} --out-dir ${md} \
|
||||
--batch-size 256 --n-steps 16 \
|
||||
--n-val-cap 5000 --n-atk-cap 10000 \
|
||||
> ${md}/phase1.log 2>&1
|
||||
fi
|
||||
|
||||
ids_out=${CROSS_DIR}/route_ac_combo_seed${seed}_to_cicids2017.json
|
||||
if [ ! -f "${ids_out}" ]; then
|
||||
echo "[gpu0 cross→ids2017] seed${seed}"
|
||||
cd ${ROOT}/Mixed_CFM
|
||||
CUDA_VISIBLE_DEVICES=0 stdbuf -oL uv run --no-sync python -u ${MIXED_CROSS} \
|
||||
--model-dir ${md} \
|
||||
--target-store ${ROOT}/datasets/cicids2017/processed/full_store \
|
||||
--target-flows ${ROOT}/datasets/cicids2017/processed/flows.parquet \
|
||||
--target-flow-features ${ROOT}/datasets/cicids2017/processed/flow_features.parquet \
|
||||
--out ${ids_out} \
|
||||
--n-benign 10000 --n-attack 10000 --seed 42 --T 64 --batch-size 256 --n-steps 16 \
|
||||
> ${CROSS_DIR}/route_ac_combo_seed${seed}_to_cicids2017.log 2>&1
|
||||
fi
|
||||
done
|
||||
echo "[gpu0 done]"
|
||||
} > /tmp/ac_eval_gpu0.log 2>&1 &
|
||||
GPU0=$!
|
||||
|
||||
# GPU 1: cross→DDoS19 for all 3 seeds
|
||||
{
|
||||
for seed in 42 43 44; do
|
||||
md=${ROOT}/artifacts/route_comparison/route_ac_combo_ciciot2023_seed${seed}
|
||||
[ -f "${md}/model.pt" ] || { echo "[wait] seed${seed} model.pt not yet"; continue; }
|
||||
|
||||
ddos_out=${CROSS_DIR}/route_ac_combo_seed${seed}_to_cicddos2019.json
|
||||
if [ ! -f "${ddos_out}" ]; then
|
||||
echo "[gpu1 cross→ddos19] seed${seed}"
|
||||
cd ${ROOT}/Mixed_CFM
|
||||
CUDA_VISIBLE_DEVICES=1 stdbuf -oL uv run --no-sync python -u ${MIXED_CROSS} \
|
||||
--model-dir ${md} \
|
||||
--target-store ${ROOT}/datasets/cicddos2019/processed/full_store \
|
||||
--target-flows ${ROOT}/datasets/cicddos2019/processed/flows.parquet \
|
||||
--target-flow-features ${ROOT}/datasets/cicddos2019/processed/flow_features.parquet \
|
||||
--out ${ddos_out} \
|
||||
--n-benign 10000 --n-attack 10000 --seed 42 --T 64 --batch-size 256 --n-steps 16 \
|
||||
> ${CROSS_DIR}/route_ac_combo_seed${seed}_to_cicddos2019.log 2>&1
|
||||
fi
|
||||
done
|
||||
echo "[gpu1 done]"
|
||||
} > /tmp/ac_eval_gpu1.log 2>&1 &
|
||||
GPU1=$!
|
||||
|
||||
wait $GPU0
|
||||
wait $GPU1
|
||||
echo "[all ac combo evals done]"
|
||||
68
artifacts/route_comparison/run_all_phase1.sh
Executable file
68
artifacts/route_comparison/run_all_phase1.sh
Executable file
@@ -0,0 +1,68 @@
|
||||
#!/bin/bash
|
||||
# Run phase1 eval on all routes after trainings complete.
|
||||
# Splits across 2 GPUs in parallel chains.
|
||||
|
||||
set -e
|
||||
ROOT=/home/chy/mambafortrafficmodeling
|
||||
UNIFIED_EVAL=${ROOT}/artifacts/verify_2026_04_24/eval_phase1_unified.py
|
||||
MIXED_EVAL=${ROOT}/Mixed_CFM/eval_phase1.py
|
||||
|
||||
cd ${ROOT}
|
||||
|
||||
# GPU 0: baselines + route_a (6 models)
|
||||
{
|
||||
for prefix in baseline_ciciot2023 route_a_causal_ciciot2023; do
|
||||
for seed in 42 43 44; do
|
||||
name=${prefix}_seed${seed}
|
||||
md=${ROOT}/artifacts/route_comparison/${name}
|
||||
[ -f "${md}/model.pt" ] || continue
|
||||
[ -f "${md}/phase1_summary.json" ] && continue
|
||||
echo "[GPU0 eval] ${name}"
|
||||
cd ${ROOT}/Unified_CFM
|
||||
CUDA_VISIBLE_DEVICES=0 stdbuf -oL uv run --no-sync python -u ${UNIFIED_EVAL} \
|
||||
--model-dir ${md} --out-dir ${md} \
|
||||
--batch-size 256 --n-steps 16 --jacobian-n-eps 4 \
|
||||
--n-val-cap 5000 --n-atk-cap 10000 \
|
||||
> ${md}/phase1.log 2>&1
|
||||
done
|
||||
done
|
||||
echo "[GPU0 done]"
|
||||
} &
|
||||
GPU0_PID=$!
|
||||
|
||||
# GPU 1: route_b + route_c (6 models)
|
||||
{
|
||||
for seed in 42 43 44; do
|
||||
name=route_b_spectral_ciciot2023_seed${seed}
|
||||
md=${ROOT}/artifacts/route_comparison/${name}
|
||||
[ -f "${md}/model.pt" ] || continue
|
||||
[ -f "${md}/phase1_summary.json" ] && continue
|
||||
echo "[GPU1 eval] ${name}"
|
||||
cd ${ROOT}/Unified_CFM
|
||||
CUDA_VISIBLE_DEVICES=1 stdbuf -oL uv run --no-sync python -u ${UNIFIED_EVAL} \
|
||||
--model-dir ${md} --out-dir ${md} \
|
||||
--batch-size 256 --n-steps 16 --jacobian-n-eps 4 \
|
||||
--n-val-cap 5000 --n-atk-cap 10000 \
|
||||
> ${md}/phase1.log 2>&1
|
||||
done
|
||||
for seed in 42 43 44; do
|
||||
name=route_c_mixed_ciciot2023_seed${seed}
|
||||
md=${ROOT}/artifacts/route_comparison/${name}
|
||||
[ -f "${md}/model.pt" ] || continue
|
||||
[ -f "${md}/phase1_summary.json" ] && continue
|
||||
echo "[GPU1 eval] ${name}"
|
||||
cd ${ROOT}/Mixed_CFM
|
||||
CUDA_VISIBLE_DEVICES=1 stdbuf -oL uv run --no-sync python -u ${MIXED_EVAL} \
|
||||
--model-dir ${md} --out-dir ${md} \
|
||||
--batch-size 256 --n-steps 16 \
|
||||
--n-val-cap 5000 --n-atk-cap 10000 \
|
||||
> ${md}/phase1.log 2>&1
|
||||
done
|
||||
echo "[GPU1 done]"
|
||||
} &
|
||||
GPU1_PID=$!
|
||||
|
||||
wait $GPU0_PID
|
||||
wait $GPU1_PID
|
||||
echo "[all phase1 done]"
|
||||
cd ${ROOT} && uv run --no-sync python artifacts/route_comparison/aggregate_results.py
|
||||
105
artifacts/route_comparison/run_cross_all.sh
Executable file
105
artifacts/route_comparison/run_cross_all.sh
Executable file
@@ -0,0 +1,105 @@
|
||||
#!/bin/bash
|
||||
# Cross-dataset eval for all 4 routes × 2 targets × 3 seeds = 24 runs.
|
||||
# Source: CICIoT2023 (where all models were trained).
|
||||
# Targets: CICIDS2017 + CICDDoS2019.
|
||||
|
||||
set -e
|
||||
ROOT=/home/chy/mambafortrafficmodeling
|
||||
UNIFIED_EVAL=${ROOT}/artifacts/verify_2026_04_24/eval_phase2_cross_cicddos2019.py
|
||||
MIXED_EVAL=${ROOT}/Mixed_CFM/eval_cross.py
|
||||
CROSS_DIR=${ROOT}/artifacts/route_comparison/cross
|
||||
mkdir -p ${CROSS_DIR}
|
||||
|
||||
# Target dataset paths
|
||||
declare -A TARGETS
|
||||
TARGETS[cicids2017_store]=${ROOT}/datasets/cicids2017/processed/full_store
|
||||
TARGETS[cicids2017_flows]=${ROOT}/datasets/cicids2017/processed/flows.parquet
|
||||
TARGETS[cicids2017_features]=${ROOT}/datasets/cicids2017/processed/flow_features.parquet
|
||||
TARGETS[cicids2017_features_spectral]=${ROOT}/datasets/cicids2017/processed/flow_features_spectral.parquet
|
||||
|
||||
TARGETS[cicddos2019_store]=${ROOT}/datasets/cicddos2019/processed/full_store
|
||||
TARGETS[cicddos2019_flows]=${ROOT}/datasets/cicddos2019/processed/flows.parquet
|
||||
TARGETS[cicddos2019_features]=${ROOT}/datasets/cicddos2019/processed/flow_features.parquet
|
||||
TARGETS[cicddos2019_features_spectral]=${ROOT}/datasets/cicddos2019/processed/flow_features_spectral.parquet
|
||||
|
||||
run_unified_eval() {
|
||||
local gpu=$1 model_dir=$2 target=$3 features=$4 out_name=$5
|
||||
local out=${CROSS_DIR}/${out_name}.json
|
||||
[ -f "${out}" ] && { echo "[skip] ${out_name}"; return; }
|
||||
echo "[gpu${gpu} eval] ${out_name}"
|
||||
cd ${ROOT}/Unified_CFM
|
||||
CUDA_VISIBLE_DEVICES=${gpu} stdbuf -oL uv run --no-sync python -u ${UNIFIED_EVAL} \
|
||||
--model-dir ${model_dir} \
|
||||
--target-store ${TARGETS[${target}_store]} \
|
||||
--target-flows ${TARGETS[${target}_flows]} \
|
||||
--target-flow-features ${features} \
|
||||
--out ${out} \
|
||||
--n-benign 10000 --n-attack 10000 --seed 42 \
|
||||
--T 64 --batch-size 256 --n-steps 16 \
|
||||
> ${CROSS_DIR}/${out_name}.log 2>&1
|
||||
}
|
||||
|
||||
run_mixed_eval() {
|
||||
local gpu=$1 model_dir=$2 target=$3 out_name=$4
|
||||
local out=${CROSS_DIR}/${out_name}.json
|
||||
[ -f "${out}" ] && { echo "[skip] ${out_name}"; return; }
|
||||
echo "[gpu${gpu} mixed eval] ${out_name}"
|
||||
cd ${ROOT}/Mixed_CFM
|
||||
CUDA_VISIBLE_DEVICES=${gpu} stdbuf -oL uv run --no-sync python -u ${MIXED_EVAL} \
|
||||
--model-dir ${model_dir} \
|
||||
--target-store ${TARGETS[${target}_store]} \
|
||||
--target-flows ${TARGETS[${target}_flows]} \
|
||||
--target-flow-features ${TARGETS[${target}_features]} \
|
||||
--out ${out} \
|
||||
--n-benign 10000 --n-attack 10000 --seed 42 \
|
||||
--T 64 --batch-size 256 --n-steps 16 \
|
||||
> ${CROSS_DIR}/${out_name}.log 2>&1
|
||||
}
|
||||
|
||||
# === GPU 0 chain: baselines + route_a, both targets ===
|
||||
{
|
||||
for prefix_route in "baseline_ciciot2023:baseline" "route_a_causal_ciciot2023:route_a_causal"; do
|
||||
prefix=${prefix_route%:*}
|
||||
short=${prefix_route#*:}
|
||||
for seed in 42 43 44; do
|
||||
md=${ROOT}/artifacts/route_comparison/${prefix}_seed${seed}
|
||||
[ -f "${md}/model.pt" ] || continue
|
||||
for target in cicids2017 cicddos2019; do
|
||||
run_unified_eval 0 "${md}" "${target}" "${TARGETS[${target}_features]}" \
|
||||
"${short}_seed${seed}_to_${target}"
|
||||
done
|
||||
done
|
||||
done
|
||||
echo "[gpu0 cross chain done]"
|
||||
} > /tmp/cross_gpu0.log 2>&1 &
|
||||
GPU0=$!
|
||||
|
||||
# === GPU 1 chain: route_b (uses spectral features) + route_c (mixed) ===
|
||||
{
|
||||
# route_b: must use flow_features_spectral.parquet
|
||||
for seed in 42 43 44; do
|
||||
md=${ROOT}/artifacts/route_comparison/route_b_spectral_ciciot2023_seed${seed}
|
||||
[ -f "${md}/model.pt" ] || continue
|
||||
for target in cicids2017 cicddos2019; do
|
||||
run_unified_eval 1 "${md}" "${target}" "${TARGETS[${target}_features_spectral]}" \
|
||||
"route_b_spectral_seed${seed}_to_${target}"
|
||||
done
|
||||
done
|
||||
|
||||
# route_c: Mixed_CFM eval (uses canonical flow_features)
|
||||
for seed in 42 43 44; do
|
||||
md=${ROOT}/artifacts/route_comparison/route_c_mixed_ciciot2023_seed${seed}
|
||||
[ -f "${md}/model.pt" ] || continue
|
||||
for target in cicids2017 cicddos2019; do
|
||||
run_mixed_eval 1 "${md}" "${target}" \
|
||||
"route_c_mixed_seed${seed}_to_${target}"
|
||||
done
|
||||
done
|
||||
echo "[gpu1 cross chain done]"
|
||||
} > /tmp/cross_gpu1.log 2>&1 &
|
||||
GPU1=$!
|
||||
|
||||
wait $GPU0
|
||||
wait $GPU1
|
||||
echo "[all cross done]"
|
||||
ls -la ${CROSS_DIR}/*.json | wc -l
|
||||
88
artifacts/route_comparison/run_full_cross_matrix.sh
Executable file
88
artifacts/route_comparison/run_full_cross_matrix.sh
Executable file
@@ -0,0 +1,88 @@
|
||||
#!/bin/bash
|
||||
# Run all missing cross-direction evals for A+C combo.
|
||||
# Targets are routed to packets-npz or full_store as appropriate.
|
||||
|
||||
set -e
|
||||
ROOT=/home/chy/mambafortrafficmodeling
|
||||
EVAL=${ROOT}/Mixed_CFM/eval_cross.py
|
||||
CROSS_DIR=${ROOT}/artifacts/route_comparison/cross
|
||||
mkdir -p ${CROSS_DIR}
|
||||
|
||||
# Target paths
|
||||
TGT_iscxtor2016_npz=${ROOT}/datasets/iscxtor2016/processed/packets.npz
|
||||
TGT_iscxtor2016_flows=${ROOT}/datasets/iscxtor2016/processed/flows.parquet
|
||||
TGT_iscxtor2016_features=${ROOT}/datasets/iscxtor2016/processed/flow_features.parquet
|
||||
TGT_iscxtor2016_label=nontor
|
||||
TGT_iscxtor2016_natk=1888
|
||||
|
||||
TGT_cicids2017_store=${ROOT}/datasets/cicids2017/processed/full_store
|
||||
TGT_cicids2017_flows=${ROOT}/datasets/cicids2017/processed/flows.parquet
|
||||
TGT_cicids2017_features=${ROOT}/datasets/cicids2017/processed/flow_features.parquet
|
||||
TGT_cicids2017_label=normal
|
||||
|
||||
TGT_cicddos2019_store=${ROOT}/datasets/cicddos2019/processed/full_store
|
||||
TGT_cicddos2019_flows=${ROOT}/datasets/cicddos2019/processed/flows.parquet
|
||||
TGT_cicddos2019_features=${ROOT}/datasets/cicddos2019/processed/flow_features.parquet
|
||||
TGT_cicddos2019_label=normal
|
||||
|
||||
TGT_ciciot2023_store=${ROOT}/datasets/ciciot2023/processed/full_store
|
||||
TGT_ciciot2023_flows=${ROOT}/datasets/ciciot2023/processed/full_store/flows.parquet
|
||||
TGT_ciciot2023_features=${ROOT}/datasets/ciciot2023/processed/flow_features.parquet
|
||||
TGT_ciciot2023_label=normal
|
||||
|
||||
run_one() {
|
||||
local gpu=$1 src=$2 tgt=$3 seed=$4
|
||||
local md=${ROOT}/artifacts/route_comparison/route_ac_combo_${src}_seed${seed}
|
||||
local out=${CROSS_DIR}/route_ac_combo_seed${seed}_${src}_to_${tgt}.json
|
||||
if [ -f "${out}" ]; then echo "[skip] ${src}→${tgt} seed${seed}"; return; fi
|
||||
if [ ! -f "${md}/model.pt" ]; then echo "[missing] ${md}/model.pt"; return; fi
|
||||
|
||||
# Resolve target args
|
||||
local tgt_args
|
||||
if [ "${tgt}" = "iscxtor2016" ]; then
|
||||
tgt_args="--target-packets-npz ${TGT_iscxtor2016_npz} --target-flows ${TGT_iscxtor2016_flows} --target-flow-features ${TGT_iscxtor2016_features} --benign-label nontor --n-attack 1888"
|
||||
elif [ "${tgt}" = "cicids2017" ]; then
|
||||
tgt_args="--target-store ${TGT_cicids2017_store} --target-flows ${TGT_cicids2017_flows} --target-flow-features ${TGT_cicids2017_features} --benign-label normal --n-attack 10000"
|
||||
elif [ "${tgt}" = "cicddos2019" ]; then
|
||||
tgt_args="--target-store ${TGT_cicddos2019_store} --target-flows ${TGT_cicddos2019_flows} --target-flow-features ${TGT_cicddos2019_features} --benign-label normal --n-attack 10000"
|
||||
elif [ "${tgt}" = "ciciot2023" ]; then
|
||||
tgt_args="--target-store ${TGT_ciciot2023_store} --target-flows ${TGT_ciciot2023_flows} --target-flow-features ${TGT_ciciot2023_features} --benign-label normal --n-attack 10000"
|
||||
fi
|
||||
|
||||
echo "[gpu${gpu}] ${src} → ${tgt} seed${seed}"
|
||||
cd ${ROOT}/Mixed_CFM
|
||||
CUDA_VISIBLE_DEVICES=${gpu} stdbuf -oL uv run --no-sync python -u ${EVAL} \
|
||||
--model-dir ${md} \
|
||||
${tgt_args} \
|
||||
--out ${out} \
|
||||
--n-benign 10000 --seed 42 --T 64 --batch-size 256 --n-steps 16 \
|
||||
> ${CROSS_DIR}/route_ac_combo_seed${seed}_${src}_to_${tgt}.log 2>&1
|
||||
}
|
||||
|
||||
# 8 missing directions × 3 seeds = 24 evals
|
||||
# Split across 2 GPUs to balance load
|
||||
{
|
||||
for dir in "ciciot2023:iscxtor2016" "cicids2017:iscxtor2016" "cicddos2019:iscxtor2016" "iscxtor2016:cicids2017"; do
|
||||
src=${dir%:*}; tgt=${dir#*:}
|
||||
for seed in 42 43 44; do
|
||||
run_one 0 ${src} ${tgt} ${seed}
|
||||
done
|
||||
done
|
||||
echo "[gpu0 done]"
|
||||
} > /tmp/cross_matrix_gpu0.log 2>&1 &
|
||||
G0=$!
|
||||
|
||||
{
|
||||
for dir in "cicids2017:ciciot2023" "cicddos2019:ciciot2023" "iscxtor2016:cicddos2019" "iscxtor2016:ciciot2023"; do
|
||||
src=${dir%:*}; tgt=${dir#*:}
|
||||
for seed in 42 43 44; do
|
||||
run_one 1 ${src} ${tgt} ${seed}
|
||||
done
|
||||
done
|
||||
echo "[gpu1 done]"
|
||||
} > /tmp/cross_matrix_gpu1.log 2>&1 &
|
||||
G1=$!
|
||||
|
||||
wait $G0
|
||||
wait $G1
|
||||
echo "[all done]"
|
||||
45
artifacts/route_comparison/run_phase1_all.sh
Executable file
45
artifacts/route_comparison/run_phase1_all.sh
Executable file
@@ -0,0 +1,45 @@
|
||||
#!/bin/bash
|
||||
# Run phase1 eval on all route_comparison models.
|
||||
# Output: <model_dir>/phase1_summary.json + phase1_scores.npz
|
||||
#
|
||||
# Usage:
|
||||
# bash artifacts/route_comparison/run_phase1_all.sh [GPU_ID]
|
||||
#
|
||||
# Default GPU_ID = 0. Each eval takes ~3-5 min with the caps below.
|
||||
|
||||
set -e
|
||||
GPU_ID="${1:-0}"
|
||||
ROOT=/home/chy/mambafortrafficmodeling
|
||||
EVAL=${ROOT}/artifacts/verify_2026_04_24/eval_phase1_unified.py
|
||||
|
||||
models=(
|
||||
baseline_ciciot2023_seed42
|
||||
baseline_ciciot2023_seed43
|
||||
baseline_ciciot2023_seed44
|
||||
route_a_causal_ciciot2023_seed42
|
||||
route_a_causal_ciciot2023_seed43
|
||||
route_a_causal_ciciot2023_seed44
|
||||
)
|
||||
|
||||
cd ${ROOT}/Unified_CFM
|
||||
for name in "${models[@]}"; do
|
||||
model_dir=${ROOT}/artifacts/route_comparison/${name}
|
||||
if [ ! -f "${model_dir}/model.pt" ]; then
|
||||
echo "[skip] ${name}: model.pt missing"
|
||||
continue
|
||||
fi
|
||||
out_dir=${model_dir}
|
||||
if [ -f "${out_dir}/phase1_summary.json" ]; then
|
||||
echo "[skip] ${name}: phase1_summary.json exists"
|
||||
continue
|
||||
fi
|
||||
echo "[eval] ${name}"
|
||||
CUDA_VISIBLE_DEVICES=${GPU_ID} stdbuf -oL uv run --no-sync python -u ${EVAL} \
|
||||
--model-dir ${model_dir} --out-dir ${out_dir} \
|
||||
--batch-size 256 --n-steps 16 \
|
||||
--jacobian-n-eps 4 \
|
||||
--n-val-cap 5000 --n-atk-cap 10000 \
|
||||
2>&1 | tee ${model_dir}/phase1.log | tail -5
|
||||
echo "[done] ${name}"
|
||||
done
|
||||
echo "[all done]"
|
||||
Reference in New Issue
Block a user