Skip to content

Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum - #4

Open
self1am with Copilot wants to merge 68 commits into
mainfrom
copilot/implement-adaptive-attacks
Open

Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum#4
self1am with Copilot wants to merge 68 commits into
mainfrom
copilot/implement-adaptive-attacks

Conversation

Copilot AI commented Feb 3, 2026

Copy link
Copy Markdown
Contributor

Adds four adaptive attack strategies that learn from defense responses and optimize evasion. Each attack implements a distinct optimization approach based on published adversarial FL research.

Implementations

Base Infrastructure (adaptive_base.py)

  • AdaptiveAttack class with feedback collection, detection/acceptance rate tracking, and adaptation hooks
  • All adaptive attacks inherit common feedback mechanism

Attack Strategies

  • stat-opt: Constrains malicious updates within statistical bounds (mean ± k·σ) of benign clients. Adapts constraint factor based on detection rate.
  • dny-opt: Q-learning with ε-greedy exploration across attack intensities and techniques (sign flip, gradient noise, scaling). Updates Q-values based on acceptance/rejection rewards.
  • min-max: Game-theoretic optimization assuming optimal defender. Evaluates attack performance across defense ensemble (Krum, trimmed mean, median). Adapts threat model weights.
  • min-sum: Minimizes sum of distances to benign updates via gradient descent optimization. Balances stealth (distance minimization) vs impact (attack magnitude).

Integration

Client Runner (client_runner.py)
Extended create_attack() to instantiate adaptive attacks from YAML configs with attack-specific parameters.

Configuration (config.py)
AttackConfig dataclass now includes parameters for all four attacks (constraint_factor, learning_rate, defense_models, distance_weight, etc.). Default initialization ensures uniform threat model weights.

Usage

from src.attacks import StatOptAttack, DnyOptAttack, MinMaxAttack, MinSumAttack

# Statistical optimization - evades statistical defenses
stat_attack = StatOptAttack(
    intensity=0.2,
    constraint_factor=1.5,  # Within 1.5σ of benign mean
    target_clients=[0, 1, 2]
)

# Q-learning adaptation
dny_attack = DnyOptAttack(
    intensity=0.15,
    learning_rate=0.1,
    exploration_rate=0.1
)

# Game-theoretic with defense ensemble
minmax_attack = MinMaxAttack(
    intensity=0.2,
    defense_models=['krum', 'trimmed_mean', 'cognitive']
)

# Distance minimization
minsum_attack = MinSumAttack(
    intensity=0.2,
    distance_weight=0.7  # 70% weight on distance minimization
)

YAML Configuration

attacks:
  - enabled: true
    attack_type: "stat_opt"
    intensity: 0.2
    constraint_factor: 1.5
    target_clients: [0, 1, 2]

Documentation

  • docs/ADAPTIVE_ATTACKS.md: Algorithm descriptions, mathematical formulations, defense evasion strategies, academic references (Fang et al. USENIX'20, Baruch et al. NeurIPS'19, Shejwalkar & Houmansadr NDSS'21, Bhagoji et al. ICML'19)
  • docs/IMPLEMENTATION_SUMMARY.md: Implementation details, architecture, integration points
  • Example configs: experiments/configs/{stat_opt,dny_opt,min_max,min_sum}_attack_test.yaml

Files Changed

  • New: 11 files (4 attack implementations, base class, 5 configs, 2 docs)
  • Modified: 5 files (client_runner, config, init, README, test_local_setup)
  • Total: +2,209 lines

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 4 commits February 3, 2026 07:28
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Copilot AI changed the title [WIP] Add implementation for adaptive attacks documentation Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum Feb 3, 2026
Copilot AI requested a review from self1am February 3, 2026 07:39
@self1am
self1am marked this pull request as ready for review February 3, 2026 07:42
Add comprehensive documentation and tooling to run production FL experiments: documentation index, architecture diagrams, execution checklist, anomaly scoring explanation, quick reference, setup summary, VM troubleshooting, and server logs analysis. Add experiment automation and helper scripts (optimize_gcp_instance.sh, run_production_experiments.sh, test_connection.sh), experiment configs for 100-client runs, analysis/visualization scripts, design assets, and supporting HTML/PNG files. Also update multiple client training log JSONs to include recent run data. These changes provide the docs, configs and scripts needed to run, monitor and analyze large-scale experiments on a GCP instance.
Switch server startup from a background thread to a multiprocessing.Process so the Flower server can register signal handlers from its own main thread. Add process-alive check, reduce server startup wait (5s -> 3s), and implement cleanup/terminate/join logic to ensure graceful shutdown.

Relax resource spawning logic in client_orchestrator: lower the minimum available memory threshold to 300MB (from 500MB) and remove the blocking CPU percent check so clients can spawn more aggressively under memory pressure.

Add a small test config (experiments/configs/test_multiprocess_fix.yaml) and a MULTIPROCESS_FIX.md summary documenting the fixes and testing instructions.

Files changed: src/orchestration/experiment_runner.py, src/orchestration/client_orchestrator.py, experiments/configs/test_multiprocess_fix.yaml, MULTIPROCESS_FIX.md.
Switch orchestrator synchronization from waiting for client processes to waiting for the server process. Add server_process parameter to ClientOrchestrator.run_experiment and ExperimentRunner to block on server_process.join(), then terminate clients after server completes. Redirect client stdout/stderr to logs/client_<id>.log and improve monitoring to surface recent client output on failures (last 500 chars). Update wait_for_completion to support indefinite waiting, longer polling interval, KeyboardInterrupt handling, and ensure clients are cleaned up. These changes fix an infinite "Waiting for N clients" loop by using server-driven completion and provide better client diagnostics.
Replace multiprocessing-based in-process server with an external subprocess to improve signal handling and capture logs. experiment_runner.py: spawn server via subprocess.Popen running run_server_with_eval.py, create logs/ directory and write server output to a per-experiment log file, switch liveness checks from is_alive() to poll(), and use terminate()/wait()/kill() with timeout handling (subprocess.TimeoutExpired). Also add sys import and remove multiprocessing usage. client_orchestrator.py: replace server_process.join() with server_process.wait() to block on subprocess termination. These changes centralize server logs and provide more robust lifecycle handling for the server process.
Introduce a new SimulationRunner (src/orchestration/simulation_runner.py) that runs Flower simulations with Ray-backed client execution. Includes an attack factory, centralized evaluation function, multiple aggregation strategy selection, and a client_fn that constructs EnhancedFLClient instances. Add a baseline experiment config for 100 clients (experiments/configs/baseline_100_clients.yaml) and update requirements to include ray>=2.0.0 to support Ray integration.
This reverts commit aa9e311.
self1am and others added 30 commits February 21, 2026 15:15
Add cloud_vm_test.md documenting steps to set up a cloud VM, create a Python venv, use tmux, and run experiment simulations. Update experiment configs (adaptive_attacks_cognitive_defence.yaml and static_attacks_cognitive_defence.yaml) to change defence.strategy from "cognitive" to "cognitive_defence" to match the updated defence naming/implementation.
Add a comprehensive set of experiment YAMLs to standardize baseline and adaptive evaluations: static label-flip, DnyOpt, StatOpt, Min-Max across defence strategies (cognitive_defence, krum, trimmed_mean, vert, none). Include a clean baseline (no attack), unified settings (seed 123, 30 rounds, 100 clients) and target client lists to make comparisons reproducible. Fix a bug in adaptive_attacks_vertical_defence.yaml where strategy was "vertical" (fell through to NoDefence) by changing it to "vert" and adding parameters (kappa, history_size, projection_dim, learning_rate, min_history_rounds). Add copilot_analysis.md containing an extensive analysis and roadmap for improving the cognitive defence and benchmarking. Also add a helper script experiments/scripts/run_baseline_experiments.sh and update a few existing attack/attack-config files to align with the new experiments. These changes prepare the repo for systematic, reproducible experiments and further development of the cognitive defence.
Clamp predictor weight updates to avoid exploding gradients/NaNs. vert_defence.py: compute grad_W = outer(error, p_input) then clip by L2 norm (threshold 1.0) before applying learning_rate step. Added test_vert.py to reproduce and monitor predictor weight norms and NaNs during aggregation with synthetic client updates. Added two small repro scripts (fix_vert.py, fix_vert2.py) that demonstrate input normalization and gradient clipping behavior during iterative updates.
Introduce a POSG-based reinforcement-learning defence and related utilities: add CognitiveDefencePOSG (sac + GRU tracker), SACAgent, and ClientTracker (src/defences/cognitive_defence_posg.py, src/defences/sac_agent.py, src/defences/client_tracker.py) and export it from defences.__init__.yaml. Update baseline experiment configs to use the new cognitive_defence_posg strategy and include SAC/tracker hyperparameters. Fix device handling to avoid MPS/CUDA tensor mismatches in enhanced_client and simulation_runner by moving tensors to the model/device and set PYTORCH_ENABLE_MPS_FALLBACK early. Add CUDA/MPS checks and deterministic CuDNN setup in utils/config. Also add a cognitive_defence_v2 branch in experiment_runner to wire a new aggregation strategy. These changes enable a learned aggregation defence, improve reproducibility, and address Apple Silicon / GPU device issues.
- Add POSG/SAC parameters to defenceConfig dataclass (max_clients,
  obs_dim, belief_hidden_dim, sac_hidden_dims, lr, gamma, reward_*,
  buffer_capacity, batch_size, device)
- Create src/server/cognitive_defence_posg_server.py that wraps
  CognitiveDefencePOSG as a Flower FedAvg strategy
- Export POSGAggregationStrategy from src/server/__init__.py
- Add cognitive_defence_posg branch in simulation_runner._create_strategy()
  so the runner can instantiate it from YAML config
- Remove extra 'config' positional argument from evaluate() override
  (Flower calls it as strategy.evaluate(round, parameters) with no config)
- Use self.evaluate_fn (FedAvg attribute) instead of self._evaluate_fn
- Call evaluate_fn with NDArrays directly, matching FedAvg parent behaviour
- Store centralized_accuracy as float for SAC reward computation
Change experiment seed from 123 to 321 in two baseline cognitive defence config files to alter randomness/reproducibility for runs. Affected files: experiments/configs/baseline/01_static_label_flip_cognitive_defence.yaml and experiments/configs/baseline/02_adaptive_dny_opt_cognitive_defence.yaml.
…pact 2xH SAC state, FLTrust pairwise cosine warmup
Add experiment configs for CogDef v2 against several attacks (dynopt, label_flip, min_max, min_sum, stat_opt). Extend CognitiveDefenceV2 with a GRU-based per-client belief tracker and a population-level PCA+gap cluster outlier detector; compute a temporal anomaly score from inter-round observation changes. Wire up CognitiveAggregationStrategyV2 in the simulation runner and add CogDef v2 parameters to defenceConfig in utils/config.py. These changes enable detection of adaptive and coordinated attackers via temporal and cluster signals and expose tunable weights/thresholds in config files.
Add _majority_consensus to compute the geometric median of the majority (honest) cluster when a cluster split is detected, preventing attacker updates from pulling the global consensus direction (fixes degraded direction signal under label-flip/high attack fractions). Wire this majority consensus into the ORIENT flow when present. Include YELLOW in the flagged-count used for posture assessment so lower-tier detections (e.g. label-flip) trigger posture escalation. Apply a light reputation penalty for YELLOW clients (30% of full severity) to prevent persistently suspicious clients from being rewarded, while keeping stronger penalties for ORANGE/RED and normal rewards for GREEN.
Flower sends full model weights. In early rounds the base model dominates
all client vectors → cosine similarity ≈ 0.9999 for every client, masking
the attack signal entirely. Both direction and cluster detectors were blind.

Fix: observe() now computes per-client delta = flat - round_mean. The
consensus direction (geometric median), _detect_direction_anomaly(), and
_detect_cluster_outliers() all operate on deltas. The base-model component
cancels out, exposing the systematic direction divergence of label-flip
attackers and making the bimodal cluster split visible in PCA space.

norm detector and aggregation modes are unaffected — they use total_norm
and full params respectively, which is correct for those purposes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…mean

Two targeted fixes for the periodic accuracy dips observed in cogdefv2_dynopt-1.log:

1. RED escalation threshold: 0.50 → 0.35
   With 40% malicious clients flagged every round (fraction=0.40), the posture
   was stuck at ORANGE and used trimmed mean.  Lowering the threshold to 0.35
   escalates to RED (Multi-Krum) which selects the tightest cluster — the honest
   majority — making it much harder for DynOpt to craft middle-band bypass updates.

2. Reputation-weighted trimmed mean in _aggregate_defensive()
   The old implementation took an unweighted mean over the middle band, so
   consistently-flagged clients (reputation ≈ 0.01 after 20+ penalty rounds)
   had equal influence to honest clients whenever they landed in the middle band.
   Now uses coordinate-wise weighted mean: band_w = reputation × sample_count,
   so long-penalised clients contribute proportionally less even when not trimmed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Label-flip attacks are semantically subtle: the adversary runs honest SGD on
mislabeled data, producing parameter updates with normal norms and plausible
full-vector directions.  The adversarial signal concentrates in the last layer
(classification head), where class-specific decision boundaries are encoded.
In the full flattened delta it is diluted across tens-of-thousands of conv
dimensions that are largely label-agnostic, making it invisible to both the
direction detector (consensus near zero due to honest client diversity) and
the cluster detector (no clear PCA gap in high-dimensional noise).

Fix: observe() now extracts a per-client head_delta = last_layer_params - mean.
direction_anomaly and cluster_outlier detectors use head_delta (highest SNR for
label-flip) rather than the full delta.  Smoke test confirms:
  honest cos_sim to head consensus: +0.993  (score ≈ 0.003)
  attacker cos_sim to head consensus: -0.996  (score ≈ 0.998)

Full delta is preserved as cluster_delta for fallback and for future ablations.
DynOpt is unaffected: adversarial perturbations also corrupt the classification
head, so head_delta still captures the attack.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ndings

Records all key discoveries, failure diagnoses, and design decisions made
during CogDef v2 development, written to inform thesis and paper writing:

- Why RL failed and how that motivates the analytical cognitive loop
- Flower full-params vs gradients finding and the delta fix
- Attack abstraction level as the determinant of detectability (novel)
- Label-flip signal concentration in the classification head (novel)
- LOF failure on coordinated attackers and the PCA+gap fix
- Trimmed mean mid-band bypass and the RED threshold + weighted trim fix
- MAPE-K self-tuning blindness without accurate underlying sensors
- Algorithm design rationale (GRU, posture hysteresis, head-delta, geom median)
- Literature positioning table and precise novel claim

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two complementary fixes for label-flip late-round collapse:

1. Convergence Resistance (new signal in temporal score)
   As the model converges, honest clients' classification-head updates shrink
   toward zero.  Label-flip attackers keep generating large head-delta updates
   because the well-trained model strongly disagrees with their flipped labels.
   Tracks per-client EMA of (head_delta_norm / pop_20th_percentile_norm).
   As honest norms shrink toward the population floor, attacker resistance
   ratio climbs → log-normalised score rises → drives fused_score into ORANGE
   territory (>0.60) → clients rejected before Multi-Krum, not just down-weighted.
   temporal_score = max(instability_score, convergence_resistance_score)

2. Cold-Start Guard (direction + cluster detectors)
   In early rounds all head deltas are near-zero (model barely trained).
   Normalising near-zero vectors amplifies noise into random directions, causing
   the direction detector and cluster detector to flag every client (observed:
   100/100 flagged at R2 in cogdefv2_label_flip-3.log).
   Fix: suppress both detectors when pop_norm_floor < 1e-3.

Smoke test (10 clients, 6 honest/4 label-flip, 15 rounds with honest convergence):
  honest: dir=0.001, temporal=0.015-0.034  (stays low throughout)
  attack: dir=0.946, temporal=0.198-0.218  (builds up and holds)

DynOpt unaffected: direction + cluster fire from R1; convergence resistance
only adds confirmation after honest clients begin to converge.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause of 78/100 over-flagging at R16:
  pop_norm_floor used 20th percentile (fastest convergers as reference).
  In real FL, honest clients have natural 5-10x norm spread between fast
  and slow convergers.  A "slow" honest client had ratio = own_norm /
  20th_pct_norm ≈ 5-8x, EMA accumulated to 5-8, log10(5-8)=0.7-0.9 —
  full false-positive temporal score.  This cascaded: honest clients
  penalised → model partially poisoned → more honest clients anomalous
  → 78/100 flagged by R16, accuracy drops from 0.647 to 0.323.

Fix 1 — use 50th percentile (median) as pop_norm_floor:
  The median represents the typical converger, not the fastest 20%.
  With shared global model in FL, honest updates are similar (5-10% spread),
  so honest ratio ≈ 1.0-1.5x median → stays below threshold.
  Attackers resisting convergence reach 5-25x median in late rounds.

Fix 2 — add RESISTANCE_THRESHOLD = 2.0 before signal fires:
  _detect_convergence_resistance() now returns 0.0 when EMA < 2.0 and
  normalises as log10(ema / 2.0) above the threshold:
    ema < 2x  → 0.0   (no signal)
    ema = 4x  → 0.30
    ema = 10x → 0.70
    ema = 20x → 1.0

Smoke test (realistic FL: shared global model, honest 0.8-1.2x spread):
  R1-R20: exactly 40/100 flagged, zero false positives, posture RED from R3.
  h_fused drops 0.03 → 0.000, a_fused stable at 0.63 throughout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sensus

Root cause of label-flip R2: 99/100 flagged catastrophe:
  The geometric median for direction consensus used raw (unnormalised)
  head delta vectors.  In early rounds, label-flip attackers generate
  large-magnitude head deltas because the barely-trained global model
  strongly disagrees with their flipped labels → large loss gradients
  (10x honest magnitude at R2).  Honest clients have small head deltas
  at the same stage.  The geometric median was magnitude-weighted and
  got pulled toward the attacker direction, making honest clients look
  like they're pointing away from consensus → direction_score ≈ 1.0
  for ~99/100 clients → catastrophic over-flagging at round 2 in every
  label-flip run observed in production.

Fix: normalise all head_delta vectors to unit sphere before computing
  geometric median in both observe() and _majority_consensus().
  Each of 60 honest clients now has equal directional vote regardless
  of update magnitude.  60 honest unit vectors vs 40 attacker unit
  vectors: geometric median converges to the honest majority direction
  in all rounds, including early rounds when norms differ by 10x.

Smoke test result (attacker scale 10x honest at R2):
  All 30 rounds: exactly 40/100 flagged, h_dir=0.004, a_dir=0.989,
  posture RED from R3.  No false positives regardless of magnitude gap.

DynOpt / StatOpt / MinMax unaffected: their attacks shift directions,
  not just magnitudes, so the normalised consensus still separates them
  from honest clients.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…N rounds

Problem: reputation recovery rate=0.03 is too slow relative to penalty.
  A client mis-flagged YELLOW for 3 rounds drops to rep≈0.38.  With flat
  recovery it takes 7+ rounds to reach rep=0.5 and 30+ rounds to reach 0.9.
  Any over-flagging cascade (like the R2 spike in label-flip) permanently
  suppresses honest clients for the entire 30-round experiment.

Fix: scale the recovery bonus by consecutive_clean rounds:
  accel = min(1.0 + 0.5 × consecutive_clean, 4.0)
  bonus = recovery_rate × accel × (1 - rep)

Recovery milestones for an honest client falsely flagged 3 rounds:
  Before fix: rep>0.5 in 7 rounds, rep>0.9 in 30+ rounds
  After fix:  rep>0.5 in 4 rounds, rep>0.7 in 8 rounds, rep>0.9 in 17 rounds

Attackers are unaffected: they never clear GREEN (fused_score stays high
every round), so consecutive_clean stays 0 → accel=1.0 → same slow
recovery they had before.  Attacker rep decays to ≈0.000 after 30 rounds
regardless.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New sections added:
  10. Magnitude-Weighted Consensus Inversion — root cause of R2:99/100
      over-flagging spike, unit-normalisation fix, thesis significance
  11. Reputation Ratchet — how false positives compound over 30 rounds,
      asymmetric penalty/recovery, accelerated recovery fix
  12. Label-Flip Detection vs Aggregation — the key thesis finding:
      we ARE detecting the 40 attackers correctly in every round, the
      model degrades due to collateral false positives, not detection failure

Section 16 (Iterative Debugging Log) added:
  Full diary of all 10 bugs found during label-flip campaign, each with
  root cause, fix, and thesis significance. Written to defend the
  iterative methodology and show systematic progress even where final
  results are pending.

Section 15 (Experimental Evidence) updated:
  DynOpt-4, StatOpt-1, MinMax confirmed results added.
  LabelFlip progression table across all 5 runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants