Open
Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum#4
Conversation
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add implementation for adaptive attacks documentation
Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum
Feb 3, 2026
self1am
marked this pull request as ready for review
February 3, 2026 07:42
self1am
approved these changes
Feb 3, 2026
Add comprehensive documentation and tooling to run production FL experiments: documentation index, architecture diagrams, execution checklist, anomaly scoring explanation, quick reference, setup summary, VM troubleshooting, and server logs analysis. Add experiment automation and helper scripts (optimize_gcp_instance.sh, run_production_experiments.sh, test_connection.sh), experiment configs for 100-client runs, analysis/visualization scripts, design assets, and supporting HTML/PNG files. Also update multiple client training log JSONs to include recent run data. These changes provide the docs, configs and scripts needed to run, monitor and analyze large-scale experiments on a GCP instance.
Switch server startup from a background thread to a multiprocessing.Process so the Flower server can register signal handlers from its own main thread. Add process-alive check, reduce server startup wait (5s -> 3s), and implement cleanup/terminate/join logic to ensure graceful shutdown. Relax resource spawning logic in client_orchestrator: lower the minimum available memory threshold to 300MB (from 500MB) and remove the blocking CPU percent check so clients can spawn more aggressively under memory pressure. Add a small test config (experiments/configs/test_multiprocess_fix.yaml) and a MULTIPROCESS_FIX.md summary documenting the fixes and testing instructions. Files changed: src/orchestration/experiment_runner.py, src/orchestration/client_orchestrator.py, experiments/configs/test_multiprocess_fix.yaml, MULTIPROCESS_FIX.md.
Switch orchestrator synchronization from waiting for client processes to waiting for the server process. Add server_process parameter to ClientOrchestrator.run_experiment and ExperimentRunner to block on server_process.join(), then terminate clients after server completes. Redirect client stdout/stderr to logs/client_<id>.log and improve monitoring to surface recent client output on failures (last 500 chars). Update wait_for_completion to support indefinite waiting, longer polling interval, KeyboardInterrupt handling, and ensure clients are cleaned up. These changes fix an infinite "Waiting for N clients" loop by using server-driven completion and provide better client diagnostics.
Replace multiprocessing-based in-process server with an external subprocess to improve signal handling and capture logs. experiment_runner.py: spawn server via subprocess.Popen running run_server_with_eval.py, create logs/ directory and write server output to a per-experiment log file, switch liveness checks from is_alive() to poll(), and use terminate()/wait()/kill() with timeout handling (subprocess.TimeoutExpired). Also add sys import and remove multiprocessing usage. client_orchestrator.py: replace server_process.join() with server_process.wait() to block on subprocess termination. These changes centralize server logs and provide more robust lifecycle handling for the server process.
…r-only mode support
Introduce a new SimulationRunner (src/orchestration/simulation_runner.py) that runs Flower simulations with Ray-backed client execution. Includes an attack factory, centralized evaluation function, multiple aggregation strategy selection, and a client_fn that constructs EnhancedFLClient instances. Add a baseline experiment config for 100 clients (experiments/configs/baseline_100_clients.yaml) and update requirements to include ray>=2.0.0 to support Ray integration.
This reverts commit aa9e311.
Add cloud_vm_test.md documenting steps to set up a cloud VM, create a Python venv, use tmux, and run experiment simulations. Update experiment configs (adaptive_attacks_cognitive_defence.yaml and static_attacks_cognitive_defence.yaml) to change defence.strategy from "cognitive" to "cognitive_defence" to match the updated defence naming/implementation.
Add a comprehensive set of experiment YAMLs to standardize baseline and adaptive evaluations: static label-flip, DnyOpt, StatOpt, Min-Max across defence strategies (cognitive_defence, krum, trimmed_mean, vert, none). Include a clean baseline (no attack), unified settings (seed 123, 30 rounds, 100 clients) and target client lists to make comparisons reproducible. Fix a bug in adaptive_attacks_vertical_defence.yaml where strategy was "vertical" (fell through to NoDefence) by changing it to "vert" and adding parameters (kappa, history_size, projection_dim, learning_rate, min_history_rounds). Add copilot_analysis.md containing an extensive analysis and roadmap for improving the cognitive defence and benchmarking. Also add a helper script experiments/scripts/run_baseline_experiments.sh and update a few existing attack/attack-config files to align with the new experiments. These changes prepare the repo for systematic, reproducible experiments and further development of the cognitive defence.
Clamp predictor weight updates to avoid exploding gradients/NaNs. vert_defence.py: compute grad_W = outer(error, p_input) then clip by L2 norm (threshold 1.0) before applying learning_rate step. Added test_vert.py to reproduce and monitor predictor weight norms and NaNs during aggregation with synthetic client updates. Added two small repro scripts (fix_vert.py, fix_vert2.py) that demonstrate input normalization and gradient clipping behavior during iterative updates.
Introduce a POSG-based reinforcement-learning defence and related utilities: add CognitiveDefencePOSG (sac + GRU tracker), SACAgent, and ClientTracker (src/defences/cognitive_defence_posg.py, src/defences/sac_agent.py, src/defences/client_tracker.py) and export it from defences.__init__.yaml. Update baseline experiment configs to use the new cognitive_defence_posg strategy and include SAC/tracker hyperparameters. Fix device handling to avoid MPS/CUDA tensor mismatches in enhanced_client and simulation_runner by moving tensors to the model/device and set PYTORCH_ENABLE_MPS_FALLBACK early. Add CUDA/MPS checks and deterministic CuDNN setup in utils/config. Also add a cognitive_defence_v2 branch in experiment_runner to wire a new aggregation strategy. These changes enable a learned aggregation defence, improve reproducibility, and address Apple Silicon / GPU device issues.
- Add POSG/SAC parameters to defenceConfig dataclass (max_clients, obs_dim, belief_hidden_dim, sac_hidden_dims, lr, gamma, reward_*, buffer_capacity, batch_size, device) - Create src/server/cognitive_defence_posg_server.py that wraps CognitiveDefencePOSG as a Flower FedAvg strategy - Export POSGAggregationStrategy from src/server/__init__.py - Add cognitive_defence_posg branch in simulation_runner._create_strategy() so the runner can instantiate it from YAML config
- Remove extra 'config' positional argument from evaluate() override (Flower calls it as strategy.evaluate(round, parameters) with no config) - Use self.evaluate_fn (FedAvg attribute) instead of self._evaluate_fn - Call evaluate_fn with NDArrays directly, matching FedAvg parent behaviour - Store centralized_accuracy as float for SAC reward computation
Change experiment seed from 123 to 321 in two baseline cognitive defence config files to alter randomness/reproducibility for runs. Affected files: experiments/configs/baseline/01_static_label_flip_cognitive_defence.yaml and experiments/configs/baseline/02_adaptive_dny_opt_cognitive_defence.yaml.
…pact 2xH SAC state, FLTrust pairwise cosine warmup
Add experiment configs for CogDef v2 against several attacks (dynopt, label_flip, min_max, min_sum, stat_opt). Extend CognitiveDefenceV2 with a GRU-based per-client belief tracker and a population-level PCA+gap cluster outlier detector; compute a temporal anomaly score from inter-round observation changes. Wire up CognitiveAggregationStrategyV2 in the simulation runner and add CogDef v2 parameters to defenceConfig in utils/config.py. These changes enable detection of adaptive and coordinated attackers via temporal and cluster signals and expose tunable weights/thresholds in config files.
Add _majority_consensus to compute the geometric median of the majority (honest) cluster when a cluster split is detected, preventing attacker updates from pulling the global consensus direction (fixes degraded direction signal under label-flip/high attack fractions). Wire this majority consensus into the ORIENT flow when present. Include YELLOW in the flagged-count used for posture assessment so lower-tier detections (e.g. label-flip) trigger posture escalation. Apply a light reputation penalty for YELLOW clients (30% of full severity) to prevent persistently suspicious clients from being rewarded, while keeping stronger penalties for ORANGE/RED and normal rewards for GREEN.
Flower sends full model weights. In early rounds the base model dominates all client vectors → cosine similarity ≈ 0.9999 for every client, masking the attack signal entirely. Both direction and cluster detectors were blind. Fix: observe() now computes per-client delta = flat - round_mean. The consensus direction (geometric median), _detect_direction_anomaly(), and _detect_cluster_outliers() all operate on deltas. The base-model component cancels out, exposing the systematic direction divergence of label-flip attackers and making the bimodal cluster split visible in PCA space. norm detector and aggregation modes are unaffected — they use total_norm and full params respectively, which is correct for those purposes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…mean Two targeted fixes for the periodic accuracy dips observed in cogdefv2_dynopt-1.log: 1. RED escalation threshold: 0.50 → 0.35 With 40% malicious clients flagged every round (fraction=0.40), the posture was stuck at ORANGE and used trimmed mean. Lowering the threshold to 0.35 escalates to RED (Multi-Krum) which selects the tightest cluster — the honest majority — making it much harder for DynOpt to craft middle-band bypass updates. 2. Reputation-weighted trimmed mean in _aggregate_defensive() The old implementation took an unweighted mean over the middle band, so consistently-flagged clients (reputation ≈ 0.01 after 20+ penalty rounds) had equal influence to honest clients whenever they landed in the middle band. Now uses coordinate-wise weighted mean: band_w = reputation × sample_count, so long-penalised clients contribute proportionally less even when not trimmed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Label-flip attacks are semantically subtle: the adversary runs honest SGD on mislabeled data, producing parameter updates with normal norms and plausible full-vector directions. The adversarial signal concentrates in the last layer (classification head), where class-specific decision boundaries are encoded. In the full flattened delta it is diluted across tens-of-thousands of conv dimensions that are largely label-agnostic, making it invisible to both the direction detector (consensus near zero due to honest client diversity) and the cluster detector (no clear PCA gap in high-dimensional noise). Fix: observe() now extracts a per-client head_delta = last_layer_params - mean. direction_anomaly and cluster_outlier detectors use head_delta (highest SNR for label-flip) rather than the full delta. Smoke test confirms: honest cos_sim to head consensus: +0.993 (score ≈ 0.003) attacker cos_sim to head consensus: -0.996 (score ≈ 0.998) Full delta is preserved as cluster_delta for fallback and for future ablations. DynOpt is unaffected: adversarial perturbations also corrupt the classification head, so head_delta still captures the attack. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ndings Records all key discoveries, failure diagnoses, and design decisions made during CogDef v2 development, written to inform thesis and paper writing: - Why RL failed and how that motivates the analytical cognitive loop - Flower full-params vs gradients finding and the delta fix - Attack abstraction level as the determinant of detectability (novel) - Label-flip signal concentration in the classification head (novel) - LOF failure on coordinated attackers and the PCA+gap fix - Trimmed mean mid-band bypass and the RED threshold + weighted trim fix - MAPE-K self-tuning blindness without accurate underlying sensors - Algorithm design rationale (GRU, posture hysteresis, head-delta, geom median) - Literature positioning table and precise novel claim Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two complementary fixes for label-flip late-round collapse: 1. Convergence Resistance (new signal in temporal score) As the model converges, honest clients' classification-head updates shrink toward zero. Label-flip attackers keep generating large head-delta updates because the well-trained model strongly disagrees with their flipped labels. Tracks per-client EMA of (head_delta_norm / pop_20th_percentile_norm). As honest norms shrink toward the population floor, attacker resistance ratio climbs → log-normalised score rises → drives fused_score into ORANGE territory (>0.60) → clients rejected before Multi-Krum, not just down-weighted. temporal_score = max(instability_score, convergence_resistance_score) 2. Cold-Start Guard (direction + cluster detectors) In early rounds all head deltas are near-zero (model barely trained). Normalising near-zero vectors amplifies noise into random directions, causing the direction detector and cluster detector to flag every client (observed: 100/100 flagged at R2 in cogdefv2_label_flip-3.log). Fix: suppress both detectors when pop_norm_floor < 1e-3. Smoke test (10 clients, 6 honest/4 label-flip, 15 rounds with honest convergence): honest: dir=0.001, temporal=0.015-0.034 (stays low throughout) attack: dir=0.946, temporal=0.198-0.218 (builds up and holds) DynOpt unaffected: direction + cluster fire from R1; convergence resistance only adds confirmation after honest clients begin to converge. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause of 78/100 over-flagging at R16:
pop_norm_floor used 20th percentile (fastest convergers as reference).
In real FL, honest clients have natural 5-10x norm spread between fast
and slow convergers. A "slow" honest client had ratio = own_norm /
20th_pct_norm ≈ 5-8x, EMA accumulated to 5-8, log10(5-8)=0.7-0.9 —
full false-positive temporal score. This cascaded: honest clients
penalised → model partially poisoned → more honest clients anomalous
→ 78/100 flagged by R16, accuracy drops from 0.647 to 0.323.
Fix 1 — use 50th percentile (median) as pop_norm_floor:
The median represents the typical converger, not the fastest 20%.
With shared global model in FL, honest updates are similar (5-10% spread),
so honest ratio ≈ 1.0-1.5x median → stays below threshold.
Attackers resisting convergence reach 5-25x median in late rounds.
Fix 2 — add RESISTANCE_THRESHOLD = 2.0 before signal fires:
_detect_convergence_resistance() now returns 0.0 when EMA < 2.0 and
normalises as log10(ema / 2.0) above the threshold:
ema < 2x → 0.0 (no signal)
ema = 4x → 0.30
ema = 10x → 0.70
ema = 20x → 1.0
Smoke test (realistic FL: shared global model, honest 0.8-1.2x spread):
R1-R20: exactly 40/100 flagged, zero false positives, posture RED from R3.
h_fused drops 0.03 → 0.000, a_fused stable at 0.63 throughout.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sensus Root cause of label-flip R2: 99/100 flagged catastrophe: The geometric median for direction consensus used raw (unnormalised) head delta vectors. In early rounds, label-flip attackers generate large-magnitude head deltas because the barely-trained global model strongly disagrees with their flipped labels → large loss gradients (10x honest magnitude at R2). Honest clients have small head deltas at the same stage. The geometric median was magnitude-weighted and got pulled toward the attacker direction, making honest clients look like they're pointing away from consensus → direction_score ≈ 1.0 for ~99/100 clients → catastrophic over-flagging at round 2 in every label-flip run observed in production. Fix: normalise all head_delta vectors to unit sphere before computing geometric median in both observe() and _majority_consensus(). Each of 60 honest clients now has equal directional vote regardless of update magnitude. 60 honest unit vectors vs 40 attacker unit vectors: geometric median converges to the honest majority direction in all rounds, including early rounds when norms differ by 10x. Smoke test result (attacker scale 10x honest at R2): All 30 rounds: exactly 40/100 flagged, h_dir=0.004, a_dir=0.989, posture RED from R3. No false positives regardless of magnitude gap. DynOpt / StatOpt / MinMax unaffected: their attacks shift directions, not just magnitudes, so the normalised consensus still separates them from honest clients. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…N rounds Problem: reputation recovery rate=0.03 is too slow relative to penalty. A client mis-flagged YELLOW for 3 rounds drops to rep≈0.38. With flat recovery it takes 7+ rounds to reach rep=0.5 and 30+ rounds to reach 0.9. Any over-flagging cascade (like the R2 spike in label-flip) permanently suppresses honest clients for the entire 30-round experiment. Fix: scale the recovery bonus by consecutive_clean rounds: accel = min(1.0 + 0.5 × consecutive_clean, 4.0) bonus = recovery_rate × accel × (1 - rep) Recovery milestones for an honest client falsely flagged 3 rounds: Before fix: rep>0.5 in 7 rounds, rep>0.9 in 30+ rounds After fix: rep>0.5 in 4 rounds, rep>0.7 in 8 rounds, rep>0.9 in 17 rounds Attackers are unaffected: they never clear GREEN (fused_score stays high every round), so consecutive_clean stays 0 → accel=1.0 → same slow recovery they had before. Attacker rep decays to ≈0.000 after 30 rounds regardless. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New sections added:
10. Magnitude-Weighted Consensus Inversion — root cause of R2:99/100
over-flagging spike, unit-normalisation fix, thesis significance
11. Reputation Ratchet — how false positives compound over 30 rounds,
asymmetric penalty/recovery, accelerated recovery fix
12. Label-Flip Detection vs Aggregation — the key thesis finding:
we ARE detecting the 40 attackers correctly in every round, the
model degrades due to collateral false positives, not detection failure
Section 16 (Iterative Debugging Log) added:
Full diary of all 10 bugs found during label-flip campaign, each with
root cause, fix, and thesis significance. Written to defend the
iterative methodology and show systematic progress even where final
results are pending.
Section 15 (Experimental Evidence) updated:
DynOpt-4, StatOpt-1, MinMax confirmed results added.
LabelFlip progression table across all 5 runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds four adaptive attack strategies that learn from defense responses and optimize evasion. Each attack implements a distinct optimization approach based on published adversarial FL research.
Implementations
Base Infrastructure (
adaptive_base.py)AdaptiveAttackclass with feedback collection, detection/acceptance rate tracking, and adaptation hooksAttack Strategies
Integration
Client Runner (
client_runner.py)Extended
create_attack()to instantiate adaptive attacks from YAML configs with attack-specific parameters.Configuration (
config.py)AttackConfigdataclass now includes parameters for all four attacks (constraint_factor, learning_rate, defense_models, distance_weight, etc.). Default initialization ensures uniform threat model weights.Usage
YAML Configuration
Documentation
docs/ADAPTIVE_ATTACKS.md: Algorithm descriptions, mathematical formulations, defense evasion strategies, academic references (Fang et al. USENIX'20, Baruch et al. NeurIPS'19, Shejwalkar & Houmansadr NDSS'21, Bhagoji et al. ICML'19)docs/IMPLEMENTATION_SUMMARY.md: Implementation details, architecture, integration pointsexperiments/configs/{stat_opt,dny_opt,min_max,min_sum}_attack_test.yamlFiles Changed
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.