@bvb563

🚨 Major Research Alert : Poisoning Attacks in Decentralised RL (GRPO) Are More Dangerous Than Expected @gensynai A new study uncovers a critical vulnerability in decentralised reinforcement learning systems specifically those built on Generalized Reinforcement Policy Optimization (GRPO). Because GRPO assigns a single scalar reward to an entire completion, even a handful of malicious nodes can secretly embed high-reward token patterns that trick the policy into updating in harmful directions. The result? 👉 Rapid model drift 👉 Widespread contamination across nodes 👉 System-level collapse in reasoning quality 🔥 Two Attack Vectors Identified 🔹 1. In-context attacks Attackers modify the actual reasoning traces, altering equations, logic steps, or chain-of-thought. This corrupts domain reasoning in math and code tasks. 🔹 2. Out-of-context attacks Attackers append irrelevant or nonsensical text that still receives high reward. This injects noise into the model’s style, structure, and distribution even when unrelated to the task. Both attack classes were shown to be highly effective in decentralized settings. 🛡️ Proposed Defenses (and When to Use Them) The paper introduces two powerful protection mechanisms: 1️⃣ Log-probability verification (for homogeneous models) Check if the model itself would plausibly generate the submitted tokens. If the sequence looks too unlikely → reject it. 2️⃣ LLM-as-a-Judge (for heterogeneous models) Use an external model to evaluate whether completions appear manipulated, corrupted, or adversarial. ✅ Results Both defenses: ✔ Block the majority of in-context and out-of-context poisoning ✔ Preserve the efficiency and scalability of decentralized GRPO ✔ Prevent malicious nodes from steering global policy updates 📌 Takeaway As decentralised RL becomes more widely adopted especially in community-driven or federated training environments robust defense mechanisms are no longer optional. Securing reward-based aggregation is essential to prevent silent model corruption and maintain long-term reliability.

0 reply

0 recast

0 reaction