Also Worth Noting - 2026-05-31

Also Worth Noting

02 [Training] Trust Region On-Policy Distillation Standard on-policy distillation quietly breaks when teacher and student distributions diverge sharply, producing unreliable policy gradients that can kill optimization entirely. TrOPD adds a KL-bounded trust region to token-level credit assignment, keeping policy updates inside a stable region even when the gap is large. The failure mode it targets is one the OPD literature has mostly treated as a footnote rather than a first-class problem. Teams running on-policy distillation at scale should treat distribution gap as a monitored training signal, not a background assumption. link

03 [Eval] SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces Models that pass standard refusal benchmarks still cause stateful workspace damage across multi-step execution sequences. SABER evaluates safety from the final environment state after a full action sequence, not from whether a single prompt was declined, and categorizes violations by root cause rather than producing a binary pass/fail. That distinction matters because prompt refusal and multi-step operational harm are measuring different things entirely. Any team deploying coding agents in production should treat SABER-style environment-state evaluation as the baseline, not an optional add-on. link

04 [Inference] LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning Training-free attention compression methods drop significantly on long-context code reasoning tasks specifically, even when they hold up on simpler retrieval benchmarks. LongAttnComp fine-tunes a lightweight cross-attention scoring layer and introduces token-level chunking with a top-p budget algorithm, enabling compression that transfers across model families without full retraining. The cross-family design is the practical differentiator: compression strategies trained on one architecture can score tokens for another. For teams serving 100k-token contexts, this is a more targeted fix than general-purpose prefill optimization. link

05 [Theory] Neural Network Compression by Approximate Differential Equivalence Compression by merging functionally equivalent neurons rather than pruning individual weights is a structurally different operation that magnitude-based methods cannot replicate. The approach encodes a trained network as a polynomial ODE system, then applies approximate lumping to identify neurons with matching induced dynamics. A single tolerance parameter controls the compression level and produces a smooth accuracy-size tradeoff curve. For practitioners who need interpretable compression budgets rather than heuristic pruning schedules, the single-parameter control is worth the added mathematical overhead. link

06 [RAG] When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrieval LLM-synthesized hard negatives are semantically plausible but distributionally mismatched to what a discriminative retriever actually needs, and naively adding them to contrastive training degrades retrieval performance rather than improving it. The paper formalizes this as a generative-discriminative gap: the distribution a language model samples from when constructing negatives does not align with the decision boundary a retriever learns to separate. Corpus-mined negatives avoid this mismatch by construction, even as they hit their own ceiling from false-positive contamination. Teams fine-tuning retrievers with LLM-generated negatives should treat distribution alignment as a prerequisite, not an afterthought. link