Code2LoRA Matches Per-Repo Fine-Tuning at Zero Inference Token Cost

Per-repository fine-tuning gives code models exactly the context they need: local APIs, import conventions, project idioms. The cost makes it impractical. At scale, maintaining one LoRA per repository means storage that grows linearly with your codebase count, retraining every time a repo evolves, and no clean path for handling a repository the model has never seen before.

Code2LoRA reframes the problem. Instead of training one adapter per repository and storing it, a hypernetwork learns to generate a repository-specific LoRA adapter from code alone. Feed the hypernetwork a repository snapshot, and it produces adapter weights on the fly. The analogy is close to how a compiler emits target-specific binaries from a single source: one generative process, many specialized outputs, nothing stored per target. The generated adapter is then applied at inference time with no additional tokens in the prompt, which means the context window stays fully available for the actual task.

The framework ships in two configurations. Code2LoRA-Static takes a single repository snapshot and produces a fixed adapter, suited for comprehension tasks on stable codebases. Code2LoRA-Evo maintains a GRU hidden state that updates incrementally per code diff, producing a continuously revised adapter as the repository changes. The GRU design matters: it means the model does not need to re-read the full repository history on every commit. It reads only the delta, updates its hidden state, and emits a revised adapter. That keeps the update cost proportional to the diff size, not the repository size.

To benchmark these configurations, the team built RepoPeftBench: 604 Python repositories, an assertion-completion static track with 40K training and 12K test tasks, and an evolution track derived from commit history with 215K training and 87K test tasks. The numbers are direct. Code2LoRA-Static reaches 63.8% cross-repo and 66.2% in-repo exact match on the static track, matching the per-repository LoRA upper bound. Code2LoRA-Evo reaches 60.3% cross-repo exact match on the evolution track, a 5.2 percentage point gain over a single shared LoRA. For teams building code intelligence tooling over large, actively maintained monorepos or multi-repo environments, the takeaway is direct: you can get per-repo adaptation quality without per-repo training infrastructure.

We're thinking: We find the GRU-backed evolution path more consequential than the headline static numbers. The static result confirms that hypernetworks can approximate expensive per-repo fine-tuning, which is useful. But the evolution track is where the real operational problem lives: most production codebases change daily, and any adapter trained on last week's snapshot is already partially stale. Code2LoRA-Evo's incremental update design directly attacks that staleness without requiring a full retraining cycle per commit. The open question is how the GRU hidden state degrades over long commit chains with large structural refactors, something the current benchmark's commit-derived splits may not fully stress. That failure mode is worth probing before any production rollout.

Key takeaways:

• A single hypernetwork generates repository-specific LoRA adapters from code snapshots or diffs, replacing the need to train and store one adapter per repository.
• Code2LoRA-Static matches per-repo LoRA upper-bound accuracy at 63.8% cross-repo exact match; Code2LoRA-Evo gains 5.2 pp over a shared LoRA on 87K commit-derived test tasks, though GRU state degradation over deep refactor histories remains untested.
• Teams shipping code completion or code intelligence features over large or frequently updated repositories should evaluate Code2LoRA-Evo as a direct replacement for RAG-based context injection, particularly where inference token budgets are constrained.

Source: Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution