Abstract
Third-party dependencies make modern software possible, but they also create a continuing obligation to adapt code as libraries, runtimes, frameworks, and package ecosystems evolve. A dependency upgrade can change more than a method name: it may alter type contracts, default behavior, module boundaries, build constraints, transitive resolution, or runtime semantics. A coding agent that edits the nearest failing line and restores compilation may therefore produce a patch that is locally plausible yet incomplete or unsafe.
This research perspective examines the emerging evidence on agent-driven software migration. BUMP established a reproducible collection of breaking dependency updates; Byam studied language-model repair of those failures; FreshBrew evaluated project-level Java migrations with semantic-preservation safeguards; and the 2026 DEPBENCH preprint tested real upgrade-repair tasks across five package ecosystems. These efforts show that dependency migration is a distinct repository-level capability, not a simple extension of code completion. They also expose recurring weaknesses: incomplete propagation of changed contracts, reliance on visible tests, ecosystem-specific performance gaps, and uncertainty about behavioral correctness.
Building on this literature, we propose a Migration Evidence Chain: a staged assurance framework that connects baseline reproducibility, dependency-graph understanding, contract discovery, repository-wide repair, behavioral verification, and release readiness. We also outline a research protocol for testing agents against real upgrade histories without rewarding downgrades, test deletion, or superficial compilation fixes. No new experiment is claimed. The objective is to make migration capability measurable in terms that matter to software teams: a supported upgrade, preserved behavior, and evidence that can be independently checked.
1. The green build that was not a safe upgrade
Imagine a service that uses a popular client library to communicate with a payment provider. A new major release changes how requests are constructed and how retryable errors are represented. A coding agent is asked to upgrade the package and repair the project. It updates the dependency declaration, replaces a deprecated import, adjusts one constructor, and runs the test suite. The build turns green.
The patch still may not be safe. A wrapper used by another service may continue to interpret the old error type. A retry path may now treat a permanent rejection as transient. A serialization default may have changed, so an object that compiles is sent with a different field representation. The agent has completed a visible programming task, but the migration has not necessarily preserved the application's contract.
This is an illustrative scenario, not a report of an observed production incident. It highlights the difference between making a repository build and maintaining the behavior that users and other systems depend on. Dependency updates are not inherently dangerous, and many are routine. The difficulty appears when a version change alters an assumption embedded in client code, configuration, tests, or deployment infrastructure.
Coding agents are increasingly asked to operate at repository scale. They inspect files, run commands, modify manifests, interpret compiler output, and iterate on tests. Those abilities make them plausible migration assistants. Yet the task is unusually easy to misgrade. A compiler can identify some incompatible signatures, but it cannot establish that a changed default still satisfies a business rule. A visible test suite may miss a path that matters only under a particular runtime condition. A patch can also make tests pass by removing assertions, skipping a module, or reverting the requested dependency.
The relevant question is therefore not simply whether an agent can update a library. It is whether it can discover the changed contract, propagate that change through the repository, preserve required behavior, and leave behind evidence that the result is safe to review.
2. What actually changes when a dependency changes?
The phrase breaking change often brings to mind a method that has been renamed or removed. That is only one part of the problem. Dependency changes can affect several connected layers, and those layers are not equally visible to an agent.
| Change layer What may change Why a local edit is insufficient |
| API surface | Classes, methods, parameters, imports, or exports | All call sites and wrappers may need coordinated updates |
| Type contract | Nullability, generics, enum values, ownership, or return types | Public interfaces and downstream consumers may still encode the old type |
| Runtime semantics | Defaults, ordering, retries, exceptions, rounding, or side effects | Code can compile while producing different results |
| Build and platform | Supported language/runtime versions, plugins, native interfaces, or module systems | Source changes alone may not repair the build or deployment contract |
| Dependency graph | Transitive versions, peer constraints, lockfiles, or conflicting requirements | The direct dependency may be correct while the resolved graph is not |
| Security and provenance | Vulnerability status, artifact origin, signatures, or policy requirements | A successful build does not establish that the selected artifact is trustworthy |
A syntactic break is often detectable by a compiler or type checker. A behavioral break can remain invisible until a particular input, sequence of calls, concurrency condition, or deployment configuration is exercised. The 2026 systematic review by Chen and colleagues synthesizes 97 primary studies across five software ecosystems and identifies a persistent gap: detection methods can perform well on syntactic breaks while behavioral breaks remain harder to detect comprehensively [5].
Scope matters as much as nature. A change may affect one direct call site, several internal wrappers, a generated client, a test fixture, or a transitive consumer. Visibility matters too. A migration guide may clearly describe a new API, while a subtle behavior change is only discoverable through release notes, source inspection, or carefully designed tests. A useful agent must reason across these dimensions rather than treating every failing line as an isolated repair request.
3. Version numbers are signals, not proof
Semantic Versioning gives developers a shared convention: increment the major version for incompatible API changes, the minor version for backward-compatible functionality, and the patch version for backward-compatible fixes [8]. The npm documentation similarly explains how package authors should communicate the expected compatibility impact of releases [9].
These conventions are useful, but they are not a substitute for compatibility evidence. They depend on maintainers applying the rules consistently, on the relevant compatibility surface being understood, and on downstream consumers using the same interpretation. A library can preserve its public signatures while changing a default or edge-case behavior. A project can also depend on a package whose release process does not follow strict SemVer.
The 2026 systematic review identifies the reliability of semantic versioning as a continuing challenge, alongside behavioral break detection and transitive dependency propagation [5]. The practical lesson for coding agents is straightforward: a version label should help prioritize investigation, not determine the repair strategy by itself.
An agent should establish the requested target version, inspect the actual resolved dependency graph, and seek evidence from version-specific migration documentation, release notes, API differences, build output, and repository usage. It should not infer that a minor update is harmless, nor should it assume every major update requires broad rewrites. The work is to identify which assumptions in this particular repository are affected.
4. What the research already tells us
The literature has moved from general code-generation benchmarks toward executable tasks that isolate software modernization. Several studies provide complementary evidence.
BUMP: reproducible breaking updates. Reyes and colleagues introduced BUMP in 2024, a collection of 571 breaking dependency updates from 153 Java projects using Maven. The dataset stores reproducible before-and-after states in Docker images. The authors report that 76% of observed failures involved compilation or test errors, while the remaining 24% included dependency-enforcer and dependency-locking failures [1]. BUMP made an important point: dependency failures are not limited to source-code errors, and reproducibility requires controlling the environment in which an update is evaluated.
Byam: repairing compilation failures with language models. Reyes and colleagues later evaluated Byam on BUMP. The 2026 study tested five language models and used build output and dependency-change information to guide repairs. Its strongest reported configuration completely fixed 27% of builds, while fixing 78% of individual compilation errors [2]. The difference between those measurements is important. Repairing an error is not the same as restoring a complete project. Several individually corrected files can still leave a build or test suite failing.
FreshBrew: project-level Java migration. May and colleagues introduced FreshBrew to evaluate migration from JDK 8 to JDK 17 across 228 repositories. Its success protocol goes beyond compilation: a candidate must compile, pass the original tests, and preserve test coverage within five percentage points of the baseline [3]. This design addresses a known weakness in automated migration evaluation: an agent should not receive credit for deleting tests or otherwise reducing the evidence used to judge correctness. FreshBrew's published abstracts have appeared in different revisions with different headline success rates; the benchmark's evaluation gates and repository scope are more stable facts to cite than a number detached from its version.
DEPBENCH: upgrade repair across ecosystems. The August 2026 preprint Update from Hell introduces DEPBENCH, with 203 real-world dependency-upgrade tasks across npm/yarn, Maven/Java, Go, Cargo/Rust, and Python. Its strongest completed model-and-harness configuration solved 104 of 203 tasks (51.2%) [4]. The authors identify incomplete repository-wide propagation as a dominant failure mode: agents may find the changed dependency surface but fail to update wrappers, types, fixtures, generated artifacts, lockfiles, or behavior consistently. This is direct evidence that dependency-upgrade repair deserves its own evaluation setting.
Change2Task: building executable tasks from repository history. Qi and colleagues describe a system that converts merged pull requests into verified coding-agent tasks on healthy modern repository revisions. Its task families include bug fixing, feature addition, test generation, API migration, and security repair. Starting from 1,130 eligible source changes, the authors report 79.6% verified task-construction success and up to 98.0% matched outcome agreement for historical and reconstructed cases [6]. This work is relevant because a realistic migration task needs more than a patch: it needs a valid initial state, a clear task specification, tools, and an executable oracle.
These studies should not be treated as a single leaderboard. They use different task definitions, languages, models, harnesses, and success criteria. Their combined message is narrower and more useful: project migration is measurable, existing agents still fail on substantial portions of realistic tasks, and the quality of the verifier strongly influences what a score means.
5. A migration is a sequence of engineering decisions
A dependency upgrade usually passes through a recognizable lifecycle. Treating the entire process as one code-editing action hides where an agent succeeds or fails.
- Establish the baseline. Identify the repository revision, runtime, package manager, lockfile, build commands, test commands, and current test health. A failing baseline should not be silently attributed to the upgrade.
- Resolve the intended target. Confirm the requested package and version. Inspect direct and transitive dependencies, peer constraints, supported runtimes, and lockfile changes.
- Understand the changed contract. Read version-specific migration notes, release notes, API documentation, type definitions, and relevant source. Record which assumptions in the repository are affected.
- Map the impact. Find direct call sites, wrappers, adapters, public interfaces, generated code, configuration, fixtures, and tests. Trace the change through the dependency graph and application boundaries.
- Apply and validate the repair. Make the smallest coherent set of changes, then run static checks, original tests, targeted migration tests, and behavioral checks.
- Prepare the result for review. Summarize the target version, resolved graph, files changed, checks performed, remaining uncertainty, and rollback considerations. High-impact changes should not be published solely because an agent reports success.
This lifecycle separates actions that can be automated from decisions that require stronger evidence. Package managers can resolve versions. Codemods can perform known syntactic transformations. Agents can inspect unfamiliar usage patterns and propose changes. None of those components, by itself, proves that the application still behaves correctly.
6. The proposed Migration Evidence Chain
We propose a Migration Evidence Chain (MEC) as a practical framework for designing, evaluating, and reviewing coding agents that perform dependency modernization. It is not a new benchmark or a claim of completed validation. It is a way to connect each migration claim to an artifact that another person or system can inspect.
Gate 1 — Baseline integrity
Before editing, capture the repository commit, dependency manifests, lockfiles, runtime versions, build configuration, and test results. Record flaky or unavailable checks rather than treating them as passes. Where possible, preserve a reproducible container or environment image.
The baseline answers a basic causal question: was the project healthy before the upgrade? Without it, a migration agent can be blamed for pre-existing failures or receive credit for a result that was already possible.
Gate 2 — Upgrade and graph integrity
The agent must preserve the requested target version and produce a coherent resolved dependency graph. The verifier should inspect manifests and lockfiles, confirm that the requested upgrade remains present, and detect silent downgrades, dependency substitution, or removal of required packages.
This gate is especially important when package managers resolve transitive dependencies or peer constraints. A direct manifest edit does not necessarily describe the graph that will actually be installed.
Gate 3 — Contract discovery
The agent should identify the relevant change evidence: release notes, migration guides, API differences, type declarations, compatibility matrices, and relevant upstream source. It should connect each important change to repository locations that may depend on it.
The output need not be a long narrative. A compact impact map is more useful: changed dependency surface, affected files, expected adaptation, evidence source, and unresolved questions. If authoritative evidence is unavailable, the agent should state that uncertainty instead of inventing a migration rule.
Gate 4 — Repository-wide repair integrity
The patch should update the full affected surface without weakening the task. This includes source files, wrappers, interfaces, generated artifacts where regeneration is required, configuration, and relevant tests. The verifier should check that tests were not removed or skipped, validation was not weakened, and the requested dependency was not reverted.
A small patch is not automatically better than a large patch. The relevant objective is a coherent patch whose scope is explained and whose changes are necessary for the migration.
Gate 5 — Behavioral evidence
Compilation and type checking are necessary for many migrations, but they are not sufficient. The evidence should include the original test suite, targeted tests for changed behavior, and—where practical—contract, property-based, differential, or integration tests.
Differential testing can compare old and new versions on inputs for which behavior is expected to remain stable. It should not assume that every output must be identical: some migrations intentionally change behavior. The specification must distinguish intended changes from regressions.
Gate 6 — Operational and security readiness
The final review should consider startup behavior, configuration compatibility, performance-sensitive paths, security scans, software bills of materials, and artifact provenance where applicable. A migration that is functionally correct in a unit-test container may still fail in the target runtime or deployment topology.
The chain ends with a reviewable handoff, not an automatic production release. The agent should report completed checks and remaining risks, and the deployment process should retain a human approval or staged rollout when the consequences of failure are significant.
| Evidence gate Minimum artifact Example failure caught |
| Baseline integrity | Pinned revision, environment, baseline test record | Pre-existing failures misattributed to the upgrade |
| Graph integrity | Target version and resolved dependency graph | Silent downgrade or conflicting transitive version |
| Contract discovery | Version-specific change and impact map | Repair based on an obsolete API assumption |
| Repair integrity | Auditable patch and unchanged validation rules | Test deletion or incomplete call-site migration |
| Behavioral evidence | Original and targeted test results | Compilation succeeds but behavior changes |
| Operational readiness | Security, runtime, performance, and rollout evidence | Vulnerable artifact or deployment-only regression |
The MEC is deliberately staged. A failed gate should be visible rather than hidden inside a single aggregate score. A team may decide that a low-risk patch can proceed with a limited evidence set, while a security-sensitive or externally exposed service requires deeper review.
7. How to construct a credible migration evaluation
A benchmark should begin with real repository histories, not synthetic prompts that merely mention a version number. Merged upgrade pull requests can provide candidate examples, but a merged pull request is not automatically a valid task. It may combine the version bump with unrelated refactoring, test cleanup, or release bookkeeping.
The task constructor should separate at least three components:
- the dependency-manifest and lockfile change;
- the source repair required to adapt to the upgraded dependency;
- the tests, fixtures, mocks, or expected outputs that verify the new behavior.
The DEPBENCH work provides a useful precedent. Its four-state oracle checks that the original repository is healthy, that the upgrade plus held-out tests exposes a failure, that the upgrade plus repair passes, and that the repair without the upgrade does not pass the same test patch [4]. This is a stronger causal contract than simply replaying a merged pull request. It helps establish that the task actually measures upgrade-induced repair.
The oracle still has limits. Passing tests cannot prove every aspect of semantic equivalence, and tests may be incomplete or flaky. The benchmark should therefore document what the tests cover and where they cannot provide assurance. FreshBrew's test-coverage safeguard is one way to make test deletion less attractive [3], but coverage itself is not a proof of correctness. Mutation testing, property checks, and human review of high-impact cases can provide additional evidence.
Task descriptions should not reveal the intended patch. A prompt generated directly from a solution diff may accidentally name the replacement API, the target file, or the exact repair path. The agent should receive the same information a developer would reasonably have at task start: the repository, the requested target, and the available documentation or issue context. Hidden repair patches and held-out tests should remain unavailable until verification.
Environment control is equally important. Pin repository revisions, toolchains, package-manager versions, container images, and dependency artifacts where licensing and registry policies permit. If the task requires fetching a new package, permit the relevant registry while blocking access to the upstream solution pull request. Record network policy, cache state, timeouts, and infrastructure exceptions. A registry outage is not an agent reasoning failure.
Finally, include negative controls. A no-op should fail when an upgrade is required. A downgrade should fail target-version validation. Deleting tests, disabling checks, weakening assertions, or changing the task's expected behavior should not earn success. These controls test whether the verifier rewards the intended migration rather than any path to a green status.
8. A scorecard that does not confuse compilation with correctness
A single pass rate is convenient for dashboards but can hide important differences. A migration evaluation should report a gate-wise scorecard and preserve hard failures as hard failures.
| Dimension Suggested measurement Interpretation |
| Target compliance | Exact requested version and lockfile state | Did the agent perform the requested upgrade? |
| Build compatibility | Install, compile, and type-check success | Can the repository be built in the target environment? |
| Test integrity | Original tests retained and executed | Did the agent preserve the validation surface? |
| Behavioral preservation | Original, hidden, and targeted contract tests | Does expected behavior remain correct? |
| Migration completeness | Affected call sites and dependency layers repaired | Did the agent propagate the new contract? |
| Security posture | Vulnerability and policy checks, SBOM changes | Did the update improve or weaken the security position? |
| Patch quality | Scope, unnecessary edits, maintainability review | Is the repair understandable and reviewable? |
| Reproducibility | Repeated runs from pinned images and artifacts | Can another evaluator reproduce the result? |
| Efficiency | Time, tokens, tool calls, and infrastructure cost | What resources were needed to complete the task? |
Coverage preservation can help detect attempts to delete tests, but coverage does not measure assertion quality. A test can execute a line without checking its result. Mutation testing can probe whether tests detect selected faults, while property-based tests can exercise broader input spaces. Both have costs and should be reported as separate evidence, not treated as universal substitutes for domain-specific specifications.
The overall result should use explicit gates. A patch that violates the target version, disables tests, or introduces a severe security regression should not compensate for that failure by scoring well on speed. Researchers can report secondary weighted summaries for comparison, but the primary result should make critical failures visible.
9. Failure modes that deserve separate labels
Migration failures are easier to diagnose when they are classified by cause rather than grouped under “agent failed.”
The one-call-site repair. The agent updates the first compiler error but misses wrappers, adapters, generated clients, or other consumers of the same API. The repository remains internally inconsistent.
The compile-first shortcut. The agent chooses a type cast, broad exception handler, or compatibility shim that suppresses a compiler error without preserving the intended contract. Static success can conceal a runtime defect.
The semantic blind spot. The new library changes a default, ordering rule, retry policy, rounding behavior, or side effect. Existing source still compiles, and the visible tests do not exercise the affected path.
The graph mismatch. The direct dependency is updated, but a transitive package, peer requirement, plugin, or lockfile entry remains incompatible. The local manifest looks correct while the resolved environment does not.
The test-oracle attack. The agent deletes tests, skips a failing suite, changes expected values without justification, or downgrades the dependency. These actions can produce a misleading pass and must be explicitly rejected by the verifier.
The documentation mismatch. The agent reads a guide for a different version or relies on a search result that describes an older API. The resulting patch may be internally consistent but based on the wrong contract.
The environment illusion. A local cache, unpinned registry response, or different runtime makes a repair appear successful on one run and fail on another. This is an infrastructure and reproducibility problem, not necessarily a reasoning problem.
Separating these cases creates a more useful research result. If an agent compiles but fails behavioral tests, the next intervention should differ from the one needed when it cannot resolve a package or misses a transitive type change.
10. Why hybrid migration agents may be more reliable
A migration agent does not need to invent every code transformation. Many migrations have well-understood mechanical steps that can be expressed as deterministic refactoring recipes. OpenRewrite, for example, represents source code as Lossless Semantic Trees and supports composable recipes. Its documentation's JUnit 4-to-JUnit 5 example combines annotation replacement, assertion changes, visibility updates, and build-file dependency changes rather than treating the migration as one text substitution [10].
This suggests a hybrid architecture:
- Agent-led reconnaissance: inspect manifests, build files, repository usage, and version-specific documentation.
- Impact analysis: produce a dependency and call-site map, then identify which changes are mechanical and which require semantic judgment.
- Deterministic transformation: use a tested codemod or migration recipe for well-specified transformations.
- Agent-led adaptation: handle project-specific wrappers, custom abstractions, and cases that do not fit existing recipes.
- Independent verification: run the baseline and post-upgrade checks through a verifier that the agent cannot modify.
- Reviewable handoff: summarize the patch, evidence, limitations, and recommended next action.
The division of labor matters. A deterministic recipe can make a repeatable transformation, but it cannot guarantee that the transformation is appropriate for every application. An LLM can interpret unfamiliar context, but it may produce inconsistent edits or overlook a downstream consumer. The combination should be evaluated against agent-only and codemod-only baselines, with the same repositories and resource budgets.
Recipe tests should also check for unnecessary changes, not only expected output. OpenRewrite's testing guidance explicitly emphasizes both [11]. This principle generalizes: a migration tool should be judged by what it changes and by what it leaves alone.
11. A research protocol for the next generation of agents
The proposed framework leads to a set of testable research questions.
- RQ1 — Ecosystem sensitivity: How does migration success vary across Java/Maven, JavaScript/npm, Go, Rust/Cargo, and Python when task difficulty and verification strength are controlled?
- RQ2 — Evidence value: Does giving an agent version-specific migration guides, API diffs, type information, and dependency-graph context improve end-to-end migration success over repository-only access?
- RQ3 — Semantic verification: How often do compile success and visible-test success disagree with held-out behavioral checks?
- RQ4 — Hybrid tools: Do agents combined with deterministic migration recipes outperform agent-only and recipe-only systems on correctness, patch quality, and cost?
- RQ5 — Change propagation: Which repository structures—wrappers, generated code, public interfaces, transitive dependencies, or configuration layers—most often cause incomplete migrations?
- RQ6 — Risk-sensitive autonomy: Can agents identify cases where evidence is insufficient and request human review before making a high-impact change?
A controlled study could begin with real upgrade histories from the five ecosystems represented in DEPBENCH [4], while adding migration families such as runtime modernization and framework transitions. Tasks should be split by repository and dependency family where possible, so closely related versions do not appear across training and test partitions. The benchmark should preserve the original task history and disclose whether public source changes could have appeared in model training data.
Each agent configuration should receive the same task image, time and tool budget, network policy, and verifier. Baselines should include a no-op, a deterministic migration tool where one exists, an agent without migration documentation, an agent with documentation and dependency-graph context, and a hybrid agent. Multiple runs are needed for stochastic systems. Report task-level outcomes, confidence intervals, failure categories, cost, and infrastructure exceptions.
A useful extension is sequential evaluation. Real repositories do not always stop after one upgrade; a framework update may force a language-runtime update, which may in turn require a test-runner or build-plugin migration. ChainSWE, an EMNLP 2026 benchmark, studies sequential dependent bug fixes in shared codebases and reports performance drops of up to 70% as issue-chain length increases [12]. It does not directly measure dependency migration, but it reinforces the need to test maintenance sequences rather than only isolated tickets.
The study should be preregistered where practical. Primary success criteria, exclusions, test thresholds, and treatment of flaky infrastructure should be defined before comparing models. Negative results are valuable: if additional documentation increases tool use but does not improve behavior, that is a finding about evidence quality and agent strategy.
12. Security, deployment, and human oversight
Dependency upgrades often have a security motivation. A project may need a newer release to address a vulnerability, remove an unsupported component, or meet a platform requirement. A migration agent must therefore avoid solving compatibility by quietly returning to the vulnerable version. Target-version compliance and security checks belong in the same evaluation.
The agent should operate inside a controlled environment. Package installation can execute scripts; repositories may contain untrusted code; and migration tasks may require network access. The evaluation harness should restrict secrets, isolate execution, record network access, and prevent the agent from changing the verifier. Where available, teams should retain artifact hashes, lockfile changes, and software-bill-of-materials evidence.
Deployment should remain a separate decision. A passing test suite is evidence, not a guarantee of safe production behavior. For high-impact services, a migration may need staged rollout, monitoring of relevant error and latency signals, and a tested rollback path. The agent can prepare the change and assemble the evidence package; a responsible engineer or deployment policy should decide whether the change proceeds.
This is not an argument against autonomy. It is a way to allocate autonomy according to evidence and consequence. A small documentation-library update may need only routine checks. A database driver, cryptography package, payment SDK, or authentication library deserves stronger scrutiny because a subtle regression can have disproportionate impact.
13. Implications for Deaimer and teams building coding-agent environments
Deaimer's public research and service pages describe work in RL environments, coding-agent evaluations, datasets, and benchmarks. Dependency modernization offers a practical task family for that work because it can be grounded in real repository histories and evaluated through executable build, test, and policy checks.
For a team commissioning a coding-agent environment, the specification should state more than “upgrade package X.” It should identify the healthy starting revision, exact target version, permitted package sources, expected behavior, protected tests, known compatibility constraints, and evidence required for completion. The verifier should reject a downgrade or weakened test suite and should distinguish an agent failure from an infrastructure failure.
The Migration Evidence Chain provides one way to organize those requirements. It can guide task design, verifier design, and post-run analysis without assuming that every migration should use the same tools or test depth. A task about a renamed API may need strong type and integration checks; a task involving changed retry semantics needs behavioral tests; a security-driven update also needs policy and artifact checks.
This section describes a proposed application of the research, not a claim that Deaimer has already implemented or empirically validated the full framework. The broader point is that a useful coding-agent benchmark should measure the maintenance outcome that a software team actually needs—not just the agent's ability to produce a plausible diff.
14. Conclusion
Dependency modernization is a demanding form of software engineering because the code being changed is only one part of the system contract. The agent must understand the requested version, resolve the dependency graph, discover changed assumptions, propagate repairs across repository boundaries, preserve behavior, and produce evidence that another party can verify.
Research has established important foundations. BUMP made breaking updates reproducible; Byam measured language-model repair at build, file, and compilation-error levels; FreshBrew added project-level migration safeguards; DEPBENCH tested upgrade repair across five ecosystems; and Change2Task demonstrated a path from repository history to executable coding-agent tasks [1–6]. These contributions are complementary, and their results should be interpreted within their own task definitions and evaluation protocols.
The next step is to connect benchmark success with migration assurance. The proposed Migration Evidence Chain makes each claim explicit: the baseline was healthy, the target version was retained, the changed contract was investigated, the repair was propagated, behavior was tested, and operational risks were reviewed. It does not eliminate uncertainty, but it makes uncertainty visible.
A coding agent should not be called successful merely because the build turns green. It should be judged by whether the requested upgrade remains in place, whether the software continues to do what it is supposed to do, and whether the evidence is strong enough for a responsible engineer to trust the change.
References
[1] F. Reyes, Y. Gamage, G. Skoglund, B. Baudry, and M. Monperrus. “BUMP: A Benchmark of Reproducible Breaking Dependency Updates.” arXiv:2401.09906, 2024. https://arxiv.org/abs/2401.09906
[2] F. Reyes, M. Mahmoud, F. Bono, S. Nadi, B. Baudry, M. Monperrus, et al. “Byam: Fixing Breaking Dependency Updates with Large Language Models.” Empirical Software Engineering, 31, article 113, 2026. https://doi.org/10.1007/s10664-026-10835-1
[3] V. May, D. Misra, Y. Luo, A. Sridhar, J. Gehring, and S. S. Ribeiro Junior. “FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration.” Proceedings of the IEEE/ACM 48th International Conference on Software Engineering (ICSE 2026). arXiv:2510.04852, 2025. https://arxiv.org/abs/2510.04852
[4] Z. Luo, R. He, P. Gao, Y. Kang, Z. Lin, M. Ma, Q. Lin, S. Rajmohan, and Y. Tian. “Update from Hell: Can Coding Agents Survive Hidden Breakage in Dependency Upgrades?” arXiv:2608.30300, 2026. https://arxiv.org/abs/2608.30300
[5] J. Chen, T. Bi, Y. Wang, and P. Thongtanunam. “Breaking Changes in Software Ecosystems: A Systematic Literature Review.” arXiv:2605.24397, 2026. https://arxiv.org/abs/2605.24397
[6] H. Qi, X. Wang, X. Gao, B. Sang, X. Zhang, M. Ma, P. Gao, Y. Kang, Q. Lin, S. Rajmohan, D. Zhang, and Q. Zhang. “Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments.” Microsoft Research, July 2026. https://www.microsoft.com/en-us/research/publication/change2task-from-repository-changes-to-executable-coding-agent-tasks-and-environments/
[7] X. Deng, J. Da, E. Pan, Y. Y. He, C. Ide, K. Garg, et al. “SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?” Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:23917–23934, 2026. https://proceedings.mlr.press/v306/deng26f.html
[8] T. Preston-Werner. “Semantic Versioning 2.0.0.” Specification. https://semver.org/
[9] npm Documentation. “About Semantic Versioning.” https://docs.npmjs.com/about-semantic-versioning/
[10] OpenRewrite Documentation. “Recipes.” https://docs.openrewrite.org/concepts-and-explanations/recipes
[11] OpenRewrite Documentation. “Recipe Testing.” https://docs.openrewrite.org/authoring-recipes/recipe-testing
[12] Q. Jin, L. Tung, K. Li, Q. Shi, Y. She, H. Jia, et al. “ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance.” Proceedings of EMNLP 2026, 8487–8505. https://aclanthology.org/2026.emnlp-main.371/

