From Legacy Code to Agentic Future: A Practical Guide to Modernizing with LLMs

Overview

Modern software development is at a turning point. The rise of agentic programming—where AI assistants autonomously perform coding tasks—has changed how we think about legacy systems. A recent retreat of software professionals, held under Chatham House Rule, revealed several practical insights. This guide translates those discussions into a step-by-step tutorial for using Large Language Models (LLMs) to modernize old codebases, verify specifications, and adopt smarter migration strategies.

We'll cover:

• How to port codebases rapidly with LLMs (like the GNU Cobol-to-Rust example)
• Using an Interrogatory LLM to check large specs
• Why reading an organization's change-control board guidelines reveals hidden risks
• The new role of Lift and Shift in legacy migration

By the end, you'll have concrete techniques to apply in your own projects—and avoid common pitfalls.

Prerequisites

• Basic understanding of software development and version control
• Familiarity with LLMs (e.g., GPT, Claude) and how to prompt them
• Access to a legacy codebase (optional, but helpful for practice)
• A test suite (even a minimal one) to validate results

Step-by-Step Instructions

1. Rapid Code Porting with LLMs

LLMs can translate entire codebases between languages. A retreat attendee reported porting 70,000 lines of GNU Cobol to Rust in just three days. Here's how you can replicate that approach.

Step 1: Prepare Your Source

Get a clean copy of the legacy code. Ensure you have a comprehensive regression test suite to verify correctness. If none exists, use the LLM itself to generate tests from the original implementation (e.g., by prompting “Generate unit tests for this Cobol function”).

# Example: If you have a Cobol program 'calc.cbl', first ask LLM to create test inputs and expected outputs.

Step 2: Prompt the LLM

Write a prompt that includes the entire source file (or chunks) and asks for a translation. Use a context window that fits. Example prompt:

"Translate the following GNU Cobol program to idiomatic Rust. Preserve all logic, variable names, and comments. Output only the Rust code."

For large files, break into modules and translate separately, then link.

Step 3: Validate with Tests

Run the regression tests on the Rust output. The attendee noted that good tests are extremely valuable—without them, you might produce buggy code. If tests fail, feed the error back to the LLM:

"The following Rust code fails this test: [test case]. Please fix."

Step 4: Iterate

Repeat until all tests pass. The three-day achievement shows this is feasible even for large codebases.

2. Interrogatory LLM: Verifying Large Specs

Large specification documents are hard for humans to review thoroughly. One expert suggested turning the LLM into an interrogator that interviews a domain expert.

How to Run an Interrogatory LLM Session

1. Load the spec into the LLM (e.g., a requirements document).
2. Prompt: "You are a technical reviewer. Ask me a series of yes/no or multiple-choice questions that verify the correctness of this specification. Start with the most critical assumptions."
3. Answer questions as the human expert. The LLM will probe ambiguous or incomplete areas.
4. Document findings: capture gaps and corrections. This method turns a passive review into an active, focused conversation.

This technique is especially useful when the spec spans hundreds of pages (like in finance or regulation-heavy industries).

3. Reading Change-Control Board Guidelines

Another participant emphasized: “The first thing I do when consulting is read the change-control board guidelines.” These documents are the scar tissue of past failures. To understand why a system is the way it is, you must understand the history.

How to Analyze CCB Guidelines

• Look for rules that seem overly strict—they often exist because of a specific disaster.
• Map each rule to a potential past incident. For example, a rule requiring two approvals for any database change might stem from a corruption event.
• Use the LLM to summarize patterns: “Read these 50 pages of CCB rules and list the top 5 risk themes.”

This analysis helps you avoid repeating old mistakes when modernizing.

4. The New Lift and Shift Strategy

Traditionally, “Lift and Shift” (rehosting with feature parity) was seen as a mediocre approach because legacy systems often contain 50% unused features (Standish Group, 2014). But LLMs change the calculus. One attendee now recommends: always do Lift and Shift as the first step—then evolve.

Why Lift and Shift First?

• Cost of porting dropped dramatically with LLMs. Porting to a modern platform (e.g., from Cobol to Rust) can be done in days.
• A modern environment (e.g., containerized microservices) makes further evolution cheaper and safer.
• Key caution: Don't stop after the shift. Use the new platform to refactor and remove unused features.

Implementation Steps

1. Port the entire legacy system using LLM (as in section 1).
2. Run tests to ensure feature parity.
3. Deploy on a modern platform (cloud, containers).
4. Analyze usage to identify dead code and processes.
5. Iteratively remove unused features and simplify.

This reduces risk and accelerates modernization.

Common Mistakes

• Assuming LLM output is production-ready – Always run thorough tests; the port may contain subtle bugs.
• Skipping the change-control review – You may miss critical risk patterns embedded in organizational rules.
• Performing Lift and Shift and stopping – You'll retain all bloat and miss the real benefits of modernization.
• Not verifying specs interactively – Passive reading of large docs overlooks ambiguities that an Interrogatory LLM can catch.

Summary

LLMs are transforming legacy modernization. You can now port massive codebases in days, verify specs through interactive questioning, mine change-control history for risks, and reconsider Lift and Shift as a starting point—not a dead end. By following this guide's steps and avoiding common pitfalls, you'll harness agentic programming to make your old systems future-ready.

Keywords: legacy migration, LLM, agentic programming, code porting, Lift and Shift, specification verification, change-control