What A Trading Firm's Dev Team Actually Has To Learn To Ship AI: Legacy Modernisation, The Verification Crunch And Model Risk - A Real-World Playbook

3-4x - Engineering productivity reported by Goldman from AI coding agents, chiefly on legacy modernisation  ·  ~100k hrs/week - Developer hours reportedly saved at Citi through automated code review  ·  3M lines / 8 weeks - COBOL converted into clear specifications in eight weeks (Publicis Sapient Slingshot)  ·  The strain - AI adoption is straining testing, model validation and auditability in trading and risk systems

The numbers coming out of banking's AI adoption are the kind that make a CTO sit up. Goldman reports 3 to 4 times engineering productivity from AI coding agents. Citi has saved on the order of 100,000 developer hours a week through automated code review and rolled AI coding tools out to some 30,000 developers. Bank of America has 18,000 developers on GitHub Copilot; CIBC well over a thousand. One firm's tooling converted three million lines of COBOL into clear specifications in eight weeks. Read alone, these say 'adopt AI coding tools and reap enormous gains'. But there is a second, harder story that every investment-banking and trading engineering team discovers, and it is the one worth learning from: as banks deploy these tools, they are facing real, growing strain on testing, model validation, software-supply-chain oversight and auditability - and that strain is most acute exactly where trading firms live, in high-volume trading, risk and real-time systems. This playbook is the practical account of what a trading firm's dev team actually has to learn.

Lesson 1: The Biggest Win Is Legacy Modernisation - Start There

The first thing a trading firm's dev team learns is where the AI productivity actually comes from, and it is not writing shiny new features - it is taming old code. The most repeatable, highest-ROI result across the industry is AI-assisted legacy modernisation: autonomous coding agents that can read, refactor, test and ship changes to the sprawling, decades-old codebases that run real trading and banking operations, and tools that turn incomprehensible legacy code into clear specifications. That is why Goldman's 3-to-4x gain is largely a legacy-modernisation story, and why converting millions of lines of COBOL to specs in weeks is such a landmark - trading firms are sitting on exactly this kind of critical, aging, poorly-documented code, and it is a huge drag on every change they want to make. The practical lesson is to point AI first at understanding and modernising the legacy estate - documenting it, generating specifications, assisting careful refactors - because that is where the biggest, most proven wins are, and it is the foundation that makes everything else faster.

Lesson 2: The Verification Crunch Is The Real Pain Point

Here is the pain point nobody puts on the slide, and the one that catches trading dev teams out: when AI lets you produce and change code far faster, the constraint moves to verifying it - and in a trading firm, verification is not optional, it is existential. Reporting that AI adoption is straining testing at major banks is the visible symptom of a structural truth: generating code got dramatically faster, but the testing, review and validation that must stand between that code and a live trading system did not, so the pressure piles onto verification. For a consumer app, weak verification means a bug; for a trading or risk system processing real money at high volume in real time, weak verification means a potentially serious financial and regulatory incident. So the essential lesson is that adopting AI coding tools without simultaneously strengthening your regression testing, validation and review capacity is not a productivity gain - it is a risk transfer, moving effort from writing to verifying without building the capacity to verify. The teams that get this right invest as hard in their testing and validation as in their AI tooling; the ones that do not accumulate unverified change in systems that cannot tolerate it.

Lesson 3: Model Risk And Auditability Are Not Optional Extras

The third lesson is one that pure-tech teams often underestimate and regulated trading teams cannot afford to: in banking and trading, you must be able to account for what your systems do, including the AI-assisted ones. Banks are under growing pressure to strengthen model validation, software-supply-chain oversight and auditability around AI-generated outputs - and for good reason, because financial regulators expect firms to understand, validate and evidence the behaviour of the models and systems making or supporting consequential decisions. For a trading firm's dev team, that translates into concrete disciplines: knowing the provenance of AI-generated code and the components it pulls in (supply-chain oversight), validating that AI-assisted changes to pricing, risk or trading logic behave correctly (model and change validation), and maintaining the audit trail to demonstrate all of it to internal risk functions and regulators. This is not bureaucratic overhead bolted onto the fun part; in regulated finance it is part of the job, and a dev team that learns to build AI adoption with validation and auditability designed in from the start moves faster in the long run than one that bolts them on after a control failure.

Lesson 4: Human Expertise Becomes More Valuable, Not Less

The final lesson runs counter to the fear that AI replaces the engineers, and trading firms are learning it firsthand: as AI handles more of the code generation, the experienced engineers who understand the domain, catch the dangerous edge cases and own the judgement become more valuable, not less. A trading system is full of subtle, high-consequence rules - about market conventions, risk limits, regulatory constraints, illegal states - that an AI agent can plausibly-but-wrongly get subtly wrong, and only someone who deeply understands the domain will catch it. The productivity gains are real precisely because they free experienced engineers from the mechanical work to focus on the judgement, verification and domain-critical decisions that AI cannot be trusted with in a trading context. So the dev team's evolution is not toward fewer engineers but toward engineers doing higher-value work - orchestrating and rigorously verifying AI output, owning the domain-critical judgement, and encoding their hard-won knowledge into the tests and guardrails that let AI work safely. In a trading firm, human expertise is the thing that makes AI adoption safe, and it appreciates in value as adoption grows.

• Point AI first at legacy modernisation - documenting, spec-generating and carefully refactoring the old code that runs the business; it is the highest, most proven ROI.
• Scale verification with generation - strengthen regression testing, validation and review as hard as you adopt the tooling, because verification is now the constraint.
• Design in model validation and auditability - provenance, change validation and audit trails are part of the job in regulated trading, not an afterthought.
• Value domain expertise - experienced engineers who catch high-consequence edge cases become more important as AI generates more code.
• Encode knowledge into guardrails - turn expert judgement into the tests, checks and constraints that let AI operate safely in trading systems.

The Bottom Line

The AI opportunity for a trading firm's engineering team is real and large - the industry's 3-to-4x productivity gains, hundred-thousand-hour savings and COBOL-to-spec breakthroughs are not hype. But capturing them safely means learning the lessons behind the headlines: aim AI first at the huge, proven win of legacy modernisation; recognise that the true constraint and pain point is now verification, and scale testing and validation to match generation; build model validation and auditability in from the start, because in regulated trading you must account for what your systems do; and treat your experienced engineers as more valuable than ever, as the domain judgement and verification capacity that make AI adoption safe. A trading dev team that learns these - the wins, the verification crunch, the model-risk discipline, the enduring value of expertise - turns AI from a source of risk into a genuine, durable advantage. That combination of ambition and regulated-grade discipline is exactly what BraivIQ helps financial engineering teams get right.

References & Further Reading

• QA Financial - AI adoption strains testing at JPMorgan: https://qa-financial.com/ai-adoption-strains-testing-at-jpmorgan/
• Yahoo Finance - Citi deploys AI coding tools to 30K developers in modernization push: https://finance.yahoo.com/news/citi-deploys-ai-coding-tools-152606091.html
• Microsoft Cloud Blog - Modern banking, reinvented: a blueprint for successful AI transformation: https://www.microsoft.com/en-us/microsoft-cloud/blog/2026/01/07/modern-banking-reinvented-cibcs-blueprint-for-successful-ai-transformation/
• The CG AI Group - Banking AI 2026: from pilots to production at scale: https://blog.thecgaigroup.com/banking-ai-2026
• US Federal Reserve - SR 11-7: Guidance on Model Risk Management: https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm