Running AI Inside The Firewall: What A Trading Firm's Dev Team Must Learn To Deploy LLMs In A Locked-Down, Air-Gapped Regulated Environment

Data stays in - The core rule: sensitive financial data never leaves the controlled environment - the model comes to the data  ·  Air-gapped - The strictest form: models, inference servers and vector databases on isolated infrastructure, no internet  ·  Log everything - Every prompt and response logged with timestamp, user identity and model version for audit  ·  Off-limits - Public tools like ChatGPT are simply not usable for sensitive data under PCI DSS, GLBA and internal policy

Ask any developer at an investment bank or trading firm about using AI at work, and you will hear the same frustration. They know AI could make them dramatically more productive - the rest of the industry is getting 3-to-4x engineering gains from it - but they cannot use the tools everyone else uses, because they cannot paste their code, their trade data, their models or their customers' financial information into a public AI service. It is not caution; it is the rules. Regulations like PCI DSS and GLBA, along with strict internal policies, flatly forbid sending sensitive financial data to public LLMs, and banking regulators require customer financial data processed by AI to remain within controlled infrastructure. For a regulated trading dev team, ChatGPT and its public peers are, for anything sensitive, simply off-limits. So the real, practical question every such team faces is: how do you get AI's productivity without breaking the rules that protect your data? This playbook is the answer trading firms are converging on - run AI inside the firewall - and what a dev team must learn to do it.

The Core Principle: The Model Comes To The Data

The whole approach rests on inverting the public-AI model. With a public tool, your data goes out to the AI - which is exactly what regulated finance cannot allow. The private approach reverses this: the AI comes to your data and runs inside your security boundary, so sensitive information never leaves the controlled environment. Concretely, that means self-hosting the language model - running it on your own infrastructure, inside your firewall - so that when it processes your code or data, all of that happens within your controlled environment and nothing sensitive is sent to an outside service. The model queries authorized data in place; the data stays where it is; the processing happens inside the boundary. This single principle - the model comes to the data, not the data to the model - is the foundation that makes AI usable in regulated finance at all, and it is the first thing a trading dev team has to internalise: the goal is not to find a public tool that is 'compliant enough', it is to bring the AI inside your walls so the compliance question of data leaving simply does not arise.

The Spectrum: From Private Cloud To Fully Air-Gapped

Running AI inside the firewall is not one thing but a spectrum of increasing strictness, and a dev team must learn where on it their workloads belong. At one end is private, self-hosted deployment - the model runs on infrastructure you control (on-premise or in a private, isolated cloud environment) rather than a shared public service, so your data stays within your boundary. At the strictest end is fully air-gapped deployment: the models, the inference servers and the vector databases all run on isolated infrastructure with no internet dependency whatsoever - the same way defence contractors run AI against controlled data. Air-gapped is the only practical option for the most sensitive, most tightly-regulated workloads, where even a private cloud's residual external connectivity is unacceptable. Between the ends sit various VPC and private-cloud arrangements offering different balances of control and convenience. The dev team's job is to match each use case to the right point on this spectrum: not everything needs full air-gapping, but the most sensitive data and the strictest regulatory requirements do, and knowing which is which is core to doing this correctly.

What The Dev Team Actually Has To Learn

Deploying and running AI inside the firewall demands a genuinely different skill set from calling a public API, and this is the heart of what a trading dev team must learn. They have to learn to deploy and operate the models themselves - self-hosting an LLM, standing up the inference infrastructure to serve it with acceptable performance, and running the vector databases and retrieval systems that ground it in the firm's own knowledge, all inside the controlled environment. They have to learn rigorous data governance: ensuring the AI only accesses authorized data, that access is controlled and appropriate, and that sensitive information is handled within the boundary. They have to build comprehensive audit logging - every prompt and response recorded with timestamp, user identity and model version - because in regulated finance you must be able to evidence exactly what the AI did, for whom, and when. And they have to manage the practical trade-offs: on-premise and air-gapped AI has real costs and operational overhead that a public API does not, so the team must weigh where the productivity is worth the infrastructure. It is more work than calling ChatGPT - but it is the work that makes AI usable at all in their environment.

• Self-host and operate the models - deploy the LLM, stand up inference infrastructure, and run retrieval/vector databases inside the boundary.
• Enforce data governance - the AI accesses only authorized data, access is controlled, sensitive information stays within the environment.
• Log everything for audit - every prompt and response with timestamp, user identity and model version, because you must evidence what the AI did.
• Match deployment strictness to the data - private cloud for some workloads, fully air-gapped for the most sensitive and regulated.
• Weigh the trade-offs - on-prem and air-gapped AI carries real cost and operational overhead; target it where the productivity justifies it.

The Payoff: AI Copilots That Are Actually Allowed

The reward for this work is significant: a trading dev team that masters running AI inside the firewall gets the AI productivity the rest of the industry enjoys, but in a form their regulators and their own policies actually permit. That includes AI code assistants that work in a fully air-gapped, on-premises way - some enterprise coding tools are built specifically to support this, precisely for regulated industries - so developers get AI help with their code without any of it leaving the environment. It includes AI copilots for operations, compliance checks and document processing that never expose sensitive data. And it includes the ability to apply AI to the firm's own proprietary knowledge and data, safely, because the model is inside the walls with it. The point worth landing is that regulated finance is not shut out of the AI productivity revolution - it just has to enter it differently, through private and air-gapped deployment rather than public tools. The trading dev teams that learn this are the ones that will capture AI's benefits without the compliance failures, data leaks or regulatory breaches that using public tools on sensitive data would invite. It is the harder path, and in their environment it is the only one that works.

The Bottom Line

For a trading firm's or investment bank's dev team, the AI productivity everyone talks about comes with a hard constraint: the public tools are off-limits for sensitive data, because policy and regulations like PCI DSS and GLBA forbid it and regulators require financial data to stay in controlled infrastructure. The answer is not to bend the rules but to run AI inside the firewall - bringing the model to the data through private, self-hosted and, for the most sensitive workloads, fully air-gapped deployment, so nothing sensitive ever leaves the controlled environment. That demands a real new skill set - self-hosting and operating models, rigorous data governance, comprehensive audit logging, matching deployment strictness to data sensitivity, and managing the cost trade-offs - but it is exactly the skill set that lets a regulated team capture AI's benefits safely, including air-gapped AI copilots that are genuinely allowed. This kind of private, compliant, high-performance AI deployment in regulated environments is precisely the work BraivIQ does for financial clients - because the productivity is real, but only if you can get it without the data ever leaving your walls.

References & Further Reading

• Knowi - private LLM BI for fintech and banking: self-hosted AI and agentic analytics in 2026: https://www.knowi.com/blog/private-ai-for-fintech-and-banking/
• ibl.ai - financial services blueprint: air-gapped AI in 90 days: https://ibl.ai/blog/financial-services-blueprint-air-gapped-ai-90-days
• IntuitionLabs - enterprise AI code assistants for air-gapped environments: https://intuitionlabs.ai/articles/enterprise-ai-code-assistants-air-gapped-environments
• Petronella Cybersecurity - private AI for CTOs: why regulated teams leave ChatGPT: https://petronellatech.com/blog/private-ai-for-ctos-why-regulated-mid-market-leaves-chatgpt/
• US Federal Reserve - SR 11-7: Guidance on Model Risk Management: https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm