Britain Built The Evals Framework The World Uses: Inspect, The AI Security Institute, And Why UK Developers Should Be Proud - And Use It

0.3.268 - The Inspect version on PyPI as of 22 September 2026 - the 246th release since open-sourcing in May 2024  ·  171 + 200 - Benchmark implementations listed in Inspect Evals, with over 200 pre-built evaluations ready to run on any model  ·  MIT, Python - Open-source under a permissive licence, built by the UK AI Security Institute with Meridian Labs  ·  8 May 2026 - Community contributions moved to a registration process with automated validation - a maintained, governed commons

Ask an engineer at a frontier lab, a safety researcher, or an enterprise team building agents how they evaluate a model, and there is a strong chance the answer involves a Python package called Inspect - and a weaker chance they know where it came from. Inspect is the open-source framework for frontier AI evaluations built by the UK AI Security Institute, the government body created to test advanced models, developed with Meridian Labs, released under the MIT licence, and maintained with a cadence most commercial projects would envy: as of 22 September 2026 the published version on PyPI was 0.3.268, the 246th release since the project was first open-sourced in May 2024. Around the core framework sits Inspect Evals, a separate MIT-licensed repository of benchmark implementations that lists 171 evaluations as of this month, with more than 200 pre-built evaluations ready to run against any model, and since 8 May 2026 community contributions have flowed through a registration process in which a bot validates each submission and derives its metadata - the mark of a project that has moved from release to governed commons. In a year in which British technology has too often been discussed in terms of standards born here and stewarded elsewhere, this is the counterpoint that deserves telling: a UK institution built, shipped, maintained and kept the stewardship of a tool the world adopted. As an AI Agency London that evaluates every agent it ships, we use it, and this educational, openly pro-UK read is why British developers should know it is theirs.

What Inspect Is, In Code

For an engineer, Inspect is best understood as a composable toolkit for turning 'is this model or agent any good at X?' into a reproducible, scored, logged experiment. Its building blocks are few and orthogonal. A dataset supplies the samples - inputs and, where applicable, targets. A solver defines how the model is driven through a sample: a plain prompt, a chain of prompting steps, or a full agent loop with tools, since Inspect supports agentic evaluations in which the model plans, calls tools and acts inside a sandboxed environment. Tools give the model capabilities during a solve - a bash sandbox, a browser, custom functions - with sandboxing so that agentic evals run safely and reproducibly. And a scorer decides how the output is judged: exact match, model-graded rubrics, pattern checks, or custom logic. Compose those and you have a task; run a task against a model and Inspect produces a structured log of every sample, every message, every tool call and every score, with a log viewer for inspecting where and why a model succeeded or failed. Because the blocks are reusable, the same dataset can be run through different solvers, the same agent through different scorers, and the same task against any model behind any provider - which is what makes it a framework rather than a script. Inspect Evals then supplies the library: over two hundred implementations of published benchmarks across coding, agentic tasks, reasoning, knowledge, behaviour and multi-modal understanding, each a task you can run in a line, and since May a registration flow through which the community adds new ones against a published paper and source repository.

• Datasets - samples with inputs and targets, from files, Hugging Face or your own data.
• Solvers - how the model is driven: single prompts, multi-step chains, or full agent loops with planning and tools.
• Tools and sandboxes - capabilities the model can use during a solve, run in isolated environments for safe, reproducible agentic evals.
• Scorers - exact match, model-graded rubrics, pattern checks or custom logic that turn outputs into scores.
• Logs and viewer - every sample, message, tool call and score recorded, with a viewer for diagnosing failures.

Why A UK-Stewarded Open Evals Standard Matters

The significance of Inspect goes beyond a convenient library, and it is worth being precise about why. Evaluation is where claims about AI meet evidence: whether a model is safe to deploy, whether an agent can be trusted with a task, whether a new release regressed - all of it rests on evals, and the framework that runs them shapes what gets measured, how, and how comparably. A framework that is open, model-agnostic and stewarded by a public safety institute has properties a vendor's internal harness never can: anyone can inspect how a score is produced, anyone can run the same test against any model, and the results are comparable across the industry rather than trapped inside one company's marketing. That is a genuine contribution to safety, because it lets the world verify rather than trust. It is also a contribution to sovereignty of a subtle and durable kind. Compute is bought and models are licensed, but the standard by which they are judged compounds - a widely adopted evaluation framework shapes the discourse, the benchmarks and ultimately the expectations of an entire field, and Britain holds that position by having done the engineering. It is precisely the kind of soft, technical influence that a national open-source strategy is supposed to secure, achieved here not by policy paper but by shipping. For UK developers the lesson is both proud and practical: the tool the world uses to evaluate AI is British, open and maintained, and using it well is how a British team holds its own systems to the same standard.

How A UK Team Should Adopt It

The practical value for a British engineering team is immediate, because Inspect is exactly the tool needed for the evaluation discipline that production agents now demand. Start by running the relevant pre-built evaluations from Inspect Evals against the models you are considering - coding, reasoning, agentic tasks - so that model selection is evidence-based rather than benchmark-marketing-based, and so that a model swap can be measured. Then build your own tasks: your real inputs as a dataset, your agent as a solver with its actual tools in a sandbox, and scorers that encode what 'correct' means for your business - which turns Inspect into the evaluation set your continuous integration gates on, with structured logs that show exactly which cases moved when a prompt, tool or model changed. Use the log viewer as your debugging surface for agent behaviour, because a failed sample with its full trace of messages and tool calls is worth an hour of speculation. And consider contributing: the registration process opened in May is how a UK team can add an evaluation that matters to its domain to the shared library, and having British engineers shaping the benchmark commons is the kind of participation that keeps stewardship meaningful. A framework built in Britain, adopted by the world, used by British teams to hold their own systems to a rigorous standard - that is the whole story, and it is a good one.

The Bottom Line

Inspect - the UK AI Security Institute's open-source framework for frontier AI evaluations, built with Meridian Labs, MIT-licensed, at version 0.3.268 after 246 releases since May 2024, surrounded by Inspect Evals' 171 benchmark implementations and over 200 pre-built evaluations and a governed community registration process since 8 May 2026 - is a piece of AI infrastructure the world runs that a British public body built, shipped, maintained and kept. In code it is a composable toolkit of datasets, solvers, sandboxed tools and scorers that turns any question about a model or agent into a reproducible, scored, logged experiment. It matters for safety, because an open, model-agnostic framework lets the world verify rather than trust, and for sovereignty, because the standard by which models are judged compounds and Britain holds it by having done the engineering - the counterpoint to a year of UK-born standards stewarded elsewhere. The caveats are honest: a framework is not a policy, maintenance is a continuing choice, and adoption is earned. For UK teams the response is practical pride: use it to select models on evidence, build your own tasks and gate your agents on them, debug with its logs, and contribute to the commons. Holding the agents we ship to that standard, with a British-built tool, is exactly how we work.

References & Further Reading

• UK AI Security Institute - Inspect documentation and evals: https://inspect.aisi.org.uk/
• GitHub (UKGovernmentBEIS) - inspect_ai: a framework for large language model evaluations: https://github.com/UKGovernmentBEIS/inspect_ai
• GitHub (UKGovernmentBEIS) - inspect_evals: collection of evals for Inspect AI (171 implementations; registration process): https://github.com/UKGovernmentBEIS/inspect_evals
• AISI - announcing Inspect Evals: https://www.aisi.gov.uk/blog/inspect-evals
• CASRAI - Inspect: UK AISI's open-source eval harness (version and release history): https://casrai.org/guides/inspect-ai-evaluation-framework-uk-aisi