OpenAI Evals Explained: What They Are and How to Use Them

Large language models can look good in a quick test and still fail when real users show up. A prompt tweak, a model swap, or a small workflow change can quietly lower quality without anyone noticing. That is why OpenAI evals matter. They give teams a more reliable way to check what an LLM is doing and whether it still meets expectations.

OpenAI Evals is an open source framework for LLM evaluation. It helps developers test outputs in a repeatable way, compare results across versions, and build their own checks with private data when needed. 

Testing AI outputs properly is one of those things most teams know they should do but never quite formalize. This guide explains how the framework works and how to start using it.

Why Informal LLM Testing Breaks Down at Scale

A lot of teams start with what OpenAI calls vibe based evals. They try a few prompts, read the answers, and decide the app looks fine. That may work for a demo. It does not work for a product that people use every day. OpenAI's own guidance flags this as an anti-pattern because model behavior varies, and the same system can pass one moment and fail the next.

The problem gets worse as the app grows. One prompt edit can affect tone, accuracy, structure, safety, or tool use. One model upgrade can improve some tasks while making others worse. Without a real test set, those regressions can slip into production and stay there until users complain. OpenAI also notes that continuous evaluation matters because AI systems are nondeterministic and need repeated checks over time.

Users do not debug your prompts. They just stop trusting the product. When something breaks in a user-facing tool, people do not care whether it came from a model swap or a workflow change. They see wrong answers, broken JSON, or replies that ignore instructions.

By the time complaints arrive, the damage is already done. That is why structured evaluation needs to be part of the process long before launch, not an afterthought once things go wrong.

What Is the OpenAI Evals Framework and What Does It Do

OpenAI Evals is an open source framework that combines a testing environment with an open registry of benchmarks. It gives teams a structured way to measure how a model performs across real tasks instead of relying on quick checks. In real life, most teams know they should test more rigorously. The problem is finding a way to produce results you can actually compare over time. That is exactly the problem this framework solves.

That comparability is where the framework earns its place. You can run the same eval across different model versions, prompts, or app configurations and see exactly where performance changed. That gives you signals you can track rather than gut feelings after a quick review. OpenAI's developer docs describe evals as tests that check whether outputs meet the style and content criteria you define, meaning you are measuring against your own standards, not vague impressions.

The framework also comes with an open registry of benchmarks, so you are not starting from scratch. Teams can pull from existing evals, adapt them, or build custom ones for private workflows. That flexibility is part of why the project has real traction.

As of April 2026, the OpenAI Evals GitHub repository has around 18,200 stars, 2,900 forks, and over 460 contributors. Those numbers reflect a framework that real teams are actively using and building on.

What Can OpenAI Evals Measure?

OpenAI Evals can test factual accuracy, tone, safety, instruction following, and structured output formatting. If you can define what a good answer looks like, the framework can measure it. What it measures depends on what you build into it, which is both its strength and the reason teams need to think carefully about their test sets.

Here is what that looks like across the most common product use cases.

• Factual accuracy. Whether the model returns correct information on a known question, or drifts into confident but wrong answers after a model update or prompt change. A medical information tool, for example, might test whether the model consistently returns accurate dosage information across different phrasings of the same question.
• Instruction following. Whether the model does what it was told. This includes format requirements, length constraints, tone rules, and task specific instructions that users or systems pass in. A support bot might be tested on whether it always asks for an account number before attempting to resolve an issue.
• Output structure. Whether the model returns valid JSON, fills required fields, or follows a template correctly. Especially relevant for tools where a downstream system depends on a predictable format.
• Safety and refusals. Whether the model handles sensitive prompts appropriately, refuses what it should refuse, and escalates or flags what it should not answer directly.
• Tone and style. Whether the output matches brand voice, communication guidelines, or audience expectations. This one usually requires model graded evaluation rather than fixed rules.
• Regression after changes. Whether a prompt edit, model swap, or workflow update quietly broke something that was working before. This is one of the most common reasons teams build evals in the first place.

The framework is only as good as the test set behind it. Teams that build evals around real user inputs catch more problems than those relying on hypothetical scenarios. If your test cases do not reflect how people actually use the product, you will pass evals that production will fail.

Ground Truth vs. Model Graded Evals: Which One Should You Use?

The choice between ground truth and model graded evals comes down to one question: does the task have a correct answer, or does quality depend on judgment? Get it wrong and you end up either over-engineering a simple test or missing something that actually needed careful review.

Ground truth evals are the right choice when there is a single correct answer, or at least a clearly defined expected result. They work well for tasks like math, code output, data extraction, classification, and multiple choice questions. In these cases, the model response can be checked against a known answer using fixed rules. A code generation tool, for example, either returns working code or it does not. That makes scoring clear, easy to read, and straightforward to debug when something goes wrong.

Model graded evals are better when the task is more open and the answer can vary. Summaries, tone checks, support replies, writing quality. These are tasks where a correct answer does not really exist. A support reply might be technically accurate and still completely miss what the customer needed. No fixed rule catches that. A grading model or human reviewer has to make the call. For sensitive or high stakes tasks, do not rely on automated scoring alone. Periodic human review catches what the grader misses.

The table below maps common use cases to the eval type that fits them best.

What's Already in the Evals Registry

Before building anything custom, it is worth knowing what already exists. The OpenAI Evals GitHub repository includes a built-in registry of benchmarks and eval templates covering a range of model abilities. For many teams, this is the right place to start.

The registry includes evals for question answering, logical reasoning, code related checks, and safety tasks. Each eval is defined through a configuration file and linked dataset, which makes them easy to inspect, adapt, and reuse. A good example is coqa.yaml, a configuration file that tells the framework which dataset to use, how to run the test, and how to score the model's answers. It works like a small instruction file. You can open it, read exactly how the eval is set up, and modify it to fit your own use case without starting from scratch.

This matters more than it might seem. Writing a good eval from zero takes time. You need to think through the task, collect representative examples, define what a correct or acceptable answer looks like, and set up the configuration correctly. The registry shortcuts a significant part of that work.

You may not find an exact match for your product, but you will almost always find a structure close enough to adapt. That is a much faster starting point than a blank file.

How to Get Started with OpenAI Evals on GitHub

If you want to start using OpenAI Evals, the OpenAI Evals GitHub repository and OpenAI's developer docs are where to begin. Clone the repo, then install it locally with pip install -e . if you plan to create or modify evals. That gets the framework running on your machine.

One thing worth knowing before you start: the registry data is stored with Git LFS, which means some eval files will not appear automatically after cloning. You need to fetch them separately. This catches a lot of people off guard and is worth doing before you spend time wondering why the data is missing.

From there, OpenAI provides build guides, custom eval docs, and a starter guide in the cookbook for teams that want to go beyond a basic test. Those resources cover the most common setup patterns and are a faster way to get oriented than reading the full codebase.

The modern path is also broader than just local CLI work. OpenAI now supports eval setup directly in the dashboard and via the Evals API. If your team prefers a visual interface, the dashboard lets you configure and run evals without touching the command line.

If you prefer a code first approach, the API gives you full programmatic control. The right starting point depends on how your team works, but both paths are well documented.

How to Create a Custom Eval with OpenAI Evals

General benchmarks rarely match what your product actually does. A support bot, a writing assistant, a sales tool, an internal AI workflow. Each of these has its own failure modes, and a generic test set will not find them. You need examples that come from real users, not ones you invented to fill a spreadsheet.

A custom eval has two parts: a dataset and an eval definition. The dataset is your collection of real inputs and expected outputs. The eval definition is the configuration file that tells the framework how to run the test. Most teams use JSONL for the dataset, one test case per line, and YAML for the configuration that connects everything together.

Here is how to build one from start to finish.

1. Define what you want to test. Start with the task. This could be answer accuracy, tone, safety, instruction following, or output format. Being specific here makes every other step easier.
2. Collect real examples. Gather prompts or requests that reflect how your product is actually used. Hypothetical test cases miss the edge cases that real users find. The closer your examples are to production inputs, the more useful the eval will be.
3. Build your dataset. For each example, include the input and the expected output, or enough context to judge whether the answer is acceptable. Save the file in JSONL format, one test case per line.
4. Create your YAML configuration file. This file connects the dataset to the framework. It tells the system where the data is stored, which model to test, and what type of eval to run.
5. Choose the eval type. Ground truth if the task has a correct answer. Model graded if quality depends on judgment. If you are unsure, revisit the previous section.
6. Use an existing template if possible. For many common use cases you do not need to start from zero. Pull an existing template from the registry and adapt it. That alone can cut setup time significantly.
7. Run the eval and review the results. Check which answers passed, which failed, and where the model made mistakes consistently. Patterns matter more than individual failures.
8. Iterate. Update your prompts, model settings, or workflow, then run the eval again. This is how progress gets tracked and regressions get caught before users do.

One more thing worth knowing. If your team works with sensitive or private data, you can build evals using that data without exposing it publicly. That makes custom evals relevant not just for research teams, but for any product team that needs quality control on real workflows.

How to Use OpenAI Evals Across the LLM Development Lifecycle

Most teams think about evals too late. Something breaks in production, users complain, and only then does structured testing become a priority. By that point the damage is done.

Evals are useful from the start. When choosing between models or testing a new prompt strategy, running a structured test beats picking based on a few good outputs. You get a real comparison, not a feeling.

Once development is underway, run evals after every significant change. A prompt edit, a model update, a workflow adjustment. Any of these can quietly degrade something that was working. Catching that before it ships is the whole point.

The same logic applies after launch. Quality is not static. Models get updated, user behavior shifts, and new edge cases appear. Teams that run evals regularly across different ChatGPT models and configurations stop relying on user complaints to tell them something went wrong. Every failure that reaches production becomes a new test case. That is how the eval set grows and gets more useful over time.

Evals in Practice: Real World Use Cases

The best way to understand what OpenAI Evals can do is to see where teams are already using it. Three areas stand out.

Healthcare is one of the clearest examples of evals applied to high stakes output. OpenAI introduced HealthBench on May 12, 2025 as a benchmark for realistic health scenarios, built with 262 physicians across 60 countries and 5,000 real health conversations. What makes it notable is the grading approach. Each response is evaluated against rubrics written by practicing physicians, not generic accuracy checks. That is what serious eval design looks like in practice.

Code generation is another natural fit. Development tools need more than fluent answers. They need working answers. That is why correctness is the only metric that matters here, and why benchmarks like HumanEval exist specifically to test functional output rather than surface quality.

Customer support and moderation are where custom evals often pay off fastest. A support team needs to know whether replies follow policy, ask the right follow up question, and return output in the correct structure. A moderation team needs consistent refusals and reliable escalation across risky prompts. These are not academic tasks. They happen every day, and evals are one of the few tools that can catch failures before users do.

Conclusion

Informal testing does not fail dramatically. It just quietly lets problems through until enough users notice. That is the real argument for OpenAI Evals, not that it is a more sophisticated tool, but that it replaces guesswork with something you can actually measure and improve over time.

The teams that get the most out of it are not always the ones with the most complex systems. They are the ones that started early, used real inputs, and treated quality as something you track rather than assume. That habit is harder to build than the technical setup. But it is also what keeps a product reliable six months after launch.

Still relying on manual checks? The framework is open source, the registry gives you a base, and your ChatGPT Settings and model choices will be much easier to evaluate with a proper test set behind them.

FAQs

What is OpenAI Evals in simple terms?

OpenAI Evals is an open source framework for testing how well an AI model performs on the tasks you actually care about. Instead of checking a few outputs and hoping for the best, you run a structured set of tests and compare results over time. It gives teams a repeatable way to measure quality rather than guess at it.

How do I create a custom eval with OpenAI Evals?

Start with a dataset that reflects your real use case. Store your prompts and expected outputs in JSONL format, one test case per line, then define the eval in a YAML configuration file that connects the data to the model and specifies what kind of check to run. For most standard use cases, OpenAI's existing templates mean you do not need to write custom evaluation code from scratch.

Do I need to write code to use OpenAI Evals?

Not always. If you follow an existing eval template, you can supply data in JSONL format and set parameters in YAML without writing any evaluation code. You still need to think carefully about your test set and grading criteria, but for common use cases the setup is straightforward.

How often should I run evals on my LLM application?

Run evals whenever something important changes. That includes prompt updates, model swaps, workflow edits, fine tuning, or new safety requirements. The goal is continuous evaluation, not a one time check. Each significant change is an opportunity for something to quietly break, and evals are how you catch that before users do.

Can OpenAI Evals detect hallucinations?

They can help, but only if your test set is built for it. Evals detect hallucinations when you include cases with known correct answers and checks for factual accuracy. A weak or generic test set will miss real failures. The framework gives you the structure, but the quality of what you can catch depends entirely on the quality of what you put in.