Authors: Bonnie Xu, Aravind Suresh, Emma Tang · Source: https://openai.com/index/inside-our-in-house-data-agent/
Published: January 29, 2026 · Fetched: 2026-09-05
Note: body below is original English text extracted from the live page. Do not treat this file as a translation.
Data powers how systems learn, products evolve, and how companies make choices. But getting answers quickly, correctly, and with the right context is often harder than it should be. To make this easier as OpenAI scales, we built our own bespoke in-house AI data agent that explores and reasons over our own platform.
Our agent is a custom internal-only tool (not an external offering), built specifically around OpenAI's data, permissions, and workflows. The OpenAI tools used to build and run it (Codex, GPT-5 flagship model, the Evals API, and the Embeddings API) are the same tools available to developers everywhere.
Our data agent lets employees go from question to insight in minutes, not days. Today, teams across Engineering, Data Science, Go-To-Market, Finance, and Research lean on the agent to answer high-impact data questions. The agent combines Codex-powered table-level knowledge with product and organizational context. Its continuously learning memory system means it also improves with every turn.
Why we needed a custom tool
OpenAI's data platform serves more than 3.5k internal users working across Engineering, Product, and Research, spanning over 600 petabytes of data across 70k datasets. At that size, simply finding the right table can be one of the most time-consuming parts of doing analysis. Even with the correct tables selected, producing correct results can be challenging (many-to-many joins, filter pushdown errors, unhandled nulls).
How it works
Our agent is powered by GPT-5.2 and is designed to reason over OpenAI's data platform. It's available as a Slack agent, through a web interface, inside IDEs, in the Codex CLI via MCP, and directly in OpenAI's internal ChatGPT app through a MCP connector.
Rather than following a fixed script, the agent evaluates its own progress. If an intermediate result looks wrong (e.g., zero rows due to an incorrect join or filter), the agent investigates what went wrong, adjusts its approach, and tries again. This closed-loop, self-learning process shifts iteration from the user into the agent itself.
Context layers
- Table Usage — schema metadata, lineage, historical query inference
- Human Annotations — curated descriptions capturing intent, semantics, business meaning, caveats
- Codex Enrichment — code-level definition of tables (Spark/Python pipelines), refreshed automatically
- Institutional Knowledge — Slack, Google Docs, Notion with permissions-aware retrieval
- Memory — corrections and nuances saved for next time (global and personal)
- Runtime Context — live warehouse queries when prior context missing/stale; talk to metadata/Airflow/Spark
A daily offline pipeline aggregates table usage, human annotations, and Codex-derived enrichment into embeddings via the OpenAI embeddings API for RAG at query time.
Workflows
After rollout, users frequently ran the same analyses for routine repetitive work. The agent's workflows package recurring analyses into reusable instruction sets (e.g., weekly business reports, table validations).
Moving fast without breaking trust
Building an always-on, evolving agent means quality can drift just as easily as it can improve. Without a tight feedback loop, regressions are inevitable and invisible. The only way to scale capability without breaking trust is through systematic evaluation.
Evals are built on curated sets of question-answer pairs. Each question targets an important metric or analytical pattern, paired with a manually authored "golden" SQL query that produces the expected result. For each eval, we send the natural language question to its query-generation endpoint, execute the generated SQL, and compare the output against the result of the expected SQL.
Evaluation doesn't rely on naive string matching. Generated SQL can differ syntactically while still being correct, and result sets may include extra columns that don't materially affect the answer. We compare both the SQL and the resulting data, and feed these signals into OpenAI's Evals grader. The grader produces a final score along with an explanation.
These evals are like unit tests that run continuously during development to identify regressions as canaries in production; this allows us to catch issues early and confidently iterate as the agent's capabilities expand.
Agent security
The agent operates purely as an interface layer, inheriting and enforcing the same permissions and guardrails that govern OpenAI's data. All access is strictly pass-through. It exposes its reasoning process by summarizing assumptions and execution steps alongside each answer.
Lessons learned
- Less is More — overlapping tools confused the agent; restricted and consolidated tool calls.
- Guide the Goal, Not the Path — highly prescriptive prompting degraded results; higher-level guidance with GPT-5 reasoning chose better paths.
- Meaning Lives in Code — schemas and query history describe shape/usage; true meaning lives in producing code. Codex crawls the codebase so the agent reasons about what each table contains.
Isolation note (for KB indexing)
Candidate = successive versions of the Data Agent application / workflows / memory-backed behavior under development. Optimizer = engineering iteration (Codex, prompt/tool consolidation, enrichment pipelines). Eval & control = fixed question set + golden SQL + Evals API grader; continuous canaries; production failures and corrections feed memory/eval growth. The golden SQL set is not rewritten by the agent under test in the same promotion cycle.