databasedmcp gives your AI assistant databases it keeps between conversations, so everything in this article works from the day you connect it. Free to start, no card.
Get started freeWhy build it yourself
A small team iterating on a prompt needs to answer one question many times a day: is this version better than the last one, and what broke? LangSmith, Langfuse and Braintrust answer it with tracing, datasets and dashboards. If you have a script that runs and scores your cases, what's missing is somewhere to keep the results.
Codex runs the eval script anyway. Have it write the run and each case's score to a database as it goes, with the prompt version and commit. "Did v4 break anything v3 got right?" becomes a join, and everyone on the account sees the same history.
Connect Codex to databasedmcp
databasedmcp is a remote MCP server that gives AI assistants private SQLite databases, with nothing to install.
Create an API key on the API keys page. Keep it in an environment variable, with export DATABASED_API_KEY=… in your shell profile. The config below names the variable, never the key.
[mcp_servers.databased] url = "https://databasedmcp.com/mcp" bearer_token_env_var = "DATABASED_API_KEY"
Restart Codex and it lists the databasedmcp tools.
Any other MCP client works the same way; the docs have the details.
Copy this prompt to build it
Paste this into Codex to set up the database, its tables and its guide in one go. The rest of this article walks through what it makes and how you use it.
Set up an eval log for me in databasedmcp, in one go. 1. Create a database called evals, described as "LLM eval log: every eval run and the score of each case in it, for comparing prompt versions". 2. Create two tables: - runs: id, model (required), prompt_version (required, the tag in prompts/), dataset (required, the case file's name), git_sha (the short commit hash), notes, ran_at (defaults to now). - results: run_id (references runs), case_id (the id from the dataset, stable across runs), score (a number from 0 to 1, required), passed (1 or 0, required), latency_ms, with one row per case per run. 3. Write the database a guide that says what each table is for: passed is 1 if the scorer said the case passed (its own threshold), and scores are logged as the scorer computed them, never re-scored here. It should say to insert the run first, then all its results in one statement with json_each. Include the query to compare runs: the last 10 runs with their prompt version, average score, cases passed and total cases, newest first. And say how to find regressions from run A to run B: cases that passed in A and failed in B. From then on: after you run the evals, log the run with the model, prompt version, dataset and commit, then every case from the script's output. When I ask whether a version is better, compare the runs and list what regressed. Show me the tables when you're finished.
Ask for the database
You don't write the schema. Describe what you want:
Create an eval log in databasedmcp: one row per eval run (model, prompt version, dataset, commit, notes) and one row per case with its score, pass or fail, and latency. After you run the evals, log the run and every case from the script's output.
Codex creates a database called evals and its tables:
CREATE TABLE runs ( id INTEGER PRIMARY KEY, model TEXT NOT NULL, prompt_version TEXT NOT NULL, dataset TEXT NOT NULL, git_sha TEXT, notes TEXT, ran_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP ); CREATE TABLE results ( run_id INTEGER NOT NULL REFERENCES runs(id), case_id TEXT NOT NULL, score REAL NOT NULL, passed INTEGER NOT NULL CHECK (passed IN (0, 1)), latency_ms INTEGER, PRIMARY KEY (run_id, case_id) );
It also writes the database a guide on how the tables are meant to be used. The guide comes back with the schema whenever an assistant looks the database up. Codex files things the same way next week as today, and so does any other assistant you connect.
# evals: runs and per-case scores - **runs**: one per eval run. `prompt_version` is the tag in prompts/, `git_sha` the short commit hash, `dataset` the case file's name. - **results**: one per case per run. `case_id` is the id from the dataset, stable across runs. `score` is 0 to 1 from the scorer; `passed` is 1 if the scorer said the case passed (its own threshold). Log scores as computed; never re-score here. After running the evals, insert the run, then all its results in one statement with json_each. Compare runs: `SELECT r.id, r.prompt_version, round(avg(x.score), 3) AS score, sum(x.passed) AS passed, count(*) AS cases FROM runs r JOIN results x ON x.run_id = r.id GROUP BY r.id ORDER BY r.id DESC LIMIT 10` Regressions from run A to B: cases with passed = 1 in A and 0 in B.
Use it
From here on you talk, and Codex writes the rows. Each request below is one statement:
Run the support-triage evals against prompt v3 and log it.
INSERT INTO runs (model, prompt_version, dataset, git_sha) VALUES (?, ?, ?, ?) RETURNING id -- params: ["claude-haiku-5-5","v3","triage-cases.jsonl","9b1e4d0"]
Codex logs the scores the script printed.
INSERT INTO results (run_id, case_id, score, passed, latency_ms) SELECT ?, json_extract(value, '$.case'), json_extract(value, '$.score'), json_extract(value, '$.passed'), json_extract(value, '$.ms') FROM json_each(?)
-- params: [1,"[{\"case\":\"refund-angry\",\"score\":0.92,\"passed\":1,\"ms\":840},{\"case\":\"login-loop\",\"score\":0.88,\"passed\":1,\"ms\":910},{\"case\":\"spam-short\",\"score\":0.41,\"passed\":0,\"ms\":610}]"]
A whole run's results go in as one statement, however many cases the dataset has.
Now try prompt v4, the one with the shorter system prompt.
INSERT INTO runs (model, prompt_version, dataset, git_sha, notes) VALUES (?, ?, ?, ?, ?) RETURNING id -- params: ["claude-haiku-5-5","v4","triage-cases.jsonl","c27f8a3","Shorter system prompt, examples moved to the end."]
Codex logs the v4 scores.
INSERT INTO results (run_id, case_id, score, passed, latency_ms) SELECT ?, json_extract(value, '$.case'), json_extract(value, '$.score'), json_extract(value, '$.passed'), json_extract(value, '$.ms') FROM json_each(?)
-- params: [2,"[{\"case\":\"refund-angry\",\"score\":0.95,\"passed\":1,\"ms\":620},{\"case\":\"login-loop\",\"score\":0.52,\"passed\":0,\"ms\":650},{\"case\":\"spam-short\",\"score\":0.83,\"passed\":1,\"ms\":480}]"]
Ask it questions
Once there's data, each report you'd pay for is a single query:
How do the last runs compare?
SELECT r.id, r.prompt_version, r.model, round(avg(x.score), 3) AS score, sum(x.passed) AS passed, count(*) AS cases, round(avg(x.latency_ms)) AS avg_ms FROM runs r JOIN results x ON x.run_id = r.id GROUP BY r.id ORDER BY r.id DESC LIMIT 10
What did v4 break that v3 got right?
SELECT a.case_id, a.score AS score_before, b.score AS score_after FROM results a JOIN results b ON b.case_id = a.case_id AND b.run_id = ? WHERE a.run_id = ? AND a.passed = 1 AND b.passed = 0 ORDER BY b.score -- params: [2,1]
Which cases fail most often across all runs?
SELECT case_id, count(*) AS runs, sum(passed = 0) AS failures, round(avg(score), 2) AS avg_score FROM results GROUP BY case_id HAVING failures > 0 ORDER BY failures DESC, avg_score LIMIT 20
You can also open the database on the website, page through its rows and run read-only SQL yourself.
What you give up
There's no tracing. LangSmith, Langfuse and Braintrust instrument your app through an SDK and record every call, prompt and token in production, with a viewer to step through them. This log holds only what your eval script computed: scores, pass or fail, latency. It doesn't run or score anything.
Instead of charts, annotation queues or side-by-side output views, you get queries and the website's table view. If you need an output, store a short excerpt in a column, not the whole transcript. When you need production monitoring, human review workflows or a dashboard for people who don't write SQL, the paid tools earn their keep.
Try it
databasedmcp is free to start, with no card. Free has 100 tool calls a week, enough to set this up and try it for a week. Starter has 1,000 for everyday use.