Context layer for AI agents: how we built ours

TL;DR A context layer is everything an AI agent knows about your data and your business, before and while it answers. A semantic layer is one part of it. Ours, behind Trackingplan's analytics agent, has five parts: what exists, what it means, what changed, what your team knows and what the industry knows. The five lessons that shaped it: put context next to the action, keep the always-on part small, explain the why and not the how, give generic examples, and let ordinary code do whatever must always happen.

In September we launched the new Trackingplan, and last week we described the agent harness for marketing data underneath it. This post goes one level down, into the part of the harness that took the most iterations: the context layer.

What a context layer is

A context layer is everything an AI agent knows about your data and your business, before and while it answers: what data exists, what it means, what changed, what your team knows that the data does not say, and the rules for using all of it.

The model brings reasoning. The context layer brings the facts that reasoning works on. Two agents on the same model answer the same question differently because their context layers differ, which is why we spend more engineering time here than on any prompt.

Context layer vs semantic layer

The two get confused because the semantic layer is the best-known part of the context layer.

A semantic layer gives every query the same meaning of revenue. It cannot tell the agent that the Meta pixel was retired on purpose last month, that a tag manager release changed the checkout yesterday, or that zero rows means the data was not loaded rather than that nobody visited.

The five parts of our context layer

Ours is built for one kind of data, marketing and analytics tracking, and has five parts.

1. What exists. The tracking plan is inferred from real traffic: every event, property and destination your sites and apps actually send, rebuilt from the hits, with the observed volumes next to it. Nobody documents it by hand, so it is never out of date. That is what Schema management does in the product.
2. What it means. A semantic layer of 18 business tables plus 10 small lookup dictionaries, with every column documented in the analyst's vocabulary. Raw names that mislead get renamed: what the raw data calls a warning is one hit that did not match its spec, while real warnings are derived later with tolerances, so the column is called spec mismatches and the agent does not report raw signals as warnings. A check in CI compares the column documentation with the live database, so the docs cannot drift from the data.
3. What changed. We keep a copy of each connected tag manager's configuration and record every change to it, so the agent can say which release changed which tag. App and SDK versions travel on the hits.
4. What your team knows. Plan context holds the facts and standing rules the traffic cannot show, such as a pixel retired on purpose or a test domain to exclude. It is shared by everyone on the plan, every entry carries its date, newer entries win, and the data wins over an entry it contradicts.
5. What the industry knows. How GA4, Adobe Analytics, Meta, consent platforms and tag managers send, name and drop data. A small core is always loaded; the rest lives in 11 table manuals, 11 metric files, 8 files of example queries and 21 skills, read only when a question needs them.

Lesson 1: put context as close as possible to the action

A rule in the system prompt competes with everything else the model has read. A rule that arrives with the result of the action it governs does not.

We learned it with ambiguous site names: the prompt told the agent to ask when several sites matched, and it picked the first one anyway. The rule now comes back inside the result of the tool that finds the site, as a verdict: resolved, no match, or several match, stop and ask. The same pattern runs through the harness:

• The tool that checks an answer before it goes out returns the known data issues on the exact events and dates the answer used, with the instruction on whether to mention them.
• The tool that loads a plan's catalog says in its description that zero rows on a scope that was not loaded means "not loaded", not "no traffic".
• When a query fails, the error comes back with a hint about the specific mistake, not a generic message.

Our internal rule is short: a tool description is the tool's contract. If the agent ignores something, we look at the prompt and the tool descriptions before we write another sentence.

Lesson 2: the larger the context, the less the model obeys

It still holds, unfortunately. The first version put every rule, table manual and metric definition in the prompt, 58,750 tokens before the first question. The agent knew everything and followed it unevenly.

We split it into a core of what every answer needs before any tool call, and manuals the agent reads on demand. The core fell to 29,026 tokens. On the 85-case suite we had then, the old flat prompt scored 66, 69 and 74 over three runs; the new layout scored 68, within that spread, with 22% fewer tokens across the suite. It made more tool calls per answer (8.2 against 5.9) and took about 25% longer, because every read is a step.

Three things we keep from it:

• Budgets, checked in CI. The core has a ceiling of 15,000 tokens, each on-demand file 7,000, each skill description 80.
• Caching does not fix it. A cached prompt is cheaper, not easier to follow: attention degrades with the size of the prompt, not its price.
• Voluntary reads get skipped. In our first measurement the agent read a table's manual before querying it 69% of the time. With the read required as a gate in the core, the frontier models we run read it in 22 of 22 cases. And splitting further has a cost: one extra read re-sends the whole conversation, about 39,000 tokens in our case, so we only move a section out if fewer than about one question in 25 needs it.

Lesson 3: instruct the why, not the how

Our early manuals read like procedures: first do this, then check that, if this then that. They broke on every case we had not listed.

What works is the heuristic plus its reason. "Never sum across accounts" is a rule the model applies mechanically. "Never sum across accounts, because a site often sends the same interaction to several accounts of one platform, for example during a property migration, and the total counts it twice" is a rule it applies to cases we never wrote down, including how to answer when someone asks for the total anyway. Frontier models plan well on their own, and a step-by-step script written once is worse than the plan they make for the question in front of them. Our guide for writing agent context says it in one line: the right altitude is the heuristic plus its reason, not an if-else recipe and not a vibe.

Lesson 4: make SQL examples generic skeletons

Exact example queries get copied: the agent reuses the same filters, the same dates and the same destination even when the question is about another one. Our examples are skeletons with placeholders, and each part carries a comment with its reason:

-- 1. Daily stats, not raw hits: cheaper, and hits are already a full-traffic estimate (no sampling weight).
SELECT event_name, sum(hits) AS hits
FROM daily_event_stats
WHERE plan_id = '{PLAN_ID}'
  AND destination = '{DESTINATION}'
  -- 2. Complete days only: today is still arriving.
  AND date BETWEEN today() - 30 AND today() - 1
GROUP BY event_name
ORDER BY hits DESC
LIMIT 25

The model learns the intent (which table, why complete days, why the limit) instead of a query to paste. The same idea applies to metrics: a page view is page_view in GA4, screen_view in Firebase and PageView in TikTok, so the metric file defines it per destination once, and tells the agent never to replace it with a flat list of names.

Lesson 5: software 1.0 still wins

Anything that must always happen should not depend on the model remembering to do it. We call it deterministic magic: ordinary code around the model that fixes or blocks things before they become a wrong answer.

• Queries are repaired automatically. When the query tool receives a query with a known dialect mistake, such as an alias that shadows the column it aggregates, the code rewrites it and runs it again before the result goes back to the model.
• Writes are impossible, not discouraged: a guard rejects anything that is not a read.
• Timeouts have a rule. After one timeout, queries run one at a time; after two, the agent answers with what it has.
• Python runs in a sandbox with no network and no credentials, and gets data only from the results of earlier queries.
• Team context has a budget. An entry that does not fit is refused, never silently trimmed, and a test checks that a personal remark ("I'm in Madrid, answer in Spanish") is not stored as a fact about the site.

What we measure

Every change to the context layer runs through the eval suite, which grades the number and the path: each case carries the SQL for the right answer, and can require or forbid tools, skills and tables. A question about consent that touches the ad spend table fails, even if the answer is right. That is how we know a smaller core did not cost accuracy, and how we found the manuals the agent was skipping.

What we would tell another team

• Treat the context layer as the product. The model is shared with everyone; this is not.
• Start with the data you have, not documentation someone has to keep current: infer what exists from the source.
• Keep the always-on part small, with a budget you enforce, and measure every read you add.
• Put each rule where its decision happens, write its reason, and move it into code if it must never be broken.
• Grade the path, not only the answer.

Frequently asked questions

What is a context layer?

A context layer is everything an AI agent knows about your data and your business, before and while it answers: what data exists, what it means, what changed, what your team knows that the data does not say, and the rules for using all of it. It sits between the model and the data, and it decides whether the model's answers are right.

What is the difference between a context layer and a semantic layer?

A semantic layer defines metrics and entities once, so every query uses the same meaning of revenue or a session. A context layer includes that and adds what an agent also needs: what exists in the data today, what changed and when, what the team knows that is not in the data, domain knowledge, and the rules that decide how the agent uses it. The semantic layer is one part of the context layer.

How do you build a context layer for AI agents?

Keep a small core the agent always reads, and put the rest in files and tools it reads when it needs them. Write the why of each rule rather than step-by-step procedures, give generic examples instead of exact ones, put rules where the decision happens, in tool descriptions and tool results, and enforce in code anything that must always happen. Then measure every change with evals that grade both the answer and the path.

Does a bigger context window make the agent better?

No. In our experience the more context the model carries, the less reliably it follows any single instruction in it. Prompt caching makes a large prompt cheaper, but not easier to follow: the degradation depends on the size of the prompt, not on its price.