Semantic memory or just Markdown?

Why Laravel Boost project rules use a generated Markdown index after we deleted semantic search.


Every team using an AI coding agent eventually sees it repeat a mistake the team already corrected.

The agent will extend the framework controller and miss the application's tenant scope. It reaches for a float type even though the project stores money as integer cents. It adds a repository layer on top of the ORM, despite the team having removed it two years ago.

All three failures come from the fact that the project has implicit conventions built up over the history of the codebase, but the agent has no durable way to remember them. It has no memory.

In this post, “memory” means the conventions a team commits, reviews, and shares through its repository.

Laravel Boost already writes Laravel guidelines into AGENTS.md or CLAUDE.md. Teams would add their own instructions and conventions to the same file. Six months later, every time you fire up an agent, you pay the context-tax for rules that rarely apply to the task at hand.

My first attempt to solve this was a semantic search tool. I deleted it five days later.

The version that finally worked was far simpler: markdown files and a generated two-column index.

The instruction file kept growing

At Laravel, both internally and for our users, we care about correct code per token spent and that the code looks like it belongs in this particular application.

Laravel-supplied guidelines cover the framework side, but project conventions need to cover choices such as base controllers, money storage, domain layers, and testing structure.

The instruction file was the obvious place to put those choices, but every addition then became part of every prompt:

  • database migration rules loaded during Livewire work

  • backend testing rules loaded while the agent edited a frontend component

The context grew unnecessarily with the team's knowledge.

Issue #606 is the clearest version of the complaint, and it was right: Boost put too much into CLAUDE.md.

Claude Code's documentation recommends keeping CLAUDE.md concise, with 200 lines as a general target. Longer files consume more context and reduce adherence. Anthropic's context-engineering guidance makes the same practical point: use the smallest set of high-signal tokens that improves the outcome.

IFScale offers supporting evidence with a caveat. It measures instruction following in business reports and provides only indirect evidence for coding agents. In its 500-instruction condition, the strongest frontier models followed 68% of the requested instructions. A large pile of good instructions still creates competition for attention.

Scoping the instructions more intelligently, which is a simple system, lets us preserve the project knowledge and control when it enters the prompt.

Thirteen agents, one format

When we built project rules, Boost supported 13 agent integrations. I read through the instruction systems for all of them. They agree on almost nothing.

Some supported glob-scoped files. Some loaded nested instruction files after the agent entered a directory. Some offered only an always-loaded project file. The trigger behavior varied enough that the same rule would load differently from one harness to the next.

Maintaining 13 implementations would have become its own product. Boost already had a portable surface in each integration: the instruction file it writes during installation. That file could point every agent toward plain Markdown in the repository.

The rules therefore had to work through files, globs, and ordinary agent tools.

I deleted the search tool

The first version I built was the clever “semantic memory” layer that exposed two MCP tools, memory-search and memory-write, backed by a 359-line MemoryRepository. The agent could record a convention and run semantic search later.

Five days after building it, I removed the search layer. It was overcomplicated and ineffective for the size of memory most teams are actually working with.

A mature Laravel application may accumulate a few dozen short convention files. A few dozen files don’t justify the coordination costs: embeddings, a vector index, and invalidation. The index could become stale and drift from the correct Markdown. It’s only a few dozen files!

Every coding agent knows how to list files and run grep.

Anthropic's Agent SDK guidance recommends starting with agentic search and adding semantic retrieval when scale or variation demands it. Boris Cherny described a similar experience building Claude Code: the team tried vector databases, ran into stale indexes and permission complexity, and found glob and grep more useful for code.

Our rule corpus fit comfortably inside the file system.

Building embeddings, an index, and an invalidation strategy for forty short Markdown files is just over-engineering. It adds a staleness failure mode to something ls or grep can just handle itself.

The rules directory was easy to ignore

The next version stored Markdown under .ai/rules. Each file declared the paths it covered:

Boost told the agent to read relevant rules before planning or editing.

In exploratory testing, agents followed that instruction inconsistently, especially on smaller models. I kept rewriting the sentence. The models kept skipping the directory.

The first action required the agent to judge relevance before it had seen the available rules. The agent knew .ai/rules existed. Its contents stayed hidden until the agent chose to inspect them. I needed to put the choices in front of it. "Check whether any rules are relevant" asks the agent to decide what relevant means, about a directory it hasn't read, before it has done anything.

So I gave it something to look at, at a different resolution. A higher-level index.

Instead of expecting it to read all the rules to see what applied, I gave it a generated two-column table that gets rebuilt on every update to the rules. Small enough to be functionally free.

The generated index

Boost now rebuilds .ai/rules/index.md after every rule write:

The index is short enough to include in the starting instructions. The detailed rules stay in their files until the task reaches the matching path.

The lookup has four steps:

  1. Identify the file being edited.

  2. Match its path against the first column.

  3. Read the file named in the second column.

  4. Apply those rules while planning and editing.

The model already knows which file it plans to touch. The table supplies the corresponding rule file. Path matching becomes the first action, and the detailed convention enters context a few lines later.

We saw agents follow this lookup more consistently than the earlier directory instruction. This observation comes from exploratory testing; a controlled evaluation remains future work.

Writing a rule

The index has to agree with the rule files. Boost records conventions through an MCP tool named record-rule, which accepts a glob, title, and note.

RuleRepository::write() performs the full update:

  1. derive an area from the stable part of the glob;

  2. find or create the rule file;

  3. merge the glob into its frontmatter;

  4. append the rule; and

  5. rebuild index.md.

A successful call updates the convention and the generated index together. The files remain ordinary human-readable Markdown, so the team can read the change and review it in a pull request.

Path lookup plus grep

Globs handle conventions tied to a part of the tree. A money-storage rule may apply in a model, action, request, and test, so its relevance comes from the concept under discussion.

Boost tells the agent to use both routes:

  • Match paths against the generated index.

  • Run grep -rin over .ai/rules for concepts that cross directories.

The two routes cover the cases we have seen in practice. We still want a proper comparison of an always-loaded rule dump, an index, grep, and the combined approach.

Anthropic arrived at the same shape

Soon after I presented this design, Taylor visited Anthropic and passed along the Claude Code team's current advice: keep project memory in a directory, tell the agent where it lives, and let the agent read the relevant files.

Anthropic's public tools use this progressive-disclosure shape in several places. Claude Code keeps its auto-memory index concise and stores details in topic files. Agent Skills describes SKILL.md as a table of contents for supporting material. Tool Search loads tool definitions as they become relevant.

Boost adds the generated index because it has to work across many harnesses and a wider range of models. The visible glob table helps models that do less exploration on their own.

The public evidence also gives us a reason to keep grep. A study of progressive disclosure found a capable tool-using agent that skipped the supplied index and inferred file paths directly. Agents may take a good route that differs from the one we supplied.

Open questions

Our current evidence comes from exploratory use. A controlled evaluation should compare:

  • every rule loaded before every task;

  • the generated index by itself;

  • the rules directory with grep;

  • the combined index and grep instructions; and

  • several models and rule-corpus sizes.

Staleness needs separate work. A committed rule can survive after the codebase changes. Pull-request review makes rules visible. An unrelated implementation change may still quietly invalidate an old instruction. We want a way to audit that drift before the agent confidently follows stale guidance.

Try project rules in Boost

Boost ships project rules as Markdown files with a generated index. The starting context contains the high-level map, while matching rules load when the task reaches them. Grep handles concepts that span the tree.

Project rules are enabled by default in Laravel Boost. Ask your agent to remember a project convention, review the generated rule, and see how it handles the next relevant change.

Laravel is the most productive way to
 build, deploy, and monitor software.

By submitting this form, you agree to our terms. You can opt-out anytime.