The hardest part of building an AI agent is not the model. It is the memory. Every agent framework I have used in the last year has its own half-baked solution for remembering what the user said three conversations ago, and every one of them starts breaking the moment you take it past a demo. Mengram is a new API that bills itself as the memory layer for AI apps, and after a few weeks of using it on a real project, I have opinions.
What memory even means here
When AI vendors say “memory,” they mean different things. The simplest is conversation history, the log of messages between the user and the model. That works for a single session and falls apart the moment the user opens a new tab. The next step is a vector store (a database optimized for finding similar items by meaning, not by exact match) of past interactions, which lets the agent recall things from prior sessions. The third step is a structured representation of who the user is, what they care about, and how they prefer to be talked to. That last one is what most people actually want and what almost nobody ships.
Mengram tries to do the third thing. The API takes in conversation snippets and outputs a structured memory graph: entities (people, projects, tools), the relationships between them, and a confidence score for each. The point is that when the agent starts a new conversation, it can ask Mengram what it knows about the user, and get back something better than a list of past messages.
This is a real problem. I have built the homegrown version of this three times, and each time the maintenance burden eventually outweighed the value. A focused API that does one thing well is appealing, especially if it spares me from writing another entity extractor.
How the API is shaped
The basic primitives are simple. You feed it text, you get back memories, and you can query for memories later. Under the hood, Mengram is doing extraction (pulling entities and facts out of unstructured text), consolidation (merging redundant memories and resolving conflicts), and retrieval (finding the memories that are most relevant to a given query).
The shape of the API is roughly:
- POST a message or a thread to ingest it as a memory source.
- GET a query, with optional filters by entity, date, or topic, and get back a ranked list of relevant memories.
- Update or forget a memory explicitly when the user asks to be forgotten, which is the GDPR-and-good-taste move.
What I appreciate is that the API does not pretend to be a model. It is a layer on top of a model, and the model choice is configurable. The vendor’s hosted product uses their own pipeline, but the open-source version (which I am running for evaluation) lets me swap in any LLM API I want for the extraction step.
What I built it into
I wired Mengram into a small agent that I use to triage my email. The agent reads an email, decides whether to file it, reply to it, or flag it for me, and then asks Mengram whether I have any context on the sender. With Mengram in the loop, the agent now knows that “Person X is working on Project Y, last emailed me about budget concerns in October.” That changes the response, and it changes it in a way I would not have caught by hand.
The implementation was about 80 lines of glue code. Most of that was a retry loop and a logger, because I am the kind of person who logs what an agent does. The actual memory calls were a handful of lines each, which is the right shape for an API that is meant to be a building block.
I also tried it in a more demanding case: a customer support agent that has to remember what a customer said across multiple tickets. The consolidation step was the part I was watching, because customer support histories have a lot of repetition, and I expected the memory to get bloated with redundant entries. Mengram’s consolidation is conservative, which I think is the right call. It merges obvious duplicates and leaves the rest alone, rather than aggressively rewriting memories and losing context.
What it does well
The thing I keep coming back to is that Mengram is honest about its failure modes. When the extraction is uncertain, the memory comes back with a low confidence score, and the API makes it easy to surface that uncertainty to the calling agent. That is the difference between a memory layer that quietly makes things up and one that lets you decide how to handle ambiguous cases.
Other things I like:
- The retrieval is fast. I am running the open-source version on a small VPS (virtual private server, basically a cloud-hosted Linux box) with a local model, and the typical query returns in under 200 milliseconds with thousands of memories in the store.
- The forget endpoint actually works. I tested it by inserting a memory, retrieving it, calling forget, and confirming it was gone. The vendor’s hosted version is also clear about data retention and deletion timelines.
- The data model is portable. I exported a memory store as JSON and re-imported it into a local instance, which is what I want from any infrastructure I am not writing myself.
- The documentation is not aspirational. It tells you what the API actually does, with the failure modes labeled.
What it doesn’t do well
I want to be specific about the parts I would not trust Mengram with:
- Long-term factual recall is not perfect. The system will sometimes conflate similar entities if you have not given it enough signal to disambiguate them. I have two clients whose names share a first word, and the memory has confused them more than once.
- The consolidation is conservative, which I called out as a feature, but it is also a limitation. If you have a memory store with thousands of entries, you will accumulate some cruft, and the API does not currently offer a “clean this up” pass.
- Pricing on the hosted version is per ingestion, which encourages a pattern of writing every message to memory. If your traffic is spiky, the cost adds up faster than the per-token model most people are used to.
- The open-source version requires a model API key for extraction, and the quality of your memories is bound to the quality of that model. A weak model produces weak memories.
Trade-offs
- You are trusting a third party (or your own instance of it) to represent your users accurately. That is a bigger commitment than logging conversation history.
- The memory graph is not a substitute for a database. If you need to query structured facts, you are still going to have a database. Mengram is for the agent’s recall, not for the application’s data layer.
- Adding a memory layer adds latency and a failure mode. Every call to the agent is now an extra round trip, and if the memory service is down, the agent has to degrade gracefully. Plan for it.
- The model choice in the open-source version is your problem. You have to pick an extraction model that is good enough, and you have to monitor the quality over time.
When to use this
Mengram is a fit if you are building an agent that needs to remember things about a user across sessions, and you do not want to write the extraction and consolidation logic yourself. It is not a fit for a one-off chatbot, a tool that does not have ongoing context, or a system that has to be auditable in the sense that “the model said so” is not a satisfying answer.
For me, the real test is whether I keep using it. I have already canceled subscriptions on a few memory-layer services that were promising in March and broken by May. Mengram has been in my stack for a few weeks, and the maintenance burden has been small. That is the bar I want a memory layer to clear, and it is the bar Mengram is currently clearing.