My Agent Has a Better Memory Than I Do
My Agent Has a Better Memory Than I Do
Every piece of content that flows through Hermes — every RSS item, every market signal, every blog post — gets stored. Indexed. Clustered. Scored.
Not by me. By an agent that never sleeps, never forgets, and gets smarter every week.
The Content Memory is TF-IDF based. No embeddings API. No vector database. No GPU. Just scikit-learn on a VPS, processing everything that passes through the system. It costs nothing. It's been running for less than a day and already knows more about what I've read than I do.
How an agent remembers
Most agents are amnesiacs. They run a task, produce output, and forget everything they learned. The Content Memory breaks that pattern.
Storage. Every piece of content gets stored in SQLite with metadata: source, URL, title, full text, publication date, tags. The RSS poller feeds it. Market scripts feed it. Blog posts feed it. Everything flows in.
Indexing. Once a week, the system rebuilds a TF-IDF index from all stored content. It extracts up to 5,000 features — words and bigrams that matter — and stores sparse vectors for every document. This is the memory. When you search for "correlation break regime change," it finds content about pattern detection even if those exact words don't appear.
Clustering. K-means clustering groups content into themes. It finds that 40 items are about commodity prices, 30 about AI agents, 15 about monetary policy. These clusters surface trends you didn't know you were tracking.
Source scoring. Every source gets a signal ratio: how many items from this source were later marked as actionable vs. how many were noise. If a feed has produced 200 items and zero of them were useful, the system flags it for deprioritization.
The learning loop
Every Sunday at 9 AM, the Learning Loop runs. It rebuilds the index, reclusters content, rescore sources, and delivers a report:
- How many items were indexed this week
- Top signal sources (ranked by usefulness)
- Low-signal sources (consider dropping)
- Content clusters (what themes are emerging)
- Prediction accuracy (for analytical agents that track outcomes)
The loop doesn't make decisions. It surfaces information. "This source has a 2% signal ratio over 200 items. You might want to drop it." The decision is yours. The data is the agent's.
Why TF-IDF instead of embeddings
Embeddings are better. But they cost tokens or local GPU RAM. TF-IDF is free.
For content retrieval and clustering, TF-IDF is good enough. It captures term importance, handles stop words, and produces sparse vectors that are fast to compare. The cosine similarity search finds related content reliably.
Could I upgrade to embeddings later? Sure. But the system works today for $0. Premature optimization is how personal projects become corporate infrastructure. Start with what works.
The architecture
- Storage: SQLite (same database as the rest of Hermes)
- Indexing: TF-IDF via scikit-learn, rebuilt weekly
- Clustering: K-means, 5-10 clusters depending on data volume
- Search: Cosine similarity over sparse vectors
- Cost: $0.00 (all local compute)
- Runtime: ~2 seconds for indexing, ~5 seconds for clustering
It's not a knowledge graph. It's not a vector database. It's a librarian that reads everything you consume, remembers it, and tells you what matters.
And it costs nothing.