I Built a Local AI Agent To Help Decide What to Cook for Dinner

I spend most of my professional time working on Data & AI systems with layered complexities: distributed data and applications, production constraints, observability, safety, and scale.

This personal project intentionally explores the opposite direction: simplicity. The inspiration? A very ordinary problem that happens to all of us: deciding what to cook for dinner with whatever is in the fridge. And if you know me, you know I am a BIG foodie. check out my food blogs linked in ABOUT

So over this weekend, I built an AI agent that reads what’s in my fridge, remembers my food preferences, suggests recipes for dinner, and learns from our conversations histories. Before starting, I imposed a constraint on myself to keep the system simple and local. The result is a small agent that runs entirely on my laptop, stores state in markdown files, and uses Ollama for local model inference. It may not be reliable or scalable, but it certainly is fun!

Me: I want to make something hearty today, maybe a stew?

This post walks through the design, the decisions, and what I didn’t build. Full code here: https://github.com/Henry-Xiao-HX/personal-chef-agent

---

DISCLAIMER: What makes this an AI agent

The term AI agent is sometimes used too loosely, often describing systems that are little more than a prompt wrapped in an API call. This project is intentionally minimal, but it still follows the core structural pattern of an LLM agent.

An AI agent is a software program that perceives its environment, reasons about it, and takes actions that affect that environment (observe -> reason -> act). The key difference is operating not in a single prompt-response step, but embedded inside a control loop that interacts with an external environment.

In my case, the environment is simply a local filesystem. The agent can read the current state (fridge contents and preferences), reason about it, and update that state when new information emerges, creating a closed feedback loop.

A couple design choices that reinforce the agentic behavior:

Persistent memory The agent maintains external state across sessions. Instead of treating each interaction independently, it accumulates knowledge over time, allowing future decisions to incorporate past preferences and inventory changes.

Reflection-based learning At the end of each session, the agent performs a reflection pass over the conversation to identify new preferences and update persistent state. This is analogous to AutoGPT style reflection, acts as lightweight memory consolidation, and resembles experience replay in reinforcement learning: distilling short-term interactions into long-term knowledge without complicating the main loop.

From an architectural perspective, this produces a minimal agent structure:

• an LLM embedded in a control loop
• tool-mediated interaction with an environment
• persistent external state
• a reflection mechanism for updating long-term memory

The implementation here is intentionally lightweight, but the pattern (observe -> reason -> act) is the same one that appears in larger agent systems.

Some architectural design

At its core, everything is just files and a controlled agent loop:

Two markdown files hold all the state:

• FRIDGE.md captures what I actually have in the fridge and pantry.
• My-Preference.md captures my food preferences

For every response, the agent

• reads both files
• generates a response using the context
• updates those files if/when new information appears in conversation

That’s it. there are no distributed databases, cloud APIs, multi-agent architecture. Just an agent loop manipulating markdown files. As an enterprise AI engineer, I spent most of my time with complex architectures. Now I just wanted something that is simple and playful.

graph LR
    %% Color Definitions
    classDef input fill:#e1f5fe,stroke:#01579b,stroke-width:2px,color:#01579b;
    classDef files fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#e65100;
    classDef logic fill:#f3e5f5,stroke:#4a148c,stroke-width:2px,color:#4a148c;
    classDef update fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px,color:#1b5e20,stroke-dasharray: 5 5;

    %% Nodes
    Q[User asks a question]:::input
    A[Agent]:::input
    
    subgraph Storage [Knowledge Base]
        F[FRIDGE.md]:::files
        P[My-Preference.md]:::files
    end

    L[Local LLM reasons over context]:::logic
    R[Return answer]:::logic
    U["Update files (optional)"]:::update

    %% Connections
    Q --> A
    A --> F
    A --> P
    F --> L
    P --> L
    L --> R
    L --> U
    
    %% Feedback Loops
    U -.-> F
    U -.-> P

    %% Styling lines
    linkStyle default stroke:#555,stroke-width:2px;

LangChain

I did use LangChain in this project to provide some scaffolding:

• A model abstraction layer, so I can swap Ollama models easily
• Filesystem tools for reading and writing markdown files

I didn’t use its agent frameworks, planner/executor abstractions, memory modules, and other more advanced features. Those can be saved for other more complex projects.

---

The agent loop

The core logic lives in a small custom class (SimpleCookingAgent):

• Loads fridge contents and preferences into every prompt
• Tracks conversation history for the session
• Uses tools to read/write markdown
• Reflects once at the end of a session to update preferences

Ollama local inference

Inference runs entirely through Ollama, using a local model (llama3.2 by default), with minimal configuration.

ollama:
  model: "llama3.2"
  temperature: 0.7

agent:
  max_iterations: 15
  verbose: true

• Temperature 0.7 gives creative but coherent recipes
• Iteration limits keep the agent bounded
• Verbose mode exposes reasoning and tool usage

Responses typically take 2–5 seconds, which is more than acceptable for this use case helping me decide dinner.

Markdown files

State lives in two markdown files because they are simple, human-readable, and easy to edit manually in a text editor. I am relying on LLM’s capability to understand natural language as opposed to a relational schema or even structured JSON. This may seem unusual compared to structured storage, but it works well for a small personal project.

Session-based learning

When you exit the agent, it performs a simple reflection step:

1. Reads the full conversation
2. Asks the LLM whether new preferences were mentioned
3. Compares against existing preferences
4. Updates My-Preference.md only if something changed
5. Timestamps the update

This allows casual statements like:

“I’m really into spicy food lately”

to become persistent knowledge without me having to tell the agent every time what I like to eat.

This pattern, session scoped reflection rather than continuous memory updates, is something I’ve found useful in larger systems as well.

---

What would change in production

Of course, all of these falls apart in a real production system: then we need to think about support for multiple users, prompt injections, scalability, reliability, and so much more good stuff.

State & concurrency

Markdown files don’t scale and doesn’t handle concurrent updates

What if you live with your partner and they updated the preferences file while you ask for a recipe - suddenly, the agent suggests mint chocolate chip ice cream, which you absolutely hate.

Production systems need structured storage, concurrency control, and versioning.

Lock the state file per session, or serialize updates to ensure only one process writes at a time. Even simple file-based locks prevent most conflicts in a small multi-user setup.

Security & tool access

Local trust doesn’t translate to shared systems. Right now, I am the only user. What if a guest “accidentally” did some prompt injection, and tricked the agent to suggest shellfish for dinner, even though I’m allergic?

In production:

• Tool access must be scoped
• Write operations must be validated
• Prompt injection becomes a real threat

Limit tool access to trusted scripts or authenticated users, and sanitize input/output. Even a rudimentary whitelist of allowed operations greatly reduces risk.

Context management

As conversation history grows, you need summarization strategies, context window management, explicit memory boundaries.

Observability

Verbose logs are great locally.

What if the agent suddenly recommends 10 eggs for a single omelet. Did it misread inventory, preferences, or mistaken me as a dinosaur with a huge appetite?

Production requires structured logs, tracing across agent steps, clear visibility into tool usage for audibility.

Add structured logs for each reasoning step and tool call. Even basic timestamped JSON logs help trace errors.

Model operations

Local models are fine for personal use. At scale, you need versioning, rollout strategies, and fallbacks.

---

Closing thoughts

This project intentionally avoids most of the complexity that usually surrounds AI systems. It’s a small, local agent with simple tools, markdown-based state, and a straightforward agent loop.

And as a side effect, I now waste less food and make better dinners - which is a satisfying outcome for a weekend AI experiment.