🧠 Reflection Agents in LangChain & LangGraph — The Ultimate Guide

Imagine you hired a brilliant junior writer named Alex. Alex can draft great content quickly but makes mistakes: missed facts, clumsy phrasing, sometimes omits crucial details. A senior editor sits beside Alex and follows this ritual:

1. Alex writes a draft.

2. The editor critiques it: gaps, errors, tone.

3. Alex revises the draft using that critique.

4. The editor either accepts the revision or asks for another iteration.

Humans improve by reflecting:

• “Did I answer correctly?”

• “How can I improve this?”

• “Where did I go wrong?”

AI can do the same.

That’s the idea behind reflection agents — systems where an AI:

1. Generates a draft

2. Critiques its own answer

3. Improves based on feedback

4. Repeats until quality is acceptable

Reflection is the foundation behind advanced agent systems like:

• Self-Refine

• Reflexion (Xu et al.)

• ReAct + Reflection

• Evaluator-based refinement

• Graph structured multi-step reasoning (LangGraph)

Reflection improves AI performance dramatically — often by 20–70% on complex reasoning tasks.

What does “reflection” mean in AI?

A reflection agent is simply:

• An AI system that improves its answer through structured self-critique.
• It does not give the final output immediately. Instead, it goes through a loop:

Draft → Critic → Improve → Evaluate → Repeat → Final

Why reflection agents exist

LLMs are powerful pattern predictors but not perfect reasoners. Common problems:

• Hallucinations (made-up facts)

• Incorrect calculations

• Hallucinations

• Incorrect calculations

• Weak reasoning

• Poor structure

• Missing context

• Biases

• Missing context or steps

• Tendency to be overly verbose or under-informative

They are used in:

• Writing assistants
• Code review agents
• Research agents
• Multi-tool LLM agents
• Long-form reasoning (math, logic, planning)

Reflection mitigates these by explicitly checking and improving outputs.

How LLM reasoning evolved (quick timeline)

• Zero-shot: Give a task and rely on model knowledge. Fast but brittle.

• Few-shot: Provide examples to shape output; improves style and format.

• Chain-of-Thought (CoT): Ask the model to show intermediate steps to improve reasoning.

• Self-Critique / Self-Refine: Model critiques its own answers, then rewrites.

• Reflexion (research concept): Agents track prior mistakes and learn from them.

• Graph Reflection: Deterministic, node-based loops (LangGraph) for controlled, repeatable reflection.

Reflection patterns 

• CoT — “think step-by-step” to reveal reasoning. Useful for math and logic.

• Self-Critique — “what's wrong with this answer?” prompts the model to find faults.

• ReAct — blends reasoning with actions (tool calls); reflection can be a special action.

• Reflexion — models that keep a memory of past errors and improve over sessions.

• Graph Reflection — explicit nodes and edges implement critique/improve loops with deterministic control.

LangChain Reflection Agent (Linear, prompt-driven) 

LangChain is excellent for connecting prompts, tools, and memory into chains. For reflection we’ll implement a straightforward draft → critique → improve loop.

When to use LangChain reflection

• You want a simple, understandable pipeline.

• Your task fits into a few reflection passes.

• You prefer prompt-driven, model-controlled iteration.

High-level architecture

User Query
   ↓
Draft (LLM) → Critique (LLM) → Improve (LLM)
   ↓
Final answer

Key tradeoffs

• Pros: Easy to implement; minimal infra; fast to prototype.

• Cons: Hard to debug loop logic; state lives in prompts; less deterministic control.

LangChain Reflection — Example

Project layout (langchain_version):

langchain_version/
 ┣ reflection_chain.py
 ┣ run_langchain.py
 ┗ requirements.txt

requirements.txt:

langchain
langchain-openai
langchain-anthropic
python-dotenv

Create .env with your keys:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=claude-...

reflection_chain.py

# reflection_chain.py
# Implements a simple reflection loop using LangChain-like prompts.
# Supports OpenAI and Anthropic via small adapter functions.

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain.prompts import PromptTemplate
import os
from dotenv import load_dotenv

load_dotenv()

# -------------------------
# Model loader (provider agnostic)
# -------------------------
def load_model(provider="openai"):
    """
    Return a model object depending on provider.
    provider: "openai" or "anthropic"
    """
    if provider == "anthropic":
        # Anthropic chat model
        return ChatAnthropic(model="claude-3-sonnet-20240229", temperature=0)
    # Default: OpenAI chat model
    return ChatOpenAI(model="gpt-4o-mini", temperature=0)

# -------------------------
# Prompts
# -------------------------
# These templates are small building blocks to control the LLM behavior.

draft_template = PromptTemplate(
    input_variables=["query"],
    template=(
        "You are an expert assistant. Produce a concise and correct draft answer to:\n\n"
        "{query}\n\n"
        "Draft:\n"
    ),
)

critic_template = PromptTemplate(
    input_variables=["draft"],
    template=(
        "You are a critical reviewer. Read the draft below and point out errors, missing facts, and unclear reasoning.\n\n"
        "DRAFT:\n{draft}\n\n"
        "Provide: 1) Concise critique bullets, 2) Exact locations (if any) of errors, 3) Suggestions to improve."
    ),
)

improve_template = PromptTemplate(
    input_variables=["draft", "critique"],
    template=(
        "You are the original author. Improve the draft using the critique below. "
        "Fix errors and expand explanations where needed. Return the improved answer only.\n\n"
        "DRAFT:\n{draft}\n\nCRITIQUE:\n{critique}\n\n"
        "Improved Answer:\n"
    ),
)

# -------------------------
# Reflection loop
# -------------------------
def run_reflection(query: str, provider: str = "openai"):
    """
    Runs a single reflection loop:
    1. Draft
    2. Critique
    3. Improve
    Returns a dict with draft, critique, improved answer.
    """
    model = load_model(provider)

    # Step 1: Draft
    # We format the draft prompt and call the model.
    draft_input = draft_template.format(query=query)
    draft_resp = model.invoke(draft_input)  # .invoke returns an object with .content
    draft = draft_resp.content.strip()

    # Step 2: Critique the draft
    critique_input = critic_template.format(draft=draft)
    critique_resp = model.invoke(critique_input)
    critique = critique_resp.content.strip()

    # Step 3: Improve the draft using the critique
    improve_input = improve_template.format(draft=draft, critique=critique)
    improved_resp = model.invoke(improve_input)
    improved = improved_resp.content.strip()

    return {"draft": draft, "critique": critique, "final": improved}

run_langchain.py:

# run_langchain.py
from reflection_chain import run_reflection

if __name__ == "__main__":
    q = input("Ask the reflection LangChain agent: ")
    res = run_reflection(q, provider="openai")  # or "anthropic"
    print("\n--- DRAFT ---\n", res["draft"])
    print("\n--- CRITIQUE ---\n", res["critique"])
    print("\n--- FINAL ---\n", res["final"])

How to run:

python run_langchain.py

Notes & tips

• You can iterate the loop multiple times by feeding the improved answer back into the critic for another pass.

• Keep prompts short and explicit. Too vague critic prompts produce weak critiques.

• For critical systems, add deterministic checks (e.g., unit tests on model output).

LangGraph Reflection Agent

LangGraph is designed for graph-based workflows. Reflection maps naturally to a graph: nodes for draft/critic/improve/evaluate and edges for flow control (including loops).

Why LangGraph for reflection?

• Deterministic flow control — edges with conditions decide next node.

• Stateful — shared state object persists across nodes.

• Debuggable — visualize graph and trace execution.

• Robust — supports retries, failure handling, and parallel nodes.

When to use LangGraph

• Multi-pass reflection with exit conditions.

• Multi-tool agents (call calculators, search, databases).

• Production systems needing observability and recovery.

High-level LangGraph reflection flow

Entry → Draft Node → Critic Node → Improve Node → Evaluate Node
                                   ↑                    |
                                   └─────(if not done)──┘
If done → Exit → Final Output

Each node reads & writes to a shared state dict — enabling a traceable flow.

---

Full LangGraph project (with Streamlit UI)

Project layout (langgraph_version):

langgraph_version/
 ┣ state.py
 ┣ nodes/
 ┃ ┣ draft.py
 ┃ ┣ critic.py
 ┃ ┣ improve.py
 ┃ ┗ evaluate.py
 ┣ graph.py
 ┣ run.py
 ┗ streamlit_app.py

requirements.txt:

langgraph
langchain-openai
langchain-anthropic
langchain-community
python-dotenv
streamlit

Create .env with keys for OpenAI and Anthropic.

state.py — typed state:

# state.py
from typing import TypedDict, Optional

class ReflectionState(TypedDict, total=False):
    query: str
    draft: Optional[str]
    critique: Optional[str]
    improved: Optional[str]
    iterations: int
    done: bool

nodes/draft.py:

# nodes/draft.py
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

def draft_node(state: dict, provider: str = "openai"):
    """
    Produce an initial draft and store in state['draft'].
    """
    if provider == "anthropic":
        llm = ChatAnthropic(model="claude-3-sonnet-20240229", temperature=0)
    else:
        llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

    prompt = f"Write a clear, correct draft for the question:\n\n{state['query']}"
    resp = llm.invoke(prompt)
    state["draft"] = resp.content.strip()
    return state

nodes/critic.py:

# nodes/critic.py
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

def critic_node(state: dict, provider: str = "openai"):
    """
    Critique the draft. Store critique in state['critique'].
    """
    if provider == "anthropic":
        llm = ChatAnthropic(model="claude-3-sonnet-20240229", temperature=0)
    else:
        llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

    prompt = (
        "You are a critical reviewer. Given the draft below, list issues, missing details, and errors.\n\n"
        f"DRAFT:\n{state.get('draft','')}\n\nReturn bullet points."
    )
    resp = llm.invoke(prompt)
    state["critique"] = resp.content.strip()
    return state

nodes/improve.py:

# nodes/improve.py
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

def improve_node(state: dict, provider: str = "openai"):
    """
    Use the critique to produce an improved draft.
    """
    if provider == "anthropic":
        llm = ChatAnthropic(model="claude-3-sonnet-20240229", temperature=0)
    else:
        llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

    prompt = (
        "Improve the draft using the critique below. Output the improved final answer only.\n\n"
        f"DRAFT:\n{state.get('draft','')}\n\nCRITIQUE:\n{state.get('critique','')}"
    )
    resp = llm.invoke(prompt)
    state["improved"] = resp.content.strip()
    return state

nodes/evaluate.py:

# nodes/evaluate.py
def evaluate_node(state: dict):
    """
    Decide whether to loop again — simple termination heuristic.
    Increase iterations, stop if iterations >= 3 or critique empty.
    """
    state.setdefault("iterations", 0)
    state["iterations"] += 1

    # If no critique or iterations exceeded, finish
    if not state.get("critique") or state["iterations"] >= 3:
        state["done"] = True
    else:
        state["done"] = False
    return state

graph.py — build and compile graph:

# graph.py
from langgraph.graph import StateGraph
from state import ReflectionState
from nodes.draft import draft_node
from nodes.critic import critic_node
from nodes.improve import improve_node
from nodes.evaluate import evaluate_node

def build_graph(provider: str = "openai"):
    """
    Build the state graph with nodes and conditional loop edges.
    """
    graph = StateGraph(ReflectionState)

    graph.add_node("draft", lambda s: draft_node(s, provider))
    graph.add_node("critic", lambda s: critic_node(s, provider))
    graph.add_node("improve", lambda s: improve_node(s, provider))
    graph.add_node("evaluate", evaluate_node)

    graph.set_entry_point("draft")
    graph.add_edge("draft", "critic")
    graph.add_edge("critic", "improve")
    graph.add_edge("improve", "evaluate")

    # If evaluation decides not done, route back to critic
    graph.add_edge("evaluate", "critic", condition=lambda s: not s.get("done", False))
    # If done, go to improve (finalize) then exit
    graph.add_edge("evaluate", "improve", condition=lambda s: s.get("done", False))

    return graph.compile()

run.py — CLI runner:

# run.py
from graph import build_graph

if __name__ == "__main__":
    provider = "openai"  # or "anthropic"
    graph = build_graph(provider)
    query = input("Enter your query: ")
    state = {"query": query, "iterations": 0, "done": False}
    result = graph.invoke(state)
    print("\nFinal improved answer:\n")
    print(result.get("improved") or result.get("draft"))

Streamlit UI (LangGraph only)

• We include a simple Streamlit interface so readers can interactively run reflection loops.

streamlit_app.py:

# streamlit_app.py
import streamlit as st
from graph import build_graph

st.set_page_config(page_title="Reflection Agent (LangGraph)", page_icon="🧠")
st.title("Reflection Agent — LangGraph")
st.write("This interactive demo runs a reflection loop (draft → critic → improve).")

provider = st.selectbox("Provider", ["openai", "anthropic"])
query = st.text_area("Enter your question", height=150)

if st.button("Run Reflection Agent"):
    graph = build_graph(provider)
    state = {"query": query, "iterations": 0, "done": False}
    result = graph.invoke(state)

    st.subheader("Final improved answer")
    st.write(result.get("improved") or result.get("draft"))

    with st.expander("Full state (debug)"):
        st.json(result)

Run UI:

streamlit run streamlit_app.py

Running locally & deployment notes

Install (conda/venv)::

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Environment (.env)::

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=claude-...

Costs & rate limits

• Reflection loops multiply LLM calls: 1 draft + 1 critique + 1 improve = 3 model calls per user query.

• Use low-temperature deterministic models for critique & evaluation.

• Consider cheaper embedding or smaller models for critic steps if quality acceptable.

Deploying UI

• Streamlit Cloud or Render are easy for deploying the LangGraph streamlit app.

• For production, wrap the graph in a serverless function or container with monitoring and API auth.

Design decisions & practical tips

• Termination Heuristic: Evaluate node checks iteration count and critique emptiness. Production systems use semantic quality checks (BLEU/ROUGE-like, factuality checks, or human-review signals).

• Tool Integration: Add nodes that call calculators, retrieval systems, or external validators — they write into state and shape flow.

• Safety: Rate-limit loops and add validation nodes to prevent runaway API costs.

• Observability: Log node inputs/outputs. LangGraph makes it easy to trace execution.

🧩 LangChain vs LangGraph (Summary Table)

Advanced improvements you can add

• Automated factuality checks: run a fact-checker node (external search) and reject outputs failing checks.

• Adaptive iteration: dynamic stop conditions based on scoring model outputs.

• Human-in-the-loop: send flagged outputs to reviewers and use their feedback as new training examples.

• Persistent learning: store critiques and successes to fine-tune future prompts or retrain small models.

🧩when to use what

• Start with LangChain if:

  • You're prototyping fast.

  • You need to test prompt strategies (CoT, self-critique).

  • You prefer simple code and quick iteration.

• Move to LangGraph when:

• You need deterministic loops, observability, retries, and multi-tool orchestration.

• You're building production agents that must be auditable.

• You want to visualize and debug the reasoning steps.

🔮 Future of Reflection Agents

• Reflection agents are the future of AI reliability.
• As models become more capable, the challenge becomes controlling, evaluating, and refining their outputs.
• LangGraph’s deterministic loops are the next evolution.

🚀 ALL AI / LangChain Post

• ALL LangChain Post