This is the difference between an app that records decisions and one that learns from them.

This post walks through how to build the latter end-to-end. We'll intercept the moment a team makes a selection, use an AI agent to capture the rationale in natural conversation, store those decision records in Amazon Bedrock Knowledge Bases, and surface that institutional memory to future teams — all inside the same app they're already using. The stack is React/Node.js on the frontend, Python 3.12+ on the backend, and AWS-native services throughout.

The Problem with Unstructured Institutional Knowledge

Most enterprise apps are good at capturing what happened. They're terrible at capturing why.

In the research site selection context, you might have a database full of selections — site IDs, dates, teams, outcomes — but none of the reasoning that shaped those choices. That reasoning is exactly what makes institutional knowledge valuable. It's the difference between "Team B selected Site 7" and "Team B selected Site 7 because the proximity to the river introduced too much ambient noise at Sites 3 and 12, and the permit turnaround at Site 7 is historically two weeks faster."

The challenge is capturing that reasoning without burdening the team. A form that pops up asking "Why did you choose this site?" will get abandoned inside a week. The better approach is to use the AI agent that's already in the workflow to conduct a brief, conversational debrief — three or four targeted questions, 90 seconds of the team's time — and then automatically package and index that reasoning so it's available to everyone who comes after them.

The Institutional Memory Loop

The core concept is a self-reinforcing cycle we'll call the Institutional Memory Loop. The same agent that helps a team make a decision is also the one that captures the reasoning behind it — and later surfaces that reasoning to the next team facing the same choice. The knowledge base is what closes the loop.

The flow has two phases. In the capture phase (top row), a team selects a site, the agent conducts a 90-second conversational debrief, and a structured decision record lands in S3. An event notification triggers a Bedrock KB ingestion job, embedding the document into an OpenSearch Serverless vector store. In the retrieve phase (bottom row), the next team's agent query hits the knowledge base, pulls relevant past decisions, synthesizes them with Claude, and surfaces the context in the same UI — attributed, with provenance. The app the next team uses looks and feels identical to the one the previous team used. The difference is that it now knows things.

Step 1: Setting Up the Bedrock Knowledge Base

Before wiring up the capture pipeline, you need a knowledge base to receive documents. Amazon Bedrock Knowledge Bases manages the entire RAG infrastructure — chunking, embedding, and storing documents in a vector store — so you don't need to operate OpenSearch directly.

Create the S3 Bucket and IAM Role

# infrastructure/setup_knowledge_base.py
import boto3
import json

s3 = boto3.client("s3", region_name="us-east-1")
iam = boto3.client("iam", region_name="us-east-1")
bedrock_agent = boto3.client("bedrock-agent", region_name="us-east-1")

BUCKET_NAME = "research-decision-records-prod"
KNOWLEDGE_BASE_NAME = "site-selection-decisions-kb"
EMBEDDING_MODEL_ARN = (
    "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
)

# Create the S3 bucket for decision records
s3.create_bucket(Bucket=BUCKET_NAME)

s3.put_bucket_versioning(
    Bucket=BUCKET_NAME,
    VersioningConfiguration={"Status": "Enabled"},
)

# IAM trust policy for Bedrock to access S3 and OpenSearch
trust_policy = {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {"Service": "bedrock.amazonaws.com"},
            "Action": "sts:AssumeRole",
        }
    ],
}

role = iam.create_role(
    RoleName="BedrockKBRole-SiteDecisions",
    AssumeRolePolicyDocument=json.dumps(trust_policy),
)

iam.attach_role_policy(
    RoleName="BedrockKBRole-SiteDecisions",
    PolicyArn="arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
)

# Create the Bedrock Knowledge Base with OpenSearch Serverless
kb_response = bedrock_agent.create_knowledge_base(
    name=KNOWLEDGE_BASE_NAME,
    description="Institutional knowledge from research site selection decisions",
    roleArn=role["Role"]["Arn"],
    knowledgeBaseConfiguration={
        "type": "VECTOR",
        "vectorKnowledgeBaseConfiguration": {
            "embeddingModelArn": EMBEDDING_MODEL_ARN,
        },
    },
    storageConfiguration={
        "type": "OPENSEARCH_SERVERLESS",
        "opensearchServerlessConfiguration": {
            "collectionArn": "arn:aws:aoss:us-east-1:ACCOUNT_ID:collection/COLLECTION_ID",
            "vectorIndexName": "site-decisions-index",
            "fieldMapping": {
                "vectorField": "embedding",
                "textField": "text",
                "metadataField": "metadata",
            },
        },
    },
)

knowledge_base_id = kb_response["knowledgeBase"]["knowledgeBaseId"]

# Add the S3 data source
bedrock_agent.create_data_source(
    knowledgeBaseId=knowledge_base_id,
    name="decision-records-s3",
    dataSourceConfiguration={
        "type": "S3",
        "s3Configuration": {
            "bucketArn": f"arn:aws:s3:::{BUCKET_NAME}",
            "inclusionPrefixes": ["decision-records/"],
        },
    },
    vectorIngestionConfiguration={
        "chunkingConfiguration": {
            "chunkingStrategy": "SEMANTIC",
            "semanticChunkingConfiguration": {
                "maxTokens": 300,
                "bufferSize": 1,
                "breakpointPercentileThreshold": 95,
            },
        }
    },
)

print(f"Knowledge Base ID: {knowledge_base_id}")

A few things worth calling out here. The SEMANTIC chunking strategy is intentional — decision records are narrative text, and semantic chunking preserves the logical boundaries of the reasoning rather than slicing arbitrarily by token count. You get better retrieval quality on conversational documents this way.

Why OpenSearch Serverless over managed OpenSearch Service? At this stage of the project, you don't need the operational depth that full OpenSearch gives you. Managed OpenSearch requires you to provision and right-size clusters, configure shard counts and replica topology, manage index lifecycle policies, and handle version upgrades. For a knowledge base that grows incrementally — a few decision records per week — that's overhead with no payoff. OpenSearch Serverless scales on demand and has zero cluster management surface. Bedrock Knowledge Bases provisions and manages the Serverless collection for you, so your interaction with the underlying vector store is exactly zero lines of infrastructure code. If you later need custom analyzers, fine-grained index control, or extremely high-throughput ingest, graduating to managed OpenSearch is a straightforward migration — but start serverless and earn that complexity.

Step 2: The AI-Assisted Capture Flow

This is the most important piece of the puzzle, because all the infrastructure in the world is worthless if teams don't actually generate decision records.

The approach here is to hook into the existing site selection flow. When a team confirms a site selection in the React app, the frontend emits a SITE_SELECTED event to the backend, which spins up a short debrief conversation via the AI agent. The agent asks three to four targeted questions, collects the answers, and then formats a structured decision record document automatically. The team never fills out a form.

React: Triggering the Capture Flow

// src/hooks/useDecisionCapture.ts
import { useState, useCallback } from "react";

interface CaptureSession {
  sessionId: string;
  messages: Array<{ role: "assistant" | "user"; content: string }>;
  isComplete: boolean;
}

export function useDecisionCapture() {
  const [session, setSession] = useState<CaptureSession | null>(null);
  const [isLoading, setIsLoading] = useState(false);

  const startCapture = useCallback(
    async (siteId: string, selectionId: string) => {
      setIsLoading(true);
      try {
        const response = await fetch("/api/decisions/start-capture", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({ siteId, selectionId }),
        });
        const data = await response.json();
        setSession({
          sessionId: data.sessionId,
          messages: [{ role: "assistant", content: data.firstQuestion }],
          isComplete: false,
        });
      } finally {
        setIsLoading(false);
      }
    },
    []
  );

  const sendAnswer = useCallback(
    async (answer: string) => {
      if (!session) return;
      setIsLoading(true);
      try {
        const response = await fetch("/api/decisions/respond", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({ sessionId: session.sessionId, answer }),
        });
        const data = await response.json();
        setSession((prev) =>
          prev
            ? {
                ...prev,
                messages: [
                  ...prev.messages,
                  { role: "user", content: answer },
                  { role: "assistant", content: data.nextMessage },
                ],
                isComplete: data.isComplete,
              }
            : null
        );
      } finally {
        setIsLoading(false);
      }
    },
    [session]
  );

  return { session, isLoading, startCapture, sendAnswer };
}

// src/components/DecisionCapture.tsx
import { useDecisionCapture } from "../hooks/useDecisionCapture";

interface Props {
  siteId: string;
  selectionId: string;
  onComplete: () => void;
}

export function DecisionCapture({ siteId, selectionId, onComplete }: Props) {
  const { session, isLoading, startCapture, sendAnswer } = useDecisionCapture();
  const [input, setInput] = useState("");

  useEffect(() => {
    startCapture(siteId, selectionId);
  }, [siteId, selectionId]);

  useEffect(() => {
    if (session?.isComplete) {
      setTimeout(onComplete, 1500);
    }
  }, [session?.isComplete]);

  return (
    <div className="decision-capture-panel">
      <h3>Help us capture why you chose this site</h3>
      <p className="subtext">
        Just a few quick questions — your answers build the team's knowledge base
      </p>
      <div className="messages">
        {session?.messages.map((msg, i) => (
          <div key={i} className={`message ${msg.role}`}>
            {msg.content}
          </div>
        ))}
      </div>
      {!session?.isComplete && (
        <form
          onSubmit={(e) => {
            e.preventDefault();
            sendAnswer(input);
            setInput("");
          }}
        >
          <input
            value={input}
            onChange={(e) => setInput(e.target.value)}
            placeholder="Type your answer..."
            disabled={isLoading}
          />
          <button type="submit" disabled={isLoading || !input.trim()}>
            Send
          </button>
        </form>
      )}
      {session?.isComplete && (
        <p className="complete">✓ Decision captured — thanks!</p>
      )}
    </div>
  );
}

The key UX principle here is framing. "Help us capture why you chose this site" is far less threatening than "Fill out this form." Keeping the capture panel lightweight and conversational is what gets adoption.

Node.js: API Routes for the Capture Session

The Node.js layer is thin — its job is to create a session ID, proxy requests to the Python Lambda, and clean up the session map when the capture completes. Two routes: POST /api/decisions/start-capture initializes the session and returns the first question, and POST /api/decisions/respond passes each answer through and returns the next prompt or the completion signal.

// src/api/decisions.ts (Express routes — full implementation in repo)
import express from "express";
import { v4 as uuidv4 } from "uuid";
import { Lambda } from "@aws-sdk/client-lambda";

const router = express.Router();
const lambda = new Lambda({ region: "us-east-1" });

// In production, replace this Map with DynamoDB or ElastiCache
const captureSessions = new Map<string, { siteId: string; selectionId: string }>();

router.post("/start-capture", async (req, res) => {
  const { siteId, selectionId } = req.body;
  const sessionId = uuidv4();
  captureSessions.set(sessionId, { siteId, selectionId });

  const result = await lambda.invoke({
    FunctionName: "decision-capture-handler",
    Payload: JSON.stringify({ action: "start", sessionId, siteId, selectionId }),
  });

  const response = JSON.parse(Buffer.from(result.Payload!).toString());
  res.json({ sessionId, firstQuestion: response.message });
});

router.post("/respond", async (req, res) => {
  const { sessionId, answer } = req.body;
  const sessionData = captureSessions.get(sessionId);
  if (!sessionData) return res.status(404).json({ error: "Session not found" });

  const result = await lambda.invoke({
    FunctionName: "decision-capture-handler",
    Payload: JSON.stringify({ action: "respond", sessionId, answer, ...sessionData }),
  });

  const response = JSON.parse(Buffer.from(result.Payload!).toString());
  if (response.isComplete) captureSessions.delete(sessionId);
  res.json(response);
});

export default router;

Step 3: The Python Lambda — Conducting the Capture Interview

This Lambda is the heart of the system. It uses the Bedrock Converse API to run a multi-turn conversation with a specific goal: extract the team's decision rationale and package it into a structured document.

# lambdas/decision_capture_handler/handler.py
import json
import os
import boto3
from datetime import datetime, timezone
from typing import Any

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
dynamodb = boto3.resource("dynamodb", region_name="us-east-1")
s3 = boto3.client("s3", region_name="us-east-1")

SESSIONS_TABLE = dynamodb.Table(os.environ["SESSIONS_TABLE"])
DECISION_RECORDS_BUCKET = os.environ["DECISION_RECORDS_BUCKET"]
MODEL_ID = "global.anthropic.claude-sonnet-4-5-20250929-v1:0"

CAPTURE_SYSTEM_PROMPT = """You are a knowledge capture specialist embedded in a research site selection application. 
Your job is to conduct a brief, friendly debrief with researchers after they select a study site.

Your goal is to extract, in natural conversation, the following information:
1. The primary factors that made this site the right choice
2. The main alternatives that were seriously considered and why they were passed over
3. Any constraints or risks that influenced the decision
4. What success looks like for this site selection

Rules:
- Ask one question at a time, never multiple at once
- Keep your questions concise and specific — researchers are busy
- After the researcher answers your 4th question, output a special JSON block formatted EXACTLY like this:

<DECISION_RECORD>
{
  "primary_factors": "summary of key selection factors",
  "alternatives_considered": "summary of alternatives and why they were rejected",
  "constraints_and_risks": "summary of constraints or risks",
  "success_criteria": "what success looks like",
  "raw_conversation_summary": "brief narrative summary of the full conversation"
}
</DECISION_RECORD>

Then end with a brief, warm closing message thanking the researcher.

Start your first message with a single, direct opening question about the primary factors behind their choice. 
Do NOT introduce yourself or explain what you're doing — just ask the question naturally."""


def lambda_handler(event: dict[str, Any], context: Any) -> dict[str, Any]:
    action = event["action"]
    session_id = event["sessionId"]
    site_id = event["siteId"]
    selection_id = event["selectionId"]

    if action == "start":
        return handle_start(session_id, site_id, selection_id)
    elif action == "respond":
        return handle_respond(session_id, event["answer"], site_id, selection_id)

    raise ValueError(f"Unknown action: {action}")


def handle_start(session_id: str, site_id: str, selection_id: str) -> dict:
    """Initialize a capture session and get the first question."""
    response = bedrock.converse(
        modelId=MODEL_ID,
        system=[{"text": CAPTURE_SYSTEM_PROMPT}],
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "text": (
                            f"A researcher just selected Site {site_id} for their study. "
                            "Please begin the debrief."
                        )
                    }
                ],
            }
        ],
        inferenceConfig={"maxTokens": 300, "temperature": 0.3},
    )

    first_question = response["output"]["message"]["content"][0]["text"]

    # Persist the conversation history to DynamoDB
    SESSIONS_TABLE.put_item(
        Item={
            "sessionId": session_id,
            "siteId": site_id,
            "selectionId": selection_id,
            "messages": [
                {
                    "role": "user",
                    "content": (
                        f"A researcher just selected Site {site_id} for their study. "
                        "Please begin the debrief."
                    ),
                },
                {"role": "assistant", "content": first_question},
            ],
            "questionCount": 1,
            "ttl": int(datetime.now(timezone.utc).timestamp()) + 3600,  # 1hr TTL
        }
    )

    return {"message": first_question, "isComplete": False}


def handle_respond(
    session_id: str, answer: str, site_id: str, selection_id: str
) -> dict:
    """Process a researcher's answer and return the next question or finalize."""
    session = SESSIONS_TABLE.get_item(Key={"sessionId": session_id})["Item"]
    messages = session["messages"]
    question_count = int(session["questionCount"])

    # Append the researcher's answer
    messages.append({"role": "user", "content": answer})

    # Build the Converse messages payload
    converse_messages = [
        {"role": msg["role"], "content": [{"text": msg["content"]}]}
        for msg in messages
    ]

    response = bedrock.converse(
        modelId=MODEL_ID,
        system=[{"text": CAPTURE_SYSTEM_PROMPT}],
        messages=converse_messages,
        inferenceConfig={"maxTokens": 600, "temperature": 0.3},
    )

    assistant_message = response["output"]["message"]["content"][0]["text"]
    messages.append({"role": "assistant", "content": assistant_message})

    # Check if the model has produced a completed decision record
    is_complete = "<DECISION_RECORD>" in assistant_message

    if is_complete:
        decision_data = extract_decision_record(assistant_message)
        closing_message = assistant_message.split("</DECISION_RECORD>")[-1].strip()

        # Write the decision record to S3 asynchronously
        write_decision_record(
            site_id=site_id,
            selection_id=selection_id,
            decision_data=decision_data,
            full_messages=messages,
        )

        # Update session state
        SESSIONS_TABLE.update_item(
            Key={"sessionId": session_id},
            UpdateExpression="SET messages = :m, questionCount = :q",
            ExpressionAttributeValues={":m": messages, ":q": question_count + 1},
        )

        return {"message": closing_message or "Thanks! Decision captured.", "isComplete": True}

    # Update session and continue the conversation
    SESSIONS_TABLE.update_item(
        Key={"sessionId": session_id},
        UpdateExpression="SET messages = :m, questionCount = :q",
        ExpressionAttributeValues={":m": messages, ":q": question_count + 1},
    )

    return {"message": assistant_message, "isComplete": False}


def extract_decision_record(text: str) -> dict:
    """Parse the JSON block the model produces when capture is complete."""
    start = text.find("<DECISION_RECORD>") + len("<DECISION_RECORD>")
    end = text.find("</DECISION_RECORD>")
    json_block = text[start:end].strip()
    return json.loads(json_block)


def write_decision_record(
    site_id: str,
    selection_id: str,
    decision_data: dict,
    full_messages: list,
) -> None:
    """Format and upload the decision record to S3 for KB ingestion."""
    timestamp = datetime.now(timezone.utc).isoformat()

    # Format as a rich markdown document — narrative text chunks better for RAG
    document = f"""# Site Selection Decision Record

**Site ID:** {site_id}
**Selection ID:** {selection_id}
**Captured At:** {timestamp}

## Why This Site Was Selected

{decision_data.get("primary_factors", "Not captured")}

## Alternatives Considered

{decision_data.get("alternatives_considered", "Not captured")}

## Constraints and Risks Acknowledged

{decision_data.get("constraints_and_risks", "Not captured")}

## Success Criteria

{decision_data.get("success_criteria", "Not captured")}

## Summary

{decision_data.get("raw_conversation_summary", "Not captured")}
"""

    s3_key = f"decision-records/site-{site_id}/{selection_id}.md"

    s3.put_object(
        Bucket=DECISION_RECORDS_BUCKET,
        Key=s3_key,
        Body=document.encode("utf-8"),
        ContentType="text/markdown",
        Metadata={
            "site-id": site_id,
            "selection-id": selection_id,
            "captured-at": timestamp,
        },
    )

A few design decisions here worth explaining. First, conversations are persisted to DynamoDB with a 1-hour TTL — this keeps the Lambda stateless while maintaining context across the multi-turn conversation without expensive in-memory solutions.

A note on DynamoDB vs. Amazon Bedrock AgentCore Memory: AgentCore Memory (formerly Bedrock Agent memory store) is purpose-built for persistent, cross-session agent memory — it's the right tool when an agent needs to remember facts about a user or ongoing project across many separate conversations over weeks or months. For this use case, each capture session is short-lived (under 10 minutes) and completely self-contained. You just need a scratchpad to hold the conversation history between Lambda invocations for a single session, and then you're done with it. DynamoDB with a 1-hour TTL is cheaper, simpler, already in most teams' AWS footprints, and doesn't add a dependency on a newer managed service. That said, if you later extend this system so the AI agent builds a persistent profile of each researcher's preferences and patterns over time — that's exactly the use case AgentCore Memory is designed for, and it would be worth revisiting then.

Second, the decision record is formatted as markdown rather than JSON because text documents chunk and embed significantly better for retrieval. The headers give the embeddings semantic structure without requiring you to write a custom chunking strategy. Third, the system prompt uses a <DECISION_RECORD> XML tag as a completion signal rather than a fixed question count — this lets the model handle researchers who are more or less verbose without cutting them off mid-thought.

Step 4: Triggering the Bedrock Knowledge Base Sync

Once the decision record lands in S3, you need to ingest it into the knowledge base. You can either run a scheduled sync or trigger it on every new document. For this use case — where capture happens infrequently but recency matters — an S3 event-triggered sync is the right call.

# lambdas/kb_sync_trigger/handler.py
import os
import boto3
from typing import Any

bedrock_agent = boto3.client("bedrock-agent", region_name="us-east-1")

KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]
DATA_SOURCE_ID = os.environ["DATA_SOURCE_ID"]


def lambda_handler(event: dict[str, Any], context: Any) -> None:
    """Triggered by S3 PutObject events under decision-records/"""
    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = record["s3"]["object"]["key"]

        print(f"New decision record: s3://{bucket}/{key}")

        # Start an ingestion job — Bedrock will pick up all new/modified S3 objects
        response = bedrock_agent.start_ingestion_job(
            knowledgeBaseId=KNOWLEDGE_BASE_ID,
            dataSourceId=DATA_SOURCE_ID,
        )

        job_id = response["ingestionJob"]["ingestionJobId"]
        print(f"Started KB ingestion job: {job_id}")

Wire this Lambda up with an S3 event notification on ObjectCreated events under the decision-records/ prefix. The ingestion job runs asynchronously — typically takes 30 to 90 seconds for small document sets — so the knowledge base is updated within a couple of minutes of each capture.

For the S3 notification configuration in CDK or CloudFormation:

# infrastructure/app_stack.py (CDK)
from aws_cdk import (
    aws_s3 as s3,
    aws_s3_notifications as s3_notifications,
    aws_lambda as lambda_,
)

decision_bucket.add_event_notification(
    s3.EventType.OBJECT_CREATED,
    s3_notifications.LambdaDestination(kb_sync_lambda),
    s3.NotificationKeyFilter(prefix="decision-records/"),
)

Step 5: Querying the Knowledge Base in the Agent

Now comes the payoff. When a team is working through site selection and the agent is providing context, it should be able to retrieve relevant past decisions and surface them naturally. Here's how to query the knowledge base and integrate the results into the agent's response.

# lambdas/site_selection_agent/knowledge_retriever.py
import os
import boto3
from dataclasses import dataclass

bedrock_agent_runtime = boto3.client("bedrock-agent-runtime", region_name="us-east-1")
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]


@dataclass
class DecisionContext:
    relevant_decisions: list[str]
    sites_referenced: list[str]
    summary: str


def retrieve_relevant_decisions(query: str, site_ids: list[str]) -> DecisionContext:
    """
    Retrieve past decision records relevant to the current selection context.
    
    Args:
        query: Natural language description of the current selection scenario
        site_ids: Site IDs being considered in the current decision
    
    Returns:
        DecisionContext with retrieved passages and a synthesized summary
    """
    # Build a rich retrieval query
    retrieval_query = (
        f"Research site selection decision: {query}. "
        f"Sites being evaluated: {', '.join(site_ids)}. "
        "What factors influenced past selections of these or similar sites? "
        "What alternatives were considered and why were they rejected?"
    )

    response = bedrock_agent_runtime.retrieve(
        knowledgeBaseId=KNOWLEDGE_BASE_ID,
        retrievalQuery={"text": retrieval_query},
        retrievalConfiguration={
            "vectorSearchConfiguration": {
                "numberOfResults": 5,
                "overrideSearchType": "HYBRID",  # semantic + keyword
            }
        },
    )

    results = response["retrievalResults"]

    if not results:
        return DecisionContext(
            relevant_decisions=[],
            sites_referenced=[],
            summary="No prior decision records found for these sites.",
        )

    passages = [r["content"]["text"] for r in results]
    sites_referenced = list({
        r["metadata"].get("site-id", "unknown")
        for r in results
        if "metadata" in r
    })

    # Use Claude to synthesize the retrieved passages into useful context
    synthesis_prompt = f"""You are summarizing past research site selection decisions to help a team make a new decision.

Retrieved decision records:
{chr(10).join(f"---{chr(10)}{p}" for p in passages)}

Current selection context: {query}

Provide a concise summary (3-5 sentences) of what past decisions reveal about the sites being considered. 
Focus on factors that would be useful to the current team. Do not invent information not present in the records."""

    synthesis = bedrock.converse(
        modelId="global.anthropic.claude-haiku-4-5-20251001-v1:0",  # Haiku 4.5 via global inference profile
        messages=[{"role": "user", "content": [{"text": synthesis_prompt}]}],
        inferenceConfig={"maxTokens": 400, "temperature": 0.1},
    )

    return DecisionContext(
        relevant_decisions=passages,
        sites_referenced=sites_referenced,
        summary=synthesis["output"]["message"]["content"][0]["text"],
    )

# lambdas/site_selection_agent/handler.py
import json
import os
import boto3
from knowledge_retriever import retrieve_relevant_decisions

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
MODEL_ID = "global.anthropic.claude-sonnet-4-5-20250929-v1:0"

AGENT_SYSTEM_PROMPT = """You are a research site selection assistant. You help research teams 
evaluate and select the most appropriate sites for their studies.

When you have access to past decision context, you should reference it explicitly — 
e.g., "Based on past selections, teams have found that Site 7's proximity to..."

Always cite the institutional context you're drawing on so teams know this is validated 
experience rather than general advice."""


def lambda_handler(event: dict, context) -> dict:
    user_message = event["message"]
    site_ids = event.get("siteIds", [])
    conversation_history = event.get("history", [])

    # Retrieve relevant past decisions from the knowledge base
    decision_context = retrieve_relevant_decisions(
        query=user_message,
        site_ids=site_ids,
    )

    # Inject the retrieved context into the system prompt
    augmented_system = AGENT_SYSTEM_PROMPT
    if decision_context.relevant_decisions:
        augmented_system += f"""

## Institutional Knowledge from Past Decisions

{decision_context.summary}

The following sites have prior decision records available: {', '.join(decision_context.sites_referenced)}.
Use this context to ground your recommendations in your team's actual experience."""

    # Build the messages payload
    messages = conversation_history + [
        {"role": "user", "content": [{"text": user_message}]}
    ]

    response = bedrock.converse(
        modelId=MODEL_ID,
        system=[{"text": augmented_system}],
        messages=messages,
        inferenceConfig={"maxTokens": 800, "temperature": 0.4},
    )

    assistant_reply = response["output"]["message"]["content"][0]["text"]

    return {
        "reply": assistant_reply,
        "contextUsed": len(decision_context.relevant_decisions) > 0,
        "sitesWithHistory": decision_context.sites_referenced,
    }

The HYBRID search mode in the retrieval call is deliberate. Pure semantic search is great for conceptual similarity but can miss exact site ID matches when a team asks specifically about Site 7. Hybrid search combines vector similarity with BM25 keyword matching, so you get both the semantic richness and the precision.

Step 6: Surfacing the Context in the UI

The final piece is making the retrieved institutional knowledge visible to the team in a way that builds trust and encourages continued use of the capture feature.

// src/components/SiteSelectionAgent.tsx
interface AgentResponse {
  reply: string;
  contextUsed: boolean;
  sitesWithHistory: string[];
}

function AgentResponseCard({ response }: { response: AgentResponse }) {
  return (
    <div className="agent-response">
      <p>{response.reply}</p>
      {response.contextUsed && (
        <div className="context-badge">
          <span className="icon">🗂</span>
          <span>
            Drawing on past decisions for{" "}
            {response.sitesWithHistory.join(", ")}
          </span>
        </div>
      )}
    </div>
  );
}

The contextUsed badge is small but important. It tells the team that the agent's response is grounded in their organization's actual history, not just general knowledge. That provenance signal is what builds confidence in the system over time — and what motivates teams to keep doing the 90-second debrief.

Best Practices and Common Pitfalls

Document formatting matters more than you'd think. The Bedrock Knowledge Base will chunk whatever you put in S3, and how you format that content directly affects retrieval quality. Markdown with descriptive headers (## Why This Site Was Selected) gives the embedding model semantic anchors. A flat JSON blob gives it nothing. Always store decision records as human-readable narrative text.

Keep capture sessions short and the questions specific. The system prompt used here targets four questions. You can tune this, but resist the temptation to ask for more. Teams that feel the debrief is too long will start dismissing the capture panel. Four focused questions with follow-ups beats ten exhaustive ones with no answers.

Use metadata filters for scoping. As the knowledge base grows, you'll want to allow queries that are scoped to specific time ranges, research programs, or geographic regions. Store those attributes as S3 object metadata and use Bedrock's filter parameter in retrieve() calls to prevent knowledge from one program contaminating recommendations for another.

Plan for knowledge base latency in the selection flow. The retrieve() call typically takes 300 to 700ms. For the selection assistant context panel — which is displayed while the team is evaluating options — this is fine. Don't put a KB retrieval call in the critical path of a page load.

Monitor ingestion job failures. The start_ingestion_job call fires and forgets. Wire up a CloudWatch alarm on the bedrock:IngestionJobFailed metric so you know when a document failed to ingest. Document parsing errors are the most common culprit, usually from encoding issues in the uploaded text.

Seed the knowledge base before launch. If you have historical meeting notes, email threads, or any documentation about past site selections, process and upload them before the app goes live. A knowledge base that returns zero results for the first three months trains users that it's useless. Even 20 to 30 bootstrapped decision records make a significant difference to early retrieval quality.

Where This Pattern Applies

Research site selection is the lens we used, but the Institutional Memory Loop is domain-agnostic. Any workflow where teams make high-stakes judgment calls that the organization wants to learn from over time is a candidate. A few concrete translations:

Clinical trial site selection. The same problem, higher stakes. CROs and sponsors evaluate dozens of sites per study and rarely document why Site A got the nod over Site B — investigator experience, patient retention history, IRB turnaround, local regulatory climate. That reasoning is enormously valuable to the next team running a similar indication. The capture and retrieval architecture is identical; only the domain vocabulary in the system prompt changes.

Architectural Decision Records (ADRs). Most engineering teams treat ADRs as a documentation chore that happens after the decision, if at all. Embedding a capture step directly into the pull request or design review workflow — where an agent asks "What alternatives did you consider and why did you rule them out?" — produces ADRs that are actually filled out, indexed, and retrievable when the next engineer is making the same tradeoff eighteen months later.

Vendor and procurement selection. Procurement teams evaluate vendors constantly and keep almost none of the comparative reasoning. Why did Security team veto Vendor X? Why did the integration story on Vendor Y win over Vendor Z's lower price? That context prevents organizations from re-evaluating the same vendor over and over, or re-learning the same lessons about a category. A post-selection debrief in the procurement tool closes that loop.

GTM and pricing decisions. Why did a drug go to market at $X? Why was a particular region sequenced before another? These decisions involve complex tradeoffs across regulatory, commercial, and manufacturing inputs, and the reasoning is almost never written down in a form that the next product team can find and learn from. A knowledge base seeded with past decision rationale gives new teams a starting point that isn't blank.

The common thread across all of these: the value isn't in the decision itself — it's in the reasoning that produced it. The Institutional Memory Loop is just a systematic way of making sure that reasoning survives the people who held it.

Wrapping Up

Three things determine whether the Institutional Memory Loop actually works in practice: making capture frictionless (which is why the AI-assisted debrief beats a form), formatting documents for retrieval quality (which is why markdown with headers beats flat JSON), and making retrieved knowledge visible and attributed in the UI so teams trust it and keep feeding it.

Systems don’t become intelligent when they store more data. They become intelligent when they remember why decisions were made. The only way that happens is if you design for it.

Code samples in this post use global.anthropic.claude-sonnet-4-5-20250929-v1:0 (Claude Sonnet 4.5 global inference profile, launched September 30, 2025) for the main agent, global.anthropic.claude-haiku-4-5-20251001-v1:0 (Claude Haiku 4.5 global inference profile) for the synthesis step, and amazon.titan-embed-text-v2:0 for embeddings. The global. prefix routes requests across AWS regions automatically for higher availability — see the Amazon Bedrock cross-region inference documentation for details. Check the Amazon Bedrock model IDs documentation for the latest available model strings in your region.

Lambda functions are deployed with Python 3.12 runtime minimum (identifier: python3.12, based on Amazon Linux 2023). Python 3.13 (python3.13, released November 2024) and Python 3.14 (python3.14, released November 2025) are also available on Lambda and both run on Amazon Linux 2023 — either is a solid choice if you want the latest language features. Python 3.11 and earlier are on an older Amazon Linux 2 base and approaching end of support; there's no reason to target them for new projects. All code in this post is compatible with 3.12 through 3.14. Regardless of runtime version, package boto3 in your deployment zip or use the AWS-provided managed layer.

Vector search is great at finding semantically similar content, but it struggles with broad keyword-heavy queries common in regulatory and clinical document search. By combining BM25 keyword matching with k-NN vector search and fusing results with Reciprocal Rank Fusion (RRF), we improved retrieval quality, particularly for recall-heavy queries common in compliance and safety workflows.

The Problem Nobody Warns You About

If you've built a RAG (Retrieval-Augmented Generation) system, you've probably followed the standard playbook: chunk your documents, embed them with a model like Titan or OpenAI, store the vectors, and retrieve the top-K most similar chunks at query time.

This works beautifully for specific questions:

"What are the contraindications for apixaban?"

The embedding model understands the semantic intent. It finds chunks that discuss apixaban contraindications even if the exact word "contraindications" doesn't appear. Vector search shines here.

But try this query:

"Which drugs have bleeding warnings?"

Suddenly, vector search struggles. Why? Because this query needs to match across many documents, finding every drug that mentions "bleeding" in a warnings context. The embedding for this query lands in a vague region of the vector space — it's semantically close to lots of things about bleeding, warnings, drugs, and adverse events, but not specifically close to any one document's bleeding warning section.

The result: you get back 3-4 chunks from one or two drugs, miss several others entirely, and the generated answer is incomplete.

This isn't a theoretical problem. In regulatory document search — FDA drug labels, clinical protocols, safety reports — the most important queries are often broad: "List all drugs with boxed warnings", "Which protocols require liver function monitoring?", "What drugs interact with anticoagulants?". These are exactly the queries where vector-only search falls short.

Why Vector Search Fails on Broad Queries

To understand the failure, think about what an embedding model actually does. It compresses a chunk of text into a fixed-size vector (say, 1024 dimensions) that captures the overall semantic meaning. Two chunks about apixaban dosing will have similar vectors. A chunk about warfarin bleeding warnings and a chunk about apixaban bleeding warnings will have somewhat similar vectors — but they'll also be similar to chunks about bleeding in general, surgical bleeding, GI bleeding from NSAIDs, and so on.

When you search for "Which drugs have bleeding warnings?", the k-NN search returns the chunks whose vectors are closest to the query vector. But "closest" in a 1024-dimensional space doesn't mean "contains the keywords 'bleeding' and 'warnings' in a drug label context." It means "semantically similar to the general concept of drugs and bleeding warnings." The distinction matters.

BM25 (the classic keyword search algorithm) doesn't have this problem. It looks for the actual terms "bleeding" and "warnings" in the text, scores documents by term frequency and inverse document frequency, and ranks and surfaces documents containing those terms, typically providing strong recall for keyword-driven queries. It's not smart about paraphrasing, but it's thorough.

The insight: Vector search is semantically precise for focused queries but can under-retrieve on recall-heavy, cross-document queries.

The Solution: Hybrid Search with RRF Fusion

Hybrid search runs both retrieval methods in parallel and merges the results. The architecture looks like this:

The key is the fusion step. Reciprocal Rank Fusion (RRF) is elegantly simple:

score(doc) = \sum \frac{1}{k + rank_i(doc)}

def rrf_fusion(result_lists, k=60):
    scores = {}

    for results in result_lists:
        for rank, doc_id in enumerate(results):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)

    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

For each result list, a document's contribution to its final score is 1 / (k + rank + 1), where k is a constant (typically 60) and rank is its position in that list. A document ranked #1 in BM25 and #3 in k-NN gets a higher fused score than a document ranked #1 in k-NN alone.

RRF has a useful property: it avoids the need for score normalization across retrieval methods. BM25 scores and k-NN distances are on completely different scales, but RRF only uses rank positions, so they combine cleanly.

Real Results

Here's what the same queries return with hybrid search enabled, against a corpus of 142 FDA drug labels (~10,800 chunks):

Broad query (BM25 strength):

"Which drugs have bleeding warnings?"

Hybrid search found Naproxen (NSAID with GI bleeding boxed warning) and Apixaban (anticoagulant with bleeding warnings and spinal hematoma boxed warning), with detailed citations from multiple label sections. Vector-only search returned only apixaban.

Cross-drug query (hybrid strength):

"Which drugs have black box warnings about suicidal thoughts?"

Hybrid search found Pregabalin, Gabapentin (antiepileptic drug warnings), Duloxetine (antidepressant warning), and Quetiapine (antipsychotic warning) — four different drug classes, each with the specific suicidality boxed warning language. Vector-only search returned only two of these.

Specific query (k-NN strength):

"What is the recommended dosage for apixaban in atrial fibrillation?"

Both approaches work well here, but hybrid search still benefits from BM25 boosting chunks that contain the exact terms "dosage", "apixaban", and "atrial fibrillation."

Choosing Your Embedding Dimensions

Amazon Titan Text Embeddings V2 offers three output dimensions: 256, 512, and 1024. This is a decision you make once — changing dimensions later means re-embedding and re-indexing your entire corpus.

Representative results from public benchmarks and AWS sample evaluations show:

Sources: MTEB Retrieval benchmarks; AWS Titan V2 sample notebooks; AWS Machine Learning Blog on Titan embeddings and OpenSearch. Exact performance varies by dataset and query distribution.

For most use cases, 512 is the sweet spot — you lose only 1% retrieval accuracy and halve your storage. But for regulatory and clinical document search, I'd argue for 1024. Here's why:

The 3.2% accuracy gap between 256 and 1024 sounds small, but in a compliance context — where missing a relevant safety finding matters — even small retrieval quality differences add up across a large corpus.

The storage argument for lower dimensions also doesn't hold at typical regulatory corpus sizes. 100,000 vectors at 1024 dimensions is ~400 MB. That's nothing for an OpenSearch cluster. You'd need millions of vectors before the storage difference between 512 and 1024 becomes meaningful.

One more thing: Titan V2 at 1024 dimensions actually outperforms Titan V1 at 1536 dimensions on retrieval benchmarks (0.51 vs 0.47 NDCG@10) (source). More dimensions isn't always better — the V2 training improvements matter more than the extra 512 dimensions.

The Chunking Problem Nobody Talks About

There's a second retrieval quality issue that's independent of vector vs. keyword search: chunk boundaries.

Most RAG tutorials show you how to split text into fixed-size windows (500 words, 1000 tokens, etc.) with some overlap. This works fine for blog posts and Wikipedia articles. It performs poorly for structured documents.

A 200-page clinical protocol has sections like Inclusion Criteria, Exclusion Criteria, Primary Endpoints, Adverse Events. With fixed 512-word chunking, a chunk boundary can land right in the middle of the Exclusion Criteria section:

Chunk 47: [...tail end of Inclusion Criteria...]
          [...first half of Exclusion Criteria...]

Chunk 48: [...second half of Exclusion Criteria...]
          [...start of Dosing Schedule...]

When someone asks "What are the exclusion criteria?", Chunk 47 is a partial match polluted with inclusion criteria text. The embedding blends both concepts. BM25 matches "criteria" in both the inclusion and exclusion parts. The answer is confused.

Section-aware chunking solves this by detecting section boundaries (numbered headings, font changes, structural markers) and chunking at those boundaries instead of at fixed word counts. Each chunk is a complete section or sub-section, with metadata recording its position in the document hierarchy:

{
  "text": "Patients are excluded if they have any of the following...",
  "sectionPath": ["6. Study Population", "6.2 Exclusion Criteria"]
}

This gives you:

• Cleaner embeddings — each vector represents one coherent concept, not a blend of two adjacent sections

• Better BM25 matching — section headers like "Exclusion Criteria" are in the same chunk as the criteria themselves

• Filtered search — you can restrict queries to specific section types ("search only in Adverse Events sections")

Recent work on hybrid and multi-stage retrieval systems shows consistent gains in top-K retrieval quality compared to single-method approaches (e.g., DIRSRT, 2026). In practice, sentence-boundary chunking often matches more complex semantic chunking approaches at lower computational cost for many workloads. It's one of the highest-impact optimizations you can make after implementing hybrid search — though the gap between strategies is typically single-digit percentage points, not transformative on its own.

Architecture at a Glance

The full system runs on AWS with minimal operational overhead:

The ingestion pipeline is fully serverless (Lambda, Step Functions, EventBridge). The search layer uses an OpenSearch managed domain — AWS handles patching, backups, and snapshots, but you're provisioning and sizing the cluster (instance types, node counts, storage). This is a deliberate choice: the managed domain gives us UltraWarm tiering for cost-effective warm storage and full control over the k-NN index configuration (HNSW engine, space type, shard count) that OpenSearch Serverless doesn't yet expose.

Key Takeaways

1. Vector search alone isn't enough for document-heavy RAG. If your queries include broad searches across many documents, you need keyword matching too.

2. RRF fusion is simple and effective. You don't need a learned ranker or complex score normalization. Rank-based fusion with k=60 works remarkably well.

3. 1024 dimensions is the right choice for Titan V2 when precision matters. The storage savings of 256 or 512 are irrelevant at typical enterprise corpus sizes. The accuracy difference isn't.

4. Chunk boundaries matter more than chunk size. For structured documents, chunk at section boundaries, not at fixed word counts. The gains are consistent but modest — expect single-digit percentage-point improvements, not a silver bullet.

5. Hybrid search + section-aware chunking is the combination that unlocks regulatory document search. Either one alone is a partial fix. Together, they handle both the broad keyword queries and the specific semantic queries that compliance teams need.

Try It Yourself

The hybrid search pattern described here works with any OpenSearch cluster that has the k-NN plugin enabled. The key components:

• Amazon Bedrock Titan V2 for embeddings (configurable dimensions)

• OpenSearch with k-NN plugin (HNSW FAISS) for vector storage + BM25

• RRF fusion — ~20 lines of code, no dependencies

def rrf_fusion(result_lists, k=60):
    scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

• Any LLM for the generation step (Claude, GPT-4, Llama, etc.)

If you're building a RAG system for structured documents — regulatory filings, clinical protocols, legal contracts, technical specifications — hybrid search with section-aware chunking is worth the investment. The improvement on broad, recall-heavy queries is often substantial in practice.

Have questions or want to discuss hybrid search architectures? Connect with me on LinkedIn.

The views expressed in this post are my own and do not represent those of my employer.

We're at the Microservices Moment for AI

The Landscape

In the early days of cloud architecture, teams built monolithic applications and eventually learned — sometimes painfully — that decomposing them into services was the right long-term bet. Agentic AI is going through exactly the same transition right now. Single all-purpose agents are giving way to orchestrated systems of specialized agents.

Gartner measured a 1,445% surge in enterprise multi-agent system inquiries from Q1 2024 to Q2 2025 (Gartner, December 2025). By end of 2026, 40% of enterprise applications are projected to embed AI agents — up from less than 5% in 2025. The frameworks and patterns you pick today will define your architectural ceiling for years.

The hard truth: Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027 due to escalating costs and inadequate risk controls. The failures originate not in bad models, but in bad orchestration design — agents that are individually capable, but poorly coordinated, still fail. Framework choice is a first-class architectural decision.

This post cuts through the noise and maps the major frameworks to the patterns they serve best — with practical guidance for teams building on AWS.

The Real Contenders in 2026

The Frameworks

The field has consolidated. LangChain is partially deprecated as a primary agent runtime — its latest versions actually run on LangGraph under the hood. The meaningful choices today are:

The core mental model: "LangGraph says: you design the graph, the model executes it. Strands says: you give the model tools and let it figure out the graph itself."

The axis isn't quality — it's developer control vs. model autonomy. Both are valid production choices. The wrong choice is picking one based on hype rather than your workflow's actual requirements.

Bedrock AgentCore: The Shift That Changes Everything

Platform Layer

The most important thing to understand about AgentCore is that it represents a categorical shift in what AWS is offering. Amazon Bedrock is no longer just a model hosting service. It is now the control plane, governance layer, and runtime that makes autonomous AI deployable in organizations with real risk profiles.

AgentCore reached general availability in October 2025 and works with any open-source framework — LangGraph, Strands, CrewAI, LlamaIndex, OpenAI Agents SDK — and any foundation model inside or outside Bedrock.

AgentCore Service Map

🔒 For regulated industries: AgentCore Policy uses Cedar — AWS's open-source policy language — to intercept every tool call in real time. Natural language policy definitions auto-convert to Cedar, making compliance boundaries auditable by non-engineers. Policy and Evaluations both reached GA in March 2026.

The 6 Orchestration Patterns You Need to Know

The Architecture

Frameworks are implementation tools. Patterns are the architecture. Production systems almost always combine two or three of these. Understanding them is what separates teams that ship from teams that stay in pilot purgatory.

1. Supervisor / Hierarchical

A central orchestrator decomposes the task, delegates to specialist sub-agents, validates outputs, and synthesizes a final result. The gold standard for most enterprise workflows. Gartner predicts that by 2027, 70% of multi-agent systems will use narrowly specialized agents, improving accuracy but increasing coordination complexity (Gartner, December 2025).

Use an expensive, capable model for the orchestrator; use cheaper, specialized models for each sub-agent.

Best implemented with: LangGraph (explicit state) or Strands + Agent Squad (model-driven routing)

2. Sequential Pipeline

Agent A hands off to Agent B. Classic for linear data transformation, document processing, or any workflow with clear stage dependencies.

Simple to debug, predictable cost profile. Critical rule: every stage must validate its inputs. Never pass garbage forward — a leaky pipeline where Stage 3 produces malformed output will have Stage 4 and 5 confidently processing garbage.

Best implemented with: LangGraph nodes with explicit state contracts

3. Parallel Fan-Out / Fan-In

Multiple agents work the same task simultaneously from different angles or specializations. A collector agent synthesizes results. Also called scatter-gather or map-reduce.

Cut latency on complex research, multi-perspective analysis, or consensus-building tasks. The initiator agent distributes work; the collector waits for all branches and produces a unified output.

Best implemented with: LangGraph parallel branches or Strands with concurrent tool calls

4. Choreography (Event-Driven)

Agents coordinate through events on a message bus — no central orchestrator. Agent A publishes research_completed, Agent B subscribes and acts, Agent B publishes analysis_ready, and so on.

High autonomy, loosely coupled, easy to add or remove agents. Trade-off: debugging is significantly harder without a centralized control flow. Best for workflows that change frequently, not for high-stakes deterministic pipelines.

Best implemented with: EventBridge + SQS + Lambda, or Kafka for higher throughput

5. Evaluator-Optimizer Loop (Reflection)

A generator agent produces output; an evaluator agent critiques it; the generator revises. The cycle repeats until a quality threshold is met.

Reflection is the most powerful pattern for accuracy-critical tasks — regulatory document authoring, code generation, clinical report drafting. Each critique round is a separate LLM call, so cost and latency multiply. Design your termination condition carefully.

Best implemented with: LangGraph cycles with conditional edges

6. ReAct (Reason + Act)

The foundational single-agent loop: Thought → Action → Observation → repeat. The model articulates its reasoning, calls a tool, observes the result, and decides the next step.

This is the basis for both Strands (which wraps this loop automatically) and most LangGraph nodes. Understanding ReAct is prerequisite to understanding every other pattern.

Best implemented with: Strands (native), LangGraph nodes, or any framework's AgentExecutor

⚡ The cascading hallucination problem: Agent A hallucinates a policy. Agent B executes against that hallucination. Agent C reports confidently on a corrupt baseline. Multi-agent systems amplify errors as much as they amplify capability. Use immutable state snapshots — each agent works with a versioned state object and produces a new version. This provides audit lineage, prevents accidental mutations, and makes replay possible.

How to Choose

Decision Framework

The Production-Ready AWS Stack

Recommended Stack

For complex, regulated, or enterprise-grade deployments on AWS, these layers complement each other without overlap:

🔭 Observability note: AgentCore Observability is OTEL-compatible and integrates natively with Langfuse. For regulated environments — pharma, healthcare, financial services — self-hosted Langfuse on AWS gives you full trace ownership with zero data leaving your VPC. This combination is current best-in-class for GxP-adjacent AI workloads.

What to Actually Do

Bottom Line

The teams winning in 2026 make a deliberate pattern choice early, instrument observability from day one, and resist the temptation to add more agents when better tools or prompts would solve the problem.

For most AWS practitioners: Start with Strands + AgentCore for speed and native integration. Add LangGraph for any sub-workflow where step auditability or strict ordering is non-negotiable. Wire Agent Squad in when your single-orchestrator topology starts to strain.

For regulated industries: AgentCore Policy (now GA) gives you Cedar-based tool call interception that compliance teams can read and audit without engineering involvement. AgentCore Evaluations (also now GA) gives you continuous quality monitoring rather than manual spot-checks. These two features alone move the "can we trust this in production?" conversation forward by months.

🧵 The sustainable pattern: The ability to leverage the reasoning capabilities of these models, coupled with the ability to do real-world things through tools, is a durable architectural bet. The specific frameworks will keep evolving. The pattern of reasoning + tool use + governance will not.

Agentic AI Architecture Guide · April 2026 · AWS · LangGraph · Strands · AgentCore

This walkthrough shows a different approach. We build a small, working AI app — a "thin slice" — but we structure it with two patterns that keep the code clean as it grows:

• BFF (Backend for Frontend) — the API layer shapes responses for the UI and hides backend complexity
• Observer — an event system that lets components react to what's happening without being coupled together

The app itself is simple: paste rough notes, get back a summary, action items, and a next step. What matters is how the pieces connect.

📦 Full source: github.com/jrgwv/thin-slice

Architecture

Each layer has one job. The frontend never talks to the model. The BFF never builds prompts. The model layer never parses responses.

Project Structure

app/
  main.py                    # FastAPI entrypoint
  routes/
    analyze.py               # BFF — request handling
  services/
    agent_service.py          # orchestration + observer events
    ai_client.py              # model integration (Bedrock Converse API)
    prompt_builder.py         # prompt construction
    response_parser.py        # raw text → structured output (with JSON extraction)
    observer.py               # event emitter/listener system
  models/
    schemas.py                # Pydantic request/response models
  static/
    index.html / app.js / styles.css

Pattern 1: BFF (Backend for Frontend)

The BFF pattern means the API layer exists for the frontend. It doesn't contain business logic. It validates input, delegates work, and shapes the response.

Here's the full route:

# app/routes/analyze.py

@router.post("/analyze", response_model=AnalyzeResponse)
async def analyze(payload: AnalyzeRequest) -> AnalyzeResponse:
    text = payload.text.strip()
    request_id = uuid.uuid4().hex[:12]

    if not text:
        raise HTTPException(status_code=400, detail="The 'text' field must not be empty.")

    if len(text) > 12000:
        raise HTTPException(status_code=400, detail="Input is too long for this prototype.")

    try:
        return run_analysis(request_id, text)
    except Exception as exc:
        logger.exception("request_id=%s — analysis failed", request_id)
        raise HTTPException(status_code=500, detail="Analysis failed.") from exc

What it does:

1. Validates input
2. Generates a request ID for tracing
3. Delegates to the agent service
4. Catches errors and returns clean HTTP responses

What it doesn't do: build prompts, call models, parse responses. That's the agent's job.

Why this matters: When you add a second frontend (mobile app, CLI, Slack bot), you add a second BFF. The agent service stays the same.

Pattern 2: Observer

The Observer pattern decouples "something happened" from "what to do about it."

The implementation is intentionally minimal:

# app/services/observer.py

_listeners: dict[str, list[EventListener]] = defaultdict(list)

def on(event: str, listener: EventListener) -> None:
    _listeners[event].append(listener)

def emit(event: str, data: dict[str, Any] | None = None) -> None:
    for listener in _listeners.get(event, []):
        listener(data or {})

Three functions: on, emit, clear. That's the whole system.

The agent service emits events at each stage of the pipeline:

# app/services/agent_service.py

def run_analysis(request_id: str, user_text: str) -> AnalyzeResponse:
    observer.emit("agent:start", {"request_id": request_id})

    prompt = build_prompt(user_text)
    observer.emit("agent:prompt_built", {"request_id": request_id, "prompt_len": len(prompt)})

    raw = call_model(prompt)
    observer.emit("agent:model_returned", {"request_id": request_id, "raw_len": len(raw)})

    result = parse_model_output(raw)
    observer.emit("agent:complete", {"request_id": request_id, "summary_len": len(result.summary)})

    return result

Right now, nothing listens. And that's fine. The point is that the hooks exist. When you need logging, tracing, metrics, or telemetry — you register a listener. The agent service doesn't change.

# Example: add structured logging later
observer.on("agent:complete", lambda data: logger.info("done", extra=data))

Why this matters: You get observability without modifying the code that does the work.

The Agent Service

This is the orchestration layer. It owns the pipeline:

1. Build the prompt
2. Call the model
3. Parse the response

It's the only place that knows all three steps exist. The BFF doesn't know about prompts. The model layer doesn't know about parsing.

If you later add tool calls, multi-step reasoning, or model routing — this is where it goes. The BFF and model layer stay untouched.

Model Integration: Bedrock Converse API

This was one of the first real lessons in the build.

The initial version used invoke_model — Bedrock's low-level API. It works, but the request and response formats are model-specific. Amazon Nova expects one body shape. Anthropic expects another. Switch models, rewrite the integration.

The fix: switch to the Converse API. It provides a unified interface across all Bedrock models.

# app/services/ai_client.py

def _invoke_bedrock(prompt: str) -> str:
    region = os.getenv("AWS_REGION", "us-east-1")
    model_id = os.getenv("MODEL_ID", "global.anthropic.claude-haiku-4-5-20251001-v1:0")
    client = boto3.client("bedrock-runtime", region_name=region)

    try:
        response = client.converse(
            modelId=model_id,
            messages=[{"role": "user", "content": [{"text": prompt}]}],
        )
    except (ClientError, BotoCoreError) as exc:
        raise RuntimeError(f"Bedrock invocation failed: {exc}") from exc

    try:
        return response["output"]["message"]["content"][0]["text"]
    except (KeyError, IndexError, TypeError) as exc:
        raise RuntimeError("Unexpected model response format.") from exc

Same request format whether you're calling Nova, Claude, Mistral, or Llama. Same response shape. Swap the model ID, everything else stays the same.

The default model is global.anthropic.claude-haiku-4-5-20251001-v1:0 — a cross-region inference profile. Bedrock routes the request to the nearest available region automatically.

Why this matters: The whole point of isolating the model layer is that you can swap models. The Converse API makes that actually true — not just in theory.

JSON Extraction: Handling Real Model Output

Here's something that works perfectly in demo mode and breaks immediately in production: assuming the model returns clean JSON.

The prompt says "Return valid JSON only." The model says "Sure! Here's the JSON:" and wraps it in markdown fences.

Calling json.loads() on that throws a JSONDecodeError. The fix is a small extraction step before parsing:

# app/services/response_parser.py

def _extract_json(raw_text: str) -> str:
    """Pull the first JSON object out of raw model output."""
    # Try stripping markdown fences first
    match = re.search(r"```(?:json)?\s*(\{.*?})\s*```", raw_text, re.DOTALL)
    if match:
        return match.group(1)
    # Fall back to first { ... }
    match = re.search(r"\{.*}", raw_text, re.DOTALL)
    if match:
        return match.group(0)
    return raw_text

It handles three cases:

1. ```json { ... } ``` — fenced with language tag
2. ``` { ... } ``` — bare fences
3. Some preamble text { ... } — JSON buried in prose

This is defensive parsing. The model is a collaborator, not a contract. You ask for JSON, you usually get JSON, but you build for the times you don't.

Why this matters: Every AI app hits this. The sooner you handle it, the fewer 500 errors you ship.

Testing the Parser

If you're parsing model output, test the weird cases — not just the happy path. These five tests cover the real-world formats models actually return:

def test_parse_clean_json():
    raw = '{"summary":"hello","action_items":["a","b"],"next_step":"ship it"}'
    result = parse_model_output(raw)
    assert result.summary == "hello"

def test_parse_markdown_fenced_json():
    raw = 'Here is the result:\n```json\n{"summary":"fenced","action_items":["x"],"next_step":"go"}\n```'
    result = parse_model_output(raw)
    assert result.summary == "fenced"

def test_parse_json_with_preamble():
    raw = 'Sure, here you go:\n{"summary":"preamble","action_items":[],"next_step":"next"}'
    result = parse_model_output(raw)
    assert result.summary == "preamble"

def test_parse_missing_fields_uses_defaults():
    raw = '{"summary":"","action_items":[],"next_step":""}'
    result = parse_model_output(raw)
    assert result.summary == "No summary returned."

def test_parse_bare_fences():
    raw = '```\n{"summary":"bare","action_items":["a"],"next_step":"done"}\n```'
    result = parse_model_output(raw)
    assert result.summary == "bare"

Clean JSON is the easy case. The ones that save you are markdown_fenced, preamble, and bare_fences — those are the formats that cause JSONDecodeError in production when you only tested with mock data.

Prompt Layer

The prompt builder wraps user input in structured instructions:

def build_prompt(user_text: str) -> str:
    return f"""You are helping turn rough notes into useful output.

Return the response in this exact JSON shape:
{{
  "summary": "short paragraph",
  "action_items": ["item 1", "item 2", "item 3"],
  "next_step": "single recommended next step"
}}

Rules:
- Be concise
- Keep action items practical
- Do not include markdown
- Return valid JSON only

Input:
{user_text}
""".strip()

This is where most of the "intelligence" lives. The model is only as good as the instructions it receives.

Frontend

The frontend is deliberately minimal: a textarea, a button, a result display. No frameworks.

const response = await fetch("/api/analyze", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ text }),
});
const data = await response.json();
renderResults(data);

The frontend doesn't know about agents, prompts, or models. It sends text, gets JSON back. That's the BFF contract.

End-to-End Flow

What This Gets You

This is ~200 lines of Python across 7 files. But the separation buys you real things:

Nothing ripples. That's the point.

Running It

git clone https://github.com/jrgwv/thin-slice.git
cd thin-slice
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
./start.sh
# Open http://127.0.0.1:8000

Demo mode is on by default — no AWS credentials needed to see it work. Set DEMO_MODE=false in .env to call Bedrock for real.

Final Thought

This isn't a production system. It's a clean, working slice of one.

That's enough to test the idea, validate the output, and decide what to build next — without untangling a mess when you do.

Build the slice. Not the system.