July 9, 2026 7 min read AWS · AI · Document Processing

Document Processing Pipelines: PDF to Structured Data on AWS

Document processing is one of the most common AI automation projects I see at clients. Invoices. Contracts. Insurance claims. Property appraisals. The pattern is always the same: PDF comes in, you need structured data out.

Most teams start with AWS Textract. It’s fast, it’s cheap, and it’s built for the job. But Textract has a ceiling. It’s great at recognizing forms and tables. It’s terrible at understanding context and extracting semantic meaning.

Then they try to glue a language model on top — Claude, GPT-4, whatever — and suddenly they have a working system.

But “working” isn’t the same as “producible.” I’m going to walk you through the architecture I’ve built for three clients. This handles hundreds of documents a month without breaking.

The Architecture

S3 Landing Zone → Lambda Processor → Claude API → DynamoDB → Validation → Notification

Here’s what each stage does:

S3 Landing Zone. Files are uploaded to s3://company-documents/inbox/. A simple bucket policy restricts who can upload. Each document gets tagged with metadata: document type (invoice, contract, claim), source system, upload timestamp.

Lambda Processor. Triggered by S3 ObjectCreated events. The function downloads the PDF, extracts text (more on this in a moment), calls Claude with structured output instructions, validates the response, and stores results in DynamoDB. If anything fails, it goes to an SQS dead letter queue.

Claude API. This is where the intelligence happens. You’re using Claude with structured output — JSON mode. You define the schema for what you want extracted, Claude returns valid JSON, and you don’t have to parse fragile regex patterns.

DynamoDB. Stores extracted data with:

Partition key: document_id (UUID of the PDF)
Sort key: timestamp
Attributes: extracted_data (the JSON), confidence_score, source_file, processing_status, errors

Validation. A second Lambda (or within the same one) validates the extracted data. Required fields present? Values in expected ranges? If validation fails, flag the document for manual review and alert someone.

Notification. Send the validated data downstream — email, webhook, Salesforce API, whatever the business process needs.

Textract vs Claude: Which One First?

This matters because it affects your cost and accuracy.

Use Textract first if:

The document is primarily a form or table (invoices, structured reports)
You need OCR for scanned documents
You need coordinates of extracted text (for highlighting in UI)
Cost is the primary constraint

Use Claude directly if:

The document is unstructured (emails, contracts, narratives)
You need semantic understanding (flagging unusual contract terms)
The document is mostly text, not tables
The PDF is already digital (not scanned)

For most of my clients, the answer is: start with Claude directly. Here’s why:

Claude handles both born-digital and scanned PDFs.
Structured outputs eliminate parsing errors.
You can ask Claude semantic questions (“Is this contract unusual? Why?”) that Textract can’t answer.
The cost difference isn’t huge if you’re smart about prompts.

If you’re processing 1,000 invoices with consistent formatting, Textract is cheaper. If you’re processing 500 diverse contracts, Claude is worth it.

The Code Pattern

# extract.py — S3-triggered Lambda: PDF in, validated JSON out
import json
import time
import boto3
import anthropic
from typing import TypedDict

s3 = boto3.client("s3")
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("documents")
client = anthropic.Anthropic()

class InvoiceData(TypedDict):
    vendor_name: str
    invoice_number: str
    invoice_date: str
    total_amount: float
    line_items: list[dict]
    payment_due_date: str

def extract_from_pdf(bucket: str, key: str) -> dict:
    # Download PDF
    response = s3.get_object(Bucket=bucket, Key=key)
    pdf_content = response["Body"].read()

    # Convert PDF to base64 for Claude
    import base64
    pdf_base64 = base64.standard_b64encode(pdf_content).decode("utf-8")

    # Call Claude with a document block + structured output schema
    message = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "document",
                        "source": {
                            "type": "base64",
                            "media_type": "application/pdf",
                            "data": pdf_base64
                        }
                    },
                    {
                        "type": "text",
                        "text": "Extract invoice data and return as JSON."
                    }
                ]
            }
        ],
        response_format={
            "type": "json_schema",
            "json_schema": {
                "name": "InvoiceExtraction",
                "schema": {
                    "type": "object",
                    "properties": {
                        "vendor_name": {"type": "string"},
                        "invoice_number": {"type": "string"},
                        "invoice_date": {"type": "string"},
                        "total_amount": {"type": "number"},
                        "line_items": {
                            "type": "array",
                            "items": {
                                "type": "object",
                                "properties": {
                                    "description": {"type": "string"},
                                    "quantity": {"type": "number"},
                                    "unit_price": {"type": "number"},
                                    "total": {"type": "number"}
                                },
                                "required": ["description", "quantity", "unit_price", "total"]
                            }
                        },
                        "payment_due_date": {"type": "string"}
                    },
                    "required": ["vendor_name", "invoice_number", "total_amount"]
                }
            }
        }
    )

    return json.loads(message.content[0].text)

def validate_invoice_data(data: dict) -> tuple[bool, list[str]]:
    errors = []

    if not data.get("vendor_name"):
        errors.append("vendor_name is required")

    if not data.get("invoice_number"):
        errors.append("invoice_number is required")

    if not isinstance(data.get("total_amount"), (int, float)) or data["total_amount"] < 0:
        errors.append("total_amount must be a positive number")

    # Validate line items sum to total
    if "line_items" in data:
        line_total = sum(item.get("total", 0) for item in data["line_items"])
        if abs(line_total - data["total_amount"]) > 0.01:
            errors.append(f"line items total ({line_total}) != invoice total ({data['total_amount']})")

    return len(errors) == 0, errors

def lambda_handler(event, context):
    bucket = event["Records"][0]["s3"]["bucket"]["name"]
    key = event["Records"][0]["s3"]["object"]["key"]
    document_id = key.split("/")[-1].split(".")[0]

    try:
        extracted_data = extract_from_pdf(bucket, key)
        is_valid, errors = validate_invoice_data(extracted_data)

        table.put_item(
            Item={
                "document_id": document_id,
                "timestamp": int(time.time()),
                "extracted_data": json.dumps(extracted_data),
                "valid": is_valid,
                "errors": errors,
                "source_file": key
            }
        )

        if not is_valid:
            send_alert(document_id, errors)  # manual review queue

        return {"statusCode": 200, "body": json.dumps({"extracted": True})}

    except Exception as e:
        sqs.send_message(QueueUrl=dlq_url, MessageBody=json.dumps({"error": str(e), "key": key}))  # dead letter queue
        raise

Cost Breakdown: 1,000 Documents/Month

Assuming 1,000 invoices per month, average 3 pages each:

Claude API cost: ~$0.30 per document (3 pages, ~1,500 tokens input, ~300 tokens output). Total: $300/month.
Lambda: ~$0.0001 per invocation. 1,000 invocations = $0.10/month.
S3: ~$0.01/month (assuming 50MB stored).
DynamoDB: ~$5/month (on-demand pricing).

Total: ~$305/month.

Compare to Textract: $2 per page for documents over 1–5 pages. So 3,000 pages × $2 = $6,000/month. Claude is roughly 20x cheaper at this volume.

The Gotchas

Multi-page documents. Claude can handle up to 20 pages per PDF document. If your invoices are longer, you’ll need to split them or use Textract first.

Scanned PDFs. Claude handles these fine, but they take longer to process (more tokens) and cost more. Consider Textract as a first-pass for text extraction if OCR quality is critical.

Validation false negatives. Your validation logic only catches structural errors. It won’t catch semantic mistakes (e.g., wrong vendor name). Build in a manual review process for high-value documents.

Rate limiting. If you’re processing 100+ documents per minute, you’ll hit API rate limits. Use SQS with a batch processor and respect rate limits. Exponential backoff is your friend.

The Observability Setup

Log these metrics to CloudWatch:

documents_processed (count)
extraction_latency_ms (histogram)
validation_failure_rate (percent)
claude_api_errors (count)
cost_per_document (dollars)

Alert: validation failure rate > 5% or extraction latency p95 > 30 seconds.

This is a production-ready pattern. I’ve used it for invoice processing, contract extraction, and insurance claim analysis. The schema changes, but the architecture doesn’t.

Get the free AI Readiness Checklist

15 questions to diagnose your team’s AI readiness, where you’ll see ROI fastest, and what to tackle first.

✓ Takes 5 minutes ✓ Actionable next steps ✓ No sales pitch

No spam. Unsubscribe anytime.

Ready to build AI that actually works?

Let’s talk about how SRE discipline transforms AI from a risky experiment into a reliable business system.

Book Your Free Discovery Call