September 21, 2026 7 min read AI · Architecture · AWS

Batch vs. Real-Time Processing: Choosing the Right Pattern

Real-time returns a result the instant a document lands. Batch queues it up and works through a backlog on a schedule. The choice between them sets your cost, your latency, and how much complexity you take on.

Invoices come in. Documents need classifying. The real question with an AI pipeline isn’t whether to process them — it’s when.

Real-time processing looks instant: a document goes in, a classification comes back. Batch processing looks slower: documents queue up and get handled on a schedule. But the choice between them sets your cost, your latency, and how much operational complexity you take on. Get it wrong and you’ll pay for it, one way or another.

Real-Time Processing (API Gateway + Lambda)

Architecture: a user makes a request, API Gateway routes it, Lambda invokes Claude, and the response comes back synchronously.

When it makes sense:

The user is waiting for the result (document upload, instant classification)
Latency is critical — a response is needed within seconds
Request volume is low to moderate, under 1,000 requests/hour
The work is quick, with the Claude call completing in under 30 seconds

Cost math for a 2,000-token input document that generates 500 tokens of output:

API Gateway: $0.0000035
Lambda (128 MB, 5 seconds): roughly $0.000009
Claude: $0.006 input + $0.0075 output = $0.0135

Total per request: about $0.0135. At 100 requests a day, that’s $4.05 a month.

A few things worth planning around. Lambda times out at 15 minutes, so if your Claude call occasionally runs long, real-time won’t hold up without retry logic that can blow past the user’s patience. Cold starts add 1-3 seconds of latency. Sub-second response needs provisioned concurrency, which runs $0.015/hour on top.

API Gateway’s burst limit is 5,000 requests/second, which is rarely the limiting factor. Your account-level API rate limits usually are. Error handling is simple (return a 500), but any degraded performance is visible to the user immediately.

Real-time works well for classification, extraction, summarization, and moderation — anywhere latency is the thing that matters most.

Batch Processing (S3 Events + Lambda)

Architecture: a user uploads a document to S3, an S3 event triggers Lambda, Lambda processes the documents, and results land in S3 or a database.

When it makes sense:

The result doesn’t need to be immediate — hours is fine
Volume is high, 10,000+ documents a day
You can batch multiple documents into one Claude call
The work tolerates failures and retries

You’re not paying per synchronous request here. S3 events are free, Lambda costs its usual invocation plus execution time, and Claude pricing is the same as above. The real savings is batching: instead of calling Claude 100 times with one document each, you send one call with 100 documents (if the API supports it) or ten calls with ten each.

For the same 100 documents:

Real-time: 100 API calls × $0.0135 = $1.35
Batch, 10 documents per call: 10 API calls × $0.135 = $1.35 — same cost if you’re not actually leveraging batching
Batch, 100 documents per call: 1 API call × $0.675 = $0.675, about 50% cheaper

The bigger savings shows up operationally. One Lambda invocation processing 100 documents costs less to run than 100 separate invocations.

The tradeoffs: you need somewhere to store results (S3, DynamoDB, Postgres) and a way to notify the user when processing finishes (SQS, SNS, a webhook, or polling). Failure handling gets more complex — a failed document needs retry logic that doesn’t lose it. And there’s a window where processing lags, sized by how often you trigger the batch.

Batch works well for high-volume document processing, content generation, bulk extraction, and scheduled reports.

The Hybrid Pattern

Real-time and batch aren’t mutually exclusive. For most AI systems, the sweet spot combines both:

A synchronous API for the fast path — the user uploads, and the system immediately returns “processing started” plus a job ID
Asynchronous processing to do the actual work — Lambda polls S3, processes documents, stores results
A status endpoint so the user can check progress by polling the job ID

# Step 1: user uploads, we return immediately
@app.post("/upload-document")
async def upload_document(file: UploadFile):
    job_id = str(uuid.uuid4())

    # Store in S3 and trigger async processing
    s3.put_object(
        Bucket='documents',
        Key=f'{job_id}/{file.filename}',
        Body=file.file.read()
    )

    return {
        'job_id': job_id,
        'status': 'queued',
        'estimated_wait': '2 minutes'
    }

# Step 2: user polls for status
@app.get("/job/{job_id}")
async def get_job_status(job_id: str):
    result = dynamodb.get_item(Key={'job_id': job_id})

    if 'Item' not in result:
        return {'status': 'processing'}

    return {
        'status': 'complete',
        'result': result['Item']['result']
    }

# Step 3: async Lambda processes documents
def lambda_handler(event, context):
    for document in get_pending_documents():
        result = call_claude(document)
        store_result(document.job_id, result)

This pattern returns instantly on the happy path, processes at your own pace without timeout risk, costs less than pure real-time since it skips provisioned concurrency, and still lets you batch for savings.

Real Cost Comparison: 10,000 Documents a Month

Pure real-time: 10,000 Claude calls ($1.35) + 10,000 Lambda invocations at 128 MB/5s (about $0.83) + API Gateway ($0.035) — total around $2.22/month.

Pure batch, 100 documents per invocation: 100 Claude calls ($1.35) + 100 Lambda invocations at 128 MB/30s (about $0.50) + free S3 events — total around $1.85/month.

Hybrid: 10,000 cheap status-poll calls ($0.035) + 100 batch Lambda invocations at 256 MB/30s (about $1.00) + 100 Claude calls ($1.35) — total around $2.39/month.

The hybrid pattern costs a little more in API Gateway charges, but it buys resilience and a better user experience: no timeout waits, and lower operational complexity than juggling pure real-time at scale.

Failure Handling

With real-time, a Claude timeout shows up as a 504 to the user. You have to decide up front: retry silently and make them wait longer, return a partial result, or ask them to try again later.

With batch, a failed document just sits in the queue. Monitoring should catch it, and you retry on a schedule with exponential backoff. The user never sees the failure — they see “still processing” until it completes or someone intervenes manually.

Batch failure handling is easier to get right, mostly because it isn’t blocking a person on the other end.

The Decision Tree

Does the user need the result immediately? Yes → real-time.
Is latency critical, under a second? Yes → real-time with provisioned concurrency. No → real-time without it.
Is volume high, over 10,000/day? Yes → batch. No → either works.
Can you batch multiple items into one Claude call? Yes, and volume is high → batch. Otherwise → hybrid.
Does the result need to be perfect, or is best-effort fine? Perfect → real-time, for synchronous feedback. Best-effort → batch.

Most of the systems I build end up hybrid. The economics and the reliability both point that way.

Get the free AI Readiness Checklist

15 questions to diagnose your team’s AI readiness, where you’ll see ROI fastest, and what to tackle first.

✓ Takes 5 minutes ✓ Actionable next steps ✓ No sales pitch

No spam. Unsubscribe anytime.

Ready to build AI that actually works?

Let’s talk about how SRE discipline transforms AI from a risky experiment into a reliable business system.

Start the Conversation

About the author

Charles Harvey is the founder of Three Moons Network and a site reliability engineer who builds production-grade AI automation for small businesses — monitoring, cost visibility, and documentation included. He writes about his hands-on AI experiments at floggingclaude.com. Connect on LinkedIn or see the code on GitHub.