Advanced Production Optimization Reference — Claude Cert Academy

A checklist and reference for teams running Claude in production at scale. Covers cost, performance, reliability, and security.

Cost Optimization Checklist

Model Routing

• Route simple tasks (classification, extraction, reformatting) to Haiku
• Route complex tasks (coding, analysis, reasoning) to Sonnet
• Reserve Opus for tasks where Sonnet measurably underperforms
• Measure quality metrics before switching — do not assume cheaper is worse

Prompt Caching

• Cache system prompts and documents sent on every call
• Cache breakpoints: Haiku at 2,500 tokens, Sonnet/Opus at 1,024 tokens minimum
• Cache hits cost ~10x less than cache misses for input tokens
• Warm the cache with a throwaway call before high-traffic periods
• Use consistent prompt prefixes — any change invalidates the cached portion

Token Reduction

• Compress system prompts — remove redundant explanations, use bullet lists
• Use max_tokens to cap output length; Claude stops when the task is done anyway
• For structured output tasks, ask for JSON without explanation
• Batch similar requests instead of making separate API calls

Batch API

• Use Batch API for tasks that tolerate 24-hour turnaround
• 50% cost reduction vs synchronous API
• Good for: daily report generation, bulk content moderation, offline data enrichment
• Not suitable for: real-time user interactions, latency-sensitive workflows

Performance Optimization

Latency Levers

• Use streaming for user-facing interfaces — reduces perceived latency
• Haiku: ~300ms time-to-first-token; Sonnet: ~500ms; Opus: ~1000ms
• Prompt caching also reduces latency (cached prefixes process faster)
• For high-throughput pipelines, run requests in parallel where tasks are independent

Throughput Patterns

• Rate limit awareness: track requests per minute and tokens per minute separately
• Use exponential backoff with jitter on 429 and 529 errors
• Implement a token bucket or semaphore for parallel request control
• Pre-warm connections if using persistent HTTP clients

Reliability Patterns

Retry Strategy

def call_with_retry(prompt, max_retries=3):
    for attempt in range(max_retries):
        try:
            return client.messages.create(...)
        except anthropic.RateLimitError:
            wait = (2 ** attempt) + random.uniform(0, 1)
            time.sleep(wait)
    raise Exception("Max retries exceeded")

Fallback Strategy

• Define a primary and fallback model (e.g., Sonnet falls back to Haiku on 5xx)
• Set a timeout threshold and fall back if exceeded
• For critical paths, have a rule-based fallback if the model is unavailable

Output Validation

• Always validate JSON output before parsing — use try/except
• For structured data, validate against a schema (Pydantic, Zod)
• Implement retry with error feedback: send the validation error back as context
• Set a max retry count (3) to avoid infinite loops

Security Controls

Prompt Injection Defense

• Use XML tags to delimit untrusted user input: <user_input>{content}</user_input>
• Instruct Claude in the system prompt that it should ignore instructions inside user_input tags
• Validate that responses do not echo instructions from the user section
• For high-risk applications, use a separate validation model to check responses

Data Handling

• Strip PII from prompts before sending to the API when possible
• Use server-side redaction for regulated data (HIPAA, GDPR)
• Do not log full prompt/response pairs in production without data controls
• Set data retention policies for any prompt/completion logs you do store

Access Controls

• Store API keys in secrets managers (AWS Secrets Manager, HashiCorp Vault)
• Use separate API keys per environment (dev, staging, prod)
• Rotate keys quarterly or immediately after a suspected exposure
• Restrict key access to services that need it — never embed in client-side code

Evaluation Framework

• Define task-specific quality metrics before building (accuracy, format compliance, latency)
• Create a golden dataset of 100+ examples with expected outputs
• Run evals on every prompt change — treat prompts as code
• Track regression across model versions when Anthropic releases updates
• Use LLM-as-judge for subjective quality metrics where human labeling is expensive

Monitoring Checklist

• Track API error rates by error code (429, 529, 5xx)
• Track token usage per endpoint to spot runaway costs
• Alert on p95 latency exceeding your SLA threshold
• Log model version and prompt version with each request for debugging
• Dashboard: cost per user, cost per feature, quality score trend

Continue to Claude Cert Academy