A 5-stage pipeline takes approximately 8 minutes to complete. Stage 4 — Claude Certified Architect – Foundations Practice — Claude Cert Academy

A 5-stage pipeline takes approximately 8 minutes to complete. Stage 4 fails intermittently due to an external API timeout, requiring the entire pipeline to restart. Which pattern best addresses this without over-engineering?

• Add retry logic inside Stage 4 with exponential backoff
• Write each stage's output to a persistent store before passing the handoff, so failed stages can resume from the last successful checkpoint
• Run all 5 stages in parallel to reduce the impact of any single stage failing
• Reduce the pipeline to 3 stages by merging adjacent stages

Answer: Write each stage's output to a persistent store before passing the handoff, so failed stages can resume from the last successful checkpoint

Checkpointing is the correct pattern for long-running pipelines with intermittent failures. Writing stage output to a persistent store before passing the handoff means a failure at Stage 4 can resume from Stage 3's output, not from the beginning. Retry logic in Stage 4 handles transient timeouts but still loses the full pipeline progress if Stage 4 fails after its retry budget is exhausted. Running stages in parallel is the wrong decomposition — these stages have data dependencies.

Continue to Claude Cert Academy