I received several DMs asking about the best testing pipeline designs that universally fit into any project.
📌 You asked, you get it!
In this post, you'll find three diagrams describing testing pipelines for different scenarios:
🟦 1. Nightly Run
Set up scheduled runs for all possible tests across all apps and services.
Since these happen overnight, utilize that time to get as much coverage as possible. Check the results first thing in the morning and notify teams about any findings.
Key benefits:
- Maximum test coverage without impacting work hours
- Early detection of issues before the workday begins
- Comprehensive validation across all systems
🟩 2. Deployment Run
These tests are automatically triggered during deployment.
Run API tests first since they're the fastest and can provide feedback in minutes. Only if all API tests pass should the rest of the tests be triggered.
⚠️ Important:
Be cautious with production runs. Make sure to clarify whether it's okay to run regression tests or any tests that generate data in production environments.
Recommended sequence:
1️⃣ API tests (fastest feedback)
2️⃣ Integration tests
3️⃣ UI tests (if API tests pass)
4️⃣ Regression suite (with appropriate environment considerations)
🟪 3. Manually Triggered Runs
These are testing pipelines that need to be triggered manually by clicking a button.
They're typically used by teams that need to test a specific service or app beyond the standard deployment runs.
Common scenarios:
- Changes were made to infrastructure with no deployment pipeline
- Testing specific features in isolation
- Ad-hoc validation requirements
Recommendation: Create as many variations and combinations as possible to give teams the freedom to pick and choose without needing to bother you to set them up.
✅️ Conclusion
These three pipeline patterns provide a solid foundation for any testing strategy. Adapt them to your specific needs, and remember: the best pipeline is one that your team actually uses consistently.
𝐏.𝐒. 🚩 𝐈𝐟 𝐲𝐨𝐮 𝐡𝐚𝐯𝐞𝐧’𝐭 𝐰𝐚𝐭𝐜𝐡𝐞𝐝 𝐢𝐭 𝐲𝐞𝐭, 𝐲𝐨𝐮𝐫 𝐧𝐞𝐱𝐭 𝐬𝐭𝐞𝐩 𝐢𝐬 𝐭𝐡𝐞 𝐅𝐑𝐄𝐄 𝟑-𝐩𝐚𝐫𝐭 “𝐌𝐚𝐧𝐮𝐚𝐥 𝐐𝐀 → 𝐒𝐃𝐄𝐓” 𝐰𝐨𝐫𝐤𝐬𝐡𝐨𝐩, 𝐚 𝐬𝐡𝐨𝐫𝐭 𝐦𝐢𝐧𝐢-𝐜𝐨𝐮𝐫𝐬𝐞 𝐭𝐡𝐚𝐭 𝐠𝐢𝐯𝐞𝐬 𝐲𝐨𝐮 𝐭𝐡𝐞 𝐟𝐮𝐥𝐥 𝐫𝐨𝐚𝐝𝐦𝐚𝐩 𝐭𝐨 𝐛𝐞𝐜𝐨𝐦𝐢𝐧𝐠 𝐚 𝐦𝐢𝐝-𝐥𝐞𝐯𝐞𝐥 𝐒𝐃𝐄𝐓 𝐚𝐧𝐝 𝐩𝐚𝐬𝐬𝐢𝐧𝐠 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰𝐬.