Don's Factory 🏭 - Using manufacturing principles to build an end-to-end AI pipeline

A folder structure based off 10 years of manufacturing experience. I've been overseeing delivery of systems from design through manufacturing and deployment in my professional work. Everyone has their own system for organizing and building their solutions; this is a natural extension of the environment I work in every day and managing these projects the same way I would any other.

The beauty is just like the lean practices inspiring this, this is a framework. All the specifics and magic in the workflow live in the context files, template details, and addition resources. Want to add in your specific quality guidelines? Throw it in config and reference it in the quality stage context file. Want to always build a specific component the same way? Add another file and reference it in design and assembly. This framework can be shaped any way to fit your workflow but the principles remain the same.

Here's a brief breakdown of the process flow as well as the files produced.

The intention is that this is the flow of objects created called "systems". A system can be anything inside a folder: an artifact, an ICM workflow, a collection of files. As long as it can fit in a folder, it’s applicable. The system folder template is:

References - inspiration or applicable documents relevant to that specific system build
Work Instructions - Where all the documents named below are stored and read from during the workflow. Documents are moved from active to processed subfolders as they are used in the sequence.
System - Where the actual output of the build is placed. This is the resulting files, or workspace produced.
travel-sheet.md - a document recording progress and a summary of every step executed on the project as it progresses.

The workflow is as follows:

01 Design:

Scope - Define the goal of the project. The outcome written in one statement. What success looks like. Produces scope.md
Design - Define a full breakdown of the intent of the system/artifact/solution to be built. What it is, what it isn't. What success looks like with a wider explanation as well as what failure looks like. The gritty details. Produces design-brief.md
Review - A sanity check of design-brief.md that all pieces are in place. Was anything missed? Will the build team need more information? Produces design-review.md
Manufacturing Kickoff - Converts design-brief.md into a condensed format for handoff to the build agents. Removes reasoning and intent for build and only provides instructions for build for context tightening. Also provides instruction for quality check and testing down the road. Produces blueprint.md, qa-brief.md, test-plan.md

02 Factory Floor

Assembly - Reads blueprint.md and builds the system. All the hard work has been done prior to this point for simplicity. Produces assembly-debrief.md for reference.
Quality Assurance - Runs through qa-brief.md to verify successful build. Also loops in general quality guidelines as a secondary safety net before moving to test. Produces quality-assessment.md
Test Stand - Runs through test-plan.md to verify function. Produces test-report.md
Acceptance - Produces acceptance-debrief.md for human review before deployment.

03 Deployment

Live Systems - Where active systems are placed once up and running. This is optional based on desired organization but the next two stages ideally would have running stages live here and worked out of here.
Field Service - Deployed systems that need rework. This has its own micro structure similar to the overall workflow: Scope - Retrofit - Test - Acceptance. Systems work through this flow then are moved back to live systems when completed.
Retired Systems - Where unused systems are placed for archiving.

The workflow is balanced between automatic processing and human intervention where optimal. The same benefits in traditional manufacturing apply:

More consistent builds based on better design
Improved quality and lessons learned
A tight paper trail of what happened. The inputs used, the outputs produced, the intentions, the history. At every step along the way.

If you made it this far, thank you! Let me know what you think. Does this process apply to solutions manufacturing or are there better ways to build?

29 comments