I read Jake's paper and built this

Built something that addresses a gap named in section 6.2 of the ICM paper. https://arxiv.org/abs/2603.16021

Van Clief and McDermott call it "observability without traceability." ICM tells you which files a stage was given. It does not tell you which passage in the output came from which file. If something is wrong, you know what was in the room but you have to read everything and work backward.

sem_debug is my attempt at that traceability layer. Point it at a stage output and the declared inputs. It maps every passage back to its source and flags anything with no detectable origin. That unattributed text is where the agent went outside the declared context.

I wrote it about a week ago and sat on it because I wasn't sure if I was being a dick. Jake may have already solved this or had something in the works. Figured sharing it was better than sitting on it.

Runs inside the ICM workspace structure. Validated on a real pipeline stage. Made the ICM workflow build the tool, because why not.

github.com/WBChain3/sem_debug

Curious if anyone has experienced the traceability problem, or if Jake has already addressed it somewhere I missed.

2 comments