ICM for data science · Clief Notes

ICM for data science

Hi all 👋

Thank you for all the great content. I’ve been lurking around for a few weeks and it’s been a lot of fun to try new things. Unfortunately I can’t share any of it but I’m hoping to try one of the competitions soon.

Most of my ICMs now have evolved to have roughly 3 rooms:

Data extraction and pre-processing
Data analysis and feature extraction
Result presentation and discussion

The projects are large enough that it is starting to feel like each room needs its own coding subagent because re-usable skills don’t seem to cut it anymore:

Engineers : each data source gets its own CLI, each analysis implements a different research angle, each user interfaces have different taste requirements…
Reviewers : Looks for red flags not just in the code but also in the data and the rendering of the presentation…
Executors : Runs my daily pipeline fast and without polluting the context window.

Most of the generic skills I find online are for software engineering; not data science.

I used to think specs folders and test folders should sit at the root of my repo but now I think they should be buried deep into each room?

Anyone else trying to use ICM for data science? Have you used ICM to migrate away from Jupyter notebooks or do you combine Jupyter with ICM?

I’m also curious if you have a particular way of forcing your ICM to invoke sub-agents and restrict their tools and/or folder access. I’m not doing any of that right now…

Cheers!

3 comments

ICM for data science