May 22 • 💬 General discussion
Bulk Data Ingestion & Synthesis Advice?
Using the frameworks + classroom materials here, I'm finally building my own Agentic OS 👏
Need to ingest a fat stack: 900+ work files (G-Drive), 700+ ChatGPT & Claude chats, iCloud mix, and a heap of meeting transcripts (docs / sheets / PDFs).
Leveraging Nate Jones' OB-1 "panning-for-gold" + "heavy-file-ingestion" skills with a stage-then-route pattern. Working, but curious if anyone has battle-tested workarounds I'm missing.
Specifically aiming for token-efficient pre-processing - anyone built deterministic batch summarizers that compress before pulling into Claude/Codex context? Trying to keep the OS lean on 900-file ingestion.
Bonus: anyone using Obsidian Bases or Dataview for query-against-local-markdown instead of cloud DB?
Any advice or suggestions appreciated!
0
0 comments
Bulk Data Ingestion & Synthesis Advice?
skool.com/cliefnotes
What we give away free beats most paid courses. Build durable AI systems with a Marine vet and Edinburgh researcher. 40+ lessons, growing.
Leaderboard (30-day)
1
+656
2
+577
3
🔥
+557
4
+549
5
+422
Powered by