[TECHNICAL CHALLENGE – Chunking & SplitInBatches Loop with Pinecone in n8n]
Hey everyone,
I’m working on building a smart agent that processes documents in n8n.
I’m doing manual chunking (not using Pinecone’s native splitter) because I want full control over chunk size, overlap, formatting, and semantic structure before sending anything to the vector store.
The pipeline does a lot: custom chunk splitting, cleaning, enrichment, reformulation, and finally vectorization in Pinecone.
Everything seems to be in place… but there’s a weird issue:
• I’m using a SplitInBatches node (batchSize: 1) to process chunks one at a time
• I feed it 72 chunks, but only the first chunk is processed, and it never moves to the next
• I’ve tried returning the loop via Merge, NoOp, and other tricks — no success
On top of that, sometimes a single chunk results in 10 empty items after passing through the LLM or post-processing nodes, which breaks the whole logic.
I’m trying to figure out:
• How do you correctly handle batch looping in n8n with SplitInBatches?
• How do you avoid the LLM or code nodes creating multiple empty outputs from a single chunk?
• If I do manual chunking, how do I ensure Pinecone gets only clean, usable, and logically separated chunks?
If anyone’s comfortable reviewing my workflow and helping fix it, I’d really appreciate it. I can share a stripped-down workflow.json file with everything in place, just needs tuning to work properly.
Thanks in advance to anyone who loves debugging split logic and complex automations.
Let’s solve this challenge cleanly.