Hey Guys
I am new to the group and would really appreciate some tips? I want to scrape a few sites so that I can create an LLM/RAG but, what is the best way of doing this so that it does all of the scraping and then automatically sends the data to the LLL/RAG? I tried unsuccessfully to use Replit and GitHub. I thought that Replit was working and I was at 750,000 cases but it turns out if was 8,000 of the same cases being duplicated :( Any help or pointing to an agent that can deliver would be amazing :)