Day #2 build: Scraper 🔥 · AI Automation Society

Day #2 build: Scraper 🔥

Scraper done! And I spent way too much time on it :D It was quite fun and I got very excited with the possibilities.

I scraped a freelancer site for AI and Automation jobs, and ran into some issues with Google Drive and wasted tokens, but got it working well in the end.

I added some additional instructions, and now it can prep me for client meetings. I give it a URL and ask it to prep me for a new client meeting, then it maps the site, scrapes only the pages worth scraping, grabs a screenshot, and pokes around the edges - reviews, socials, competitors - before writing up what it found. Then publishes to Google Drive as proper Docs and Sheets, because nobody wants to read a .md file on their phone in a client's car park.

I can see this as a standard part of a business development pipeline. No more hunting for facts and insights on web pages and copy/pasting into documents.

I was very impressed with how Claude learned from each scrape and improved the recipe.

14 comments

Day #2 build: Scraper 🔥