For this exercise, I scraped the US government public bids website (sam.gov) for listings matching our products' NAICS code over the last 5 days.
Claude Code really deserves full credits on this one! Interestingly, it determined from the outset that using sam.gov 's API was a better option than Fire Crawl due to the many authentication layers on sam.gov. Upon using the API, it then discovered that the keyword filtering parameter was not working, and turned to Fire Crawl to cross-reference the website's search fields to determine other possibilities. Turns out that the API can only return a date range of data, with no filtering capabilities. We decided it was best to fetch the data, then perform keyword filtering offline.
This scraping performed periodically can be packaged into a weekly leads report for our sales team to follow up on.