I asked perplexity: heres what It said. I wondered if this is worth some thought?
Q: Can a windows log identify if an LLM has been to my site? A: Yes, Windows server logs (such as those from IIS) can reveal if an AI or large language model (LLM) bot has visited your website, provided you know what to look for in the logs.passion+3
What Windows/IIS Logs Record
Windows web servers—including IIS—create log entries for every request to your site. These entries include:
The visitor’s IP address.
The requesting user agent (the software, bot, or browser making the request).
The URL accessed.
Timestamp.
Other request/response data (status code, referrer, etc.).learn.microsoft+2
How to Identify AI Bots
User Agent Strings: Many AI bots identify themselves with named user agents, such as GPTBot, ClaudeBot, OpenAI-User-Agent, PerplexityBot, or similar. You can search for these user agent strings in your logs.dev+3
Unusual Patterns: AI bots often make rapid, repeated requests over a short period, access deep parts of your site, or use user agents that don’t match major browsers or known search engines.tencentcloud+2
Non-human Behavior: Sessions without cookies, requests with no/little engagement, and traffic from data center IP ranges are additional signals.hogtheweb+2
Practical Steps
Download and review your raw server logs from IIS.
Filter by User Agent to spot known AI bots or any unusual/crawler-like identification.
Cross-reference with published lists of AI/LLM user agents and watch for updates as new bots emerge.momenticmarketing+1
Use tools (e.g., ELK Stack, GoAccess, AWStats) for easier parsing and pattern detection.botify+1
Limitations
Some AI bots may spoof user agents or use generic browser strings to avoid easy detection.
Not all bot traffic will identify itself; look for suspicious activity patterns as well.passion+1
JavaScript-based analytics won’t catch bots that never execute scripts—server logs are required for complete visibility.hogtheweb
Overall, by regularly reviewing and analyzing your server logs for specific user agents and patterns, you can reliably determine if AI/LLM bots are crawling your website.otterly+3