SpoorFox Sites
The SpoorFoxSites fetcher
SpoorFoxSites reads a website only when that website's owner, a SpoorFox customer, asks SpoorFox Sites to start their new site from their current one. It reads the site once, then never again.
Its user agent
If you found SpoorFoxSites in your server log, this is what it looks like and why it was there.
SpoorFoxSites/1.0 (+https://spoorfox.com/sites/bot)
What it reads
- Public pages of that one website: at most 30 HTML pages, up to 3 clicks from the start page.
- The photos those pages show: at most 40.
- One request at a time, at most 2 a second, and the whole read is done within 3 minutes.
What it does not do
- It does not run JavaScript, log in, submit forms, or read frames and widgets.
- It never tries to get past a bot check or a CAPTCHA. A website behind one is simply not read.
What happens with what it reads
Nothing it reads is used to train AI models. The texts and photos go only into that customer's new site draft.
It honours robots.txt
SpoorFoxSites follows the group in your robots.txt for SpoorFoxSites, or the group for * when there is none. Disallow is respected, and Content-Signal: ai-input=no stops the read.
How to block it
Add this to the robots.txt of your website:
User-agent: SpoorFoxSites Disallow: /
Or, if you only want to say no to AI input:
User-agent: * Content-Signal: ai-input=no
Questions
Write to [email protected] or use the contact page.