ToS;DR

Spidering, crawling, or accessing the site through any automated means is only allowed if you respect machine-readable instructions set by this service like robots.txt


Service: Google
Status: PENDING
Case: Spidering, crawling, or accessing the site through any automated means is not allowed
Changes: 3
Source: link
Author: Dr_Jeff (20149) Staff


using automated means to access content from any of our services in violation of the machine-readable instructions on our web pages (for example, robots.txt files that disallow crawling, training, or other activities)</p></li> <li><p>


Comments:
On 2026-02-24 13:37:34 UTC, shadowwwind (28460) Staff wrote:

changes-requested: Change title to include "that does not respect machine-readable instructions set by this service."



We track editorial changes to analyses and updates to a point's status and display the previous versions here as part of an effort to promote transparency regarding our curation process.

Version 3: 2026-02-25 19:19:52 UTC by Dr_Jeff (20149) Staff

Previous Title: Spidering, crawling, or accessing the site through any automated means is not allowed

Updated Title: Spidering, crawling, or accessing the site through any automated means is only allowed if you respect machine-readable instructions set by this service like robots.txt

Previous Analysis: No changes recorded

Updated Analysis: No changes recorded

Previous Status: CHANGES REQUESTED

Updated Status: PENDING

Version 2: 2026-02-24 13:37:34 UTC by shadowwwind (28460) Staff

Previous Title: No changes recorded

Updated Title: No changes recorded

Previous Analysis: No changes recorded

Updated Analysis: No changes recorded

Previous Status: PENDING

Updated Status: CHANGES REQUESTED

Version 1: 2026-02-23 19:25:20 UTC by Dr_Jeff (20149) Staff

Previous Title:

Updated Title: Spidering, crawling, or accessing the site through any automated means is not allowed

Previous Analysis:

Updated Analysis: Generated through the annotate view

Previous Status:

Updated Status: PENDING