Spidering or crawling is not allowed

Classification: neutral

Weight: 10



Service Title Rating Status Author
SpigotMC "Do not access the forums via automated mechanisms.</li>" approved Photon
Cloudant "You may only use a crawler to crawl this Web site as permitted by this Web site’s robots.txt protocol, and IBM may block any crawlers in its sole discretion. The use authorized under this agreement is non-commercial in nature (e.g., you may not sell the content you access on or through this Web site.) All other use of this site is prohibited.</p>" declined import script
privacytoolsIO "You may not automate access to the forum, or monitor the forum, such as with a web crawler, browser plug-in or add-on, or other computer program that is not a web browser." approved user 434
Grammarly "a.</strong> copy, modify or create derivative works based on the Site, Services, Software or Documentation, or any portion(s) of any of the foregoing (individually and collectively, “Grammarly IP”);" declined Photon
Wattpad "Don’t use any kind of software, device or method (whether it’s manual or automated) to “crawl”, “spider” or otherwise remove any content from any part of the Site or Services." approved LuneSirius
Tapas "You agree not to use or launch any automated system, including without limitation, "robots," "spiders," or "offline readers," that accesses the Service in a manner that attempts to forge the presence of a user accessing the site for the purpose of artificially increasing "pageviews," "subscribers," or "likes" or other elements that contribute to a series popularity or advancement within any programs that Tapas Media offers to users.</p>" approved LuneSirius
indiegogo "or use any kind of software to “crawl” or “spider” any part of the Site" approved LuneSirius
ShapeShift "(iii) other than as specifically approved in writing, any copying, distributing, or disclosing any part of the Service in any medium, including without limitation by any automated or non-automated “scraping”. (iv) other than as specifically approved in writing, using any automated system, including without limitation “robots,” “spiders,” “offline readers,” etc., to access the Service in a manner that sends more request messages to the ShapeShift servers than a human can reasonably produce in the same period of time by using a conventional on-line web browser" approved barelyaware
LinkedIn "Develop, support or use software, devices, scripts, robots, or any other means or processes (including crawlers, browser plugins and add-ons, or any other technology) to scrape the Services or otherwise copy profiles and other data from the Services" approved radiorahim
yelp " You also agree not to, and will not assist, encourage, or enable others to: <ol> <li>Violate the Terms;</li> <li>Modify, adapt, appropriate, reproduce, distribute, translate, create derivative works or adaptations of, publicly display, sell, trade, or in any way exploit the Site or Site Content (other than Your Content), except as expressly authorized by Yelp;</li> <li>Use any robot, spider, site search/retrieval application, or other automated device, process or means to access, retrieve, scrape, or index any portion of the Site or any Site Content;" approved okfifi
allrecipes "(e) you shall not use any manual or automated software, devices or other processes (including but not limited to spiders, robots, scrapers, crawlers, avatars, data mining tools or the like) to "scrape" or download data from the Services (except that we grant the operators of public search engines revocable permission to use spiders to copy materials from the Website for the sole purpose of and solely to the extent necessary for creating publicly available searchable indices of the materials, but not caches or archives of such materials)." approved Peepo
Alignable "(but not through scraping, spidering, crawling or other technology or software used to access data without the express written consent of Alignable or its Users)" approved Esmooth
Quake "You agree that you will not: (a) use any robot, spider, other automatic device or manual process to monitor or copy our web pages or the content contained herein without Id Software’s prior written permission." approved Quited255
boingo " Further, the use of manual or automated software, devices, or other processes to “crawl,” “scrape,” or “spider” any portion of the Services is strictly prohibited." approved Peepo
McDonald's "Use or launch any unauthorized technology or automated system to access the online services or extract content from the online services, including but not limited to spiders, robots, screen scrapers, or offline readers;" approved Peepo
New York Times "<p>(ii) use robots, spiders, scripts, service, software or any manual or automatic device, tool, or process designed to data mine or scrape the Content, data or information from the Services, or otherwise access or collect the Content, data or information from the Services using automated means." approved Peepo
Pornhub "<p>Additionally, you agree not to:</p> <p>• use the Websites in any manner that could disable, overburden, damage, or impair the site or interfere with any other party’s use of the Websites, including their ability to engage in real time activities through the Websites;</p> <p>• use any robot, spider or other automatic device, process or means to access the Websites for any purpose, including monitoring or copying any of the material on the Websites without our prior written consent;" approved Peepo
Social Blade Spidering or crawling is not allowed pending Mixer


Comments:
No comments found