On 2018-05-18 13:49:30 UTC, Deleted wrote:

imported from 2314

On 2018-05-23 12:41:12 UTC, Deleted wrote:

https://groups.google.com/forum/#!msg/tosdr/VByLSF4QPlk/1jlgG8zfBC0J It appears that some sites are blocking tosback2's crawler -- we might
consider adding user-agent spoofing to deal with this.

They appear to include:

http://www.cooks.com/rec/privacy.html
http://www.peoplesmart.com/?_act=privacy
http://www.ticketmaster.com/h/privacy.html
http://pinterest.com/about/privacy/

Also, Jimm, I know you're in the process of moving, but can you (or
someone else) upload some more recent crawl data? -- the last one is
from May 6th, a week ago...

--
tosdr.org | twitter.com/tosdr | github.com/tosdr
---
You received this message because you are subscribed to the Google Groups "Terms of Service; Didn't Read" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tosdr+unsubscribe@googlegroups.com.
To post to this group, send email to tosdr@googlegroups.com.
Visit this group at http://groups.google.com/group/tosdr?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 2018-05-30 12:50:11 UTC, Deleted wrote:

imported status as declined