Scrape Users Fail/Request

[mlmron313]

Running test now.
Post with 44k notes
Max on SETTTINGS tab set to 9999

and I got 10k results as requested

am updating to give logging so should be able to find out what is going on your side

I just tried it on a post with 30k and another with 100k with the settings you posted above and am still only getting ~2k per scrape. Any ideas?

submit your logs
HELP > LOGS > SUBMIT


It could be that a page fails for you, so no new items are found, thus the process stops. On higher scrapes, tumblr may be detecting too many calls from your machine and returns nothing (a normal user probably won't pull 5000 pages in quick succession)


But the logs should tell

Log submitted. ID: logs_MYEMAIL_31038

[martin@rootjazz]

your logs don't appear to have the logs that should be present.

You did run the scrape after updating?


You should be on 1.407

HELP > ABOUT

to see your version number.


If you can confirm your version, then run the scrape again WITHOUT any other actions running at that time

[mlmron313]

your logs don't appear to have the logs that should be present.

You did run the scrape after updating?


You should be on 1.407

HELP > ABOUT

to see your version number.


If you can confirm your version, then run the scrape again WITHOUT any other actions running at that time

Version is 1.407. I just tried again and still not getting over 1k or so per scrape. Log ID: logs_myemail_50120

And I did not have any other actions running at the same time.

[martin@rootjazz]

The only reason that exists why your scrape would stop short is if the code to reload the page with more notes was not found on the previous pull.


This could be
a) no more notes
b) the call previous failed


I will attempt multiple calls if the pull fails. Perhaps that will fix it for you

[martin@rootjazz]

I just attempted the scrape from your logs and got 2k users before I killed it

[martin@rootjazz]

try this

http://rootjazz.com/tumblingjazz/updatetesting.html


1.408