Scrape Users Fail/Request

[Scurzo]

When I try to use the option
"Scrape users who liked/reblogged a Post"

It sometimes works for posts with a small number of notes. Ex. It works for posts with 2000 notes or less (at least for me)
The problem arises when I try to scrape the users from posts that have 10k-50k or more notes, the bot only scrapes a small number of users (like 500-1500)

I don't know if the bot is coded to fail at large requests or if it's a problem.
I would really appreciate a fix for this

here's a tumblr account with significant notes (for testing purposes)
http://jessepnkman.tumblr.com/archive

[martin@rootjazz]

I tested this a couple of weeks ago and pulled 20k. There might be a hard limit setting so you don't pull too many pages, let me check...

[mlmron313]

I tested this a couple of weeks ago and pulled 20k. There might be a hard limit setting so you don't pull too many pages, let me check...

I'm having the same problem. Only able to get about 5000 or so scrapes no matter how large the note is. Any idea how I can get a full scrape from a large note?

Thanks!

[martin@rootjazz]

Maybe something has changed on tumblr with regards to getting the higher numbers.

Can you send me the post URL you are trying to use. I don't have any posts with high numbers noted down, I will try and find some, but if you read this, might save some time if you can send one

[mlmron313]

300k notes - http://rico-bear.tumblr.com/post/117692794888
175k notes - http://proletarianrevenge.tumblr.com/po ... e-truth-to
240k notes - http://sandandglass.tumblr.com/post/117 ... tor-at-the

Please let me know after you take a look.

Thanks!

[martin@rootjazz]

Running test now.
Post with 44k notes
Max on SETTTINGS tab set to 9999

and I got 10k results as requested

am updating to give logging so should be able to find out what is going on your side

[martin@rootjazz]

http://rootjazz.com/tumblingjazz/updatetesting.html

[mlmron313]

http://rootjazz.com/tumblingjazz/updatetesting.html

Hmm, previously set my max to 999,999 and I would only get a max of 5k scrape regardless of the number of notes on the post. I'll try what you said. But can you try to scrape 50k+ and see if it works?

Thanks!

[mlmron313]

Running test now.
Post with 44k notes
Max on SETTTINGS tab set to 9999

and I got 10k results as requested

am updating to give logging so should be able to find out what is going on your side

I just tried it on a post with 30k and another with 100k with the settings you posted above and am still only getting ~2k per scrape. Any ideas?

[martin@rootjazz]

Running test now.
Post with 44k notes
Max on SETTTINGS tab set to 9999

and I got 10k results as requested

am updating to give logging so should be able to find out what is going on your side

I just tried it on a post with 30k and another with 100k with the settings you posted above and am still only getting ~2k per scrape. Any ideas?

submit your logs
HELP > LOGS > SUBMIT


It could be that a page fails for you, so no new items are found, thus the process stops. On higher scrapes, tumblr may be detecting too many calls from your machine and returns nothing (a normal user probably won't pull 5000 pages in quick succession)


But the logs should tell