Scrape Follower Limit?

[remiremiremiremi]

Hi All,

There have been a few threads that are asking close to this question, but they either haven't been answered adequately or are out of date. When I try to run a follower scrape on a large account, I'm only getting 10,000 results. I've raised both the maximum in the Scrape tab and the Settings tab to 999,999, and that doesn't seem to do anything. Looking online, it says in many threads that there is an 8,000 result cap on these sorts of queries from Soundcloud, but I'm getting 10,024 so I don't think that's the issue.

Anyone got some help for me? Alternatively, is there any way I can do serial searches with a 10,000 item result but without duplicates. As it is now, I would have continuously run these small searches and then also filter out the duplicates.

Thanks!

[martin@rootjazz]

It used to be 8,000 but it got removed by soundcloud.

Now I am not aware of a limit, and you can certainly scrape a lot more.

As to why your scrapes are stopping at 10,000, I don't know. Do you have the action logs of the scrape? If yes, send them over, if not submit app logs

HELP > LOGS > SUBMIT

then send your logs ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)

[remiremiremiremi]

Hi Martin,

Thanks for the quick response. Sent my logs, first 4 of the ID# is 3692

[martin@rootjazz]

nothing useful unfortunately

15:55:51: scrapedfollowersof: scraped enough, drop out10024/0

the program is thinking it has scraped enough to meet requested amount. But is not saying what limit is being met. I will have to add some debugging code to figure out what is stopping it.

But, in the mean time, did you check the MAX RESULTS on the settings tab?

[martin@rootjazz]

actually I think your global limit is limiting the results. Please confirm your SETTINGS TAB : MAX SERP RESULTS value