scraping results

Post Reply
zapper_18
Posts: 20
Joined: Wed Mar 14, 2018 5:19 am

scraping results

Post by zapper_18 » Fri Jul 13, 2018 2:54 am

Hi I am wondering if i set the max scrape results to 90000 and I am doing follow followers of user X which have followers 90k. Will instadub really scrape all the followers or is there actually a limit? Because I feel like instadub reach the max scraped too soon. thanks

User avatar
martin@rootjazz
Site Admin
Posts: 17728
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: scraping results

Post by martin@rootjazz » Fri Jul 13, 2018 1:20 pm

I think there is a limit, but not sure what it is.

However, for some accounts, ISG will only show 7k followers. I worked through an issue with another user, who couldl only scrape 7k of their own followers. Yet I could scrape all 36k with my account. A bit of searching discovered this is something ISG do for no known reason. Rather than page all results to go 200 at a time, they dump 7k on you and will not allow any more paging to get more.

As to the 90k, I would ask you to think - do you need them? Are you able to use them?
In some cases, the answer is yes. In other cases, the answer is no. I have a theory for the above mentioned 7k limit it is caused by too many followers scrapes, so it is best to scrape the amount of results you need and not needlessy grab loads just because you can

Post Reply