Failed to scrape all reported followers:

Post Reply
Bri
Posts: 407
Joined: Fri Nov 13, 2015 10:53 am

Failed to scrape all reported followers:

Post by Bri » Wed Jan 13, 2021 1:54 pm

Failed to scrape all reported followers:

I#m trying to unfollow, tried different options, still the same results...

tried a few times, any ideas?

logs: 74964

User avatar
martin@rootjazz
Site Admin
Posts: 26467
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Failed to scrape all reported followers:

Post by martin@rootjazz » Wed Jan 13, 2021 6:37 pm

Program tries to pull the page, IG returns success but it includes no profiles
13:50:37: Results page: 0
13:50:37: Existing: 3482: Scraped new: 0
13:50:37: Scraped followers data: 3482
13:50:37: Details report: #followers: 19256 but could only scrape: 3482
It appears failed to scrape all followers, cancelling action to avoid unfollowing followers, please run action again
Maybe you have run this too often and IG don't like it.

There should be an option to check each unfollow before it is processed to see if they are a follower, this means you don't need to scrape all your followers before the action, just one for each item is there a check.

If your database is uptodate and in good order, this other method should be fine. Test it with a test unfollow of 5 items, if it all goes fine, you'll be ok

Bri
Posts: 407
Joined: Fri Nov 13, 2015 10:53 am

Re: Failed to scrape all reported followers:

Post by Bri » Wed Jan 13, 2021 11:14 pm

That seems to be working better :lol:

User avatar
martin@rootjazz
Site Admin
Posts: 26467
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Failed to scrape all reported followers:

Post by martin@rootjazz » Thu Jan 14, 2021 4:00 pm

Bri wrote:
Wed Jan 13, 2021 11:14 pm
That seems to be working better :lol:
Just to explain things better incase others find this post.

There are two ways to check you don't unfollow followers
1) scrape all followers and make sure the items to unfollow are not in that list. This works best if you do not have too many followers

2) check before the unfollow, so the program prepares it's list of who to unfollow, then just before unfollowing, it makes a single request "is_this_profile_a_follower". This does mean more requests overall, as if you want to unfollow 100 profiles, it must make 100 individual relationship checks. However, they are spread out and these checks are usually made, so not really an issue, but is a bit slower than (1) due to more checks. Whereas (1) can grab 100 followers per request by scraping the followers endpoint, but for large numbers the issue is, IG doesn't like it and stops returning results for large follower numbers



Regards,
Martin

Bri
Posts: 407
Joined: Fri Nov 13, 2015 10:53 am

Re: Failed to scrape all reported followers:

Post by Bri » Fri Jan 15, 2021 8:47 am

No, because for some reason, when I did try to scrape my followings, that also kept failing (after 3594 ish items, out of 6k), hence the support thread :)


It would be good to allow this feature when also 'do not unfollow if following you' is unchecked because when this option is unchecked, it doesn't allow you to check each follow, it scrapes all the followings before it does the process, do you understand me? because this option also fails me

User avatar
martin@rootjazz
Site Admin
Posts: 26467
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Failed to scrape all reported followers:

Post by martin@rootjazz » Fri Jan 15, 2021 1:18 pm

Bri wrote:
Fri Jan 15, 2021 8:47 am
No, because for some reason, when I did try to scrape my followings, that also kept failing (after 3594 ish items, out of 6k), hence the support thread :)
No what?
IG is not letting you scrape your results, they return an empty page of results instead of continuing to serve the requests. Unsure why. Maybe is a new limit. Maybe they don't like your account (too frequently scraping), maybe something else

It would be good to allow this feature when also 'do not unfollow if following you' is unchecked because when this option is unchecked, it doesn't allow you to check each follow, it scrapes all the followings before it does the process, do you understand me? because this option also fails me
ah you're referring to the CHECK FOLLOWING BEFORE UNFOLLOW, to make sure the database is in in a good state? Let me check on this, I didn't think that scraped all followings, but made a single check...

ajaa
Posts: 2512
Joined: Fri Jun 02, 2017 10:38 pm

Re: Failed to scrape all reported followers:

Post by ajaa » Fri Jan 15, 2021 3:57 pm

martin@rootjazz wrote: Maybe they don't like your account (too frequently scraping)
Lol, that's what ISG do always for us.

Bri
Posts: 407
Joined: Fri Nov 13, 2015 10:53 am

Re: Failed to scrape all reported followers:

Post by Bri » Fri Jan 15, 2021 10:29 pm

ajaa wrote:
Fri Jan 15, 2021 3:57 pm
martin@rootjazz wrote: Maybe they don't like your account (too frequently scraping)
Lol, that's what ISG do always for us.
fucking sucks, I need to grow :lol: :lol: :lol:

Bri
Posts: 407
Joined: Fri Nov 13, 2015 10:53 am

Re: Failed to scrape all reported followers:

Post by Bri » Fri Jan 15, 2021 10:58 pm

martin@rootjazz wrote:
Fri Jan 15, 2021 1:18 pm
Bri wrote:
Fri Jan 15, 2021 8:47 am
No, because for some reason, when I did try to scrape my followings, that also kept failing (after 3594 ish items, out of 6k), hence the support thread :)
No what?
IG is not letting you scrape your results, they return an empty page of results instead of continuing to serve the requests. Unsure why. Maybe is a new limit. Maybe they don't like your account (too frequently scraping), maybe something else
Yes IG hates me :lol:
It would be good to allow this feature when also 'do not unfollow if following you' is unchecked because when this option is unchecked, it doesn't allow you to check each follow, it scrapes all the followings before it does the process, do you understand me? because this option also fails me
ah you're referring to the CHECK FOLLOWING BEFORE UNFOLLOW, to make sure the database is in in a good state? Let me check on this, I didn't think that scraped all followings, but made a single check...

<edit: fixed your terrible quoting :p>

User avatar
martin@rootjazz
Site Admin
Posts: 26467
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Failed to scrape all reported followers:

Post by martin@rootjazz » Tue Jan 19, 2021 6:09 pm

martin@rootjazz wrote:
Fri Jan 15, 2021 1:18 pm

It would be good to allow this feature when also 'do not unfollow if following you' is unchecked because when this option is unchecked, it doesn't allow you to check each follow, it scrapes all the followings before it does the process, do you understand me? because this option also fails me
ah you're referring to the CHECK FOLLOWING BEFORE UNFOLLOW, to make sure the database is in in a good state? Let me check on this, I didn't think that scraped all followings, but made a single check...
ok so have checked, using the CHECK FOLLOWING BEFORE UNFOLLOW it does NOT pull all followings, but performs a check before the unfollow.

Can you show me why you think this is not the case. Include logs showing the program scraping all followings to perform a "is still following check" and a screenshot of the options to create the unfollow action


Regards,
Martin

Post Reply