Why it didn't scrape ALL profile details?

Post Reply
Bartekef
Posts: 605
Joined: Thu Sep 22, 2016 12:24 pm

Why it didn't scrape ALL profile details?

Post by Bartekef » Thu Sep 12, 2019 9:57 am

Hey. Could you please take a look at logs and tell my why it didn't scrape profile details of ALL followers of this account?

(account has 3869 followers and I only have profile details of 1657 accounts returned).

In logs I see this:
Got user details: 50226177:trombix11
*FAILED: Not public email:
* FILTER: Exclude item: trombix11
So I guess it's not getting profile details of this account because it has no public email? But I need profile details of every profile, even those without emails (it's only my guess though).

This is how I set this action up:
Image

Logs: 59618

PS. Any way to do this action quicker?

Thank you.

User avatar
martin@rootjazz
Site Admin
Posts: 22573
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Why it didn't scrape ALL profile details?

Post by martin@rootjazz » Thu Sep 12, 2019 3:04 pm

Bartekef wrote:
Thu Sep 12, 2019 9:57 am
Hey. Could you please take a look at logs and tell my why it didn't scrape profile details of ALL followers of this account?

(account has 3869 followers and I only have profile details of 1657 accounts returned).

In logs I see this:
Got user details: 50226177:trombix11
*FAILED: Not public email:
* FILTER: Exclude item: trombix11
So I guess it's not getting profile details of this account because it has no public email? But I need profile details of every profile, even those without emails (it's only my guess though).
Then don't set a filter

Also be aware, when you want FULL DETAILS, a separate request PER result (so for 1000 results you need to 1000 additional requests, at some point IG will say "no that's enough, temp scrape block"

Whether that is 100k or 500k I am not sure, but it will happen sooner or later if you push up the numbers and not supplying enough scrape accounts.

Bartekef
Posts: 605
Joined: Thu Sep 22, 2016 12:24 pm

Re: Why it didn't scrape ALL profile details?

Post by Bartekef » Thu Sep 12, 2019 5:07 pm

martin@rootjazz wrote:
Thu Sep 12, 2019 3:04 pm
Bartekef wrote:
Thu Sep 12, 2019 9:57 am
Hey. Could you please take a look at logs and tell my why it didn't scrape profile details of ALL followers of this account?

(account has 3869 followers and I only have profile details of 1657 accounts returned).

In logs I see this:
Got user details: 50226177:trombix11
*FAILED: Not public email:
* FILTER: Exclude item: trombix11
So I guess it's not getting profile details of this account because it has no public email? But I need profile details of every profile, even those without emails (it's only my guess though).
Then don't set a filter

Also be aware, when you want FULL DETAILS, a separate request PER result (so for 1000 results you need to 1000 additional requests, at some point IG will say "no that's enough, temp scrape block"

Whether that is 100k or 500k I am not sure, but it will happen sooner or later if you push up the numbers and not supplying enough scrape accounts.
But there was no filter tho
And I believe I used like 5 accs for it (might be wrong)

Oh man there was a filter. I'm 99% sure I never set it if I did that was by mistake. Plus it stated " no filter " near "user filter" so that's why I was mistaken.

User avatar
martin@rootjazz
Site Admin
Posts: 22573
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Why it didn't scrape ALL profile details?

Post by martin@rootjazz » Thu Sep 12, 2019 6:18 pm

Bartekef wrote:
Thu Sep 12, 2019 5:07 pm


But there was no filter tho
And I believe I used like 5 accs for it (might be wrong)

There was a filter:
*FAILED: Not public email:
* FILTER: Exclude item: trombix11
Oh man there was a filter. I'm 99% sure I never set it if I did that was by mistake. Plus it stated " no filter " near "user filter" so that's why I was mistaken.
Not sure what happened then, I tested and add a filter it says FILTER SET, remove filter and shows NO FILTER

Bartekef
Posts: 605
Joined: Thu Sep 22, 2016 12:24 pm

Re: Why it didn't scrape ALL profile details?

Post by Bartekef » Thu Sep 12, 2019 8:28 pm

martin@rootjazz wrote:
Thu Sep 12, 2019 6:18 pm
Bartekef wrote:
Thu Sep 12, 2019 5:07 pm


But there was no filter tho
And I believe I used like 5 accs for it (might be wrong)

There was a filter:
*FAILED: Not public email:
* FILTER: Exclude item: trombix11
Oh man there was a filter. I'm 99% sure I never set it if I did that was by mistake. Plus it stated " no filter " near "user filter" so that's why I was mistaken.
Not sure what happened then, I tested and add a filter it says FILTER SET, remove filter and shows NO FILTER
Might be my fault, although I'm 99.9% sure I never touched that. Well, problem solved. Thanks.

Post Reply