Hey, so I'm doing scrape actions across multiple accounts with different search settings, and while I do have the
"Global unique results for search regardless of account / filters used"
checked, it will still give duplicate results across different search terms / settings.
The only filter I have enabled is "Can DM".
Now my issue is that each of the accounts has a follow limit of 5000. So once that limit is reached i will have to unfollow everybody and start over again, and I'm afraid that I will scrape many people that I have scraped already.
My idea was to put all of them in a list, so that I can add the filter "Exclude if user is in list" to make sure they don't get scraped, as I assume there is no Global Unique Results across different search terms in the scraping tool.
I hope this is somewhat easily understandable. Feel free to let me know if you want me to explain a little better
Filter out people in a specific list from scrape action
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
- martin@rootjazz
- Site Admin
- Posts: 34640
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Filter out people in a specific list from scrape action
Yes, the unique options is per searcheddielp123 wrote: ↑Fri Jul 23, 2021 6:32 pm Hey, so I'm doing scrape actions across multiple accounts with different search settings, and while I do have the
"Global unique results for search regardless of account / filters used"
checked, it will still give duplicate results across different search terms / settings.
so profile_a_followers can return the same results as profile_b_followings. But if you run profile_a_followers search multiple times you won't get the same results
That's correct. What you want to do, isn't really possible at this time. I suppose you could "try" the IGNORE words filter and enter the username of each previous result BUT you will need to use the username token:as I assume there is no Global Unique Results across different search terms in the scraping tool.
username:<input>
As the ignore words function is only for BIO at this time (next update will allow for bio + username however).
But again, I am not sure how this is going to work if you try and paste thousands and thousands of results in.
So it isn't really possible in all honesty
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Re: Filter out people in a specific list from scrape action
Yup exactly and that's the issue because I mostly have to scrape multiple different filters to get to the amount of results that i desiremartin@rootjazz wrote: ↑Fri Jul 23, 2021 9:37 pm
Yes, the unique options is per search
so profile_a_followers can return the same results as profile_b_followings. But if you run profile_a_followers search multiple times you won't get the same results
Well it would definitely be a start and I could definitely test it out. But then again I don't know if it would be an API call each time that it checks for the ignore usernames or if it would be a bot internal process like in the other "IGNORE" processes.martin@rootjazz wrote: ↑Fri Jul 23, 2021 9:37 pm What you want to do, isn't really possible at this time. I suppose you could "try" the IGNORE words filter and enter the username of each previous result BUT you will need to use the username token:
username:<input>
As the ignore words function is only for BIO at this time (next update will allow for bio + username however).
But again, I am not sure how this is going to work if you try and paste thousands and thousands of results in.
So it isn't really possible in all honesty
Is it in no way possible to do something such as the "global processed logs" that work in other tools such as the FOLLOWING tool but for the SCRAPING tool?
Thanks
- martin@rootjazz
- Site Admin
- Posts: 34640
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Filter out people in a specific list from scrape action
Your suggestion has been noted and added to the feature suggestions list. Depending on the demand for the change / feature will alter how quickly it gets implemented.eddielp123 wrote: ↑Fri Jul 23, 2021 10:40 pm
Is it in no way possible to do something such as the "global processed logs" that work in other tools such as the FOLLOWING tool but for the SCRAPING tool?
Thanks