Scrape only users with their DM open?
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Scrape only users with their DM open?
Hey, when performing a filtered user search, whether it is purely for scraping purposes or to follow these users, is there any filter that allows me to only scrape "messageable" users and filter out anybody who isn't? Meaning that they have their DMs open and I will be able to message them even if they didn't follow back?
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Re: Scrape only users with their DM open?
I also just saw in the changelog
so it seems possible that there is a "can dm" filter I'm just wondering if its possible to be able to include that as a filter option in the User Filter settings for scraping, following, DMing etc.3.226 13/05/2021
DM action will test for userdetails endpoint to confirm profile can receive DM before attempting
- martin@rootjazz
- Site Admin
- Posts: 34712
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape only users with their DM open?
Makes sense and is a two minute update. I'll add it for next update and let you know when readyeddielp123 wrote: ↑Mon Jun 21, 2021 1:11 pm I also just saw in the changelog
so it seems possible that there is a "can dm" filter I'm just wondering if its possible to be able to include that as a filter option in the User Filter settings for scraping, following, DMing etc.3.226 13/05/2021
DM action will test for userdetails endpoint to confirm profile can receive DM before attempting
Regards,
Martin
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Re: Scrape only users with their DM open?
amazing!!martin@rootjazz wrote: ↑Mon Jun 21, 2021 2:25 pmMakes sense and is a two minute update. I'll add it for next update and let you know when readyeddielp123 wrote: ↑Mon Jun 21, 2021 1:11 pm I also just saw in the changelog
so it seems possible that there is a "can dm" filter I'm just wondering if its possible to be able to include that as a filter option in the User Filter settings for scraping, following, DMing etc.3.226 13/05/2021
DM action will test for userdetails endpoint to confirm profile can receive DM before attempting
Regards,
Martin
- martin@rootjazz
- Site Admin
- Posts: 34712
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Re: Scrape only users with their DM open?
just tried it out, it doesnt seem to work, it will just skip/ignore users even if they're dm-able.
i submitted logs with a test run - all people that are being scraped here are 100% dm-able (and of course in the setting i had the "Can DM" enabled)
but none of them are being scraped and instead itll say * IGNORE: not allowed tro DM
log id 53886
- martin@rootjazz
- Site Admin
- Posts: 34712
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape only users with their DM open?
let me test, code looks correct, maybe the property is not being set by twitter...
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Re: Scrape only users with their DM open?
I tried it out in the follow tool, scrape tool, dm tool, always the same issue unfortunatelymartin@rootjazz wrote: ↑Tue Jun 22, 2021 2:33 pm let me test, code looks correct, maybe the property is not being set by twitter...
- martin@rootjazz
- Site Admin
- Posts: 34712
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape only users with their DM open?
ok, seems like the CAN_DM option only appears on a full user_details endpoint scrape, the program was scraping a less user object and not detecting the property wasn't set.
The next update will fix this. I shall let you know when it is ready.
Regards,
Martin
The next update will fix this. I shall let you know when it is ready.
Regards,
Martin
-
- Posts: 510
- Joined: Tue Feb 11, 2020 2:53 pm
Re: Scrape only users with their DM open?
perfect, thank you so much.martin@rootjazz wrote: ↑Tue Jun 22, 2021 5:38 pm ok, seems like the CAN_DM option only appears on a full user_details endpoint scrape, the program was scraping a less user object and not detecting the property wasn't set.
The next update will fix this. I shall let you know when it is ready.
Regards,
Martin