Scraping Profile Under Twitter Search (people) and (recent)

[papamax]

I have been trying without success to scrape profiles from the People search on twitter like below:



Isn't there a way to do that? I can't find any suitable option on Custom Search, nothing that will give a Profile URL as direct output for a search.

----

Additionally, a second issue is when I try to scrape profiles using tweet search(recent / video / img) using custom search to output creator of Tweet like below:



Here is what I have on the scraper page, note that there is no user filter and the account performing the search is open for sensitive content and even posts it.



It gives pretty much no results back when there is literally hundreds available:



I have seen this same issue multiple times before as well when trying to scrape tweets from some user... When they have 1000's of tweets in the account and the search also shows endless numbers if I scroll down the scraper give me back 40-50 items when I am requesting 500 for example.

Needless to say I believe a scraper tutorial is in order, tool has been in the market for a long time and there isn't a single one. I believe that would help a lot of users and decrease the questions you answer here.

Thanks

[martin@rootjazz]

I don't think PEOPLE is supported actually

what you are doing is scraping the RECENT tab, and getting the profiles that posted the tweets


As for why it isn't working, don't know will need to test. I won't lie, I am massively backlogged at the moment due to kids on school holidays and I'm on my own.


You can look at the ADVANCED SEARCH help page:
viewtopic.php?f=30&t=6985&p=66649#p66649

and try and figure out why it isn't working yourself, or wait for me to get round to it (will be a few days, sorry)

[papamax]

Any updates on the issue of scraping profiles using tweet search(recent) using custom search to output creator of Tweet like explained above?

I have tried figuring it out and testing a lot of different settings, it is always the same issue... I think it has to do with NSFW account. It generally works with regular accounts.

Thanks

[martin@rootjazz]

Any updates on the issue of scraping profiles using tweet search(recent) using custom search to output creator of Tweet like explained above?

working fine here

Start search: Tweet Search (recent) with: fitness min_faves:50 -filter:replies(100) using: <user>
Search: fitness min_faves:50 -filter:replies with: <user>: max100 return: RETURN_URLS
tweets: 24
Search pages: 1
Paused for 4secs
Results: 24
tweets: 23
Search pages: 2
Paused for 4secs
Results: 47
tweets: 24
Search pages: 3
Paused for 5secs
Results: 71

STEPS
tweet search (recent)
creator

input: fitness min_faves:50 -filter:replies

I guess it still doesn't work for you? try a different account maybe? What happens then?

[papamax]

I figured something out here...

Most of my issues were using searches related to NSFW terms.

Here is what fixed is, you may want to add it to the software if the user selects sensitive content on filter:



then



and



The HIDE SENSITIVE tick box on Search Settings MUST be unchecked for it to work... And it is checked by default! Even if the account is NSFW set and even if the time line is enabled to Sensitive content... Search settings are a separate setting.

Thanks

[martin@rootjazz]

thanks for the info. I thought the site settings NSFW changed that... Let me check on that and see why it isn't

[JACKX25]

How can make it automatically for bulk accounts by (Twitter dub - Bot) pls ?
How can i make all accounts IN Search section
"changes to buttons OFF (hide sensitive) ?
There is a place to click in bot for filtre that section in The BOT ?

(Bcs i looking about it in tutorial i can't find)

[martin@rootjazz]

How can make it automatically for bulk accounts by (Twitter dub - Bot) pls ?

Sorry I do not understand your question can you explain please

How can i make all accounts IN Search section

On the SEARCH tab in the ACCOUNTS SELECTION form, bulk select accounts, by using shift / cntrl keys for normal multiple selection

"changes to buttons OFF (hide sensitive) ?

UPDATE ACCOUNTS tab

There is a place to click in bot for filtre that section in The BOT ?

Again, sorry I don't understand the question can you explain please

[JACKX25]

i mean I'm interested in (nsfw post) and in twitter can't see that (nsfw posts) "Only" if you make FILTRE by (search) settings, for me i do it manually and that take time and bored when i do it one by one , so i want know if there is an option how can i select all accounts i login in twitterdub and make all filter accounts in (Hide sensitive) as pictures below..

[martin@rootjazz]

ok seems Twitter have changed how this work. Will get it updated