Scrape profiles that tweet daily and more requirements

[kjn]

Hi,

Wonderful software, but please fix the sound on the Youtube tutorials, I can't even hear what you say on the 'Twitterdub custom search follow action' video !

However, back to my question:

How can I scrape:
- Accounts that tweet daily
- Which have two or more hashtags in their tweets
- and eventually also: which have a good following / followers rating

If your software can do this kind of stuff, it would be great!

PS: also, do you have a dedicated video on scraping, using files etc?

[martin@rootjazz]

Hi,

Wonderful software, but please fix the sound on the Youtube tutorials, I can't even hear what you say on the 'Twitterdub custom search follow action' video !

My apologies, the sound is quite awful. They really do need making again

However, back to my question:

How can I scrape:
- Accounts that tweet daily

You would need to be more specific. What does daily mean? Posted once a day every day for 100 days, posted yesterday, something else?

- Which have two or more hashtags in their tweets

Any hashtag or specified?

- and eventually also: which have a good following / followers rating

Use the filter tab and you can set min:max followers / followings and the ratio

PS: also, do you have a dedicated video on scraping, using files etc?

Don't think so, just load in a text file, one search term per line and it will bulk create actions /searches one per line.
Or if you want to load in a load of results to be used in a custom search, set custom search first to:
ID / URL / USER




Regards,
Martin

[kjn]

Hi,

Wonderful software, but please fix the sound on the Youtube tutorials, I can't even hear what you say on the 'Twitterdub custom search follow action' video !

My apologies, the sound is quite awful. They really do need making again

That would be great, I am sure I am the first to see them all because not all buttons and input fields are obvious what they mean. You could also put a (?) icon after the fields to explain them in the program itself, that would be super helpful.

However, back to my question:

How can I scrape:
- Accounts that tweet daily

You would need to be more specific. What does daily mean? Posted once a day every day for 100 days, posted yesterday, something else?

I am trying to get Twitter profiles which are active, that means that I have to measure how active they are. If they were active, in let's say the last 6 days, I know that would be a good user to follow/retweet/like in the hopes that such an active user will follow me back and eventually spread the word.

- Which have two or more hashtags in their tweets

Any hashtag or specified?

Any hashtags will do for me, but if it could be specified with wildcards like #*dog* or something it would be even more powerful.

- and eventually also: which have a good following / followers rating

Use the filter tab and you can set min:max followers / followings and the ratio

Thanks!

PS: also, do you have a dedicated video on scraping, using files etc?

Don't think so, just load in a text file, one search term per line and it will bulk create actions /searches one per line.
Or if you want to load in a load of results to be used in a custom search, set custom search first to:
ID / URL / USER

Regards,
Martin

Thanks, I did understand the first line, but the bold part I did not quit get. Could you maybe explain you to do that, maybe with an annotated screenshot if I do not ask to much?


Your support is really good, thanks for coming back that soon!

[martin@rootjazz]

I am trying to get Twitter profiles which are active, that means that I have to measure how active they are. If they were active, in let's say the last 6 days, I know that would be a good user to follow/retweet/like in the hopes that such an active user will follow me back and eventually spread the word.

If you just want to know if they have posted in the last 6 days, use the FILTER: Posted in last(n) days = 6

Any hashtags will do for me, but if it could be specified with wildcards like #*dog* or something it would be even more powerful.

Filter wildcards not supported. If you want to search for specific hashtags, use HASHTAG search.

The program fully supports - advanced search:
https://twitter.com/search-advanced?

So if you can build your complex search there, the program can do it, but is not a good idea to filter in on multiple hashtags as it means you will ignore such a high percentage of results you could get into oversearching blocks. But if you wanted to, you can run a HASHTAG SEARCH on #hash1 then do a filter include on #hash2

As for filter must include (n) hashtags (any) that is not supported.

Thanks, I did understand the first line, but the bold part I did not quit get. Could you maybe explain you to do that, maybe with an annotated screenshot if I do not ask to much?

If you want to bulk create 100 searches from 100 hashtag inputs, you could create a text file with a 100 lines.
On the SEARCH TAB, select HASHTAG search as your search, then input the text file. This will create 100 searches, one for each line.

Lets says you have a text file of 100 profiles and you want to run a complex custom search on those results. You DO NOT WANT 100 indvidual searches, but you want the file to be your step1.

i.e. you want the program to read your file of 100 profiles
from each of those 100 profiles you want to scrape the most recent tweet
from each most recent tweet you want to scrape all likers / retweeters / commenters
and you want it all performed in a SINGLE action

Your custom search would be
ID URL USER
USER TWEETS
NOTES (likers / commenters / retweeters)

In this case, ID URL USER is used a "load file" step.

IF you just create the custom search as:
USER TWEETS
NOTES

The program would loop over the file and create 100 individual actions, one each per line.

Basically if your FIRST step is
ID URL USER
ID URL TWEET

then the file is read WITHIN the search in a single action.

IF you input a file and DO NOT include an ID URL first step then you can 1 action per line of your file



Regards,
Martin

[martin@rootjazz]

Your support is really good, thanks for coming back that soon!

You're welcome. I am here all day every day (except sundays and saturday afternoon). Support is answered every morning / lunch and end of day (GMT-5hrs).

So if you have any questions, any thing you are not sure on (Especially with the tutorial audio issue), any thing doesn't seem to work too well, Please do post up any questions you have.



Regards,
Martin

[kjn]

Thanks! That made much more sense. Only one thing is not clear, you said:

On the SEARCH TAB, select HASHTAG search as your search, then input the text file. This will create 100 searches, one for each line.

Did you mean the Scraper tab? There is no specific hash search. Did you mean the Scrape Profiles input?

Sorry if I sound stupid, just trying to understand the program

[martin@rootjazz]

Thanks! That made much more sense. Only one thing is not clear, you said:

On the SEARCH TAB, select HASHTAG search as your search, then input the text file. This will create 100 searches, one for each line.

Did you mean the Scraper tab?

yes SEARCH tab / SCRAPER tab. IS the same thing

There is no specific hash search. Did you mean the Scrape Profiles input?

Either SCRAPE PROFILES or SCRAPE TWEETS, in the custom search, if you create a simple search with one step HASHTAG SEARCH(recent) (or maybe is called TWEET SEARCH (recent) I don't recall.

Sorry if I sound stupid, just trying to understand the program

Not at all, I am not explaining things well. If there is something you don't understand, you are most likely not the first or the last to be confused, so please do keep asking for clarification if you don't understand something.


But basically, create a scrape tab search action, with a single step: followers_of / hashtag search whatever, set the input to a file and you will bulk create actions, one per line in your file.