Page 1 of 1

Scrape pinned tweets?

Posted: Mon Dec 25, 2017 1:39 pm
by Bartekef
Merry christmas everyone and your families!

Hey

I don't expect you to reply to me now, I just wanted to create that topic now as now I do remember about it :lol:

I wonder if is there a way to scrape full text of pinned tweets of selected twitter profiles? (from file or just followers/followings of any profile)

If not, could you please develop such funcionality??

(I want to scrape pinned tweets text + combine it with info of which profile is this pinned tweet taken from)
For example:

Profile julia, https://twitter.com/julia has her pinned tweet: 'Hey I'm Julia! Visit my website xxx to check my photos!'

TwitterDUB gives me processed file in txt:

1. Profile name: Julia; Profile URL: https://twitter.com/julia; Pinned tweet text: 'Hey I'm Julia! Visit my website xxx to check my photos!'
2. Another pinned tweet of another profile
3. Another pinned tweet of another profile
4. (...)

Thank you!


>>

2. Also to scrape Profile details like SCM does it, but to twitter - so I can have twitter profile's BIO's and I can search for my keywords in them (scrape profile info from .txt file of profile URL's)

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 6:29 am
by support@rootjazz
2. Also to scrape Profile details like SCM does it, but to twitter - so I can have twitter profile's BIO's and I can search for my keywords in them (scrape profile info from .txt file of profile URL's)
Please check
https://gyazo.com/6ebb2e521118472a8b945b1672e9c105
1. Profile name: Julia; Profile URL: https://twitter.com/julia; Pinned tweet text: 'Hey I'm Julia! Visit my website xxx to check my photos!'
2. Another pinned tweet of another profile
3. Another pinned tweet of another profile
4. (...)

Thank you!
What do you mean by pinned tweet? I have visited that profile and i see no pinned tweet. Can you please post screenshot? it will help us to identify the thing you are talking about.

Note: I have not followed that account.

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 2:06 pm
by Bartekef
Ah this Julia profile was just an example, not the best though, as it seems it doesnt have pinned tweet. Anyways.

You can pin any tweet of yours, to be displayed on the top of your Twitter page. This way, anyone who visit your profile, see this tweet first:

Image


Information in pinned tweets is valuable for me. I can filter users in the way of manually finding my keywords in their pinned tweets.

For example, on image above: I want to find people who put 'address' phrase in their pinned tweets. I set TwitterDUB to do the following:
1. Scrape pinned tweets text from profile URL's from text file with profile URL's

I open returned text file and manually search for 'address'. If I found that in specific line, I copy that user's URL to another text file, for my further actions.

So it's crucial to have profile url next to it's pinned tweet text; one user with tweet text per line.

The text file, returned by TwitterDUB after performing this action, could look like this:

Image

Then I simply copy and save URLs which have keywords I'm looking for in their pinned tweets, for further actions.

And,

My bad - there is option to actually scrape profile details with bio included. BUT you can only do it from file of Twitter ID's. I need option to do it from Twitter URLs (or a way to bulk turn thousands of URLs to IDs):

Image

... because I scrape them from soundcloud using Scrape Profile Details, and it returns me URLs, not IDs.

--------------------------------

Summary:
1. Need pinned tweet scrapper, to return user profile url and pinned tweet's text, each user and his tweet's text per line (image 1 and 2).
2. Need 'Scrape Profile Details' from URLs or a way to turn URLs to IDs in bulk (image 3).

I hope it's possible. I really, really need that. I've found new way to filter my potential customers to get those who are most likely to buy.
Highly targeted....you understand that's the milestone for me.
I can filter out 1-10% of users, from 10-100k (maybe more) profile URLs this way, which I would contact directly and interact with manually after.

Reviewing 10-100k profiles manually and contacting them daily - or doing the same to 100-1k highly-targeted people - that's a very big difference, esepcially when I need talk to each of them, one on one.

ALSO, I will be annoying with this: viewtopic.php?f=1&t=5485, as this feature allows me to do almost the same with targeting. I need to determine, which users from my .txt files of URLs, have playlists, albums or both. So please, please!

Thank you :)

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 5:47 pm
by martin@rootjazz
My bad - there is option to actually scrape profile details with bio included. BUT you can only do it from file of Twitter ID's. I need option to do it from Twitter URLs (or a way to bulk turn thousands of URLs to IDs):

[s]Should work with profile URLs OR IDs.[/s]

No it doesn't work with URLs, as each URL requires a request to get the ID. I will need to see about adding a URL => ID routine. Remind me in the new year

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 5:48 pm
by martin@rootjazz
am looking into pinned tweets

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 6:08 pm
by Bartekef
martin@rootjazz wrote: Fri Dec 29, 2017 5:48 pm am looking into pinned tweets
Doesn't :(

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 8:09 pm
by martin@rootjazz
pinned tweets not scrapable with API.

probably possible via website scrape though. Will add it to the suggestions list

Re: Scrape pinned tweets?

Posted: Fri Dec 29, 2017 8:58 pm
by Bartekef
martin@rootjazz wrote: Fri Dec 29, 2017 8:09 pm pinned tweets not scrapable with API.

probably possible via website scrape though. Will add it to the suggestions list
Thank you!!!

Re: Scrape pinned tweets?

Posted: Mon Jan 01, 2018 11:04 am
by bitcoin
Interesting strategy. I don't know if I have use for it, but interesting :)

Re: Scrape pinned tweets?

Posted: Thu Jan 11, 2018 1:38 am
by Bartekef
bitcoin wrote: Mon Jan 01, 2018 11:04 am Interesting strategy. I don't know if I have use for it, but interesting :)
Isn't it ? :)