Scrape pinned tweets?

Support / help / discussion forum for twitter bot
Post Reply
Bartekef
Posts: 677
Joined: Thu Sep 22, 2016 12:24 pm

Scrape pinned tweets?

Post by Bartekef »

Merry christmas everyone and your families!

Hey

I don't expect you to reply to me now, I just wanted to create that topic now as now I do remember about it :lol:

I wonder if is there a way to scrape full text of pinned tweets of selected twitter profiles? (from file or just followers/followings of any profile)

If not, could you please develop such funcionality??

(I want to scrape pinned tweets text + combine it with info of which profile is this pinned tweet taken from)
For example:

Profile julia, https://twitter.com/julia has her pinned tweet: 'Hey I'm Julia! Visit my website xxx to check my photos!'

TwitterDUB gives me processed file in txt:

1. Profile name: Julia; Profile URL: https://twitter.com/julia; Pinned tweet text: 'Hey I'm Julia! Visit my website xxx to check my photos!'
2. Another pinned tweet of another profile
3. Another pinned tweet of another profile
4. (...)

Thank you!


>>

2. Also to scrape Profile details like SCM does it, but to twitter - so I can have twitter profile's BIO's and I can search for my keywords in them (scrape profile info from .txt file of profile URL's)
support@rootjazz
Posts: 2317
Joined: Tue Feb 02, 2016 1:58 pm

Re: Scrape pinned tweets?

Post by support@rootjazz »

2. Also to scrape Profile details like SCM does it, but to twitter - so I can have twitter profile's BIO's and I can search for my keywords in them (scrape profile info from .txt file of profile URL's)
Please check
https://gyazo.com/6ebb2e521118472a8b945b1672e9c105
1. Profile name: Julia; Profile URL: https://twitter.com/julia; Pinned tweet text: 'Hey I'm Julia! Visit my website xxx to check my photos!'
2. Another pinned tweet of another profile
3. Another pinned tweet of another profile
4. (...)

Thank you!
What do you mean by pinned tweet? I have visited that profile and i see no pinned tweet. Can you please post screenshot? it will help us to identify the thing you are talking about.

Note: I have not followed that account.
Bartekef
Posts: 677
Joined: Thu Sep 22, 2016 12:24 pm

Re: Scrape pinned tweets?

Post by Bartekef »

Ah this Julia profile was just an example, not the best though, as it seems it doesnt have pinned tweet. Anyways.

You can pin any tweet of yours, to be displayed on the top of your Twitter page. This way, anyone who visit your profile, see this tweet first:

Image


Information in pinned tweets is valuable for me. I can filter users in the way of manually finding my keywords in their pinned tweets.

For example, on image above: I want to find people who put 'address' phrase in their pinned tweets. I set TwitterDUB to do the following:
1. Scrape pinned tweets text from profile URL's from text file with profile URL's

I open returned text file and manually search for 'address'. If I found that in specific line, I copy that user's URL to another text file, for my further actions.

So it's crucial to have profile url next to it's pinned tweet text; one user with tweet text per line.

The text file, returned by TwitterDUB after performing this action, could look like this:

Image

Then I simply copy and save URLs which have keywords I'm looking for in their pinned tweets, for further actions.

And,

My bad - there is option to actually scrape profile details with bio included. BUT you can only do it from file of Twitter ID's. I need option to do it from Twitter URLs (or a way to bulk turn thousands of URLs to IDs):

Image

... because I scrape them from soundcloud using Scrape Profile Details, and it returns me URLs, not IDs.

--------------------------------

Summary:
1. Need pinned tweet scrapper, to return user profile url and pinned tweet's text, each user and his tweet's text per line (image 1 and 2).
2. Need 'Scrape Profile Details' from URLs or a way to turn URLs to IDs in bulk (image 3).

I hope it's possible. I really, really need that. I've found new way to filter my potential customers to get those who are most likely to buy.
Highly targeted....you understand that's the milestone for me.
I can filter out 1-10% of users, from 10-100k (maybe more) profile URLs this way, which I would contact directly and interact with manually after.

Reviewing 10-100k profiles manually and contacting them daily - or doing the same to 100-1k highly-targeted people - that's a very big difference, esepcially when I need talk to each of them, one on one.

ALSO, I will be annoying with this: viewtopic.php?f=1&t=5485, as this feature allows me to do almost the same with targeting. I need to determine, which users from my .txt files of URLs, have playlists, albums or both. So please, please!

Thank you :)
User avatar
martin@rootjazz
Site Admin
Posts: 34361
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape pinned tweets?

Post by martin@rootjazz »

My bad - there is option to actually scrape profile details with bio included. BUT you can only do it from file of Twitter ID's. I need option to do it from Twitter URLs (or a way to bulk turn thousands of URLs to IDs):

[s]Should work with profile URLs OR IDs.[/s]

No it doesn't work with URLs, as each URL requires a request to get the ID. I will need to see about adding a URL => ID routine. Remind me in the new year
User avatar
martin@rootjazz
Site Admin
Posts: 34361
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape pinned tweets?

Post by martin@rootjazz »

am looking into pinned tweets
Bartekef
Posts: 677
Joined: Thu Sep 22, 2016 12:24 pm

Re: Scrape pinned tweets?

Post by Bartekef »

martin@rootjazz wrote: Fri Dec 29, 2017 5:48 pm am looking into pinned tweets
Doesn't :(
User avatar
martin@rootjazz
Site Admin
Posts: 34361
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape pinned tweets?

Post by martin@rootjazz »

pinned tweets not scrapable with API.

probably possible via website scrape though. Will add it to the suggestions list
Bartekef
Posts: 677
Joined: Thu Sep 22, 2016 12:24 pm

Re: Scrape pinned tweets?

Post by Bartekef »

martin@rootjazz wrote: Fri Dec 29, 2017 8:09 pm pinned tweets not scrapable with API.

probably possible via website scrape though. Will add it to the suggestions list
Thank you!!!
bitcoin
Posts: 924
Joined: Tue Jul 04, 2017 1:25 am

Re: Scrape pinned tweets?

Post by bitcoin »

Interesting strategy. I don't know if I have use for it, but interesting :)
Bartekef
Posts: 677
Joined: Thu Sep 22, 2016 12:24 pm

Re: Scrape pinned tweets?

Post by Bartekef »

bitcoin wrote: Mon Jan 01, 2018 11:04 am Interesting strategy. I don't know if I have use for it, but interesting :)
Isn't it ? :)
Post Reply