Hi Rootjazz
With twitterdub I am wondering if the program can spot topics/keywords if they begin trending and then run a few automated commands or would I need to use twitterdub with another program (ala scrapebox) to get this working?
Here are my needs.
Twitterdub would need to spot certain keywords and/or hashtags when they begin trending and retweet the trending topic and scrape any links to see if the webpages have twitter tweet functions on them and tweet it out.
So for example, if the keyword black friday started trending could Twitterdub automatically see which accounts were tweeting? And let's say it's a store like Macy’s or Amazon, could twitterdub then scrape the website the tweet links to, look for a twitter retweet function and compose a tweet and send it out? Or could twitterdub simply retweet the Amazon/Macy's tweet?
Or would I need scrapebox do the scraping part and have twitterdub compose the tweet and retweet?
Thanks
Can twitterdub track trending topics, then scrape and retweet?
- martin@rootjazz
- Site Admin
- Posts: 34388
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Can twitterdub track trending topics, then scrape and retweet?
You can access trending tags on a per country basis using the trend hash token.
As for "spotting" trends, the program doesn't. But if you can define what you mean in steps, the program could be made to do it. But there needs to be some instructions on what you mean by "spot trending".
As for "spotting" trends, the program doesn't. But if you can define what you mean in steps, the program could be made to do it. But there needs to be some instructions on what you mean by "spot trending".
Re: Can twitterdub track trending topics, then scrape and retweet?
How does this access trending tags and can I define which tags to look for?martin@rootjazz wrote: ↑Mon Nov 18, 2019 12:44 pm You can access trending tags on a per country basis using the trend hash token.
I ran into a program recently that could "spot" when a tag is trending by using what it called a twitter stream agent. That combined with a peak detector, it looks at the twitter stream constantly for when a term begins to "peak" or trend. Then it spits out an email to the user. In this case I would want twitterdub to perform an action as described in the op.martin@rootjazz wrote: ↑Mon Nov 18, 2019 12:44 pm As for "spotting" trends, the program doesn't. But if you can define what you mean in steps, the program could be made to do it. But there needs to be some instructions on what you mean by "spot trending".
Here is a link to a video here that explains it better: https://vimeo.com/tectonic/huginn-introduction. Take a look at about 2;09 in.
- martin@rootjazz
- Site Admin
- Posts: 34388
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Can twitterdub track trending topics, then scrape and retweet?
Log into your twitter account in teh website, top right on the home page (I think) it shows "trending hashtags for your region".lsberic wrote: ↑Mon Nov 18, 2019 7:07 pmHow does this access trending tags and can I define which tags to look for?martin@rootjazz wrote: ↑Mon Nov 18, 2019 12:44 pm You can access trending tags on a per country basis using the trend hash token.
There is nothing to specify. You can access those trending tags for your country (as specified in settings).
It is using the twitter API stream endpoint. TD does NOT use the API so cannot perform this. Plus, unless the streaming endpoint has changed since I looked at it, I would doubt it is actually detecting trends, just returning what it finds on the stream connected to based on some rules set by the program. As I said, it needs to be defined exactly to have the program do it.I ran into a program recently that could "spot" when a tag is trending by using what it called a twitter stream agent. That combined with a peak detector, it looks at the twitter stream constantly for when a term begins to "peak" or trend. Then it spits out an email to the user. In this case I would want twitterdub to perform an action as described in the op.martin@rootjazz wrote: ↑Mon Nov 18, 2019 12:44 pm As for "spotting" trends, the program doesn't. But if you can define what you mean in steps, the program could be made to do it. But there needs to be some instructions on what you mean by "spot trending".
Here is a link to a video here that explains it better: https://vimeo.com/tectonic/huginn-introduction. Take a look at about 2;09 in.
watch this endpoint
when a word is mention (n) times in (y) minutes, mark it as trending. But is that actually trending, or just something spammed out?
For actual trends, use the trending tags. Twitter is telling you these are currently trending.
Re: Can twitterdub track trending topics, then scrape and retweet?
Ok, I'll take a look at it. I assume a tag can be a keyword. Once that's discovered, can twitterdub perform an action like:martin@rootjazz wrote: ↑Mon Nov 18, 2019 9:06 pm
For actual trends, use the trending tags. Twitter is telling you these are currently trending.
- send out a tweet regarding that keyword or tag.
- retweet tweets that contain the keyword or tag.
- scrape tweet that contain the keyword/tag and look for links to webpages, then find the tweet function on the webpage and send out a tweet?
- martin@rootjazz
- Site Admin
- Posts: 34388
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Can twitterdub track trending topics, then scrape and retweet?
yes, just include the token in your tweet text. The form instructions should list itlsberic wrote: ↑Tue Nov 19, 2019 4:03 pmOk, I'll take a look at it. I assume a tag can be a keyword. Once that's discovered, can twitterdub perform an action like:martin@rootjazz wrote: ↑Mon Nov 18, 2019 9:06 pm
For actual trends, use the trending tags. Twitter is telling you these are currently trending.
- send out a tweet regarding that keyword or tag.
- retweet tweets that contain the keyword or tag.
Code: Select all
#trend_hash#
So you want to scrape tweets that contain the #trend_hash#scrape tweet that contain the keyword/tag and look for links to webpages, then find the tweet function on the webpage and send out a tweet?[/list]
then scrape links contained (if contained) in that tweet
then??? I get confused here, you wrote " then find the tweet function on the webpage"
So go to the link contained in the tweet and find the tweet function on that webpage? That does not make sense (to me anyway)
Eitherway, no the program cannot extract URLs from tweets and perform anything based on that webpage, so I guess whatever it is you want to do the program cannot do it
Re: Can twitterdub track trending topics, then scrape and retweet?
Hey Martin,martin@rootjazz wrote: ↑Wed Nov 20, 2019 1:43 pm So you want to scrape tweets that contain the #trend_hash#
then scrape links contained (if contained) in that tweet
then??? I get confused here, you wrote " then find the tweet function on the webpage"
So go to the link contained in the tweet and find the tweet function on that webpage? That does not make sense (to me anyway)
Thanks for indulging me in this thread.
So, what I was referring to is a function similar to scrapebox which has a social plugin. With that plugin, scrapebox can look at a set of links and find the social urls on a given webpage.
That function is displayed here. https://youtu.be/S8RxlzdBAjU
I'm not sure why a program could not scrape the links in a tweet, but I don't know a lot about how twitter links are formulated on the backend.martin@rootjazz wrote: ↑Wed Nov 20, 2019 1:43 pm
Eitherway, no the program cannot extract URLs from tweets and perform anything based on that webpage, so I guess whatever it is you want to do the program cannot do it
But just so I am clear, using the twitter trends tag, I can automate twitterdub to monitor when keyword is trending and set it to tweet a new message or like or retweet an existing twitter post automatically?
- martin@rootjazz
- Site Admin
- Posts: 34388
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Can twitterdub track trending topics, then scrape and retweet?
I meant, it could if i coded it to do it. But I haven't so it cannot. Not that it is impossible.lsberic wrote: ↑Wed Nov 20, 2019 5:11 pmHey Martin,martin@rootjazz wrote: ↑Wed Nov 20, 2019 1:43 pm So you want to scrape tweets that contain the #trend_hash#
then scrape links contained (if contained) in that tweet
then??? I get confused here, you wrote " then find the tweet function on the webpage"
So go to the link contained in the tweet and find the tweet function on that webpage? That does not make sense (to me anyway)
Thanks for indulging me in this thread.
So, what I was referring to is a function similar to scrapebox which has a social plugin. With that plugin, scrapebox can look at a set of links and find the social urls on a given webpage.
That function is displayed here. https://youtu.be/S8RxlzdBAjU
I'm not sure why a program could not scrape the links in a tweet, but I don't know a lot about how twitter links are formulated on the backend.martin@rootjazz wrote: ↑Wed Nov 20, 2019 1:43 pm
Eitherway, no the program cannot extract URLs from tweets and perform anything based on that webpage, so I guess whatever it is you want to do the program cannot do it
But I don't see the point, why do you want scrape social URLs contained in a webpage that was containined in a tweet.
yes you can use the token #trend_hash# as input to your searches.But just so I am clear, using the twitter trends tag, I can automate twitterdub to monitor when keyword is trending and set it to tweet a new message or like or retweet an existing twitter post automatically?
So you can scrape recent tweets containing #trend_hash# and like them
Re: Can twitterdub track trending topics, then scrape and retweet?
Hi Martin,martin@rootjazz wrote: ↑Thu Nov 21, 2019 12:28 am
I meant, it could if i coded it to do it. But I haven't so it cannot. Not that it is impossible.
But I don't see the point, why do you want scrape social URLs contained in a webpage that was containined in a tweet.
Thanks for the clarification.
But the main thing is to be able grab trends sorted by keywords and then being able to retweet useful information in an automated fashion. If twitterdub cannot scrape web pages linked in that tweet, that's fine.
I was hoping there was a function like scrapebox social plugin so that if I had that additional data I could make use of it.
- martin@rootjazz
- Site Admin
- Posts: 34388
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Can twitterdub track trending topics, then scrape and retweet?
Sorry for not understanding, but I don't understand what you are trying to do
1) get a list of trending topics
2) manually create tweets for each trending hashtag
3) schedule those curated tweets to post out that day?
a) scraping tweets
b) checking for web links
c) scraping weblinks and looking for links back to social sites.
I cannot think of any use that would be. If I am going to add a feature, I need to understand it and agree that it will be beneficial to multiple users or can be promoted as a unique feature that could drum up more sales.
So you want toBut the main thing is to be able grab trends sorted by keywords and then being able to retweet useful information in an automated fashion.
1) get a list of trending topics
2) manually create tweets for each trending hashtag
3) schedule those curated tweets to post out that day?
Again, maybe I just don't understand you correctly. But I cannot see any benefit toIf twitterdub cannot scrape web pages linked in that tweet, that's fine.
I was hoping there was a function like scrapebox social plugin so that if I had that additional data I could make use of it.
a) scraping tweets
b) checking for web links
c) scraping weblinks and looking for links back to social sites.
I cannot think of any use that would be. If I am going to add a feature, I need to understand it and agree that it will be beneficial to multiple users or can be promoted as a unique feature that could drum up more sales.