Hi Martin,
I want to get help on two areas of scraping:
1. How to get tweets of an individual between a particular date range? I have gone to advanced twitter search option and created a query:
URL: https://twitter.com/search?q=(from%3Aso ... 2020-12-01
Search String: (from:soundcloud) until:2020-12-31 since:2020-12-01
and then I went to Twitterdub Scrape Tweet section and tried to paste both the url and the search string but it wanted me to select custom search button and provide a search option. Then I picked Tweet URL but even then it didn't work.
Can you please advise two things. To run this query:
a. Do I put URL or Search String in the tweet scrape option?
b. What should be the custom search option?
2. The first line on the scrape tab is: Scrape Profiles containing Emails. How does this work?? Do we need to put in Profile URL / Profile URLs in a file? If yes, how do we find the profiles with email address? Can you please help us on how progress that functionality.
Really appreciate your help.
Cheers,
Shaily
Scrape Help
- martin@rootjazz
- Site Admin
- Posts: 34701
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
- martin@rootjazz
- Site Admin
- Posts: 34701
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Help
Just the search params explained in the link aboveCan you please advise two things. To run this query:
a. Do I put URL or Search String in the tweet scrape option?
TWEET SEARCHb. What should be the custom search option?
Re: Scrape Help
Hi Martin,
I did exactly that... and that's what I explained above... and it didn't work.
Cheers,
Shaily
I did exactly that... and that's what I explained above... and it didn't work.
Cheers,
Shaily
- martin@rootjazz
- Site Admin
- Posts: 34701
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Help
Logs showing attempt please
HELP > LOGS > SUBMIT
then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)
HELP > LOGS > SUBMIT
then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)
Re: Scrape Help
Hi Martin! Is there a tutorial how to use Scraper function? I can't find anything except the link that you posted above..martin@rootjazz wrote: ↑Mon Feb 22, 2021 10:03 pm Logs showing attempt please
HELP > LOGS > SUBMIT
then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)
- martin@rootjazz
- Site Admin
- Posts: 34701
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Help
cyclop wrote: ↑Fri May 13, 2022 1:11 pmHi Martin! Is there a tutorial how to use Scraper function? I can't find anything except the link that you posted above..martin@rootjazz wrote: ↑Mon Feb 22, 2021 10:03 pm Logs showing attempt please
HELP > LOGS > SUBMIT
then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)
Twitterdub custom search tutorial
- Explains how to build a search and specify how many results per step (optional)
https://www.youtube.com/watch?v=D1W2KnNciPI
Getting Started with Twitterdub (new for 2022)
https://www.youtube.com/watch?v=rVfGB1_ ... swlBFQBn8J
All new tutorials: (new for 2022)
https://www.youtube.com/watch?v=rVfGB1_ ... rQItYJO4bU
I do still have a list of tutorials to make, so can add any new tutorials you want to the list. Until then, explain what you are trying to do, what you don't understand, what is confusing and I can do my best to explain
Regards,
Martin
Re: Scrape Help
Thanks! I have looked those but haven't understood it completely yet.. Sorry if these questions are silly.martin@rootjazz wrote: ↑Mon Feb 22, 2021 10:03 pm I do still have a list of tutorials to make, so can add any new tutorials you want to the list. Until then, explain what you are trying to do, what you don't understand, what is confusing and I can do my best to explain
Regards,
Martin
1) If i have a list of tweets urls. How can i scrape their bodies? (I always got url in the scrape file.. Trying it in "Scrape Tweets" fields)
2) How can I scrape last 10 tweet bodies of a profile?
- martin@rootjazz
- Site Admin
- Posts: 34701
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Help
SCRAPE TABcyclop wrote: ↑Fri May 13, 2022 4:52 pmThanks! I have looked those but haven't understood it completely yet.. Sorry if these questions are silly.martin@rootjazz wrote: ↑Mon Feb 22, 2021 10:03 pm I do still have a list of tutorials to make, so can add any new tutorials you want to the list. Until then, explain what you are trying to do, what you don't understand, what is confusing and I can do my best to explain
Regards,
Martin
1) If i have a list of tweets urls. How can i scrape their bodies? (I always got url in the scrape file.. Trying it in "Scrape Tweets" fields)
SCRAPE TWEETS
check DETAILS, it will get all the details in a CSV, one of those will be the tweet body
With your list of tweet URLS in a file, one per line
CUSTOM SEARCH
input: your_file.txt
STEPS:
ID URL OF TWEET (this also takes a file)
Assuming you have a list of profiles in a file, one per line2) How can I scrape last 10 tweet bodies of a profile?
STEPS:
ID URL PROFILE
USER TWEETS (10)
Then same as above. SCRAPE TAB > SCRAPE TWEETS > OUTPUT: DETAILS
Regards,
Martin