Hello,
I've scraped many tweets using this software and one thing I've noticed is how difficult it can be to work with the scraped tweets.
Here's the biggest challenge I noticed. Scraped Tweets that are multi-lined come out ill formatted,
If I tweeted something like
Roses are red.
Violets are blue.
The Output may either come out as:
Roses are red.Violets are blue.
Or
Roses are red.
Violets are blue.
Since the updates, I've noticed its more the latter than the former. It maybe it was easier while the former since i could just enter it into a google doc & edit it via spell check.
The problem is when there are multiple tweets, we can't distinguish when a new line or tweet is established from the scraped tweet data.
I suggest an improvement to this function, as I find this essential for myself & others, to meet the modern style of tweeting where many include [several] line breaks into tweets. In other words: Keep the Original Tweet Format when Scraping
I could see this in multiple ways:
- Twitterdub Tweet format where the <br> is included to signal line breaks
- Some kind of list function where the start of tweets have a number or a dash before them, like how these two lines are functioned.
Scrape Tweet Function Improvements Suggested For Multilined Tweets
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Tweet Function Improvements Suggested For Multilined Tweets
Can you let me know which exact function you are using that causes the bad formatting output.
Re: Scrape Tweet Function Improvements Suggested For Multilined Tweets
From the Scrape function, the "Scrape body" function-> the one next to "Scrape Tweets"
In the past, "Scrape Tweets" wouldn't produce what I sought no matter which selection I made between "id" "url" & "detail" but the body did.
The "Scrape Body" function is where I experience what I describe in the OP
In the past, "Scrape Tweets" wouldn't produce what I sought no matter which selection I made between "id" "url" & "detail" but the body did.
The "Scrape Body" function is where I experience what I describe in the OP
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Tweet Function Improvements Suggested For Multilined Tweets
\r and \n are replaced in the results. I cannot see how it is possible to get a single tweet saved over two lines, it doesn't happen in my tests and code appears to handle it correctly.
Please provide example searches you run, along with the results file and logs showing this issue.
For future reference, please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information
https://rootjazz.com/forum/viewtopic.php?f=23&t=1634
Regards,
Martin
Please provide example searches you run, along with the results file and logs showing this issue.
For future reference, please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information
https://rootjazz.com/forum/viewtopic.php?f=23&t=1634
Regards,
Martin
Re: Scrape Tweet Function Improvements Suggested For Multilined Tweets
Input: User Tweets.martin@rootjazz wrote: ↑Thu Jul 18, 2019 10:15 pm \r and \n are replaced in the results. I cannot see how it is possible to get a single tweet saved over two lines, it doesn't happen in my tests and code appears to handle it correctly.
Please provide example searches you run, along with the results file and logs showing this issue.
For future reference, please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information
https://rootjazz.com/forum/viewtopic.php?f=23&t=1634
Regards,
Martin
Input username
Cap the scrape at 500 results
No other filters
Scrape Body
Results: Sent via PM
Logs: 2197
I'll work on utilizing that more often!
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Tweet Function Improvements Suggested For Multilined Tweets
What usernames?bmanfacts wrote: ↑Fri Jul 19, 2019 10:04 pmInput: User Tweets.martin@rootjazz wrote: ↑Thu Jul 18, 2019 10:15 pm \r and \n are replaced in the results. I cannot see how it is possible to get a single tweet saved over two lines, it doesn't happen in my tests and code appears to handle it correctly.
Please provide example searches you run, along with the results file and logs showing this issue.
For future reference, please, read through this post and submit your issue accordingly. Not everything will be relevant to your issue, but have a read as it gives an idea of the information you can provide to help me help you as quickly as possible without delays in having to request additional information
https://rootjazz.com/forum/viewtopic.php?f=23&t=1634
Regards,
Martin
Input username
Which appears wrong to you? I scrolled the save file, but without knowing what you think is wrong, had to guess which lineResults: Sent via PM
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Scrape Tweet Function Improvements Suggested For Multilined Tweets
nevermind, figured it out. The next update will fix this. I shall let you know when it is ready.
Regards,
Martin
Regards,
Martin
- martin@rootjazz
- Site Admin
- Posts: 34706
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact: