Custom search fails

[janojebo]

I have added tweet url , set custom search tweet interactions and getting results 0 also tried with retweetedby same results
logs 99465

Code: Select all

Perform custom search
Setup custom search controller
Custom search run: search: https://twitter.com/***
Perform custom search: #chain/total: 1/1
Start search: Tweet Interactions with: https://***
Extract ID from URL: 625**40 : https://twi*
Extract ID from URL: 625**90240 : https://twitter.c*
Results of search: Tweet Interactions with: https://twitter.*
End of chain: Store results: 16
* ERROR: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Results: 0
Saving to: C:\Users\*\rootjazz\Twitterdub\saved_data\search_ht**90240_2015-07-27_6.txt

Next tried insetr usename url, custom search recenttweetsof+tweetinteractions results:

Code: Select all

* ERROR:  Term: 625538737452240899 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625538468714815488
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625538468714815488
Results of search: Tweet Interactions with: 625538468714815488
End of chain: Store results: 17
* ERROR:  Term: 625538468714815488 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625533800827764736
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625533800827764736
Results of search: Tweet Interactions with: 625533800827764736
End of chain: Store results: 18
* ERROR:  Term: 625533800827764736 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625533452616642560
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625533452616642560
Results of search: Tweet Interactions with: 625533452616642560
End of chain: Store results: 15
* ERROR:  Term: 625533452616642560 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625532135022161920
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625532135022161920
Results of search: Tweet Interactions with: 625532135022161920
End of chain: Store results: 18
* ERROR:  Term: 625532135022161920 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625530631716474880
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625530631716474880
Results of search: Tweet Interactions with: 625530631716474880
End of chain: Store results: 18
* ERROR:  Term: 625530631716474880 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625530380167327744
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625530380167327744
Results of search: Tweet Interactions with: 625530380167327744
End of chain: Store results: 14
* ERROR:  Term: 625530380167327744 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625529827785883648
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625529827785883648
Results of search: Tweet Interactions with: 625529827785883648
End of chain: Store results: 16
* ERROR:  Term: 625529827785883648 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625529759880118272
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625529759880118272
Results of search: Tweet Interactions with: 625529759880118272
End of chain: Store results: 22
* ERROR:  Term: 625529759880118272 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625525322277109760
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625525322277109760
Results of search: Tweet Interactions with: 625525322277109760
End of chain: Store results: 14
* ERROR:  Term: 625525322277109760 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625522007971635200
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625522007971635200
Results of search: Tweet Interactions with: 625522007971635200
End of chain: Store results: 19
* ERROR:  Term: 625522007971635200 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625518122259324928
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625518122259324928
* ERROR:  Term: 625518122259324928 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625516460174090240
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625516460174090240
Results of search: Tweet Interactions with: 625516460174090240
End of chain: Store results: 15
* ERROR:  Term: 625516460174090240 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625512795069652992
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625512795069652992
Results of search: Tweet Interactions with: 625512795069652992
End of chain: Store results: 0
* ERROR:  Term: 625512795069652992 Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Perform chained search: [1] 625512639221866496
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625512639221866496
* ERROR:  Term: 625512639221866496 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625512599233372160
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625512599233372160
* ERROR:  Term: 625512599233372160 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625512524885172224
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625512524885172224
* ERROR:  Term: 625512524885172224 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625512198744506368
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625512198744506368
* ERROR:  Term: 625512198744506368 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625510972485160960
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625510972485160960
* ERROR:  Term: 625510972485160960 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625510379771326464
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625510379771326464
* ERROR:  Term: 625510379771326464 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625509750407622656
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625509750407622656
* ERROR:  Term: 625509750407622656 Exception of type 'System.OutOfMemoryException' was thrown.
Perform chained search: [1] 625509631578767361
Perform custom search: #chain/total: 2/2
Start search: Tweet Interactions with: 625509631578767361
* Cancelled: 
Results: 0
Saving to: C:\Users\*\AppData\Roaming\rootjazz\Twitterdub\saved_d*-07-27_2.txt

after some time i have stopped the process because it was consuming 800+MB ram.
Im not sure if im doing something wrong or its bug.
I want to get same as on TJ scrape users from notes but since twitter have likes and retweet separated what i need to setup to get both users ? tweet interactions (if not what are tweet interactions)?

[martin@rootjazz]

Please include the exact search parameters you used, including the initial tweet URL

[martin@rootjazz]

fixed - next update

[janojebo]

Updated to 1.110 (http://rootjazz.com/twitterdub/updatetesting.html you forgot to change number there is in changelog 2x 1.109)
Scrape is working now but im getting always only 16 results. i have set 3k max results also on global max scrape results i have 150k
tweet link https://twitter.com/SexualGif/status/625892213013278720 almost 1k rewteets and 2k likes(favorites)
settings


also tried retweetedby option only but same results

Sample of results

Code: Select all

Perform custom search
Setup custom search controller
Custom search run: search: https://twitter.com/SexualGif/status/625892213013278720
Perform custom search: #chain/total: 1/1
Start search: RetweetedBy with: https://twitter.com/SexualGif/status/625892213013278720
Extract ID from URL: 625892213013278720 : https://twitter.com/SexualGif/status/625892213013278720
Results of search: RetweetedBy with: https://twitter.com/SexualGif/status/625892213013278720
End of chain: Store results: 16
StoreResults: checking: 16
Stored: 16
Total Results so far: 16 / 16
Results: 16
Saving to: C:\Users\*\AppData\Roaming\rootjazz\Twitterdub\saved_data\search_httpstwittercomsexualgifstatus625892213013278720_2015-07-28_9.txt

Asking again what i need to setup in custom search to get users who liked(favorited) and retweeted tweet?

[martin@rootjazz]

Tweet interactions doesn't work well, this is tweet not providing the full information. They will not give the likers at all. Retweeters is also limited in how many they reply, seems quite random.

Go to the tweet page, click retweeters, twitter itself only shows 26


Similar issue for repliers. Twitter doesn't make that information public, so have to try and work it out by requesting all mentions, then checking for tweet ID. Which doesn't always return the replies for the requested tweet due to request limits.


Let me see if something better can be done. But at best, the only information that can be pulled is from the website. Likers / retweeters / repliers cannot be more than what is shown on the website - ie. about 20 or so for each

[janojebo]

Ah i see thanks for information. I hope these will be way to make this possible.

[martin@rootjazz]

added webpage scraping. Must you will get is about 120+ items. That is all there is on the webpage - replies / likers / retweeters

next update

[martin@rootjazz]

http://rootjazz.com/twitterdub/updatetesting.html




Regards,
Martin