Scrapes not returning many results

Support / help / discussion forum for twitter bot
Post Reply
joeybad47
Posts: 57
Joined: Thu Jun 15, 2017 12:51 am

Scrapes not returning many results

Post by joeybad47 »

Hey,

I am up to the latest version of Twitterdub (2.006), but I am finding the scrape feature is not working nearly as well as it did prior to 2.0. I am only getting about 40 results when I perform the following scrape, whereas before I could get as many results as I wanted virtually, even specifying up to 10,000.

Custom Search > Tweet Search (Recent) > Creator of Tweet

Input Keyword: Youtube
Return Results: URL's

The scrape only returns 40 or so results, when of course I'm sure thousands of people are tweeting 'youtube' every minute.

Getting this error:

No new page URL could be built - ending search

Logs: 29423

Thanks! Totally understand the frustration that must come with a full rewrite of the program :shock:
User avatar
martin@rootjazz
Site Admin
Posts: 34375
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrapes not returning many results

Post by martin@rootjazz »

checking now
User avatar
martin@rootjazz
Site Admin
Posts: 34375
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrapes not returning many results

Post by martin@rootjazz »

ok, twitter is doing something funky to the results to confuse things so the cursor to the next item isn't it the normal location. Will need to do some processing to get around it.

Next update will handle this
joeybad47
Posts: 57
Joined: Thu Jun 15, 2017 12:51 am

Re: Scrapes not returning many results

Post by joeybad47 »

martin@rootjazz wrote: Thu Aug 16, 2018 2:23 pm ok, twitter is doing something funky to the results to confuse things so the cursor to the next item isn't it the normal location. Will need to do some processing to get around it.

Next update will handle this
great, thanks so much :)
Post Reply