Hi Admin,
Here is the submit log code: 38654
Here is the screenshot: https://i.imgur.com/CL6YGYF.png
This is not a new issue I am facing with but I know I have not asked before - but I want to increase the limit far beyond 5000 scraping data.
How do I do that? The reason I want to scrap lakhs of data is to analyse it and set it for another tasks like retweet, follow, like, etc.
I need to increase the limit to unlimited. Where is the option or what is the method?
Bot not scraping more than 5000 data?
-
- Posts: 130
- Joined: Sun Feb 09, 2020 3:29 pm
- martin@rootjazz
- Site Admin
- Posts: 34771
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Bot not scraping more than 5000 data?
No idea. But from your logs screenshot cannot tell.chupakabra wrote: ↑Fri Mar 26, 2021 5:39 am I just learnt this: viewtopic.php?f=30&t=9641&p=62031&hilit=5000#p62031
Is that true? Let me know.
What limit did you set for the search?
What search is it?
What is your MAX SEARCH on the SETTINGS tab?
-
- Posts: 130
- Joined: Sun Feb 09, 2020 3:29 pm
Re: Bot not scraping more than 5000 data?
Oh I just found that my Max Scrape is at 5000 under settings. SO if I increase it to like 50000 it will scrape 50k results?
- martin@rootjazz
- Site Admin
- Posts: 34771
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: Bot not scraping more than 5000 data?
Maybe. Twitter may have a limit it will not serve any more results for. I've haven't tried to scrape 50k results for a long time, so cannot say. But set the value as high as you want, and you will get all the results twitter is prepared to send youchupakabra wrote: ↑Sun Mar 28, 2021 6:04 am Oh I just found that my Max Scrape is at 5000 under settings. SO if I increase it to like 50000 it will scrape 50k results?
Regards,
Martin