Hi admin
from mixcloud website using this query(http://www.mixcloud.com/search/?mixcloud_query=a)
i get 3819260 username results.
But using your mixcloud bot "search term scrape" option , i got only around 260+ username results with same query(a).
How to scrape all usernames?
thank you
search query option doubt
-
martin@rootjazz
- Site Admin
- Posts: 34978
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
mixcloud bot search results max return
I believe this is because there is a max return value. Otherwise scraping a gazillion results would take a fair while and if you are pulling a gazillion pages / 10 (number of tracks per page) you are risking an IP ban or being flagged.
Confirmed, there is a hardcoded value of a maximum of a 100 pages of results, so this *should* give you a the max results per query of a 1000.
Will need to run further tests as to why your query only returns 250+. If could be that mixcloud are not returning results as you are trying to pull too many pages. But I find out and get back to you
Confirmed, there is a hardcoded value of a maximum of a 100 pages of results, so this *should* give you a the max results per query of a 1000.
Will need to run further tests as to why your query only returns 250+. If could be that mixcloud are not returning results as you are trying to pull too many pages. But I find out and get back to you