Rootjazz

Posted: **Fri Oct 11, 2013 4:52 am**

Hi admin
from mixcloud website using this query(http://www.mixcloud.com/search/?mixcloud_query=a)
i get 3819260 username results.
But using your mixcloud bot "search term scrape" option , i got only around 260+ username results with same query(a).

How to scrape all usernames?

thank you

Posted: **Sat Oct 12, 2013 6:58 pm**

I believe this is because there is a max return value. Otherwise scraping a gazillion results would take a fair while and if you are pulling a gazillion pages / 10 (number of tracks per page) you are risking an IP ban or being flagged.

Confirmed, there is a hardcoded value of a maximum of a 100 pages of results, so this *should* give you a the max results per query of a 1000.

Will need to run further tests as to why your query only returns 250+. If could be that mixcloud are not returning results as you are trying to pull too many pages. But I find out and get back to you

Rootjazz

search query option doubt

search query option doubt

mixcloud bot search results max return