Page 1 of 1

Advice on How to Scrape More Users?

Posted: Thu Jul 05, 2018 7:29 pm
by joeybad47
Hi Martin,

I have been using the 'Scrape Tracks: Term or Path' feature now for many months in order to scrape users who have uploaded a track in the last day. I have a .txt file with over 100+ keywords, including all genres, top SoundCloud search terms, and other common music related keywords.
I have been able to consistently scrape about 28,000 unique results daily, of users who have uploaded a track in the last day.

My ideal goal however would be to scrape ALL users who upload tracks in the past 24 hours. I was wondering if you had any advice on how to go about getting more results? SoundCloud states users upload about 12 hours of audio every minute, so there must be many more users uploading than just 28,000 in a 24 hours period.

Just to note; I have my max scrape results set to 99,000 in settings, so I am not being limited by this.

Re: Advice on How to Scrape More Users?

Posted: Mon Jul 09, 2018 7:13 pm
by martin@rootjazz
Sorry for late replied. I must have missed this on Thursday / Friday.
SoundCloud states users upload about 12 hours of audio every minute
Soundcloud may lie. More users, more funding they can go for.

But what you doing is probably the best. Scraping via keyword genre / tag. The only way would be more keywords I guess.

Re: Advice on How to Scrape More Users?

Posted: Tue Jul 10, 2018 3:51 pm
by joeybad47
martin@rootjazz wrote: Mon Jul 09, 2018 7:13 pm Sorry for late replied. I must have missed this on Thursday / Friday.
SoundCloud states users upload about 12 hours of audio every minute
Soundcloud may lie. More users, more funding they can go for.

But what you doing is probably the best. Scraping via keyword genre / tag. The only way would be more keywords I guess.
Thanks for your input