Advice on How to Scrape More Users?
Posted: Thu Jul 05, 2018 7:29 pm
Hi Martin,
I have been using the 'Scrape Tracks: Term or Path' feature now for many months in order to scrape users who have uploaded a track in the last day. I have a .txt file with over 100+ keywords, including all genres, top SoundCloud search terms, and other common music related keywords.
I have been able to consistently scrape about 28,000 unique results daily, of users who have uploaded a track in the last day.
My ideal goal however would be to scrape ALL users who upload tracks in the past 24 hours. I was wondering if you had any advice on how to go about getting more results? SoundCloud states users upload about 12 hours of audio every minute, so there must be many more users uploading than just 28,000 in a 24 hours period.
Just to note; I have my max scrape results set to 99,000 in settings, so I am not being limited by this.
I have been using the 'Scrape Tracks: Term or Path' feature now for many months in order to scrape users who have uploaded a track in the last day. I have a .txt file with over 100+ keywords, including all genres, top SoundCloud search terms, and other common music related keywords.
I have been able to consistently scrape about 28,000 unique results daily, of users who have uploaded a track in the last day.
My ideal goal however would be to scrape ALL users who upload tracks in the past 24 hours. I was wondering if you had any advice on how to go about getting more results? SoundCloud states users upload about 12 hours of audio every minute, so there must be many more users uploading than just 28,000 in a 24 hours period.
Just to note; I have my max scrape results set to 99,000 in settings, so I am not being limited by this.