Rootjazz

Hey,

The email scraper in SCM randomly stopped working for me this morning. I've tried from multiple PC's on different networks, with/without proxies. I keep getting this error message: * FAILED: 87345 : Couldn't scrape profile ID

I'm scraping from users who've uploaded a track based on a certain search term, which I usually do successfully everyday. This problem just started happening randomly, not after a particular update or anything; so maybe this was due to a change done by SoundCloud?

* ERRROR: building profile details of: https://soundcloud.com/davemell: Error converting value {null} to type 'System.Int32'. Path 'reposts_count', line 1, position 715.
Failed pulled details for: https://soundcloud.com/davemell
* FAILED: 87345 : Couldn't scrape profile ID

Update went out earlier today

https://soundcloudmanager.com/updatetesting.html

Great, thanks

Hi, sorry if i should make a new topic, but im scraping emails, and its not finishing the whole amount, am i missing something? see screen shot



I also have another processing running, and its still going, been going for 14 hours + lol

if you are trying to scrape 1.5 million posts you can expect an IP ban from Soundcloud. No one has a reasonable use case for pulling 1.5 million pages. That is why there is a proxy option to specify proxies to spilt the page requests up.


But even with proxies, it will take a while, that is a lot of pages to pull

Ok, just some compete with all, and some dont, thanks for the reply.

What do you think it won't compete eventually?

Complete*

some BIG scapes finish with all results, and some dont.

Don't suppose you still have the logs?

Or you performed them within the last 7 days so you could submit logs?

Could be somewhere in my logs folder. But on with other stuff now, keep up the amazing work. I love your stuff. Glad you never started the vine bot. Haha. Vine = R.I.P.....