Couldn't pull page: 403

Discussions to do with Soundcloud Manager. Do not use for support, use the dedicated support forum for help requests
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Couldn't pull page: 403

Post by martin@rootjazz »

Seems scraper tab wasn't enforcing you to select accounts, next update it will.


Test here all working. Try again with accounts, if it still doesn't work, specify the search running and submit logs showing it

HELP > LOGS > SUBMIT

then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)
bmsietta
Posts: 45
Joined: Wed Mar 04, 2020 11:33 pm

Re: Couldn't pull page: 403

Post by bmsietta »

martin@rootjazz wrote: Tue Feb 21, 2023 1:44 pm
bmsietta wrote: Mon Feb 20, 2023 2:46 pm is there any way this scraper can use proxies?
scrape requires accounts, proxies are just for email scrape from profile page. But for scraping, you should be specifying hte accounts to use
really? I trust you more than almost anyone on the internet but when I access profiles from a window that's not logged in, I can still see the info I need (links and bio)

because the other scraper will only scrape the links if there is an email in it. makes sense that it should be able to still scrape the links if an email is not there
bmsietta
Posts: 45
Joined: Wed Mar 04, 2020 11:33 pm

Re: Couldn't pull page: 403

Post by bmsietta »

the problem is that accounts are very expensive and considering i need to scrape 300k records I will need to spend at least 500 accs = 250$ while backconnect proxies are 15$ a month
bmsietta
Posts: 45
Joined: Wed Mar 04, 2020 11:33 pm

Re: Couldn't pull page: 403

Post by bmsietta »

update: scraping links works now, fixed by itself.

still think it would be very helpful to Not need accounts since I can view the data while not logged in
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Couldn't pull page: 403

Post by martin@rootjazz »

bmsietta wrote: Tue Feb 21, 2023 2:51 pm
martin@rootjazz wrote: Tue Feb 21, 2023 1:44 pm
bmsietta wrote: Mon Feb 20, 2023 2:46 pm is there any way this scraper can use proxies?
scrape requires accounts, proxies are just for email scrape from profile page. But for scraping, you should be specifying hte accounts to use
really? I trust you more than almost anyone on the internet but when I access profiles from a window that's not logged in, I can still see the info I need (links and bio)

There was a change recently, you went from being able to scrape unlimited numbers to only a few before being stopped by SC server, hence the update enforcing accounts (apparently implemented badly)

because the other scraper will only scrape the links if there is an email in it. makes sense that it should be able to still scrape the links if an email is not there
as the email scraper is just pulling the profile page, it can be done via proxies. it's not a search
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Couldn't pull page: 403

Post by martin@rootjazz »

bmsietta wrote: Fri Feb 24, 2023 2:55 pm update: scraping links works now, fixed by itself.
What search are you running?
still think it would be very helpful to Not need accounts since I can view the data while not logged in
sure, but searching doesn't work without accounts so...
bmsietta
Posts: 45
Joined: Wed Mar 04, 2020 11:33 pm

Re: Couldn't pull page: 403

Post by bmsietta »

what I want to scrape is this data:
https://i.imgur.com/0DVZxx7.png

using this scraper:
https://i.imgur.com/tkKh62n.png

this data is available without having to log in. buying accs is more inconvenient and expensive than using proxies. could it be done using only proxies? if you use backconnect proxies you have a pool of 100k + IPs so it should do the trick.
these proxies work perfectly with this scraper:
https://i.imgur.com/oxUZPvy.png

which seems to pull the same data but only for accounts that have email.
I want to scrape also accounts that don't have email but social links
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Couldn't pull page: 403

Post by martin@rootjazz »

bmsietta wrote: Mon Feb 20, 2023 2:46 pm is there any way this scraper can use proxies? I loaded my proxies but it ends up scraping with my home IP, thus getting blocked after 300 requests. i find it easier than using accounts
Is it possible to add that functionality, sure, can the app do it now? No


The suggestion has been noted and added to the feature suggestions list. Depending on the demand for the change / feature will alter how quickly it gets implemented.

As others have asked for the option to prioritise their feature, I'm just letting you know (it's by no means compulsory) if the feature is very important to you, we do offer the chance to sponsor the feature to get it developed ASAP. If interested write in with your budget (it's simple, the more you pay bigger the priority will be :-) for further discussion: support[at]rootjazz[dot]com
Post Reply