Having some trouble with the search scraper

Ask any support / help / issues / problem or question related to Soundcloud Manager
PrismaPhonic
Posts: 66
Joined: Thu Dec 03, 2015 5:36 am

Having some trouble with the search scraper

Post by PrismaPhonic »

Ok, changed this post complete because suddenly this feature is now working for me. Scrape followers of seems to now be working correctly, even though it wasn't for a bit. not sure why? but good now. in the process of testing though I encountered a different bug that does seem to be presistent. Read below

EDIT:
Also, one more thing I tested and keep getting an error on. In Scrape lists I tried creating a custom search: Latest track of > Users who commented on track and select the Result type as "Profiles." I then paste in the URL for the artists page and hit search. This is what happens when I do that:
Processing search: custom_search
* ERROR: Object reference not set to an instance of an object.
* UNKNOWN ERROR OCCURRED - see logs for more info.
If the problem continues, please submit logs to support.
Object reference not set to an instance of an object.

Object reference not set to an instance of an object.
PrismaPhonic
Posts: 66
Joined: Thu Dec 03, 2015 5:36 am

Re: Having some trouble with the search scraper

Post by PrismaPhonic »

Alright, more problems I'm encountering.

Seems that if I run "Scrape Followers Of" in the Scrape Lists pane I get nothing passing my filter at all, and in fact it only compares a few followers before stopping. For example, I run the test on artist 20syl whom has a sizeable following of 167,000 followers.
Processing search: https://soundcloud.com/20syl scrape_followers
* FAILED: Track number filter: >1 <300 (0)
* IGNORE: Failed Filter: https://soundcloud.com/beatniix
* FAILED: Track number filter: >1 <300 (0)
* IGNORE: Failed Filter: https://soundcloud.com/djbish-noize
* FAILED: Followers number filter: >30 <1000 (6)
* IGNORE: Failed Filter: https://soundcloud.com/wynford-hanson
* FAILED: Followers number filter: >30 <1000 (6)
* IGNORE: Failed Filter: https://soundcloud.com/wynford-hanson
0: saved scraped followers list to
You'll also notice at the end that it compared the same user against my filter twice. Now, if I use this same filter in the "Follow" pane on "Follow Followers of" I end up getting tons of results of followers who pass my filter. But it seems to be broken in the scrape lists pane?

Also, it should be noted that this process ran VERY slowly

EDIT: I've submitted my logs. ID 13598
User avatar
martin@rootjazz
Site Admin
Posts: 34712
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Having some trouble with the search scraper

Post by martin@rootjazz »

sorry, I am getting a bit confused in what you are saying.


If you can note your issues completely separately - as clearly as possible

ISSUE 1
SCRAPER tab
scrape followers of box
enter value: https://soundcloud.com/example
set filter: min = 1 max: 100
problem is: 0 results
logs to show issue
<logs>
submitted logs id <id>


ISSUE2
.....


That way there is no confusion at all
User avatar
martin@rootjazz
Site Admin
Posts: 34712
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Having some trouble with the search scraper

Post by martin@rootjazz »

in my test

scrape tab
scrape followers of
with filter


all processes correctly

https://soundcloudmanager.com/updatetesting.html
PrismaPhonic
Posts: 66
Joined: Thu Dec 03, 2015 5:36 am

Re: Having some trouble with the search scraper

Post by PrismaPhonic »

ISSUE 1
Scrape Lists tab
Scrape Followers of Box
enter value: https://soundcloud.com/20syl
set Artist Filter: min track 1, max track 300, min followers 30, max followers 1,000

problem is: 0 results. File is created but nothing is in the file at all. Furthermore, it appears that SCM does compare a few followers of the artist but stops after only a few comparisons.

Logs:
Processing search: https://soundcloud.com/20syl scrape_followers
* FAILED: Track number filter: >1 <300 (0)
* IGNORE: Failed Filter: https://soundcloud.com/beatniix
* FAILED: Track number filter: >1 <300 (0)
* IGNORE: Failed Filter: https://soundcloud.com/djbish-noize
* FAILED: Followers number filter: >30 <1000 (6)
* IGNORE: Failed Filter: https://soundcloud.com/wynford-hanson
* FAILED: Followers number filter: >30 <1000 (6)
* IGNORE: Failed Filter: https://soundcloud.com/wynford-hanson
0: saved scraped followers list to
PrismaPhonic
Posts: 66
Joined: Thu Dec 03, 2015 5:36 am

Re: Having some trouble with the search scraper

Post by PrismaPhonic »

ISSUE 2
Scrape Lists tab
Custom Search Box
Custom Search Parameters: Latest track of > Users who commented on track.
Set results to “Profiles”
enter value: https://soundcloud.com/20syl
set Artist Filter: min track 1, max track 300, min followers 30, max followers 1,000

problem is: process fails.

Logs:
Processing search: custom_search
* ERROR: Object reference not set to an instance of an object.
* UNKNOWN ERROR OCCURRED - see logs for more info.
If the problem continues, please submit logs to support.
Object reference not set to an instance of an object.

Object reference not set to an instance of an object.
User avatar
martin@rootjazz
Site Admin
Posts: 34712
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Having some trouble with the search scraper

Post by martin@rootjazz »

PrismaPhonic wrote:ISSUE 2

Processing search: custom_search
* ERROR: Object reference not set to an instance of an object.
* UNKNOWN ERROR OCCURRED - see logs for more info.
If the problem continues, please submit logs to support.
Object reference not set to an instance of an object.

Object reference not set to an instance of an object.
HELP > LOGS > SUBMIT

then send your logs ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)




I will test all issues now - thanks for detailing, you don't know how much easier it is to debug from a nice bullet list than a sprawling block of text.


Did you also update before testing?


https://soundcloudmanager.com/updatetesting.html
User avatar
martin@rootjazz
Site Admin
Posts: 34712
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Having some trouble with the search scraper

Post by martin@rootjazz »

issue 1 should already be fixed in latest update

issue 2 confirmed - fix ready in a few hours
User avatar
martin@rootjazz
Site Admin
Posts: 34712
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Having some trouble with the search scraper

Post by martin@rootjazz »

issue2 fixed - update soon
Post Reply