Scrape filter didn't work

Ask any support / help / issues / problem or question related to Soundcloud Manager
sunder321
Posts: 74
Joined: Thu Nov 02, 2017 10:55 am

Scrape filter didn't work

Post by sunder321 »

Hi. Why scrape filter doesn'twork? SCM only scrape and save results but didn't filter it. Logs submitted: 88609

Here is standard scrape logs. I set User filter for user
active last day, user has avatar and user is PRO acc but SCM add all users to final result and don't filter it.

Code: Select all

Starting: 30.05.2023 21:35 
Search: https://soundcloud.com/leonardo-pereira-679451162
Max: 20000
Processing search: https://soundcloud.com/leonardo-pereira-679451162 custom_search
Followings Of
Max: 20000 globalmax: 10000
Max Results Wanted: 20000
Setup custom search controller
Custom search run: search: https://soundcloud.com/leonardo-pereira-679451162
Followings Of
Perform custom search: 'Followings Of' #chain/total: 1/1 using: 
One search stage detected, setting default per item per stage value to max: 20000
Start search: Followings Of with: https://soundcloud.com/leonardo-pereira-679451162(20000) using: No account
profile reports followings: 16
Scrape process url :https://api-v2.soundcloud.com/users/371956007/followings?app_version=1685012490&offset=0&client_id=vvWDsm284DNe9CDlRlfb3wuseloIl1RS&limit=25&linked_partitioning=1
Checking filter on: 16 items
Adding: https://soundcloud.com/racionais-mcs-cosa-nostra
Adding: https://soundcloud.com/wl-relikia-2
Adding: https://soundcloud.com/jonas-kj
Adding: https://soundcloud.com/oficialmattd
Adding: https://soundcloud.com/bolinho22
Adding: https://soundcloud.com/reidoparquinho
Adding: https://soundcloud.com/djdgounico
Adding: https://soundcloud.com/yoshi-dj-sucessada
Adding: https://soundcloud.com/djlacosteirojuninho
Adding: https://soundcloud.com/sopagodaooficial
Adding: https://soundcloud.com/guilherme-santos-353705229
Adding: https://soundcloud.com/yoshi-sucessada-unico
Adding: https://soundcloud.com/betuels2
Adding: https://soundcloud.com/skid-row-12
Adding: https://soundcloud.com/xande-de-pilares
Adding: https://soundcloud.com/jonas-gomes-220960605
Total items: 16
scrapedfollowingsof: 16
scrapedfollowingsof: invalid nexturl - drop out: null?client_id=vvWDsm284DNe9CDlRlfb3wuseloIl1RS&app_version=1685012490
Results of search: 16 Followings Of with: https://soundcloud.com/leonardo-pereira-679451162
Handle results: 16 nextstep: 2/1
End of chain: Store results: 16
Finished search: results: 16
Saved: 16 to: C:\Users\38068\AppData\Roaming\rootjazz\Soundcloud Manager\saved\custom_httpssoundcloudcomleonardopereira679451162_2023-05-30.txt
Started: 30.05.2023 21:35 
Finished: 30.05.2023 21:35 
ID: 9150129a-adc8-4c2f-84f4-4a28ace014df
Action ran for: 0hr:0min:6s
And Here is Filter from path logs. I set User filter for user
active last day, user has avatar and user is PRO acc but SCM add all users to final result and don't filter it.

Code: Select all

****************
Starting: 30.05.2023 21:46 
loaded: 2 urls
Fillter Processing: https://soundcloud.com/leonardo-pereira-679451162
Fillter Processing: https://soundcloud.com/fakefruitmusica
* FAILED: 2398472938472938472938 : Bot is null
* FAILED: 2398472938472938472938 : Bot is null
Started: 30.05.2023 21:46 
Finished: 30.05.2023 21:46 
ID: 41961c1a-785f-4255-bffc-756b1e477af3
Action ran for: 0hr:0min:1s
How I can fix this issue?
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape filter didn't work

Post by martin@rootjazz »

let me take a look, can you submit logs, incase they are required

HELP > LOGS > SUBMIT

then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)




Regards,
Martin
sunder321
Posts: 74
Joined: Thu Nov 02, 2017 10:55 am

Re: Scrape filter didn't work

Post by sunder321 »

Logs id: 71457
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape filter didn't work

Post by martin@rootjazz »

More details please, I just tried a TRACK SEARCH with TRACK FILTER and it works fine
TRACK SEARCH get users with ARTIST FILTER also fine


What exact search did you run, probably is an issue with that specific search
sunder321
Posts: 74
Joined: Thu Nov 02, 2017 10:55 am

Re: Scrape filter didn't work

Post by sunder321 »

I use old version of Soundcloud Manager Version: 3.7.1.664 because it works better for me. In this SCM version I didn't use any accs for scraping so I just run scrape users and thats enough. For example I used Filter if I want to filter users from txt file path. In this path I have 1 url to filter. Here is result of filtering:

Code: Select all

Starting: 07.06.2023 04:29 
loaded: 1 urls
Fillter Processing: https://soundcloud.com/didi-o_o/block-gets-hot-like-niger-rip
* FAILED: 3209472389479284 : bot is null
Started: 07.06.2023 04:29 
Finished: 07.06.2023 04:29 
ID: 6320358c-5e6e-4cdf-a2ee-9062c851295a
Action ran for: 0hr:0min:1s
In another example I used Build Custom Search as main tool for user scrape. Here I paste path of track urls and set Users Who Liked a Track - Following of as criteria for searching and also I set Users Is Pro and User is Active in last 1 day in Track Filter and User Filter too. After that in logs you can see that SCM add to filter users but didn't filter all results. I checked all saved results from txt file and search this users manually trough my browser. And I saw that SCM just save all results without filtering because this results doesn't meet my criteria of User is Pro and User is Active in last 1 day.

Code: Select all

Checking filter on: 0 items
Total items: 0
scrapedfollowingsof: 0
scrapedfollowingsof: invalid nexturl - drop out: null?client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&app_version=1685974193
Results of search: 0 Followings Of with: https://soundcloud.com/user-340205889
Handle results: 0 nextstep: 3/2
End of chain: Store results: 0
Handle Result:https://soundcloud.com/user-342277635-183401502  step: 1/2 Result: 1/73
Perform chained search: [1] https://soundcloud.com/user-342277635-183401502
Perform custom search: 'Followings Of' #chain/total: 2/2 using: 
Last Search Step with no per step results specified - attempt max: 20000
Start search: Followings Of with: https://soundcloud.com/user-342277635-183401502(20000) using: No account
Get User ID: https://soundcloud.com/user-342277635-183401502
Resolve Url: 'https://soundcloud.com/user-342277635-183401502' to ID
GET: https://soundcloud.com/user-342277635-183401502
GET RESP: 200
https://soundcloud.com/user-342277635-183401502
Got ID: 1256157001 for: https://soundcloud.com/user-342277635-183401502
profile reports followings: 0
Scrape process url :https://api-v2.soundcloud.com/users/1256157001/followings?app_version=1685974193&offset=0&client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&limit=25&linked_partitioning=1
Checking filter on: 0 items
Total items: 0
scrapedfollowingsof: 0
scrapedfollowingsof: invalid nexturl - drop out: null?client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&app_version=1685974193
Results of search: 0 Followings Of with: https://soundcloud.com/user-342277635-183401502
Handle results: 0 nextstep: 3/2
End of chain: Store results: 0
Handle Result:https://soundcloud.com/user-852230430  step: 1/2 Result: 1/73
Perform chained search: [1] https://soundcloud.com/user-852230430
Perform custom search: 'Followings Of' #chain/total: 2/2 using: 
Last Search Step with no per step results specified - attempt max: 20000
Start search: Followings Of with: https://soundcloud.com/user-852230430(20000) using: No account
Get User ID: https://soundcloud.com/user-852230430
Resolve Url: 'https://soundcloud.com/user-852230430' to ID
GET: https://soundcloud.com/user-852230430
GET RESP: 200
https://soundcloud.com/user-852230430
Got ID: 449643552 for: https://soundcloud.com/user-852230430
profile reports followings: 4
Scrape process url :https://api-v2.soundcloud.com/users/449643552/followings?app_version=1685974193&offset=0&client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&limit=25&linked_partitioning=1
Checking filter on: 4 items
Adding: https://soundcloud.com/user-650642666-62570162
Adding: https://soundcloud.com/charliesib
Adding: https://soundcloud.com/user-98139452
Adding: https://soundcloud.com/murkuk
Total items: 4
scrapedfollowingsof: 4
scrapedfollowingsof: invalid nexturl - drop out: null?client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&app_version=1685974193
Results of search: 4 Followings Of with: https://soundcloud.com/user-852230430
Handle results: 4 nextstep: 3/2
End of chain: Store results: 4
Handle Result:https://soundcloud.com/user-567162682-458995312  step: 1/2 Result: 1/73
Perform chained search: [1] https://soundcloud.com/user-567162682-458995312
Perform custom search: 'Followings Of' #chain/total: 2/2 using: 
Last Search Step with no per step results specified - attempt max: 20000
Start search: Followings Of with: https://soundcloud.com/user-567162682-458995312(20000) using: No account
Get User ID: https://soundcloud.com/user-567162682-458995312
Resolve Url: 'https://soundcloud.com/user-567162682-458995312' to ID
GET: https://soundcloud.com/user-567162682-458995312
GET RESP: 200
https://soundcloud.com/user-567162682-458995312
Got ID: 898908079 for: https://soundcloud.com/user-567162682-458995312
profile reports followings: 7
Scrape process url :https://api-v2.soundcloud.com/users/898908079/followings?app_version=1685974193&offset=0&client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&limit=25&linked_partitioning=1
Checking filter on: 7 items
Adding: https://soundcloud.com/user-101864965
Adding: https://soundcloud.com/tobyjbj
Adding: https://soundcloud.com/user-657292745
Adding: https://soundcloud.com/user-402121258
Adding: https://soundcloud.com/user-353865347-354406082
Adding: https://soundcloud.com/keynan-gaudion
Adding: https://soundcloud.com/theo_green
Total items: 7
scrapedfollowingsof: 7
scrapedfollowingsof: invalid nexturl - drop out: null?client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&app_version=1685974193
Results of search: 7 Followings Of with: https://soundcloud.com/user-567162682-458995312
Handle results: 7 nextstep: 3/2
End of chain: Store results: 7
Handle Result:https://soundcloud.com/didi-o_o  step: 1/2 Result: 1/73
Perform chained search: [1] https://soundcloud.com/didi-o_o
Perform custom search: 'Followings Of' #chain/total: 2/2 using: 
Last Search Step with no per step results specified - attempt max: 20000
Start search: Followings Of with: https://soundcloud.com/didi-o_o(20000) using: No account
Get User ID: https://soundcloud.com/didi-o_o
Resolve Url: 'https://soundcloud.com/didi-o_o' to ID
GET: https://soundcloud.com/didi-o_o
GET RESP: 200
https://soundcloud.com/didi-o_o
Got ID: 1004233591 for: https://soundcloud.com/didi-o_o
profile reports followings: 2
Scrape process url :https://api-v2.soundcloud.com/users/1004233591/followings?app_version=1685974193&offset=0&client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&limit=25&linked_partitioning=1
Checking filter on: 2 items
Adding: https://soundcloud.com/user-30258525
Adding: https://soundcloud.com/merijn-stam
Total items: 2
scrapedfollowingsof: 2
scrapedfollowingsof: invalid nexturl - drop out: null?client_id=AwJSFpPRsaHU9v4TbjcezVAZVRdlvDSF&app_version=1685974193
Results of search: 2 Followings Of with: https://soundcloud.com/didi-o_o
Handle results: 2 nextstep: 3/2
End of chain: Store results: 2
Finished search: results: 344
Saved: 288 to: C:\Users\38068\AppData\Roaming\rootjazz\Soundcloud Manager\saved\custom_httpssoundcloudcomdidiooblockgetshotlikenigerrip_2023-06-07_2.txt
Started: 07.06.2023 04:40 
Finished: 07.06.2023 04:44 
ID: 3aca5a95-4f41-4393-aac5-2caf8d613795
Action ran for: 0hr:4min:29s
You can also check some results url from this logs in browser and you will see that not all users meet searching criteria. SCM in my case just save all results without filtering.
sunder321
Posts: 74
Joined: Thu Nov 02, 2017 10:55 am

Re: Scrape filter didn't work

Post by sunder321 »

Latest logs id: 23802 Submitted
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape filter didn't work

Post by martin@rootjazz »

sunder321 wrote: Wed Jun 07, 2023 2:01 am Latest logs id: 23802 Submitted
Sorry, you will need to update to the latest version and see if that fixes it. I am not able to apply fixes to an old version
sunder321
Posts: 74
Joined: Thu Nov 02, 2017 10:55 am

Re: Scrape filter didn't work

Post by sunder321 »

Ok. But I can't select accounts to perform scraping in the new version because SCM crashed after I selected accounts. Here is logs of this problem in updated version LOGS ID: 39298 SUBMITTED
sunder321
Posts: 74
Joined: Thu Nov 02, 2017 10:55 am

Re: Scrape filter didn't work

Post by sunder321 »

Any solution?
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Scrape filter didn't work

Post by martin@rootjazz »

sunder321 wrote: Wed Jun 07, 2023 11:32 pm Ok. But I can't select accounts to perform scraping in the new version because SCM crashed after I selected accounts. Here is logs of this problem in updated version LOGS ID: 39298 SUBMITTED
open app
goto scraper tab
select accounts
...

crash? Is that it?
Post Reply