following numbers way down

[dariush90025]

Hi, I've set SCM to follow the followers of a user with 16K followers. I set SCM to follow 225 a day. It's been processing this job since the 19th. The first couple of days it did all 225. The last two days it only did 15 to 20 a day. Here's the processing info for the last day. How can I fix this? I assume it's some kind of issue with my settings but then I can't see why it would work fine the first two days then malfunction if settings were the problem.

Follow: nigel-popoola
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/nigel-popoola with: malkovichmusic
Made: 5 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: nigel-popoola (successes: 6 fails: 0)
Total Followed: 6/225
Pausing: 3
Made: 6 of limit: 225: action: Follow: using: malkovichmusic
Follow: cy18291
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/cy18291 with: malkovichmusic
Made: 6 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: cy18291 (successes: 7 fails: 0)
Total Followed: 7/225
Pausing: 1
Made: 7 of limit: 225: action: Follow: using: malkovichmusic
Follow: sa-dew
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/sa-dew with: malkovichmusic
Made: 7 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: sa-dew (successes: 8 fails: 0)
Total Followed: 8/225
Pausing: 2
Made: 8 of limit: 225: action: Follow: using: malkovichmusic
Follow: djitruth
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/djitruth with: malkovichmusic
Made: 8 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: djitruth (successes: 9 fails: 0)
Total Followed: 9/225
Pausing: 2
Made: 9 of limit: 225: action: Follow: using: malkovichmusic
Follow: claudiosnd
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/claudiosnd with: malkovichmusic
Made: 9 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: claudiosnd (successes: 10 fails: 0)
Total Followed: 10/225
Pausing: 2
Made: 10 of limit: 225: action: Follow: using: malkovichmusic
Follow: vicinglis
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/vicinglis with: malkovichmusic
Made: 10 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: vicinglis (successes: 11 fails: 0)
Total Followed: 11/225
Pausing: 3
Made: 11 of limit: 225: action: Follow: using: malkovichmusic
Follow: user-205644317
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/user-205644317 with: malkovichmusic
Made: 11 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: user-205644317 (successes: 12 fails: 0)
Total Followed: 12/225
Pausing: 2
Made: 12 of limit: 225: action: Follow: using: malkovichmusic
Follow: gtjam
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/gtjam with: malkovichmusic
Made: 12 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: gtjam (successes: 13 fails: 0)
Total Followed: 13/225
Pausing: 2
Made: 13 of limit: 225: action: Follow: using: malkovichmusic
Follow: florian-smola
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/florian-smola with: malkovichmusic
Made: 13 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: florian-smola (successes: 14 fails: 0)
Total Followed: 14/225
Pausing: 2
Made: 14 of limit: 225: action: Follow: using: malkovichmusic
Follow: pierreanomalymusic
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/pierreanomalymusic with: malkovichmusic
Made: 14 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: pierreanomalymusic (successes: 15 fails: 0)
Total Followed: 15/225
Pausing: 2
Made: 15 of limit: 225: action: Follow: using: malkovichmusic
Follow: nate-mojica-1
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/nate-mojica-1 with: malkovichmusic
Made: 15 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: nate-mojica-1 (successes: 16 fails: 0)
Total Followed: 16/225
Pausing: 3
Made: 16 of limit: 225: action: Follow: using: malkovichmusic
Follow: courtney-franklin-770001888
Play Before: False Download Before: False Like Played: False Repost Played: False Follow Played: False
Follow: https://soundcloud.com/courtney-franklin-770001888 with: malkovichmusic
Made: 16 of limit: 225: action: Follow: using: malkovichmusic
* SUCCESS: Followed: courtney-franklin-770001888 (successes: 17 fails: 0)
Total Followed: 17/225
Pausing: 2
Processed: 17
NumSuccess: 17
NumFiltered: 0
NumFailed: 0
Successes: https://soundcloud.com/pissyrogers,http ... -770001888

Fails:
17 users followed
Finished processing
Repeat every: 24
Scheduled to: 11/23/2015 10:57 AM

[martin@rootjazz]

so you are scraping the followers of, then following, repeating every 24 hours?

what version are you on?

2.x

or the latest 3.x branch?


If on teh 2.x branch, this is the cause. You are scraping 225 items from this search, then trying to process (checking those 225 and ignoring if already followed - your earlier logs should show this than the ones you posted

the 3.x branch will perform that check DURING the search so you always get 225 items to process.


Another way would be to
a) scrape ALL items first (use the scrape tab)
b) follow from list (which contains the 8k max results

then those skipped from previously processed would not be an issue

[dariush90025]

I'm using 3.0.3.7.
I just put the name of the desired account into Follow Followers Of then sent that to the Processor using the above mentioned settings.
Am I supposed to be scraping all the followers first? If so, then what's the point of the Follow Followers Of function?

[martin@rootjazz]

Run your action as normal, when it completes (or if has completed recently) submit your logs

HELP > LOGS > SUBMIT

then send your logs ID - the first 4 numbers is sufficient (displayed after successful uploading of logs)





Maybe more information contained there. This shouldn't be an issue on the v3

[martin@rootjazz]

I'm using 3.0.3.7.
Am I supposed to be scraping all the followers first? If so, then what's the point of the Follow Followers Of function?

No, in v2 there was the issue as described above, shouldn't occur in v3, which was a redesign to incorporate all new features added in the last 2+years of development which lead to an ad-hoc flow and led to the issue as mentioned

[dariush90025]

I see. So any suggestions on what may be causing my problem?

[martin@rootjazz]

I see. So any suggestions on what may be causing my problem?

Not until you do as requested and submit your logs

[arcan210]

Are you always using the same user when you do the search/follow? SoundCloud limits the number of results that it will return, and will generally always return the same list when you search. It sounds like you may just be running out of users in that list, even though there are still thousands more users left on the target.

Try changing to a different user and see if your problem persists. If numbers increase with a new target, then you know you are running up against a soundcloud limit.

here's another thing you can try. Put that user's URL in the "scrape followers of" box on the scrape tab, and press the recursive button. That should scrape a nice heavy list of profile URLS. Then feed that list into the search/follow box rather than having SCM try to scrape a new list every time.

Does any of that make sense? Hope it's helpful.

Cheers

Arcan

[dariush90025]

Yeah, one account in particular has a lot of followers that I know would follow me so I was focused on that one. I just did what you said regarding scraping with Recursive. Per Recursive is set to 100. Is that good? How long will the scraper take, keeping in mind there's about 15K followers? Where do I find the resulting scrape file?

[martin@rootjazz]

you cannot scrape more than 8000 from ANY SINGLE SEARCH, if the user has 2million followers, you will only get 8000. This is a soundcloud.com limit NOT SCM


Recursive means

scrape 100 userA followers
now scrape a 100 followers from the followers of userA
now scrape a 100 followers from the follows of the followers
now scrape a 100 followers from the follows of the followers of the followers

.......

until you hit the max results limit - so is basically unlimited (until SC ban you for scraping too much - around 500,000 page pulls ime