Following issue

Support / help / discussion forum for twitter bot
Post Reply
Danny000
Posts: 130
Joined: Mon Jan 15, 2018 12:19 am

Following issue

Post by Danny000 »

Hi there

The Follow tab is kind of weird:

For a small test, like a few users follow dozens, with limited threads (3-5 threads), it works well.

BUt when scale it up, 1000 users following, 40 threads, TD will got lots errors, problems, skip, and failed.

OS: win server 2102 R2
TD: v1.441


Logs submitted: 70043

Part of the Realtime log:
Follow: https://twitter.com/Ariss13489080 using: megan_white2338
* IGNORE: https://twitter.com/2pk8u
PROBLEM: user returned null: JeyyYyej
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/JeyyYyej
PROBLEM: user returned null: Reyajah7
Checking for twitter API error...
PROBLEM: user returned null: MadhoshL
Checking for twitter API error...
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
PROBLEM: user returned null: MELANIE_MELON1
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/Reyajah7
* FAILED to get ID from : https://twitter.com/MadhoshL
PROBLEM: user returned null: Jonatha29579569
Checking for twitter API error...
PROBLEM: user returned null: DickLoversGrdr
Checking for twitter API error...
* IGNORE: https://twitter.com/JeyyYyej
PROBLEM: user returned null: Ariss13489080
Checking for twitter API error...
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* FAILED to get ID from : https://twitter.com/MELANIE_MELON1
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* FAILED to get ID from : https://twitter.com/Jonatha29579569
* FAILED to get ID from : https://twitter.com/DickLoversGrdr
* FAILED to get ID from : https://twitter.com/Ariss13489080
* IGNORE: https://twitter.com/Reyajah7
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/MadhoshL
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/MELANIE_MELON1
* IGNORE: https://twitter.com/Jonatha29579569
* IGNORE: https://twitter.com/DickLoversGrdr
* IGNORE: https://twitter.com/Ariss13489080
Get TwitterUser: 1002875760951087104 using: purnomo8963
*TWITTER ERROR: <empty>
* FAILED: 678353 : Failed to auth account: oa: 987231722990223362-3BGwykljCnHKHWe0TKi29B4mD1pSQA7 secret: qeYizLJ9dbEUKQfHvxf6TQsaYB8WOAYNjfU0owCKg3MI8 un: ClintForbis6686 Err: <empty>
* FAILED : https://twitter.com/BuwaLuvlLugcoza user: ClintForbis6686
Follow: https://twitter.com/gzg39009461 using: Weaver7192Julie
Failed: 2 items in a row
Check for pause: 60,90
Pausing for: 75
Follow: https://twitter.com/its_syarif using: william57035453
PROBLEM: user returned null: its_syarif
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/its_syarif
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/its_syarif
PROBLEM: user returned null: gzg39009461
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/gzg39009461
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/gzg39009461
* ERROR: verify action: Bad Gateway - Twitter is down or being upgraded.
* ERROR Info: <html><head><title>502 Bad Gateway</title></head>
<body><h2>502 Bad Gateway</h2><h3>Failed to connect parent proxy</h3></body></html>

* FAILED : https://twitter.com/abae13marseillr user: purnomo8963
Failed: 3 items in a row
Check for pause: 60,90
Pausing for: 63
*TWITTER ERROR: <empty>
* FAILED: 678353 : Failed to auth account: oa: 986289351490322432-INWhIM3OAecb7QWYto87z4bOIunPKrQ secret: 86IghbJPvGkdWFcRnQod2C4RrICDmxQFBBLAsFnhpES6m un: Love5233Melissa Err: <empty>
* FAILED : https://twitter.com/Ayyip3 user: Love5233Melissa
Failed: 4 items in a row
Check for pause: 60,90
Setting:

Image
support@rootjazz
Posts: 2317
Joined: Tue Feb 02, 2016 1:58 pm

Re: Following issue

Post by support@rootjazz »

Danny000 wrote: Tue Jun 05, 2018 12:45 am Hi there

The Follow tab is kind of weird:

For a small test, like a few users follow dozens, with limited threads (3-5 threads), it works well.

BUt when scale it up, 1000 users following, 40 threads, TD will got lots errors, problems, skip, and failed.

OS: win server 2102 R2
TD: v1.441


Logs submitted: 70043

Part of the Realtime log:
Follow: https://twitter.com/Ariss13489080 using: megan_white2338
* IGNORE: https://twitter.com/2pk8u
PROBLEM: user returned null: JeyyYyej
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/JeyyYyej
PROBLEM: user returned null: Reyajah7
Checking for twitter API error...
PROBLEM: user returned null: MadhoshL
Checking for twitter API error...
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
PROBLEM: user returned null: MELANIE_MELON1
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/Reyajah7
* FAILED to get ID from : https://twitter.com/MadhoshL
PROBLEM: user returned null: Jonatha29579569
Checking for twitter API error...
PROBLEM: user returned null: DickLoversGrdr
Checking for twitter API error...
* IGNORE: https://twitter.com/JeyyYyej
PROBLEM: user returned null: Ariss13489080
Checking for twitter API error...
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* FAILED to get ID from : https://twitter.com/MELANIE_MELON1
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* FAILED to get ID from : https://twitter.com/Jonatha29579569
* FAILED to get ID from : https://twitter.com/DickLoversGrdr
* FAILED to get ID from : https://twitter.com/Ariss13489080
* IGNORE: https://twitter.com/Reyajah7
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/MadhoshL
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/MELANIE_MELON1
* IGNORE: https://twitter.com/Jonatha29579569
* IGNORE: https://twitter.com/DickLoversGrdr
* IGNORE: https://twitter.com/Ariss13489080
Get TwitterUser: 1002875760951087104 using: purnomo8963
*TWITTER ERROR: <empty>
* FAILED: 678353 : Failed to auth account: oa: 987231722990223362-3BGwykljCnHKHWe0TKi29B4mD1pSQA7 secret: qeYizLJ9dbEUKQfHvxf6TQsaYB8WOAYNjfU0owCKg3MI8 un: ClintForbis6686 Err: <empty>
* FAILED : https://twitter.com/BuwaLuvlLugcoza user: ClintForbis6686
Follow: https://twitter.com/gzg39009461 using: Weaver7192Julie
Failed: 2 items in a row
Check for pause: 60,90
Pausing for: 75
Follow: https://twitter.com/its_syarif using: william57035453
PROBLEM: user returned null: its_syarif
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/its_syarif
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/its_syarif
PROBLEM: user returned null: gzg39009461
Checking for twitter API error...
* FAILED to get ID from : https://twitter.com/gzg39009461
* SKIP: Exception of type 'LibUtil.ExSkipItem' was thrown.
* IGNORE: https://twitter.com/gzg39009461
* ERROR: verify action: Bad Gateway - Twitter is down or being upgraded.
* ERROR Info: <html><head><title>502 Bad Gateway</title></head>
<body><h2>502 Bad Gateway</h2><h3>Failed to connect parent proxy</h3></body></html>

* FAILED : https://twitter.com/abae13marseillr user: purnomo8963
Failed: 3 items in a row
Check for pause: 60,90
Pausing for: 63
*TWITTER ERROR: <empty>
* FAILED: 678353 : Failed to auth account: oa: 986289351490322432-INWhIM3OAecb7QWYto87z4bOIunPKrQ secret: 86IghbJPvGkdWFcRnQod2C4RrICDmxQFBBLAsFnhpES6m un: Love5233Melissa Err: <empty>
* FAILED : https://twitter.com/Ayyip3 user: Love5233Melissa
Failed: 4 items in a row
Check for pause: 60,90
Setting:

Image


Thanks for the required data submission. I couldn't replicate the issue as i don't have that much accounts and proxy.
I am forwarding the log number and the thread to admin for better investigation. He will reply back here after investigation. Please be patient.



Regards
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Following issue

Post by martin@rootjazz »

14:17:26: Bad Gateway - Twitter is down or being upgraded.
502

0 - <html><head><title>502 Bad Gateway</title></head>
<body><h2>502 Bad Gateway</h2><h3>Failed to connect parent proxy</h3></body></html>
how many accounts are on unique proxies?

BUt when scale it up, 1000 users following, 40 threads, TD will got lots errors, problems, skip, and failed.
can you check it on 3-4 threads, does that work ok?
Danny000
Posts: 130
Joined: Mon Jan 15, 2018 12:19 am

Re: Following issue

Post by Danny000 »

martin@rootjazz wrote: Tue Jun 05, 2018 2:55 pm
14:17:26: Bad Gateway - Twitter is down or being upgraded.
502

0 - <html><head><title>502 Bad Gateway</title></head>
<body><h2>502 Bad Gateway</h2><h3>Failed to connect parent proxy</h3></body></html>
how many accounts are on unique proxies?

BUt when scale it up, 1000 users following, 40 threads, TD will got lots errors, problems, skip, and failed.
can you check it on 3-4 threads, does that work ok?
2 accounts per static residential proxy.
3-4 threads work fine.

To me it seems more like problem of thread number.
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: Following issue

Post by martin@rootjazz »

BUt when scale it up, 1000 users following, 40 threads, TD will got lots errors, problems, skip, and failed.
so those 1000 users share 250 proxies. So we can assume sometimes the 40 threads will be duplicating proxy use.

What is the deal with the residential proxies? How many can you connect to at any one time? What appears to happen is that when you crank up the threads, many requests fail - no response, not errors, just no response. Which "points" to the network
Post Reply