Rootjazz

Hey first time posting, happy to be here.

I've recently tried scrapping avatar and banner images using the avatar scraper and I was getting error response from twitter it seems,

Failed JSON response: 403
https://twitter.com/i/api/1.1/users/loo ... ilosopheRR
{"errors":[{"code":18,"message":"Too many terms specified in query."}]}
ERROR RESP FROM TWITTER: 18 Too many terms specified in query.
* TESTING for known failure text: Too many terms specified in query.
* FAILED: 92374087236487692374627 : Too many terms specified in query.

after that it indicated progress failed.
did twitter do some changes to the avatar or banner urls?
also wanted to know is this the only way to scrape avatar and banner images?

I think i get what is going on here, it tries to send to Twitter api all the scraped IDs at one query is that it? Should it be split somehow to allow the api to fetch the avatars and banners one by one? Not really sure how to do that though...

SOLVED
I've tried it again with limit on followersOf to 5 and the scrapping succeeded this time around.
Guess as the api error stated too many requests, meant too many requests in one call...

Just documenting my results for others if they encounter the same, or for my owner reference in case I forget.

its2l82die wrote: ↑Fri Jul 29, 2022 3:57 pm I think i get what is going on here, it tries to send to Twitter api all the scraped IDs at one query is that it? Should it be split somehow to allow the api to fetch the avatars and banners one by one? Not really sure how to do that though...

yes seems, seems the request should be page.

Can you just explain the search you were running that triggered this bug. I'll try and cause it now, but if you have instructions on how to make it fail, might save me some time

never mind, figured it out. twitter changed how long the URL can be to request user details in a single request.

The next update will fix this. I shall let you know when it is ready.



Regards,
Martin

https://rootjazz.com/twitterdub/updatetesting.html



Regards,
Martin

Thank you for the update Martin really appreciate it