Problem with scraper feature

[mynameismyname]

Hi, I'm trying to scrap my followers by language, but is seems like it's broken, whether I type "ru" "en" or "es" in "user filter" the results are always the same. The same accounts shows up in "raw logs"

Btw I noticed the "Assign default default lang to search: es" whether I type "ru" or "en" is it normal?

I'm trying to block all russians/english/spanish followers I have but right now I don't know how to do it, please help me

[martin@rootjazz]

Btw I noticed the "Assign default default lang to search: es" whether I type "ru" or "en" is it normal?

Did you set es?

double click account to view ACCOUNT DETAILS form, then click ADVANCED and check the language specified, does it say es?

Hi, I'm trying to scrap my followers by language, but is seems like it's broken, whether I type "ru" "en" or "es" the results are always the same. The same accounts shows up in "raw logs"

Seems that Twitter have stopped returning the language property for profiles

Code: Select all

"lang":null

not sure if this is new / a bug with twitter or something. But if twitter are not returning that property, there isn't much we can do with it

[mynameismyname]

"Did you set es?"
I did it first, but then I changed it to "ru", but the raw logs keeps showing "es"

"double click account to view ACCOUNT DETAILS form, then click ADVANCED and check the language specified, does it say es?"
how do I do that? please explain me te complete path

"Seems that Twitter have stopped returning the language property for profiles
not sure if this is new / a bug with twitter or something. But if twitter are not returning that property, there isn't much we can do with it"

can you check it yourself to see if I made a mistake or if it's really a problem from twitter please?

[mynameismyname]

I figured something, I keep seeing the same profil name because the scrap features always starts from my latest followers and go to the newest ones and always list them all...

Next question, when the scrap is done, how can i export the result to the block feature?

I don't understand what should be under "Scrape Profiles"? Because when I run "Scrape Profiles" it says "please enter a search term"

And whats the difference between selecting "Details" or "ID" next to "User Filter"?

It's really a pain to use, everything is hard to understand with the UI, it should be a lot simpler to use...

[mynameismyname]

Ok I give up for today I'm already tired of this software. For some odd reason the "Assign default default lang to search: es" remain in all scrap even if I removed it long time ago. I can choose any user filter it won't return any result ever. I even tried to update it to the latest beta, still the same issues. I give up, it's a pain in the ass to use.

[martin@rootjazz]

"Did you set es?"
I did it first, but then I changed it to "ru", but the raw logs keeps showing "es"

"double click account to view ACCOUNT DETAILS form, then click ADVANCED and check the language specified, does it say es?"
how do I do that? please explain me te complete path

If you restart the program and do NOT run the processor (make sure it is off).
go to the account details form (double click account)
check the advanced box top right to see all options
check the language, does it still show es? If so, change it to ru, save the form
Then reopen the form and check the language again, what does it show? es? ru? (it should be ru)

Just tested changing languages, working fine for every account and every language.

"Seems that Twitter have stopped returning the language property for profiles
not sure if this is new / a bug with twitter or something. But if twitter are not returning that property, there isn't much we can do with it"

can you check it yourself to see if I made a mistake or if it's really a problem from twitter please?

I don't think you understood me. Twitter are not returning any information for the language property. This is nothing to do with the app. Twitter, are returning "null" (means empty) for the language property when you use the website, they used to return the object containing the language information, now they are returning nothing. Twitter are not providing any language information on any profile at this time

[martin@rootjazz]

I figured something, I keep seeing the same profil name because the scrap features always starts from my latest followers and go to the newest ones and always list them all...

Yes, it will scrape all the followers as you see them if you go to the list of your followers on the website then scroll through them.

Each time you go to that webpage, twitter shows you the most recent followers. If you check the page again 2 minutes later, twitter shows you the list from the start. That is what the program is scraping.

Next question, when the scrap is done, how can i export the result to the block feature?

The results are saved to your HELP > SAVED folder (the path is listed at the end of the action logs.

But are you sure you want to BLOCK all your current followers? What are you trying to achieve?

I don't understand what should be under "Scrape Profiles"? Because when I run "Scrape Profiles" it says "please enter a search term"

The program does not know you want to search for YOUR followers, so you must enter YOUR PROFILE so the program can run the FOLLOWERS scrape on YOUR PROFILE.

And whats the difference between selecting "Details" or "ID" next to "User Filter"?

ID will return the results as a list of IDs. DETAILS will return a CSV of details for each result

It's really a pain to use, everything is hard to understand with the UI, it should be a lot simpler to use...

Any software requires learning, especially one with many options and functions. Sure if the program just scraped your followers and added them to block, it could be made simpler, but it would not be fit for purpose for the other 99.99% of users.

[martin@rootjazz]

I can choose any user filter it won't return any result ever.

Just tested a filter, working fine. If no results are returned, what filter are you setting, what search are you trying to run?

[mynameismyname]

Thanks for your help, I finally understand how to properly use it. So what I'm trying to achieve is really simple, I am french, I have a lot of fake and foreign followers, I want to block them all. I only want to keep real french followers. Since I tweet in french, I know the foreign followers won't interact with my tweets. I want to keep a healthy/real follower count. What I've noticed with the fake followers/profiles, they're following 2000 people and have 10 followers and haven't tweeted since a few years. What activity ratio should I apply to them?

Can I scrap my followers from the oldest one instead of the newest ones? It would easier for the software to find the fake ones since around 3/4k of my latest followers are real

[martin@rootjazz]

Thanks for your help, I finally understand how to properly use it. So what I'm trying to achieve is really simple, I am french, I have a lot of fake and foreign followers, I want to block them all. I only want to keep real french followers. Since I tweet in french, I know the foreign followers won't interact with my tweets.

ok, so I see why you wanted the language filter, which is unfortunately no longer viable.

I want to keep a healthy/real follower count. What I've noticed with the fake followers/profiles, they're following 2000 people and have 10 followers and haven't tweeted since a few years. What activity ratio should I apply to them?

I think it would be easier to just filter on:

Following: min=1000
Followers: max= 20

Unfortunately: "Days since Last tweet" is not going to help as it will ignore those that you want.

But the above following / followers filters will ONLY save profiles following MORE than 1000 and has less than 20 followers, which should get most of the fake followers

Can I scrap my followers from the oldest one instead of the newest ones? It would easier for the software to find the fake ones since around 3/4k of my latest followers are real

Not possible. Just as you cannot do it on the website

goto
https://twitter.com/<username>/followers

where <username> is your username, you can ONLY view your followers in that order