Scraping profile error

[aritwitter]

Hello,

I've been encountering some error when scraping accounts for details. I want to get the name, loc, bio/description, number of followers, number of followings, date of creation. I have previously done this and got the right output but now I am only getting this:

screenname url name location description profile_url followers_count friends_count statuses_count urls location created_date
https://twitter.com/account1
https://twitter.com/account2
https://twitter.com/account3

As shown above, there are columns for the details but those columns are blank.

For input, I have a text file containing the url of the accounts I want to scrape:
https://twitter.com/account1
https://twitter.com/account2
https://twitter.com/account3

For the custom search I chose 'A user ID or URL' then I selected 'details' so it can be saved in a CSV.

Please let me know if I'm doing it right and why I am not getting the right output.

[martin@rootjazz]

Let me know exactly how you are setting up the action. Instruction set to replicate exactly or screenshots be helpful

Then submit logs showing the failed action

HELP > LOGS > SUBMIT

then send your logs ID - the numbers are sufficient (displayed after successful uploading of logs)

viewtopic.php?f=32&t=12078



Regards,
Martin

[aritwitter]

Sorry, my bad. it's not an error per se. But the result of the scrape is not what is expected. The content of the txt file didn't have data for the other columns, it only has the url of the account and nothing else which is what I sent last time:

screenname url name location description profile_url followers_count friends_count statuses_count urls location created_date
https://twitter.com/account1
https://twitter.com/account2
https://twitter.com/account3

I was expecting that there will be data under each column.

Here's the steps of how I did it:

1. clicked scrape tab
2. Selected the account which will do the scraping process
3. Clicked custom search chose "A user Id or url' for option then "A user Id or url' again under selections
4. Input the filepath of the text file containing the url of the accounts I want to scrape
5. Selected 'Details' to save result to a text file
6. Then lastly clicked "scrape profile"

log: 32009

[martin@rootjazz]

log: 32009

thanks that's what I needed, testing...

[martin@rootjazz]

Tested, it's working here for normal searches

3. Clicked custom search chose "A user Id or url' for option then "A user Id or url' again under selections
4. Input the filepath of the text file containing the url of the accounts I want to scrape

But I think this is what the issue is...

yes confirmed, this is the issue.

The next update will fix this. I shall let you know when it is ready.



Regards,
Martin

[martin@rootjazz]

https://rootjazz.com/twitterdub/updatetesting.html


Regards,
Martin

[aritwitter]

Hi, I'm getting another error. It says proxy authentication required. Then proxy issue detected: 407

Let me know what this is all about and how I can fix it.

Log: 19736

[martin@rootjazz]

Hi, I'm getting another error. It says proxy authentication required. Then proxy issue detected: 407

Let me know what this is all about and how I can fix it.

Log: 19736

you need to authorise the proxy, speak to your proxy provider

[aritwitter]

The issue with proxy has already been resolved, thanks!

As for the error in the scraping output, it seems like I am still getting the same result, just the url, even after updating the app.

Log: 37781

[martin@rootjazz]

As for the error in the scraping output, it seems like I am still getting the same result, just the url, even after updating the app.

Log: 37781

It's working here (for a new action I just tested)

What version are you on?

HELP >ABOUT

Might have been a build issue, not pulling in the update. I Will do a new build and let you know when ready for testin



Regards,
Martin