New and clueless on TwitterDub - LOCATION SCRAPPING

Support / help / discussion forum for twitter bot
Post Reply
cheeky_monkey
Posts: 1
Joined: Thu Oct 22, 2015 10:42 pm

New and clueless on TwitterDub - LOCATION SCRAPPING

Post by cheeky_monkey »

Hello everyone!

Hope everyone's day is going well. I've search in the documentation and can't find the answer I'm looking for. I hope someone can please help.

1) I'm trying to create a txt/csv file of all the people who live in the SANTA BARBARA area who have either in their PROFILE BIO or LOCATION that they are from here. How the heck do I do this?

2) Once I do successfully scrap this data, how do I export this file?

Thank you everyone!
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: New and clueless on TwitterDub - LOCATION SCRAPPING

Post by martin@rootjazz »

You cannot perform a scrape of profile data. Twitter doesn't provide this functionality.

You can FILTER your results by what is in the bio though, but you must provide a search to get results, then check if the resulting users have the word in their bio. Obviously this reduces your results drastically


As to search by location, you would need to perform a GEOSEARCH



Scrape tab - profile search
Image


setup your search
GEO SEARCH - return tweets
CREATOR OF TWEET - gets the author of the tweet

Image


Then in the scrape tab, in the search box enter your address

Santa barbara, California

OR whatever address you want, the program can take any form of address, so you can be as detailed as you want

32 Street, Santa Barbara, California, 2761287

Or whatever an actual address looks like for that region :-)


You will need to specify an account to perform the search

And you can limit the number of results you want.

Click the SEARCH button to create your search, it will be added to the PROCESSING TAB

goto to the processing tab, make sure the processor is running (click RUN)

Then you can double click the action to view the logs, or what is happening.

The results are saved to:

HELP > SAVED DATA



I ran a test scrape for 10 items:

Code: Select all

HeilalaX	https://twitter.com/HeilalaX	lorna.	san francisco		http://t.co/VbKWcaJe7O
Sandoval4Sergio	https://twitter.com/Sandoval4Sergio	Sergio Sandoval	Santa Barbara, CA		
Endobariatric	https://twitter.com/Endobariatric	Dr. Alvarez ™	Piedras Negras, Mexico	Official account of Dr Guillermo Alvarez, Weight Loss #Surgeon of excellence, #Author, #Speaker and #cyclist. Follow my life @ Snapchat: gmoalvarez	http://t.co/d7UVz9Jezk
bubariosb	https://twitter.com/bubariosb	Martín Ríos Benítez	Corrientes - Argentina	Tenis	
_jadebree	https://twitter.com/_jadebree	Giada B.	Thuggin in Los Angeles	Susie Carmichael in a world full of Angelicas. Live & Grow September 25	
jtkwood	https://twitter.com/jtkwood	Jason Kirkwood	I'm Everywhere 	I'm just an extraordinary guy doing extraordinary things.	
train805	https://twitter.com/train805	thomas trigo	Ventura, ca	stay at home dad, angry birds enthusiast, former crappy wakeboarder.	http://t.co/nyRZxGX7ym
APhatJ	https://twitter.com/APhatJ	Phat J	Santa Barbara, CA	Phat J | On-Air Word Slayer | 92.9 KjEE	http://t.co/vgdxFDZdNG
tmj_CAA_NURSING	https://twitter.com/tmj_CAA_NURSING	TMJ-CAA Nursing Jobs	Santa Barbara, CA	Follow this account for geo-targeted Healthcare-Nursing job tweets in Santa Barbara, CA from TweetMyJobs. Need help? Tweet us at @TweetMyJobs!	http://t.co/QBUDvJ81Sq
_jdhudson	https://twitter.com/_jdhudson	JD Hudson	Redding, CA	Jack of all trades and a master at some.	http://t.co/MqAkKPAOWV
tmj_CAA_health	https://twitter.com/tmj_CAA_health	TMJ-CAA Health Jobs	Santa Barbara, CA	Follow this account for geo-targeted Healthcare job tweets in Santa Barbara, CA from TweetMyJobs. Need help? Tweet us at @TweetMyJobs!	http://t.co/KDnJmoDl8c
WesleyArnold	https://twitter.com/WesleyArnold	Wesley Arnold	Atlanta, GA	Filmmaker | Photographer | http://t.co/pYOtU3fiXj	http://t.co/BYuYe8Luaw
Maria__Granberg	https://twitter.com/Maria__Granberg	Maria Granberg	Los Angeles, CA	Creative Mind From Swedenland | Model | Actress |	https://t.co/z02WYZXbzj
kilhopark_photo	https://twitter.com/kilhopark_photo	kilho park		Photographer, family man. Live, love and take killer images. Cheers.	http://t.co/mKOx1fvBXk


where the CSV is something like
username, profileURL, name, location, bio, url
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: New and clueless on TwitterDub - LOCATION SCRAPPING

Post by martin@rootjazz »

from the next testing update, you will also be able to specify a TWEET search specifying

search term
Location coords lat,long: ie 10.123,248.1231
Radius around the coords
Date From
Date To
Search type {recent|mixed|popular}

The format will require a JSON input


{
"hashtag":"#search_term#",
"geopoint":"#geopoint#",
"radius":"#radius#",
"date_from":"#date_from#",
"date_to":"#date_to#",
"search_type":"#search_type#"
}



There will also be a way to build the above required format, making it easy for you to input
User avatar
martin@rootjazz
Site Admin
Posts: 34360
Joined: Fri Jan 25, 2013 10:06 pm
Location: The Funk
Contact:

Re: New and clueless on TwitterDub - LOCATION SCRAPPING

Post by martin@rootjazz »

https://rootjazz.com/twitterdub/updatetesting.html

Check SCRAPER tab for how to build the new query, which you copy and paste into the search boxes
Post Reply