Hello everyone!
Hope everyone's day is going well. I've search in the documentation and can't find the answer I'm looking for. I hope someone can please help.
1) I'm trying to create a txt/csv file of all the people who live in the SANTA BARBARA area who have either in their PROFILE BIO or LOCATION that they are from here. How the heck do I do this?
2) Once I do successfully scrap this data, how do I export this file?
Thank you everyone!
New and clueless on TwitterDub - LOCATION SCRAPPING
-
- Posts: 1
- Joined: Thu Oct 22, 2015 10:42 pm
- martin@rootjazz
- Site Admin
- Posts: 34360
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: New and clueless on TwitterDub - LOCATION SCRAPPING
You cannot perform a scrape of profile data. Twitter doesn't provide this functionality.
You can FILTER your results by what is in the bio though, but you must provide a search to get results, then check if the resulting users have the word in their bio. Obviously this reduces your results drastically
As to search by location, you would need to perform a GEOSEARCH
Scrape tab - profile search
setup your search
GEO SEARCH - return tweets
CREATOR OF TWEET - gets the author of the tweet
Then in the scrape tab, in the search box enter your address
Santa barbara, California
OR whatever address you want, the program can take any form of address, so you can be as detailed as you want
32 Street, Santa Barbara, California, 2761287
Or whatever an actual address looks like for that region
You will need to specify an account to perform the search
And you can limit the number of results you want.
Click the SEARCH button to create your search, it will be added to the PROCESSING TAB
goto to the processing tab, make sure the processor is running (click RUN)
Then you can double click the action to view the logs, or what is happening.
The results are saved to:
HELP > SAVED DATA
I ran a test scrape for 10 items:
where the CSV is something like
username, profileURL, name, location, bio, url
You can FILTER your results by what is in the bio though, but you must provide a search to get results, then check if the resulting users have the word in their bio. Obviously this reduces your results drastically
As to search by location, you would need to perform a GEOSEARCH
Scrape tab - profile search
setup your search
GEO SEARCH - return tweets
CREATOR OF TWEET - gets the author of the tweet
Then in the scrape tab, in the search box enter your address
Santa barbara, California
OR whatever address you want, the program can take any form of address, so you can be as detailed as you want
32 Street, Santa Barbara, California, 2761287
Or whatever an actual address looks like for that region
You will need to specify an account to perform the search
And you can limit the number of results you want.
Click the SEARCH button to create your search, it will be added to the PROCESSING TAB
goto to the processing tab, make sure the processor is running (click RUN)
Then you can double click the action to view the logs, or what is happening.
The results are saved to:
HELP > SAVED DATA
I ran a test scrape for 10 items:
Code: Select all
HeilalaX https://twitter.com/HeilalaX lorna. san francisco http://t.co/VbKWcaJe7O
Sandoval4Sergio https://twitter.com/Sandoval4Sergio Sergio Sandoval Santa Barbara, CA
Endobariatric https://twitter.com/Endobariatric Dr. Alvarez ™ Piedras Negras, Mexico Official account of Dr Guillermo Alvarez, Weight Loss #Surgeon of excellence, #Author, #Speaker and #cyclist. Follow my life @ Snapchat: gmoalvarez http://t.co/d7UVz9Jezk
bubariosb https://twitter.com/bubariosb Martín Ríos Benítez Corrientes - Argentina Tenis
_jadebree https://twitter.com/_jadebree Giada B. Thuggin in Los Angeles Susie Carmichael in a world full of Angelicas. Live & Grow September 25
jtkwood https://twitter.com/jtkwood Jason Kirkwood I'm Everywhere I'm just an extraordinary guy doing extraordinary things.
train805 https://twitter.com/train805 thomas trigo Ventura, ca stay at home dad, angry birds enthusiast, former crappy wakeboarder. http://t.co/nyRZxGX7ym
APhatJ https://twitter.com/APhatJ Phat J Santa Barbara, CA Phat J | On-Air Word Slayer | 92.9 KjEE http://t.co/vgdxFDZdNG
tmj_CAA_NURSING https://twitter.com/tmj_CAA_NURSING TMJ-CAA Nursing Jobs Santa Barbara, CA Follow this account for geo-targeted Healthcare-Nursing job tweets in Santa Barbara, CA from TweetMyJobs. Need help? Tweet us at @TweetMyJobs! http://t.co/QBUDvJ81Sq
_jdhudson https://twitter.com/_jdhudson JD Hudson Redding, CA Jack of all trades and a master at some. http://t.co/MqAkKPAOWV
tmj_CAA_health https://twitter.com/tmj_CAA_health TMJ-CAA Health Jobs Santa Barbara, CA Follow this account for geo-targeted Healthcare job tweets in Santa Barbara, CA from TweetMyJobs. Need help? Tweet us at @TweetMyJobs! http://t.co/KDnJmoDl8c
WesleyArnold https://twitter.com/WesleyArnold Wesley Arnold Atlanta, GA Filmmaker | Photographer | http://t.co/pYOtU3fiXj http://t.co/BYuYe8Luaw
Maria__Granberg https://twitter.com/Maria__Granberg Maria Granberg Los Angeles, CA Creative Mind From Swedenland | Model | Actress | https://t.co/z02WYZXbzj
kilhopark_photo https://twitter.com/kilhopark_photo kilho park Photographer, family man. Live, love and take killer images. Cheers. http://t.co/mKOx1fvBXk
username, profileURL, name, location, bio, url
- martin@rootjazz
- Site Admin
- Posts: 34360
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: New and clueless on TwitterDub - LOCATION SCRAPPING
from the next testing update, you will also be able to specify a TWEET search specifying
search term
Location coords lat,long: ie 10.123,248.1231
Radius around the coords
Date From
Date To
Search type {recent|mixed|popular}
The format will require a JSON input
{
"hashtag":"#search_term#",
"geopoint":"#geopoint#",
"radius":"#radius#",
"date_from":"#date_from#",
"date_to":"#date_to#",
"search_type":"#search_type#"
}
There will also be a way to build the above required format, making it easy for you to input
search term
Location coords lat,long: ie 10.123,248.1231
Radius around the coords
Date From
Date To
Search type {recent|mixed|popular}
The format will require a JSON input
{
"hashtag":"#search_term#",
"geopoint":"#geopoint#",
"radius":"#radius#",
"date_from":"#date_from#",
"date_to":"#date_to#",
"search_type":"#search_type#"
}
There will also be a way to build the above required format, making it easy for you to input
- martin@rootjazz
- Site Admin
- Posts: 34360
- Joined: Fri Jan 25, 2013 10:06 pm
- Location: The Funk
- Contact:
Re: New and clueless on TwitterDub - LOCATION SCRAPPING
https://rootjazz.com/twitterdub/updatetesting.html
Check SCRAPER tab for how to build the new query, which you copy and paste into the search boxes
Check SCRAPER tab for how to build the new query, which you copy and paste into the search boxes