Thursday, March 26, 2015

Twitter Data Acquisition in Python



Prerequisites:
 Install Python at https://www.python.org/downloads/

 Create a Twitter Application
1)      Register a Twitter Application at https://apps.twitter.com/
2)      After you have successfully created a Twitter Application, write down your CONSUMER_KEY, CONSUMER_SECRET, Access_TOKEN, and Access_TOKEN_SECRET.
 Install necessary python libraries
1)      Go to https://pip.pypa.io/en/latest/installing.html, download the get-pip.py;
2)      Add Installation Folder/Python27/Scripts to the Path Variable in My Computer/ Properties/ Advanced system settings/ System Environment Variables
3)      Right click on the downloaded get-pip.py , choose Edit with IDLE, Run … Run Module (F5)
4)      Go to Windows/Start, in the Search programs and files type cmd
5)      In the pop-up window, type pip install twitter, the twitter library will be installed automatically.

6)      Type pip install dbf, the dbf library will be installed automatically
7)    Download the  GenerateDBFTable.py and GetTweetByQuery.py at https://github.com/xbwei/GetTwitter/tree/master/Twitter

Create dbf table: 

 Right click on the GenerateDBFTable.py file, choose Edit with IDLE, Run … Run Module (F5), a dbf table named Tweet will be created

Customize python script:
    Open the GetTweetByQuery.py file with IDLE (right click on the file, choose Open with IDLE)
    Fill in your CONSUMER_KEY, CONSUMER_SECRET, OAUTH(Access)_TOKEN, and OAUTH(Access)_TOKEN_SECRET in the OAUTH section

In the define query section, modify the following parameters:
1)      q: define the text that contained in the collected tweets returned by REST API
2)      count: define the maximal number of collected tweets returned by REST APT
3)      lang: specify the language of the tweets returned by the REST API
4)      geocode: define the latitude, longitude and radius where the tweets will be collected by the REST API


Collect Tweets:
Right click on the customized GetTweetByQuery.py, choose Edit with IDLE, Run … Run Module (F5)
    Open the Tweet.dbf in Excel to view the collected tweets.