Twitter is an online social networking service that enables users to send and read short 140-character messages called “tweets”. [1]
\nTwitter users are tweeting about different topics based on their interests and goals.
\nA word, phrase or topic that is mentioned at a greater rate than others is said to be a “trending topic”. Trending topics become popular either through a concerted effort by users, or because of an event that prompts people to talk about a specific topic. [1]
\nThere is wide interest in analyzing of trending data from Twitter.
\nAnd in this post we will look at searching and downloading the tweets related to specific hashtag. We will use Python and Twitter API. Our example will be search tweets related to “deep learning”. After downloading Twitter data we will also look at some data manipulations with the data.<\/p>\n
The example of downloading of Twitter data is based on the work [2]
\nBelow is source code:<\/p>\n
import twitter
\nimport json
\nCONSUMER_KEY =”xxxxxx”
\nCONSUMER_SECRET =”xxxxxx”
\nOAUTH_TOKEN = “xxxxxx”
\nOAUTH_TOKEN_SECRET = “xxxxxx”
\nauth = twitter.oauth.OAuth (OAUTH_TOKEN, OAUTH_TOKEN_SECRET, CONSUMER_KEY, CONSUMER_SECRET)
\ntwitter_api= twitter.Twitter(auth=auth)
\nq=’#deep learning’
\ncount=100
\nsearch_results = twitter_api.search.tweets (q=q, count=count)
\nstatuses=search_results[‘statuses’]
\nfor _ in range(5):
\n print “Length of statuses”, len(statuses)
\n try:
\n next_results=search_results[‘search_metadata’][‘next_results’]
\n except KeyError, e: #result does not exist
\n break
\n kwargs=dict( [kv.split(‘=’) for kv in next_results[1:].split(“&”)])
\n search_results = twitter_api.search.tweets(**kwargs)
\n statuses += search_results[‘statuses’]
\n# Show one sample search result by slicing the list
\nprint json.dumps(statuses[0], indent=10)
\nhashtags = [ hashtag[‘text’]
\n for status in statuses
\n for hashtag in status[‘entities’][‘hashtags’] ]
\nurls = [ urls[‘url’]
\n for status in statuses
\n for urls in status[‘entities’][‘urls’] ]
\ntexts = [ status[‘text’]
\n for status in statuses
\n ]
\n#Created_at is date time when created
\ncreated_ats = [ status[‘created_at’]
\n for status in statuses
\n ]
\nprint json.dumps(hashtags[0:50], indent=1)
\nprint json.dumps(urls[0:50], indent=1)
\nprint json.dumps(texts[0:50], indent=1)
\nprint json.dumps(created_ats[0:50], indent=1)
\n# Now we append some data into the file
\nwith open(“data.txt”, “a”) as myfile:
\n for w in hashtags: myfile.write(w)
\n myfile.write(“\\n”)
\n# count of word frequencies
\nwordcounts = {}
\nfor term in hashtags:
\n wordcounts[term] = wordcounts.get(term, 0) + 1
\nitems = [(v, k) for k, v in wordcounts.items()]
\nfor count, word in sorted(items, reverse=True):
\n print(“%5d %s” % (count, word))
\n# in case we need extract date or month or year
\nfor x in created_ats:
\n print x
\n print x[4:10]
\n print x[26:31]
\n print x[4:7]<\/p>\n
Output example for last for loop (just one cycle)
\nWed Mar 30 02:10:20 +0000 2016
\nMar 30
\n2016
\nMar<\/p>\n
Any comments or suggestions are welcome.<\/p>\n
References<\/b>
\n[1] https:\/\/en.wikipedia.org\/wiki\/Twitter Twitter, From Wikipedia
\n[2] http:\/\/www-scf.usc.edu\/~aupadhya\/Mining.pdf MINING DATA FROM
\nTWITTER by Abhishanga Upadhyay, Luis Mao, Malavika Goda Krishna<\/p>\n","protected":false},"excerpt":{"rendered":"