Continued adventures in learning Python (started here):

A function gives you a return value based on an argument that you input.  The function len will give you a return value of the number of elements in an object.  The argument you provide is the name of the object.  In this example the object is called “names.”

You create a function by using the syntax def (define).  This function is called “Mary” (it could be called anything).  The function doubles the argument.

This tutorial uses Tweepy software to use Python to get data from Twitter.  This tutorial was presented today at the Community Data Science Workshop @UW.  The first tutorial, an introduction to APIs, is here.

1) Get a a Twitter API key & access token

• First, you need to set up Twitter authentication so Twitter will give you the data.  
• Authentication steps are here: https://openhatch.org/wiki/Community_Data_Science_Workshops/Twitter_authentication_setup

1) Then get a bunch of Python programs for Twitter

• Download this zip file from GitHub to your desktop: https://github.com/makoshark/twitter-data-examples/archive/master.zip
• Change your directory in Python so you can access these files directly by using the “cd” command

2) Put the authentification information into a Python program

• Download a text editor like Smultron
• Open a new file and type your keys and tokens so it looks like this:

• Save the file as “twitter_authentication.py” to the file of Twitter programs you just downloaded from GitHub.  (Replace the existing file with this name.)

3) Get tweets from Twitter

• In your terminal run the command “python twitter1.py”.
• Tweets should appear.

• This means you successfully used the Twitter API in a basic way.
• This Twitter data looks messy and that is something you need to get used to.

• 99% data science is using your brain to figure out what the data you are looking at means.
• 1% of data science is using statistics to interpret the data (or so says Guy)
• Then you use additional commands in Python to extract pieces of information (like time zone) that you may want to analyze.

Notes and reflections from the Community Data Science Workshop, presented by Benjamin Mako Hill and friends today at the University of Washington, Dept. of Communication.

Contents

• Introduction
• Using sample APIs
• Putting data in a format you can use

Introduction

What is an API?

• Term stands for Application Programming Interface
• It is a standard (or protocol).  It’s not a piece of software.
• It’s a way for one program to talk to another program.
• It’s a way to get data from online platforms about what people are doing on that platform.

Why should I care as an activism researcher?

• Sometimes people are using those platforms for activism.
• You can learn something (not everything) about activism activity on the platform by looking at the traces of that activity that individuals leave in the form of tweets, follows, edits, and more.
• These traces are the data that you access through the API

What can I do with an API?

• ask for data (almost always asking a URL)
• get data back (almost always in a file format called JSON)
• build a dataset of content to study (using the data you got through the API)

Challenges

• When a platform changes its architecture, the structure of data can also change.
• Different platforms have different APIs.  Though they have similar features, you will need to learn each platform’s API separately
• The API structure and documentation will be better for platforms that make money off their API, like Twitter
• For those that don’t care about how people access their data, the API will not be well-structured
• However, for platforms whose data is commonly accessed via API (like Twitter), there will be existing Python modules that have been created to make your task easier.

Using Simple APIs

Use Python to go out onto the web, grab some data, and show it to you

• Use Python, which is a program you probably already have in your computer and can access with your terminal (here’s how)
• Python example code, which scrapes the HTML code from the website http://www.python.org

• In this example, the API is the standard that allows you to pull data (HTML or other) from a website by using the code above.

Put data from the web into a file on your computer

• This code puts the html code from http://www.python.org into a file on your computer

• Now the file is on your computer (you can find it by searching your computer for a file named python.html)
• Use the command os.chdir to set your directory, which is the place on your computer where the file is placed
• The default directory is your personal directory on your computer.  For example, mine is called mjoyce.

Use a simple API that involves kittens

• Go to placekitten.com, a place to get images of kittens of particular dimensions for use as placeholders in web design
• The dimensions of the pic are in the URL, such as http://placekitten.com/200/200 and http://placekitten.com/400/200
• Here’s code to pull a 300×300 pixel cat image from placekitten.com and put it in a file on your computer called “kitten.jpg”

• And here’s what the file you created looks like on your computer

• In this example, the API is the standard that you can get an image of a kitten with certain dimensions by using a URL with the dimensions at the end of that URL

Putting Data in a Format You Can Use

Why should I care about JSON?

• When you get data from a website using an API, it will most often be in JSON format.

What is JSON?

• It is a language for structuring data.  It is a format used by programs for programs.
• Here’s an example of what a JSON file looks like: http://json.org/example.html
• Here’s a simpler example by Mako: http://mako.cc/cdsw.json
• It has a very similar data format to Python

Interpreting a JSON file

• This is an entry about a pet fish with a name, age, and favorite color

Importing a JSON file into Python

• This code displays a JSON file from the web in Python

• This imports a file for interpreting JSON into Python.

• This code names the json file “data” and displays the contents of the file in Python

Putting data from a JSON file into a spreadsheet

• This code puts certain data in the JSON file from the web into a .csv spreadsheet file so it is easier to work with.

• And this is what the .csv file you created looks like

• These are the basics of what you need to do to be a data scientist, either for the study of activism or any other activity carried out online.

Grad school is a pyramid scheme.  There are lots of smart students and only a few academic jobs for us when we graduate.  Oh, and then there’s the fucking internet, which promises to to have only a few of the most talented instructors teach all the children on earth.  Which means even fewer academic jobs.  Fuck them!

Actually, TedEd seems awesome.  But grad students like myself are still screwed.

I’ve just added a new Data section to this site.  I’ll be sharing data files resulting from my research on global digital activism, all of which have a Creative Commons Attribution-NonCommercial-ShareAlike license.

The first group of files is related to the second version of the Global Digital Activism Data Set, which I created while working at the Digital Activism Research Project, and which I am continuing to work on and improve.

Image: Digital Activism Research Project

I’ve just added a new Open Methods section to this site.  I’ll be sharing research materials related to my current study on digital activism effectiveness, all of which have a Creative Commons Attribution-NonCommercial-ShareAlike license.

Ch-ch-ch-check it out!

Image: Flickr/Kris Krug

Even though Occupy emerged and faded almost two years ago, the scholarship of Occupy continues.  Occupy publications in the last year include a paper that explores the values, attitudes, and beliefs of occupiers as they relate to their use of technology and another that looks at changes in participant engagement, interests, and social connectivity on Twitter.  Another paper looks at the implication of the occupations for public space.

In the British Journal of Sociology, the American sociologist Craig Calhoun wishes to evaluate Occupy, but clarifies that he does not wish to critique it too harshly.  “[T]here is no shame in being more moment than movement,” he writes. “It is no denigration of Occupy Wall Street (or the Occupy movement(s) more generally) to say it may not have a future as such.”

Yet if an activist mobilization doesn’t “have a future,” that certainly isn’t a good thing.  Why not say so? Why this hedging (which is not particular to Calhoun)? The great challenge of analyzing activism success is that the people who analyze activism outcomes as intellectuals also care about activism outcomes as emotional beings. 

The danger here is that we as scholars find ourselves looking for effects rather than measuring effects. The former is a result of post-facto analysis, Monday morning quarter-backing.  There is always some outcome to be found which can be construed as successful (“changing the discourse,” for example). The latter method seeks to record what a campaign was initially trying to achieve and be willing to say that it did not succeed in that attempt.

When we as scholars care about changing the world – and most scholars of activism do – there is a great danger of the former type approach, in which emotion clouds empiricism. And there is often too little of the latter approach, in which unsentimental empiricism leads to recognition of a disappointing reality.

As a scholar who wants activists to succeed I know that I run the risk of producing the former type of analysis, and I am trying very hard to produce the latter. This means remembering that activism campaigns are trying to change the state of the world in some way, and the extent to which they do or do not bring about this change must be the primary criteria for evaluating success.

This is the logic being the variables I’m currently creating. I know this will frustrate some people, because I am missing the lesser effects of (failed) campaigns. But I think it is the right way to go. If we want to make activists more effective we need to be rigorous and unsentimental. Calling a failure a success may make us feel better, but it’s unlikely to lead to better activism and real change.

Image: Wikipedia

Now that the Republicans trolling American democracy have recently (and temporarily) been vanquished, it seems as good a time as any to think about the nature of trolling.

Why people become horrible trolls online, taunting and harassing others for spite, joy, or profit?  John Suler of Rider University has some answers.  In a 2004 article entitled “The Online Disinhibition Effect,” he theorizes that people “say and do things in cyberspace that they wouldn’t ordinarily say and do in the face-to-face world” for six reasons, cumulatively called the online disinhibition effect.  I find 1 through 3 to be the most convincing, but here’s the whole list.

Suler is careful to argue that trolls are not necessarily jerks offline as well.  The online context, combined with inherent personality traits, brings forth the habits of trolling.

Image: Troll Hunter (film)