Though they often use it as a weapon against each other, cyber-optimists and cyber-pessimists agree on one thing: digital activism cannot be effective without connecting back to the real world. Because all our institutions of power still exist offline, in order to influence real-world outcomes, digital actions must have real-world effects. Yet we are far more empirical in our study of online activity than of offline context, resulting in an incomplete picture of digital activism as a whole.

In their forthcoming book chapter, “Contingency and Hybridity in Advocacy Networks: Implications of the Egyptian Protest Movement,” Christopher Wilson and Alix Dunn of The Engine Room make a case for the importance of analysis both online and offline factors – a methodology they call hybridity. While needing to weigh both online and offline factors is not new, they are trying to systematize this type of analysis:

Though intuitively of great importance, there has been little study of the joint mobilization of digital and traditional media, or communication bridging digital and grounded networks, and what consequences this might have for how we understand the interaction between online and offline activity in digital advocacy. The characteristic of hybridity is an attempt to counteract this inattention, by identifying objects of analysis in which online and offline activity interact per se, in which that interaction is measurable and comparable, and in which that interaction is meaningful.

Subjective Hybridity: Case Studies

While all researchers – and even casual analysts – know intuitively that they must consider offline context and consequences when analyzing digital activism, scaling hybridity analysis is one of the great challenges of digital activism research.

At one end of the spectrum, we have case studies that do an excellent job of hybridity analysis, but in most hybrid case studies N=1 and only a single instance is analyzed. Most case studies are also stronger on qualitative than quantitative data, because it is easier for the untrained analyst to work with, but there are also examples quantitative case studies, like Zeynep Tufekci’s work on preferential attachment on Twitter during the Egyptian revolution, which is more empirical than most qualitative case study work because it follows the exigencies of network science. Still, the amount of offline context included is up to the writer of the individual case study.

There are also large-N hybrid studies, based mostly on the collections of qualitative case study and anecdote, like the work of Clay Shirky and Evgeny Morozov. In their work, Shirky and Morozov consider hybridity through personal interpretation and analysis, as is the case with the writers of N=1 case studies, of which their work is composed. Both do a good job of hybridity, though they create their own standards about what information about offline context to include and analyze.

Empirical Non-Hybridity: Quantitative Analysis of Digital Data

At the other end of the spectrum we have large-N analyses of online phenomena, like the Berkman Center’s country blogosphere maps and the Web Ecology Project’s analysis of Iranian protest tweets that mix qualitative content analysis with quantitative analysis of link behavior and frequency. These studies are exhaustive, and thus statistically significant, in analyzing all available data without the narrow scope of the study. In these cases offline context is considered, but not using the methodological rigor and thoroughness of the analysis of online content.

Why Empirical Hybridity is Hard

Simply put, hybridity analysis of large-N data sets is hard. Why? Here are the main challenges:

1. Ease/Difficulty of Information Access: A tweet, by its very nature, is made public online, and thus easy for a researcher to grab. Information about strategic thinking and other factors of causality and outcome are less likely to be published for reasons of self-protection, lack of motivation, or time constraint. While much data about online actions can be collected using a simple scraping program, the same is rarely true of information about offline political economy.
2. Differences in Marginal Cost of Data Collection: Thousands of tweets can be collected easily by a scraper. The marginal additional effort to record an extra tweet is vanishingly small, and is measured in computing time or server cost, not in researcher time or effort. In fact, for online data, the marginal cost of collecting an additional piece of data decreases for each additional tweet. If you spend 2 hours setting up a scraping program, that number does not increase whether you collect 100 tweets or 10,000. The financial cost is also low. Collecting offline data is quite different. It may take hours to track down one tweeter to interview and that interview may not even be probative. It may be necessary to travel to the location of the event, a great financial expense.
3. Different Levels of Abstraction: Larry Lessig defines digital as “perfect copies, freely made,” a boon to researchers. Tweet and Facebook postings are encoded uniformly and have a limited range of content types. The data can be copied directly to a spreadsheet without the need for abstraction, alteration, or interpretation. With the help of software, every tweet with the hashtag #Jan25 can be individually analyzed in its complete and original form. A strategy or outcome, on the other hand requires significant interpretation on the part of the researcher and activist. Even if the researcher is quoting the activist directly, the questions asked will shape the type of information obtained. The activist herself may forget, misinterpret, or withhold information. In addition, some important contextual variables, like the relative power of different political actors, can only be comparatively quantified in the extreme abstract, with available information dramatically flattened and subjectively interpreted. Digital information comes in pre-packaged and copy-able analytic units of binary code. The same cannot be said for many types of offline information, much of which is lost at the moment of capture because of the complexity of offline phenomena and the limits of the ability of the researcher to perceive and accurately record (symbolize) all information.
   
4. Ethical Concerns: Activists may unwisely publish personal data, but if the researcher records and analyses that data, it does not put the activist in any additional danger, or at the very least is working with the same information as repressive governments. By seeking offline information, however, the researcher runs the risk of making public what was previously private and putting the activist at risk. It would be foolish and irresponsible to try to conduct in-person interviews with digital activists in Bahrain or Syria right now, but even remote interview techniques, like Skype or email questionnaires, may be tracked by the authorities and used to persecute activists. Sometime offline data is not only difficult to obtain, it is also dangerous to do so.

Suggestions on Hybridity Methodology

How do we scale and systematize hybridity analysis beyond the N=1 of the case study? Here are some ideas:

1. Study Groups of Case Studies: The easiest and cheapest (in time and money) way to scale hybridity analysis is to study groups of case studies. Since any case study worth its salt weighs online and offline factors, comparing large numbers of case studies will result in large data sets that also have the characteristic of hybridity. This is one of the methodological principles behind our Global Digital Activism Data Set. This methodology is far from perfect, of course. There is little regularity in the way digital activism case studies are written, making comparability difficult. Even when the same questions are asked of each case study during a coding process, some cases will provide that information more completely than others, and in some it will be totally absent or conflicting. So even if groups of case studies are subjected to uniform analysis, they may not all include the information necessary to provide equally reliable information across cases. The level of abstraction is also high, and so is the subjectivity of the hybridity of each case. Still, I think this is a good place to start. Reaching consensus on the type of information a digital activism case study should contain will also make this type of work easier.
2. Automate the Collection of Offline Data: One of the problems of collecting offline information is that it is expensive and time-consuming to the researcher. In order to scale this type of work, researchers should find ways to automate this type of data collection wherever possible. For example, rather than interviewing an activists personally, the researcher could direct multiple activists to an online questionnaire that includes questions designed to capture the unanticipated, as in an in-person interview. Having all activists answer the same questionnaire could also improve comparability. Though there might be a lower conversion rate, a system of incentives, even payment, could be developed to encourage activists to complete the form. Methodologies like this would allow interviewing to be less costly in both time and money, lowering the bar to entry into this type of research and increasing the number of activists interviewed.
3. Be Creative with Online Data: Some offline questions can be answerable by the creative and skilled analysis of online data – just ask the repressive governments that search activists’ Facebook friend networks in search of their offline collaborators. In other cases information about offline context is recorded online and can be scraped. We need to think more deeply about what kinds of questions about offline context can be answered through the analysis of online data and what the best methodologies are.
4. A Digital Efficiency Does Not Always Exist: Sometimes it will not be possible to automate offline data collection or reveal this information through analysis of online sources. Sometimes it will simply be necessary to get on a plane and spend painstaking hours interviewing activists in situ or using other methods of offline data collection. The Internet age has made instant and low-cost omni-presence the default, but sometimes analog research methods – time-consuming and expensive though they are – are still the only way to get the whole story.

Finding Methodologies to Meet the Exigencies of Empiricism

The uniformity of digital data, compared to the messiness of offline data, encourage collection of large-N digital data sets and discourages (or does not facilitate) the collection of large-N analog ones. The digital network has many affordances for researchers, but tougher offline work is still necessary to get a complete and accurate picture of digital activism phenomena and their effectiveness. We need to start thinking more critically about our research methodologies, requiring higher levels of empirical proof for characterizations of digital activism, and finding practical means for achieving this empiricism.