How to interpret collective aggregated judgments?

Why is it important for us to be able to explain social laws and patterns? Perhaps the most basic answer is that we want to be able to explain social laws because, ultimately, we can change them — Alan Garfinkel, Forms of Explanation (1981, 180)

Our digital society increasingly relies in the power of others’ aggregated judgments to make decisions. Questions as diverse as which film we will watch, what scientific news we will decide to read, which path we will follow to find a place, or what political candidate we will vote for are usually associated to a rating that influences our final decisions.

These aggregated judgments may have a herding effect and are topic-dependent so they affect the way we interpret collective aggregated opinions. In what sense is this aggregated information (which is related to the items’ quality) useful? What kind of collective biases determine our individual perception? These questions are relevant because in some cases the pernicious herding effect may produce suboptimal market outcomes and rich-get-richer dynamics that increase human inequality.

The journal Science recently published the article “Social Influence Bias: A Randomized Experiment.” This article analyses the results of a large-scale randomized experiment on a social news aggregation website (a site where users  contribute to new articles and engage into discussions with one another). This experiment was designed by Lev Muchnik, Sinan Aral and Sean J. Taylor in order to investigate if the knowledge of aggregated opinions distorts decision-making.

Apparently, some previous research had confirmed the existence of significant biases in individual behaviour. This information may not be surprising, what is striking is that the experiment by Muchnik, Aral and Taylor showed that the influence generated by aggregated opinions has an asymmetric effect.

The positive opinions do not have the same effect as the negatives: positive social influence is accumulated creating a tendency to the ‘ratings bubbles’, while social negative influence is neutralized by the crowds’ gradual and summative corrections.

The results obtained suggest that social influence substantially biases rating dynamics in systems designed to harness collective intelligence. In the social news aggregation website the positive herding effect was clearly topic-dependent and was deeply influenced by whether the judgment had been made by ‘friends’ or ‘enemies’.

Beyond the intelligent and sophisticated design of the experiment and the complex descriptions of the authors’ work, in my opinion their merit consists in helping us ask ourselves two relevant questions.

The first question is related to what I will call here the reputation heuristic. The authors ask how we can distinguish if the popular products are popular because of the irrational effect of past positive ratings or if the best products become popular because they are of the highest quality. Distinguishing between these two explanations reveals itself as relevant when what is important is to know if social influence produces an irrational herding effect.

The second question has to do with the limitations that the authors observe in their own experiment. I will denominate it here the challenge of living environments. Is it possible to explore individual mechanisms and aggregated opinions in living environments? Is it possible to improve our collective intelligence modifying our ability to interpret collective judgments in order to be able to avoid biased collective judgments?

The living environments referred to by the authors are no others than those in which past, present and future humanity lives. If the irrational herding effect that influences our cognitive biases has also effects on markets’ evolution, political discourse planning or our health it is due to its close relation with aspects related with our current way of life.

For this reason, to those who want to continue thinking about the philosophical, sociological and political aspects of this topic, I would recommend without doubt the fascinating book Humanity 2.0: What it Means to be Human Past, Present and Future (Steve Fuller, 2011).

No doubt, this post could be read also as an aggregated opinion written after reading the article by Muchnik, Aral and Taylor. My suggestion would be to ask ourselves not just how we should interpret but also how we should change our understanding (of the laws and patterns) of social influence bias.

Article published in Social Epistemology Review and Reply Collective, 2013 vol. 2, no. 11, 27-28

Steve Fuller (2011). Humanity 2.0: what it means to be human past, present, and future. Basingstoke, Palgrave Macmillan DOI: 10.5860/CHOICE.49-5743