Exploding Growth of Indicator Scorecards

Using data to rate countries doesn’t just help us understand their behavior, it actually changes it. Take a look at just one example. The maps below show how countries have changed their laws to criminalize human trafficking since 2001, when the United States Department of State first published the Trafficking in Persons report, to 2014.

Figure 1: Trends in trafficking legislation

Source: Kelley. Scorecard Diplomacy. Data, code, analysis and case studies available here.

The same is true for elections. Our experience with international election monitors shows that observing and reporting on elections improves their quality. Social scientists call this phenomenon the Hawthorne effect: When people are aware of being observed, they change their behavior.

A bad ranking can shame countries, but “scorecard diplomacy” goes further than just calling out failures. The scorecards also outline actions that can lead to a better score in the next cycle. Countries tend to pass new trafficking laws, for example, just in time for the deadline for the next Trafficking in Persons report. A country’s GPA—global performance assessment—can attract attention, shape debate, and, as the data indicates, change their actions. In this way, scorecards are a form of “soft power,” used in combination with, or instead of, “hard power,” such as economic sanctions or military force. They combine both “carrots and sticks.”

As new forms of information technology have made it possible to analyze and spread data globally, instantly, and at a minimal cost, the number of these types of rankings have exploded. The World Bank has the Ease of Doing Business Index, Transparency International has the Corruptions Perceptions Index, and, as mentioned, the State Department has the Trafficking in Persons Report. Indices and rankings exist now in nearly every domain—from gender equality to military power. They are one more tool for international actors to influence domestic politics.

The Evolving Fight against Human Trafficking

I explore this phenomenon in the context of U.S. foreign policy on human trafficking in a new book, Scorecard Diplomacy: Grading States to Influence the Reputation and Behavior, (Cambridge University Press, 2017).

The evolution of the fight against human trafficking illustrates the power of scorecards. While the data is unreliable, the International Labor Organization estimates there are more than 20 million trafficking victims worldwide. It is not only a horrific crime; it is also difficult to combat.

Anti-trafficking measures are costly, and countries often deny the problem. Policymakers may not see human trafficking victims as constituents, since the victims are often children, immigrants, or part of marginalized groups. Governments may even be complicit in the exploitation of workers, such as Japan’s worker trainee program, or Uzbekistan’s cotton production program. Cultural norms can also lead to different ideas of what constitutes trafficking. In the Middle East, foreign children as young as four were formerly bought to serve as jockeys in camel races.

In the early 2000s, two important developments occurred in the global fight against human trafficking. First, the United Nations adopted the Palermo Protocol to Prevent, Suppress, and Punish Trafficking in Persons. It established a definition of human trafficking that included sexual exploitation, forced labor, and organ removal.

Second, the U.S. issued its first Trafficking in Persons report. It covers three areas of action: protecting victims, prosecuting traffickers, and prevention. The best ranking, Tier 1, is for countries that make efforts to address the problem. The worst offenders are graded as Tier 3, and there is a Tier 2 and a “watch list” Tier 2 subcategory, which serves as a warning to those at risk of a Tier 3 grade.

Diplomats and other concerned parties can now combine the public release of the report with quieter forms of diplomacy. These include the use of grants, the possibility of sanctions (so far, rather ignored), and collaboration with other nongovernmental and intergovernmental organizations.

How Well Does it Work? Ask WikiLeaks

Confirmation of the effectiveness of scorecard diplomacy comes from an unexpected source: WikiLeaks. The archive includes a quarter-million diplomatic cables from 2000 to 2010, a treasure trove of data that is usually unavailable, offering a peek behind the scenes to understand how officials react in private conversations.

Analysis of more than 8,500 cables that discuss human trafficking—plus interviews of 90 people from NGOs and IGOs, as well as U.S. politicians and bureaucrats—shows that at least on the topic of human trafficking, countries are very concerned with their reputations, especially how they are ranked within a peer group and who they were compared with.

A French official noted, incredulously, “You’re telling me that France could easily be in Tier Two, with Nigeria?”

Analysis shows that some nations have hired public relations firms to help improve their image and that countries in the bottom tier are more likely to pass anti-trafficking laws in an effort to improve their rankings.

As any teacher knows, disappointed students frequently contest grades. Similarly, countries have accused the U.S. of arrogance and meddling in their internal affairs. In 2006, Belize’s Prime Minister Said Musa said on the evening news, “But those who seek to judge us should perhaps examine their own decadent societies before they come and pass judgment on us.”

Countries in Tier 3 or on the Watch List often try to save face by criticizing the validity of the rankings. Sometimes they have good cause. The policy is not perfect; nonetheless, it has been effective. Numbers, rating, and rankings provoke concern for international reputations, demonstrating the power of strategically deployed information.

Scorecard diplomacy uses reputation like a sculptor might use a chisel. By using it to target his or her force, the sculptor delivers power more effectively than the use of a hammer alone.