"Human Freedom" versus "Social Progress"


Two country rankings -- the Human Freedom Index (HFI) and the Social Progress Index (SPI) -- are created by people who have some sharply opposing sociopolitical views. Surprisingly, however, the two indices produce very similar country rankings. They also both show strong correlations to democracy scores and GDP per capita, but no strong relationships to income inequality or happiness.

The similarity of the two indexs' rankings is due to the fact that statistically the behavior of both the HFI and the SPI can be largely explained by just one of their sub-components which measures personal freedom (a philosophical value both indices agree on). Illustrating how index sub-components can drive overall index behavior is a cautionary tale of why these types of “preference indices” are misleading guides for public policy.

The two indices close correlation means their composite country ranks cannot be effectively used to quantitatively differentiate between their creators’ philosophies. A "reverse-engineering" exercise which compares their country rankings against individuals’ responses to a wide variety of questions indicates both indices seem to end up differentiating countries largely based on cultural views towards discrimination and religious authority.

Two Different Social Philosophies

Human Freedom Index

The philosophical underpinnings of the HFI are candidly described in the Human Freedom Index 2016 report. They state that the HFI’s goal is to measure “negative liberty”:

“In the simplest terms, negative liberty means non-interference by others. Berlin contrasts that type of liberty with positive liberty, which requires an individual removing of constraints that impede one’s personal improvement or the fulfillment of his potential. When positive liberty, however, is imposed by others, it undermines negative liberty since individuals naturally have conflicting views on whether and how to achieve self-improvement. As in the case of the totalitarian systems of the 20th century, this allows rulers to ignore the wishes of people and commit torture and other atrocities in the name of some higher form of freedom. Berlin further warned, as did Friedrich Hayek, against the not-uncommon tendency to call “other good things”—think of income or housing, for example—“freedom,” since this merely causes confusion.

Negative liberty is the only kind that can be adequately measured. That is because “it comes in only one flavor—the lack of constraint imposed on the individual,” whereas positive freedom is far more likely to mean different things to different people and thus cannot be measured independent of the goals that conflicting ideologies or groups might identify with freedom.”

The index is co-produced by the Cato Institute, the Fraser Institute, and the Liberales Institut at the Friedrich Naumann Foundation for Freedom. All three of these groups are described generally as “conservative” and “libertarian.” The Cato Institute was founded by the Charles Koch Foundation in 1974; the Koch brothers are considered the "arch enemy" of the left.

The index is equally weighted between “Personal Freedom” and “Economic Freedom.” The components of both sub-indices are illustrated in the HFI country page for Venezuela (which ranks near the bottom in their country index):


The key components of Personal Freedom are straightforward and relate to protection of personal safety, free speech and the right to make all other personal decisions free of interference. Their Economic Freedom index reflects a desire for small government and regulation, strong property rights and preservation of the value of a country’s currency. The HFI priciples closely align with the stated platform of the US Libertarian Party which received 4.4 million votes (3%) in the 2016 presidential election.

As with all of these preference indices (there are at least 12 not counting the SPI), the underlying data are not their own. They are mixing data recorded by other organizations like the World Bank, OECD, University of Maryland and the International Gay and Lesbian Alliance (they are all described in Appendix B of the 2016 report linked above). All of their data is freely downloadable.

Social Progress Index

The philosophy behind the SPI is based on the writings of Amartya Sen, Douglass North, and Joseph Stiglitz. Stiglitz, an economist, served as President Clinton’s Chairman of the President’s Council of Economic Advisers. His writings assert that free markets do not necessarily lead to social justice or market efficiency. Economist Amartya Sen is also critical of free market dynamics. Echoing the Cato Institute’s description of “positive” and “negative” liberty, Sen’s capabilities approach is reported to focus “... on positive freedom, a person's actual ability to be or do something, rather than on negative freedom approaches, which are common in economics and simply focuses on non-interference.” While not as directly aligned with a US political party, the SPI values overlap with concerns found within the platform of the Democratic party which received 65.5 million votes (48%) in the 2016 presidential election.

The SPI’s four design principles are described on its FAQ page:

  • “Exclusively social and environmental indicators: our aim is to measure social progress directly, rather than utilize economic proxies. By excluding economic indicators, we can, for the first time, analyze the relationship between economic development (measured for example by GDP per capita) and social development rigorously and systematically. Prior efforts to move “beyond GDP” have comingled social and economic indicators, making it more difficult to disentangle cause and effect.
  • Outcomes not inputs: our aim is to measure the outcomes that matter to the lives of real people. For example, we want to measure the health and wellness achieved by a country, not how much effort is expended nor how much the country spends on healthcare.
  • Actionability: the Index aims to be a practical tool that will help leaders and practitioners in government, business and civil society to implement policies and programs that will drive faster social progress. To achieve that goal, we measure outcomes in a granular way that links to practice. The Index has been structured around 12 components and 52 distinct indicators. The framework allows us to not only provide an aggregate country score and ranking, but also supports granular analyses of specific areas of strength and weakness. Transparency of measurement using a comprehensive framework helps change-makers identify and act upon the most pressing issues in their societies.
  • Relevance to all countries: our aim is to create a holistic measure of social progress that encompasses the health of societies. Most previous efforts have focused on the poorest countries, for understandable reasons. But knowing what constitutes a healthy society for higher-income countries is indispensable in charting a course to get there.”

The SPI was “...designed by Professor Michael E.Porter and The Social Progress Imperative working in collaboration with economists at the Massachusetts Institute of Technology (MIT) and leading international organizations in social entrepreneurship, business, philanthropy, and academia including Cisco, Deloitte Touche Tohmatsu Limited (DTTL), Skoll Foundation, FundaciĆ³n Avina, and Compartamos Banco.”[1]

The index is equally weighted between “Basic Human Needs”, “Foundations of Wellbeing” and “Opportunity.” The sub-indices are illustrated below (their index methodology is very well described and their underlying data is freely downloadable ):
2016 Social Progress Index   Social Progress Imperative.png

Preference Indices

I used the term “preference indices” to describe both the HFI and the SPI. What I mean by this is that neither index measures an objective fact like rainfall by country. They are both a type of “beauty contest” designed to measure how close a country meets the index designers’ “ideal” country.

To take it out of the realm of politics and social policy, imagine I believe the “best” horse was Roy Rogers' horse "Trigger." You on the other hand, are a huge fan of the thoroughbred "Seabiscuit." I take objective measurements of hide color, height, weight and so forth to measure how every horse in the world compares to Trigger. You do the same for your "Seabiscuit" index. We then each average our measurements by country to rank countries on their "Trigger-ness" or "Seabiscuit-ness." Our country rankings will be quite different, but neither index will help answer the esoteric question of which horse type is the best. We have simply just constructed indices that measures how closely a country fits our horse preferences. To settle the dispute between us we might make reference to other characteristics we did not include in our indices that people might care about. For example we could correlate our rankings to average country "horse speed" or "ability to do movie tricks." We will see that both the sponsors of the HFI and the SPI are interested in their index’s correlations to other social data they did not directly measure within their indices.

Politically inspired “beauty contest” country rankings have been around for awhile and are subject to criticism that their main purpose is to advance the values and policy goals of their sponsors. For example, in 1941 President Roosevelt allegedly encouraged the formation of Freedom House -- which ranks countries on various freedom criteria -- as a way to convince US citizens to enter World War II. The authors of both the HFI and the SPI state clearly that their goal is to help shape public policy.

Similarity of Country Rankings

On the face of it, one might expect the country rankings of these two indices to be quite different based on the philosophies of their founders. One group believes in small government and free markets; the other distrusts markets and believes government is critical to ensuring social welfare. I have no data to back it up, but I suspect the personnel involved with the two indices don’t sit on each other’s boards, contribute to each other’s organizations, vote for the same political candidates, or mix socially.

Nevertheless, Figure 1 shows that there is an 86% positive correlation between the HFI and SPI country ranks, meaning that if the SPI ranks one country as better than another, the HFI agrees almost all of the time. The SPI explains about 70% of the variation of the HFI. If the two indices were a “where do you want to live” guide, both groups would want to live in about the same place (SPI top 5 ares: Norway, Sweden, Switzerland, Iceland and New Zealand; HFI top 5 are: Hong Kong, Switzerland, New Zealand, Ireland, and Denmark). Both indices agree that Canada is more desirable than the UK; and that both of those countries are more desirable than the United States.

Figure 1: HFI versus SPI

Thus it is not surprising that both indices show similar relationships to GDP/capita, democracy, income inequality (as measured by the GINI ratio), and average World Value Survey Life Satisfaction.

Figure 2 shows that both are highly correlated to GDP/capita, although the SPI is much more so (R-squared to log[GDP/Capita] is 79% versus 34% for HFI). The HFI 2016 report is pleased by the relationship and cites it as evidence that human freedom leads to prosperity. The SPI 2016 report also shows a graph of the relationship but does not make a claim about causality. The SPI has a higher correlation with GDP/Capita because wealth is required to pay for some of the sub-components it includes in its index (e.g., Internet Users - R-squared 82% with GDP/capita; Access to Advanced Education R-squared 77%; Access to piped water - R-squared 76%; Access to improved sanitation facilities - R-squared 75%). The HFI index does not include sub-components that are so obviously connected with wealth (the HFI sub-component most correlated with GDP/capita is Civil Justice with an R-squared of 58%).

Figure 2: Relationships with World Bank GDP/Capita (PPP version) 2010-2014 Average

Figure 3 shows that both are highly correlated to democracy ratings from the Economist Intelligence Unit’s Democracy Index (2014).

Figure 3: Relationships with Democracy

Again the HFI 2016 report includes a graph of this relationship and is pleased to use it as evidence that democracy is an important component of human freedom. The 147 page SPI 2016 does not bring the relationship up; the word “democracy” only occurs twice -- once in a discussion of Nepal and the other because the name of the institute of one of its Advisory Board members includes the word “democracy”.

As shown in Figure 4, neither the HFI or the SPI have a strong relationship with income inequality as measured by the GINI coefficient (R-squared of 11% and 9% respectively).

Figure 4: Relationships with Income Inequality (World Bank GINI Coefficient 2010-2014 Average)

Conversely to the case with democracy, it is the HFI 2016 report that is silent on the absence of a relationship of their index to income inequality (the words “inequality” or “GINI” do not appear in their report). The SPI report mentions “income inequality” once (in the context of hoping their index will be useful in understanding it). However its website FAQ explicitly acknowledges the weak relationship between the SPI and income inequality. Oddly (to me) they counter the weak inequality relationship by stating the SPI does provide insight into inequality because it has a strong correlation with global variations in the percentage of people living in extreme poverty. Given that Figure 2 shows a strong relationship between the SPI and GDP/capita, that result is hardly surprising. One does not need their index to know that GDP/capita -- and thus extreme poverty -- varies considerably around the world.

Finally, Figure 5 shows that neither index has a strong relationship with happiness as measured the the World Value Survey (Wave 6) Life Satisfaction question (see an earlier post for a discussion of happiness measurements). The SPI’s R-squared is 18%, while the HFI’s is 6% and likely statistically insignificant.[2]

Figure 5: Relationships With Happiness (WVS Wave 6 Life Satisfaction)

The HFI 2016 report is silent on happiness except for an assertion that the creation of the index has spawned literature on the link between freedom and happiness (no citations were given and I have not found literature explicitly discussing happiness and the HFI). The SPI report also does not discuss the relationship, but it’s website FAQ has a link which does:
Once one controls for GDP, there is no separate impact of the Basic Human Needs or Foundations of Wellbeing dimensions on subjective wellbeing; there is, however, a quite robust and independent impact of Opportunity on life satisfaction.”
In fact if you break it down to the individual sub-components within Opportunity, it is specifically “Tolerance for homosexuals” and “Freedom over life choices”[3] that are key drivers of the SPI’s overall relationship with Life Satisfaction.

This last observation -- that we need to look underneath the hood down to the individual sub-components to understand the behavior of these indices -- contains the seeds for understanding two things: 1) Why the policy implications of these two different country rankings are misleading, and 2) why the two indices are so closely correlated.

The Composite Index Soup

The HFI and SPI are two different soups made up of “equally weighted” ingredients. If one wanted to try and use these indices as something more than just a reflection of their creators’ values, one needs to correlate them to other topics of interest like happiness or fostering democracy. However, even if one can find a correlation, it generally driven by individual ingredients in the soup -- it is not an endorsement of the whole soup.

For example, an undisciplined thinker is likely to see the HFI’s strong correlation with democracy as an endorsement of the entire HFI policy package: small government, free markets and individual rights. However of the seven HFI sub-components, statistically only “Rule of Law”, “Expression & Information” and “Legal System and Property Rights” are significant drivers of that relationship. The other four policy prescriptions could be ignored if you principally care about democracy. You don’t have to buy the whole philosophical package.[4]

Similarly all of the SPI’s correlation with democracy is driven by the “Opportunity” sub-component (just as its weak correlation with happiness is). The other policy prescriptions -- “Basic Human Needs” and “Foundations For Well Being” -- have no statistically significant relationship with democracy. Again from a statistical perspective, to foster democracy you don’t need to buy the whole policy package that the SPI adds on to the Opportunity sub-component to arrive at its country ranking.[5]

Close Correlation Of HFI and SPI Results

Understanding the key role of the individual index sub-components is also key for understanding why the HFI and SPI are so closely correlated. The relationship between the HFI Personal Freedom sub-component and the SPI Opportunity sub-component explains the close correlation of the overall HFI and SPI results. The chain of correlation is as follows: 1) HFI is largely driven by its sub-component Personal Freedom (PF), 2) the SPI can be almost entirely re-created by only knowing its Opportunity (O) sub-component, 3) the PF and O measure similar traits and are very highly correlated, thus 4) the HFI and SPI are closely correlated. Concisely:

  • HFI is driven largely by PF
  • SPI can be nearly recreated only knowing O
  • O≅PF, so…
  • HFI ≅ SPI.

HFI Is Heavily Driven By Personal Freedom: While ostensibly the HFI is a 50/50 blend of its Personal Freedom and Economic Freedom sub-indices, the “Personal Freedom” (PF) half of the HFI has 73% more variability than the “Economic Freedom” (EF) half. Because EF values are so similar (and somewhat correlated with PF), it is the widely different PF scores that are playing the biggest role in separating countries. By itself the PF Index score has a correlation of 0.94 (where 1.0 is perfect correlation) with the HFI; PF on its own explains 88% of the variability of the HFI. It is not clear to me that the HFI’s creators intended PF to be so dominant as their 2016 report talks conceptually about PF and EF being “equally important.”

Figure 6: Greater variability of HFI’s Personal Freedom sub-index

The SPI Opportunity sub-component explains 88% of the SPI’s Rankings: While the SPI is not dominated by one of its three sub-components (they all show similar variability), all three of the sub-components are highly correlated with each other.[6] That means if one knows just one of the sub-component values, one can nearly recreate the entire SPI country score. Figure 7 illustrates this.

Figure 7: Recreating SPI only knowing Opportunity sub-component

Close correlation of SPI Opportunity and HFI Personal Freedom: Very early on you probably realized that the HFI Personal Freedom and the SPI Opportunity sub-components share a significant number of common values. They both measure the rights of individuals to make choices related to sexuality and relationships, the value of a free press, and the value of personal safety. It does not seem to me that SPI authors would philosophically disagree with anything within the HFI’s “Personal Freedom” half of the HFI index. Similarly I don’t believe HFI authors would have much disagreement with anything within the SPI’s “Opportunity” third of the SPI index. Figure 8 shows the close correspondence of the two sub components (Opportunity has a correlation of 0.87 with Personal Freedom and explains 75% of its variance).

Given this strong correlation, plus the fact that these components explain so much of their parent indexes -- it is not surprising that the HFI and SPI are so closely correlated.

Figure 8: Agreement between SPI Opportunity and HFI Personal Freedom

Cream of Mushroom Soup
Consequently it is very difficult to use the two indices to compare the validity of the index creators’ philosophies. It is as if two people argue about whether mushrooms taste better than broccoli and decide to prove their respective positions by serving a room full of people one cup of cream of mushroom soup and one cup of cream of broccoli soup. To their dismay they find the same people like both the mushroom soup and the broccoli soup and vice versa. Because cream is the dominant flavor and texture, all they have discovered is who likes cream soups and nothing about preferences for mushrooms versus broccoli.  In this case, because Personal Freedom/Opportunity is the dominant flavor of the HFI and SPI indices, their creators have largely only discovered where Personal Freedom/Opportunity is high and where it is low. They have not discovered much about their competing preferences: small government/free markets versus provision of social needs and environmental values.
Cream of Broccoli Soup

So What does the HFI or SPI tell you?

We have discussed the methodological and conceptual problems of an amalgam of measurements designed to measure how countries measure up against a set of stated preferences. If the creators of the SPI and HFI have not been successful in designing a way of distinguish between the merits of negative liberty versus positive liberty, what message is embedded within their similar country rankings?

Let’s imagine someone handed you both sets of country rankings but told you absolutely nothing about how the rankings were derived. For all you know they could be based on country views about religion, or perhaps support for income redistribution. To gain an insight into what these mysterious rankings could be related to, you could compare them to the answers of the 400+ questions that were asked of 80,000+ people around the world in the World Value Survey (Wave 6 2010-2014). Table 1 does this (but see footnote [2]):

Table 1: Highest relationships between World Value Survey question responses and HFI and SPI country rankings. Questions in grey appear for both indices

Human Freedom Index Correlation To World Value Survey Questions
R-squaredIssueHigh Ranked CountriesQuestion
51%SexismDisagreeOn the whole, men make better business executives than women do
51%SexismDisagreeOn the whole, men make better political leaders than women do
50%SecularDisagreeDemocracy: Religious authorities should interpret the laws.
47%AgismDisagreeCompanies that employ young people perform better than those that employ people of different ages
46%Family AuthorityDisagreeOne of my main goals in life has been to make my parents proud
46%DemocracyDisagreeDemocracy: The army should take over when government is incompetent.
45%SexismDisagreeIf a woman earns more money than her husband, it's almost certain to cause problems
44%SexismDisagreeWhen jobs are scarce, men should have more right to a job than women
42%SexismDisagreeA university education is more important for a boy than for a girl
42%SecularDisagreeWhenever science and religion conflict, religion is always right
Social Progress Index Correlation To World Value Survey Questions
R-squaredIssueHigh Ranked CountriesQuestion
64%Gay RightsAgreeJustifiable: Homosexuality is justifiable
64%Free PressAgreeInformation source: Internet important source of news
63%SexismDisagreeOn the whole, men make better business executives than women do
63%AgismDisagreeCompanies that employ young people perform better than those that employ people of different ages
62%SecularAgreeJustifiable: Divorce
59%SexismDisagreeOn the whole, men make better political leaders than women do
59%SecularAgreeJustifiable: Sex before marriage
57%SexismDisagreeA university education is more important for a boy than for a girl
56%SexismDisagreeIf a woman earns more money than her husband, it's almost certain to cause problems
56%Gay RightsDisagreeWould not like to have as neighbors: Homosexuals

Based on this “reverse-engineering”, both indices seem to end up differentiating countries largely based on cultural views towards discrimination and religious authority. It is striking how many of the same questions are highly correlated to both indexes. Highly ranked countries in both indices are tolerant, care about women's' equality, and prefer secular values. It is also interesting that economic questions referring to wealth, the role of the government or income inequality -- areas that would be disputed by the HFI and SPI creators -- do not appear in the top 10 most highly correlated questions.

Transparency and reproducibility:  All of the labeled Figures and Tables can be generated by my code using the free, publicly-available R program to analyze the free publicly available data available from the links in the article.  

[2]  Unfortunately the World Value Survey only covers about 60 countries, so only about half of the SPI and HFI country rankings can be used in this analysis.
[3]  The percentage of respondents answering satisfied to the question, “Are you satisfied or dissatisfied with your freedom to choose what you do with your life?”
[4]  Conversely it is “Economic Freedom” that is responsible for the HFI’s strong correlation with GDP/capita, not “Personal Freedom”
[5] As with the HFI, it is “Basic Human Needs” and “Foundations For Well Being” that drive the relationship with GDP/capita, not “Opportunity”.
[6] Both Opportunity and Basic Human Needs have a 0.89 correlation with Foundations of Well Being; Opportunity and Basic Human needs have a correlation between themselves of 0.82


Popular Posts