2 Literature Review
2.1 Conceptualization
The conceptual breadth of democracy is a very controversial topic, where “it is rather unlikely to expect agreement on the ‘one’ concept” (Geissel et al., 2016, p. 572). However, this is still important to discern. Both narrow and broad (and everything in between) conceptions have their merit and pitfalls (Gründler & Krieger, 2020, 2021; Munck & Verkuilen, 2002; O’Donnell, 2001). Depending on the kind of research, a narrower approach might be more appropriate as to avoid conceptual overlap and avoid insufficient data availability. A narrow concept, or a too restrictive criteria selection, might lead an indicator do lose discriminatory power and overlook developments in democratization or democratic backsliding (Gründler & Krieger, 2020, 2021; Munck & Verkuilen, 2002). “If the definition of democracy involves only a low standard, then differences among established democracies can hardly be identified. In other words, gradations of democratic quality cannot be detected” (Geissel et al., 2016, p. 572). This contribution, hence, did not intend to outright answer this controversial question, but to shed a light on the perils of a low-standard selection criteria for democratic characteristics (Geissel et al., 2016; Gründler & Krieger, 2016; Merkel, 2004).
For the purposes of this contribution Wolfgang Merkel’s (Merkel, 2004) concept of Embedded Democracy was employed. It presupposes, that democratic systems are embedded in two ways. For one, it is embedded internally, where “the specific interdependence/independence of the different partial regimes of a democracy secures its normative and functional existence” (Merkel, 2004, p. 36). Furthermore, it is also embedded externally, where “these partial regimes are embedded in spheres of enabling conditions for democracy that protect it from outer as well as inner shocks and destabilizing tendencies” (Merkel, 2004, p.36).
2.1.1 Internal Embeddedness
There are 5 partial regimes of a liberal democracy: political rights, civil rights, horizontal accountability, the effective power to govern, and a democratic electoral regime (Merkel, 2004). These regimes are embedded internally, meaning that one regime is dependent on other regimes. They support each other’s functioning, e.g. civil rights and rights of participation supporting the conduction of democratic elections. However, they also guarantee that the political actors of one does not infringe on the other, e.g. by guaranteeing horizontal accountability between powers, the likelihood of a breach of civil rights or effective power to govern can be dramatically reduced. Meaning that these regimes are simultaneously independent and interdependent (Merkel, 2004).
For starters, the political rights regime is a precondition for elections (Merkel, 2004). At its core it relates to the freedom of expression and freedom of association. Societies are complex and this regime enables diverse opinions and positions to be discussed publicly and to coordinate in order to influence the political landscape. Public and private media should not only exist, but also be independent from political pressure (Merkel, 2004). Furthermore, citizens should have the freedom to form organizations and coordinate in order to influence political outcomes, without interference. This includes political parties, which should have the freedom to coordinate and express themselves, without politically motivated restrictions (Merkel, 2004). This guarantees that the recurring check on political power, embodied by elections, is supplanted by a soft and steady check in between them (Merkel, 2004).
The civil rights regime protects individual rights from the reach of the state or even a ‘tyranny of the majority’ (Merkel, 2004). Rule of law has to be guaranteed, i.e. the state’s power is limited by upholding the laws effectively according to clearly delineated prerogatives, as well as protecting certain laws from the reach of the legislature, the executive and even a majority of citizens. Basic constitutional rights lie at the core of the rule of law (Merkel, 2004). “Individual rights of protection grant legal protection of life, freedom and property – the threefold meaning of Locke’s term property – as well as protection against illegitimate arrest, exile, terror, torture or unjustifiable intervention into personal life, both on behalf of the state and on behalf of private or anti-establishment forces and individual actors. Equal access to the law and equal treatment by the law are basic civil rights” (Merkel, 2004, p. 40). However, this alone does not suffice. To uphold the three powers, i.e. legislature, executive and judiciary, need to be distinct and accountable to each other (Merkel, 2004).
The regime of division of powers and horizontal accountability makes sure that no power encroaches on the political and civil liberties of the citizens (Merkel, 2004). The exercise of executive power is particularly limited in this case. To that end an independent and functional judiciary is necessary, which can review executive and legislative action. Though, in the concept of Embedded Democracy, both civil rights and horizontal accountability require an independent judiciary (Merkel, 2004), for the purposes of this contribution the independence of the judiciary was included exclusively in horizontal accountability, as it relates more closely to how the three powers relate to each other. Vertical accountability, i.e. elections, control the government periodically. Civil rights create a barrier against state abuse towards the individual. Horizontal accountability implies that the three bodies check on each other reciprocally, however none of them should dominate or interfere with the core-sphere of responsibility of others (Merkel, 2004). These constitutional powers maintain a constant check on the responsiveness and accountability of the government (Merkel, 2004).
The partial regime of the effective power to govern, as the name already suggests, underscores the requirement that those who are elected to high office are also those that govern in the end (Merkel, 2004). While in a not-too-distant past this might have seemed self-evident, in the present more complex power structure make it so that this partial regime cannot be taken for granted. Extra-constitutional actors, e.g. the military or other powerful actors not subject to democratic accountability, should not have final decision-making power (Merkel, 2004).
Lastly, but arguably most importantly, the electoral regime grants access to positions of state power incumbent on the results of open and competitive electoral contests (Merkel, 2004). This regime is positioned at the center of all other regimes. It is the sine qua non that at its simplest separates potential democracies from autocracies (Merkel, 2004). This regime borrows from Dahl’s (1989) concept of polyarchy and hence presupposes that democratic elections are supported by 4 elements: elected representatives, free and fair elections, universal and active suffrage, as well as universal and passive right to vote (Merkel, 2004). Elections are periodical sanctioning mechanism and when coupled with the partial regime of political rights, it embodies the essence of vertical accountability. By virtue of being periodic, though, it does not alone suffice to guarantee a democratic regime, as mentioned previously. While politicians might align themselves politically with the priorities of citizens, in the interest of getting reelected, this on its own is not reliable safeguard for democratic government (Merkel, 2004).
2.1.2 External Embeddedness
Furthermore, democracies are embedded externally (Merkel, 2004). They exist in the socioeconomic and civic-societal environment. These environments are not composed of defining characteristics of a democracy, however they encompass, enable and stabilize a democratic regime (Merkel, 2004).
The Socio-Economic Context
For one, a democracy benefits substantially from a stable and fruitful socioeconomic environment, or as Wolfgang Merkel calls it the “Socio-Economic Context” (Merkel, 2004, p. 44). In short, the “more well-to-do a nation, the greater is the chances that it will sustain democracy” (Lipset, 1959). This relationship has been observed often in the last four decades and has proven remarkably stable (Cheibub et al., 1996; Cutright, 1963; Dahl, 1971; Lipset, 1959, 1993; Merkel, 2004; Vanhanen, 1984, 1989; Welzel, 2000). Beyond the usual metrics of economic development, e.g. gross domestic product (GDP), there is also the influence played by socioeconomic inequality. Unequal distribution of economic resources can lead to a deterioration in the quality of a democracy if a consistent part of the populace is pushed below the poverty line (Merkel, 2004). The rationale is that extreme socioeconomic inequality can serve as an effective hurdle for political equality. Citizens will more likely be able to form their own opinions as active members of the citizenry, if they do not fear for their physical and economic security and if they are sufficiently educated through a sufficiently developed social and economic status (Merkel, 2004). The economic underprivileged part of the populace, in the exercise of civil and political rights, are at a disadvantage (Merkel, 2004).
There are caveats to be made, though. First, the influence of inequality has notable exceptions, namely developing countries and countries in the so-called third world, tend to have an economically disadvantaged population that shows “low-intensity citizenship” (Merkel, 2004; O’Donnell, 1998). Second, a developed economy is not the sine qua non for democratic systems. Alone, it cannot predict whether a country is democratic or not (Merkel, 2004). Neither is there a direct correlation with the level of democratization. Furthermore, economic development cannot on its own predict thresholds or transition zones for democratization (Merkel, 2004).
However, these factors can create a socioeconomic environment that is conducive to a healthy democratic regime (Merkel, 2004). “To sum up, a developed economy, the prevention of extreme poverty, the pluralization of the social structure, and the fair distribution of the material and cognitive resources of society create a shield for democracy and, in most cases, enhances the quality of democracy with regards to the rule of law and participation. Inversely, both the lack of a well-developed economy and abrupt downward economic change endanger the stability and the quality of a liberal democracy” (Merkel, 2004, p. 45).
Civil Society
The notion that a well-developed civil society strengthens democratic systems is not novel. It is based on arguments brought forth by philosophers of the past, as well as more modern theorists (Merkel, 2004). The environment of civil society has to fulfill certain functions to support a democratic regime (Merkel, 2004). For starters, it has to protect individuals from a state wielding its power arbitrarily (Merkel, 2004). Second, it should support the balance of powers and rule of law (Merkel, 2004). Third, civil society should adequately educate the citizenry, as well as recruit political elites (Merkel, 2004). Lastly, it should establish a ‘pre-institutional’ public sphere of critical discourse, where self-determined forms of participation are able to influence political agendas beyond the traditional political and business interests (Merkel, 2004). “In a strict sense, civil society does not belong to the defining core of a constitutional democracy. It is rather outside of this core and therefore may be regarded as externally embedding it. The functions civil society carries out, however, have considerable implications for the sustainability of democratic constitutional institutions” (Merkel, 2004, p. 47).
International and Regional Integration
The stability and quality of a democracy can be affected by the integration of nations into international as well as regional organizations, that stand on an economic or politically democratic basis. There are caveats, nonetheless. Military or security focused alliances do not impart a similar effect on democratic systems, even if largely occupied by a democratic member base or seemingly committed to democratic principles (Merkel, 2004). Arguably the most prominent case in that regard is the North Atlantic Treaty Organization (NATO) (Merkel, 2004). Historically, the organization has been dominated by democratic states (Merkel, 2004; North Atlantic Treaty Organization, 2020). Furthermore, the alliance was founded as bulwark against an authoritarian state, namely the Union of Soviet Socialist Republics (USSR) (North Atlantic Treaty Organization, 2022). This is also officially embodied by the North Atlantic Treaty, which states in its preamble that its parties are “determined to safeguard the freedom, … founded on the principles of democracy, individual liberty and the rule of law” (North Atlantic Treaty Organization, 2022), as well as the Membership Action Plan published in 1999 stipulated that as a requirement for prospective members (North Atlantic Treaty Organization, 1999). These are both, however, non-binding and can be ignored at will, which the alliance has not refrained from doing in the past. While dominated by democracies, the pool of founding members included Portugal, which in 1949 was notably under the leadership of dictator António Salazar (Wagner, 2010). More recently and perhaps notably, Turkey’s democracy has deteriorated substantially in the last decade (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Alizada, et al., 2021; Elçi, 2019; Merkel, 2004). Both of these cases have largely been ignored, likely due to their strategic importance, as foreign policy security is the focus of the alliance (Merkel, 2004).
On the other hand, organizations that are founded or promote on democratic principles have a higher likelihood to positively influence the stability of democracy. That is no guarantee though. For an instance the promotion of democracy, human rights and the rule of law are mentioned prominently in the constitutive act of the African Union (AF) and the institution has also implemented many instruments to that end (African Union, 2000; Mangu, 2014). However, since its founding, the continent has seen multiple, recurring breaches of constitutional rights, election integrity and human rights (Mangu, 2014; Sarkin, 2018). A similar dynamic applies to Association of Southeast Asian Nations (ASEAN), for instance. Despite multiple references to democracy and human rights as core principles of the organization, most members are not considered democratic and a measly half of them have commissions focused on human rights (Poole, 2019).
MERCOSUR1 at first might seem like an exception to that trend. Though not part of its founding document (MERCOSUL, 1991), in 1998 the MERCOSUR adopted the Ushuaia Protocol, which established democratic principles as a cornerstone of the organization as it envisions healthy democratic institutions as “essential condition for the development of integration processes between States party to the … Protocol” (MERCOSUL, 1998). Furthermore, it establishes sanctioning mechanisms against states that rupture their own democratic order, as other member states can vote to suspend rights accordingly (MERCOSUL, 1998; van der Vleuten & Hoffmann, 2010). However, in 2012 the MERCOSUR underwent its first expansion when Venezuela joined the organization. This was controversial on its own, since Venezuela not only lacked a lot of the characteristics of a healthy democracy, it had also been on a steady downward trend for the preceding decade, in stark contrast to the other member states (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Alizada, et al., 2021; García-Guadilla & Mallen, 2019). Four years after its accession, though, the other member states made use of the suspension power against the left-wing government in Venezuela indefinitely. However, this action came after right-wing governments took office in Brazil and Argentina, hence “the politics of enforcement shouldn’t be conflated with the politics of adoption of democracy clauses” (Closa & Palestini, 2018, p. 26). Furthermore, many of its members have experienced downward trends in many democracy indices in the recent past, including its arguably most important members, i.e. Brazil and Argentina (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Alizada, et al., 2021; Freedom House, 2021).
Merkel (2004, p. 48) argues that neither “the Association of South-east Asian Nations (ASEAN) nor South America’s MERCOSUR and NATO have comparable effects [to the EU’s], because they are not committed to the principle of democratic values in the same way”. The de facto European Union was established with the signing of the Treaty on European Union2 in 1992 (European Union, 1992), however its predecessor, the European Economic Community (EEC), already embodied some of its tenets, specifically on matters pertinent to the economy and trade (European Economic Community, 1958). However, it does not include commitments towards a democratic political order (Kubicek, 2003). This changed with the Treaty of Maastricht which established the promotion and maintenance of democracy, the rule of law, and human rights a core objective of the EU, as well as the 1993 Copenhagen Criteria, which established democratic criteria for membership (European Union, 1992; Kubicek, 2003).
This democratic tradition, despite being only formalized in the early 1990s (Kubicek, 2003), has been rooted in the roster of members of the EU. Since its founding literally all member states scored above 0.5 in the EDI and were ranked as “free” in FH indicator. This only changed in 2018 when Hungary got an EDI score of 0.489 and FH ranked Hungary as “partly free”, which means it stands alone amongst all 28 current and former EU members to do so (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Alizada, et al., 2021; Freedom House, 2021). A decline has also been seen in Poland. Once in office the ruling parties of both countries, i.e. the Fidesz party in Hungary and Law and Justice (PiS3), began bearing its influence on key independent public institutions to impede their use of veto power (Kovács & Scheppele, 2018). First and foremost, their sights were set on the constitutional judiciaries. Once neutralized, they were unable to serve as a check on the power of those in the executive and legislative. Furthermore, this allowed ordinary judiciaries to be molded as to make it more difficult for individuals and opposition group to mount a challenge against the government (Kovács & Scheppele, 2018). In short, “[j]udicial independence, once quite strong in both Poland and Hungary, is now a thing of the past” (Kovács & Scheppele, 2018). This encroachment on the separation of power has taken place in the form of constitutional reform that amended the role of higher courts, changing selection procedures for constitutional judges and election rules of presidents of the court, hence putting more power in the hands of the legislature (usually dominated by the ruling party), increasing the number of judges (so-called “court packing”), restricting the competencies of the court by restricting the power of constitutional review, nullifying case law established by the Constitutional Court prior to the constitutional reform, removing abstract review (once the most common type of constitutional review) , by which any citizen could bring a constitutional matter to the court, reducing the age of retirement for judges, hence packing lower courts, amongst other changes (Kovács & Scheppele, 2018).
These developments were not left without a response, albeit belated and generally ineffective. As Hungary’s “attacks on the independence of the judiciary continued over eight long years, some of the European Union institutions took note, made repeated criticisms, but ultimately did not succeed in altering the course of events substantially” (Kovács & Scheppele, 2018). When Hungary replaced judges by lowering the retirement ages and firing sitting judges en masse, the European Commission charged Hungary with violating the prohibition on age discrimination. Though the commission won the case, this sort of violation only incurs a compensation fine, which Hungary paid to the aggrieved party, but it did nothing to reverse the decision or penalize Hungary in any other way (Kovács & Scheppele, 2018). However, they couldn’t go further legally, since, at least the Commission perceived it then, the treaties that EU members are bound to presume that member states have independent judiciaries, so there was no explicit provision in EU law that would give the Commission the power to charge Hungary with interference of the judiciary (Kovács & Scheppele, 2018). As the Hungarian government consolidated its power over the judiciary, the European Parliament did pass several resolution denouncing its actions, but they were largely ignored by the Commission and the Council (Kovács & Scheppele, 2018). In 2018 the Parliament decided with a supermajority to trigger Article 7, which could strip Hungary of its voting rights though the procedure is still ongoing (Makszimov, 2022; Staudenmaier, 2018), and in 2022 the Commission triggered the Rule of Law Framework against Hungary, which in turn could suspend its EU funding (Deutsche Welle, 2022).
When Polish President Andrzej Duda, however, kicked off a constitutional fight with the President of the Constitutional Tribunal over the decision to force PiS to swear in three judges nominated by the previous ruling party, the European Commission, unlike with Hungary, acted rather quickly (Kovács & Scheppele, 2018). Soon thereafter, and after the Polish government ignored recommendations by the Commission, it invoked its then new tool namely Rule of Law Framework. Despite it being designed to be implemented against Hungary, which it ultimately was, its first target was the newly inaugurated PiS government (Kovács & Scheppele, 2018). These actions, rather, did little to deter the Polish Government, which quickly took over the Constitutional Tribunal and in turn allowed the PiS Government to make changes to the judiciary in general, e.g. baseless firing of judges and reduction of retirement age (Kovács & Scheppele, 2018). All throughout this process, the Commission escalated its implementation of Rule of Law Framework, the (rather quiet) Council approved the decision of the Commission and the Parliament passed resolutions against the aforementioned actions (Kovács & Scheppele, 2018). However, the Polish government “barreled ahead, unchecked” (Kovács & Scheppele, 2018). This reached its zenith when, in 2017, the Commission triggered Article 7 proceedings against Poland, which are, much like in Hungary’s case, still on-going (Bobinski, 2022; Kovács & Scheppele, 2018).
These facts present a problem for international organizations as indicators of democracy. While both MERCOSUR and the EU have overwhelmingly members that could arguably be considered democracies, both organizations have proven ineffective at preventing democratic backsliding within their ranks and, in the case of the former, not accepting member with considerable breaches of democratic principles (Closa & Palestini, 2018; García-Guadilla & Mallen, 2019; Kovács & Scheppele, 2018). The latter has had its ranks filled with democratic states. However, it has proven itself ineffective at preventing backsliding, as the cases of Poland and Hungary show (Kovács & Scheppele, 2018). The reasons for it are many. For starters, though some agreements of the EU do put an emphasis on democracy and its inherent principles, the organization does not have the necessary tools to enforce them (Kovács & Scheppele, 2018). The EU “was designed to protect Member States from an overreaching Union rather than to protect the Union from failing Member States” (Kovács & Scheppele, 2018). Hence, the European Commission has few tools at its disposal to intervene. Furthermore, member states are very reluctant to judge each other since that opens space for the judgement to be turned on themselves. While action has come from the Parliament and the Commission, the Council has remained nearly absolutely silent (Kovács & Scheppele, 2018). It has also largely ignored calls from the aforementioned institutions to do so. Lastly, but perhaps more importantly, the EU is not a federation. EU institutions were given competencies by member states, however national constitutional structure does not fall under that purview (Kovács & Scheppele, 2018). The opposite is in fact the case, i.e. national constitutions are protected from EU interference by the EU’s own legal framework (Kovács & Scheppele, 2018).
Hence while Merkel (2004, p. 48) argued that “the European Union (EU) and its precursor organizations have proven the most successful in the international embedding of democracies. … This economic and institutional embedding into community-based, strong common interests and values will also considerably stabilize the ten central and eastern European new member countries after 2004”, this contribution concludes that recent developments have shown that said stabilization effect is not guaranteed, that there is no international organization today that fulfills the expectation of this externally embedded context, and hence it was not considered in further sections.
2.2 Aggregation
A significant issue in measuring democracy, as alluded to previously, is how to aggregate those characteristics. In their Working Paper, Gründler and Krieger indicated that a machine learning based index can avoid underestimating changes in the degree of democratization, when compared to standard methods of aggregation, i.e. additive, multiplicative, additive and multiplicative, and a Bayesian latent variable approach (Gründler & Krieger, 2020). Their published research indicated that a machine learning index can produce much more consistent and plausible results, with a high degree of detail (Gründler & Krieger, 2016, 2021).
However, their previous approaches have points that can be improved upon. First of all, while the SVMDI has been compared to other approaches, the index used entirely different conceptions of democracy and variables. Hence, one could not isolate the aggregation function in order to assess its potential benefits or issues in relation to an actual existing index. Furthermore, both SVMDIs, i.e. the one created in 2016 (Gründler & Krieger, 2016) and the one created in 2021 (Gründler & Krieger, 2021), had their own specific set of issues.
While the first attempt used a relatively broader concept of democracy, the theoretical underpinning of the data selection was not well-grounded. First of all, since the variables come from a variety of different sources (Gründler & Krieger, 2016), there is a potential for conceptual overlap (Gründler & Krieger, 2021). Second, some of these variables are aggregated themselves (Gründler & Krieger, 2016), which defeats the purpose of using SVM to aggregate the data. Third, it’s timespan is rather restricted as it only encompasses 30 year, from 1981 to 2011 (Gründler & Krieger, 2016), an issue the authors themselves bring up in their later publication (Gründler & Krieger, 2021). Lastly, for a relatively broader concept of democracy, the number of variables is relatively low, i.e. 11 (Gründler & Krieger, 2016). For context, the SVM index with the narrowest conception in this contribution, based on Merkel’s (2004) Electoral Regime, also had 11 variables. This does not mean it is necessary better, but it does indicate that various characteristics of democracy might have been overlooked or simplified.
The updated SVDMI solves some of these issues, such as the restricted timespan and previously aggregated variables (Gründler & Krieger, 2021). However, in their effort to avoid conceptual overlap with other fields of research and the variable selection related to Dahl’s (1989) concept of Polyarchy, they ran into the issue raised by Geissel et al. (2016), i.e. successfully assessing gradations of democratic quality, a weakness that was further explored in this contribution in Section 6.
2.2.1 Indicator Review
One question arises, namely what is the purpose of another democracy index? Beyond the ones hitherto mentioned, there are already several ones available (Gründler & Krieger, 2021). Some of which have already been used in this contribution to exemplify variations in the level of democracy between countries. Furthermore, they are by no means monolithic. The indices of democracy use conceptions of democracy range from Dahl’s (1989) minimalistic polyarchy to broader concepts. They also vary in aggregation methods and timespans (Gründler & Krieger, 2021). This section, inspired by and updating the work done by Gründler and Krieger (2016, 2021), attempted to answer this question by going over prominent democracy indices, reviewing them and, at last, explain what gaps a new machine-learning focused index with a wider as well as modular conceptualization of democracy might fill. This section does not intend to address all existing indices or all debates around them, but this is supposed to give a short overview of the state of affairs and pinpoint possible weaknesses.
V-Dem’s Electoral Democracy Index
One of the two indices already mentioned and used in this contribution, V-Dem’s EDI4 is arguably one of the most renowned. The research project responsible for it, i.e. V-Dem, aims at providing institutional information on all countries worldwide (Gründler & Krieger, 2021). In its entirety, the V-Dem database contains more than 450 regime characteristics, which are well grounded in theory, and some of which are used to create five distinct democracy indices (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Altman, et al., 2021; Gründler & Krieger, 2021). Arguably the most prominent under these indices is the EDI (Gründler & Krieger, 2021; Teorell et al., 2019). It also serves as a basis for all other democracy indices in the V-Dem roster (Gründler & Krieger, 2016). This index is continuous, offers a confidence interval, is available for a total of 202 countries between 1789 and 2021, and uses an arguably minimalistic approach. It is built out of five characteristics, all of which are expert-based, with the exception of one, namely suffrage. The others are extent to which executive and legislative are elected, associational autonomy, free and fair elections, and freedom of expression and alternative sources of information (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Altman, et al., 2021; Gründler & Krieger, 2021; Teorell et al., 2019). They themselves, apart from suffrage, are composed of several sub-indicators, the aggregation of which depends on whether the sub-indicator in question is additive or multiplicative. The index itself is then calculated by taking the mean of the indices (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Altman, et al., 2021; Coppedge, Gerring, Knutsen, Lindberg, Teorell, Marquardt, et al., 2021; Gründler & Krieger, 2021; Teorell et al., 2019). This is explained in further detail in section 6.
And herein lies a few weaknesses. Combining additive and multiplicative indicators does not suffice to outright remove their individual methodological weakness (Gründler & Krieger, 2020, 2021). Furthermore, Teorell et al. (2019) do not provide a clear-cut reasoning for why these indicators should be equally weighed and additively combined (Gründler & Krieger, 2021). Lastly, this index is arguably minimalistic. The aforementioned characteristics do not include factors such as an independent judiciary, economic stability, socioeconomic inequality, or the role of civil society. Though, as mentioned previously, this does not necessarily mean it’s worse, it might lead the index to overlook certain developments, e.g. loss of judicial independence.
Freedom House (FH)
The second index mentioned in this contribution is Freedom House’s Freedom in the World Index. In actuality it is composed of two indices, i.e. a graded index as well as an ordinal index, though the latter is just an extension of the former. The grading ranges from 100 (most democratic) to 0 (most autocratic)5 and the ordinal scale contains three categories, namely free, partly free and not free (Freedom House, 2022; Gründler & Krieger, 2021). To come up with these indices it separates between two distinct aspects of democracy, i.e. Political Rights (PR) and Civil Liberties (CL). They are then added to form the final index. Each of these aspects are formed by 10 and 15 characteristics, respectively, each being given a score between zero and four. In the end PR can have a maximum of 40 and CL a maximum of 60 points6 (Freedom House, 2022; Gründler & Krieger, 2021).
The index includes nearly all independent countries and self-governing territories. The timespan however is much shorter than the EDI, since it starts in 1972 (Gründler & Krieger, 2021). Conceptually it is broader, encompassing characteristics such as minority rights, the existence of safeguards against corruption, and judicial independence (Freedom House, 2022; Gründler & Krieger, 2021). However, this indicator suffers from methodological weaknesses. Chief amongst these is its aggregation rule as it is a simple additive function and hence can create dubious classifications at the lower end of the spectrum (Gründler & Krieger, 2020; Munck & Verkuilen, 2002; Teorell et al., 2019). A country can theoretically score high on certain aspects but score the lowest possible on what one could argue are necessary conditions for democracy, such as free and fair elections. Furthermore, 25 expert-based regime characteristics are only publicly available from 2006 onwards (Gründler & Krieger, 2021). Finally, Freedom House tend to follow American research on democracy, which neglects “the connection between poverty and low intensity citizenship. Thus, while Germany has been scolded or even downgraded for surveying a dubious quasi-religious sect like Scientology with its intelligence service, the fact that almost 20 per cent of US citizens (predominantly African Americans) are disadvantaged through poverty in the exercise of their civil and political rights is completely ignored” (Merkel, 2004, p. 58). So while the concept might be relatively broad, when compared to the EDI, even that breadth misses components that might affect the quality of a democratic system.
Boix–Miller–Rosato indicator (BMR)
The BMR is a dichotomous indicator that categorizes a regime as a democratic, if it meets three conditions, i.e. the majority of male and female citizens are eligible to vote, the executive was directly or indirectly elected in popular elections and is responsible to voters or to a legislature, and the legislature (as well as the executive) was chosen in free and fair elections (Boix et al., 2013, 2018; Gründler & Krieger, 2021). This indicator has a few major strengths. For instance, the database is relatively comprehensive, including indices for 222 countries spanning the timespan between 1800 to 2015, and thus one of the best measures in terms of data coverage. It also is methodologically consistent, as they argue why the aforementioned institutional aspects are considered, then operationalized, and they aggregate them in accordance with their conceptual assumptions (Gründler & Krieger, 2021).
However, this indicator also possesses major weaknesses. First of all, it is dichotomous, which, as mentioned previously, lacks a lot of discriminating power. This becomes problematic, when assessing the cause or effect of democracy, as well as its development over time. It is hard (if not impossible) to distinguish gradual processes of democratization or democratic backsliding (Gründler & Krieger, 2021). Furthermore, since the conditions that form the BMR are also theoretically dichotomous, the use of subjective conditions becomes problematic, e.g. free and fair elections. For some observations that might be clear, but for many it isn’t. In such cases, there should be at least an uncertainty dummy variable, which would indicate that coders were not certain about their conclusion. One could go further and provide a confidence interval that reflects the measurement of uncertainty. However, the BMR provides neither, therefore making the identification of borderline cases impossible (Boix et al., 2013, 2018; Gründler & Krieger, 2021). The suffrage condition also only stipulates that half of the combined male and female populations have the right to vote. This leaves space for the exclusion of parts of the citizenry, e.g. minorities, from the electoral process, without technically breaking this condition. Finally, Boix et al. (Boix et al., 2013, 2018) do not provide their raw data, as recommended by Munck and Verkuilen (2002), hence impeding users of the indicator from checking whether “their results depend on the choice of the aggregation method nor whether they are caused by a specific aspect of democracy” (Gründler & Krieger, 2021, p. 4).
Democracy-Dictatorship Indicator (DD)
Much like the BMR the DD is a binary indicator, its consistency between the definition of democracy and the aggregation procedure is a positive characteristic, and it uses four conditions to classify a regime as democratic, i.e. the head of government was elected directly or indirectly, the legislature was elected through a popular elections, the parliamentary elections must be disputed by more than one party, and an alternation in power should be under the regiment of the same rules that brought the incumbent government into office (Bjørnskov & Rode, 2020; Cheibub et al., 2010; Gründler & Krieger, 2021; Merkel, 2004). The latest edition of the index encompasses 192 countries, 16 self-governing territories, 96 colonies and a timespan between 1950 to 2020. Despite being less comprehensive than the BMR in that regard, it does provide which conditions a specific country violates, if it’s not classified as a democracy (Bjørnskov & Rode, 2020; Gründler & Krieger, 2021). The obvious transparency benefit notwithstanding, this allows users to pinpoint whether their results are due to a specific area of democracy (Gründler & Krieger, 2021).
This indicator, by virtue of being dichotomous, is plagued by the issues therewith attached and shared by the BMR (Gründler & Krieger, 2021). Furthermore, the indicator has a couple of conceptual issues. For one, it does not have any requirement on political participation, e.g. suffrage, which can lead to misclassifications (Gründler & Krieger, 2021; Munck & Verkuilen, 2002). Furthermore, the requirement relating to an alternation in power can also lead to misclassification, as young democracies in a transitional phase might be disregarded as there might not have been enough time for one to take place (Gründler & Krieger, 2021; Knutsen & Wig, 2015).
Polity IV and Polity V
Much like FH, this indicator is graded, ranging from -10 (most autocratic) to +10 (most democratic) and it is composed of five institutional aspects, i.e. constraints on chief executive, competitiveness of the chief executive recruitment, openness of the chief executive recruitment, regulation of participation, and competitiveness of participation. Every country with more than 500 inhabitants is evaluated in relation to these characteristics and that is available from 1800 to 2018 (Gründler & Krieger, 2021; Marshall et al., 2019; Marshall & Gurr, 2020). The characteristics are also publicly available, meaning users can check whether their results are affected by one of these particular aspects of democracy (Gründler & Krieger, 2021).
There are a few conceptual and methodological issues, though. For starters, many observations are given so called “standardized authority scores” (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Altman, et al., 2021; Gründler & Krieger, 2021; Marshall & Gurr, 2020). These include countries with a systemic interruption due to foreign influence, cases of interregnum or anarchy, and countries in systemic transition. They are given categorical values outside of the normal range of the index. An alternative variable is proposed by Marshall and Gurr (2020), which would treat cases of foreign interruption as “system missing”, cases of interregnum or anarchy are given a neutral value of 0, and cases of transition are given equidistant scores in between the initial and final values (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Altman, et al., 2021; Marshall & Gurr, 2020). However, these solutions might still lead to spurious changes in the level of democracy (Gründler & Krieger, 2021). The database also does not offer confidence intervals or any form of measurement uncertainty, which makes it difficult to classify anocracies (Gründler & Krieger, 2021). Finally, a major issue with this index is its aggregation rule. All regime characteristics are weighted equally, the aggregation rule is simply additive, and the aggregation procedure in general lacks theoretical foundation, which results in doubtful classification, particularly at the lower end of the spectrum (Cheibub et al., 2010; Gründler & Krieger, 2020, 2021; Munck & Verkuilen, 2002; Teorell et al., 2019; Treier & Jackman, 2008).
Acemoglu–Naidu–Restrepo–Robinson indicator (ANRR)
This dichotomous indicator varies substantially from the previously mentioned indices, as it combines multiple indicators, i.e. Polity IV, FH7, DD, and BMR (Gründler & Krieger, 2021). A regime is democratic when the Polity index is greater than zero and the FH index is below 10. Should one of these indices not be available for the country in question, then the country also has to be to categorized by the remaining index, as well as be categorized as democratic by either DD or BMR (Gründler & Krieger, 2021). Should both FH and Polity not be available, then the country has to be categorized as democratic by both DD and BMR. Should none of these conditions apply, then the country is categorized as autocratic (Acemoglu et al., 2019; Gründler & Krieger, 2021).
This database contains indices for 184 countries between 1960 and 2010 (Gründler & Krieger, 2021). There is caveat to this though. Since all the underlying measures are available to the public, users can extend the timespan with relative ease, should it be needed (Gründler & Krieger, 2021). However, there are shortcomings. For one, the concepts of democracy underlying these indicators are by no means the same. FH, for instance, has a much broader definition of democracy than any of the other three (Gründler & Krieger, 2021). Furthermore, since FH only publishes data as far back as 1972, there are many countries that are not on FH. This in turn means there is the possibility that an observation is categorized as a democracy by the other indices, which would otherwise not be categorized as democracy by FH but is not on its database (Gründler & Krieger, 2021). Lastly, the classification creates spurious regime changes, a factor that Acemoglu et al. (2019) themselves pointed out (Gründler & Krieger, 2021). To overcome this issue, they manually recoded transitions based on whether they were plausible or not from their point of view, however there is no selection criteria and hence it is rather arbitrary (Acemoglu et al., 2019; Gründler & Krieger, 2021). Finally, there are no uncertainty measure for this indicator (Gründler & Krieger, 2021).
Unified Democracy Score (UDS)
The UDS is a composite index. It uses a variety of indices8 and its original version offers data for a timespan from 1946 to 2000, though it has been relatively recently updated to expand it all the way back to 1815 (Gründler & Krieger, 2021; Marquez, 2022; Pemstein et al., 2010). A major strength is that the UDS produces a measure for each country-year combination and also provides a measure of uncertainty. That being said, the confidence intervals increase towards the extreme ends of the spectrum and narrow for hybrid regimes, which, as pointed out by Teorell et al. (2019), is counterintuitive as the level of uncertainty increases towards the middle of the spectrum. Hence, these confidence levels should be interpreted with caution (Gründler & Krieger, 2021).
Lexical Index of Electoral Democracy (LIED)
Developed by Skaaning et al. (2015), distinguishes between seven political regime types, i.e. regimes without elections, with no-party or one-party elections, with multiparty parliamentary elections, with multiparty parliamentary and presidential elections, minimally competitive multiparty election for legislature and executive, with minimally competitive multiparty election and full male or female suffrage for legislature and executive, and lastly with minimally competitive multiparty election and full male and female suffrage for legislature and executive (Gründler & Krieger, 2021).
This indicator possesses one of the widest temporal coverages of all by starting in 1789 and also provides data on all sovereign countries in the world (Gründler & Krieger, 2021). Despite being complimented by clear coding rules and a well-grounded aggregation procedure (Gründler & Krieger, 2021). However, it can hardly be used in applied studies effectively. The main problem lies it its categorical scale. If used as a single variable, it presents an issue for a regression model, since the “democratic distance” between categories is not equidistant (Gründler & Krieger, 2021). There is a negligible jump, if any, from a country without elections to a country with no-party/one-party elections, in terms of its degree of democratization (Gründler & Krieger, 2021). However, that changes dramatically when a country has multiparty elections. This presents a challenge for users of that index (Gründler & Krieger, 2021).
2.3 Summary
Many of these indicators are frequently used in the scientific literature. Building upon the work done by Gründler and Krieger (2021), this contribution has reviewed and updated the overview of eight prominent democracy indicators. Further, it has pointed out weaknesses in many of them. These usually revolve around the aggregation method, or the conceptual choices therewith attached. This contribution attempted to alleviate the aggregation issue, by using SVM to aggregate characteristics, and to an extent the conceptual issue by applying a conception of democracy, i.e. Embedded Democracy (Merkel, 2004), that can increases in scope but distinguishes between different important aspects of democracy. Hence, the SVM index in this contribution avoids overlap in characteristics within itself, as well as offer researchers the opportunity to avoid conceptual overlap with their own fields, while still being able to identify gradations of democratic quality.
Also referred to as MERCOSUL in the Portuguese acronym.↩︎
Also referred to as the Treaty of Maastricht.↩︎
in its Polish acronym↩︎
Often referred to as Polyarchy Index (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Altman, et al., 2021; Coppedge, Gerring, Knutsen, Lindberg, Teorell, Marquardt, et al., 2021)↩︎
It is possible for a country or territory’s total political rights score to be less than zero (between –1 and –4) if it receives mostly or all zeros for each of the 10 political rights questions and it receives a sufficiently large negative score for the political rights discretionary question (Freedom House, 2022).↩︎
“Prior to the 2020 edition, Freedom in the World assigned a country or territory two ratings—one for political rights and one for civil liberties—based on its total scores for the political rights and civil liberties questions. Each rating of 1 to 7, with 1 representing the greatest degree of freedom and 7 the smallest degree of freedom, corresponded to a specific range of total scores. … The ratings are still included in the raw data available for download” (Freedom House, 2022).↩︎
Acemoglu et al. (2019) uses the FH ratings as they were prior to 2020, i.e. from 2 (most democratic) to 14 (most autocratic) (Freedom House, 2022; Gründler & Krieger, 2021).↩︎
It employs the DD, and FH, Polity, as well as further indicators proposed by Vanhanen (2000) Arat (1991), Bollen (1999), Bowman et al. (2005), Coppedge & Reinicke (1990), Hadenius (1992), and Gasiorowski (1996) though these have fallen into disuse (Gründler & Krieger, 2021; Pemstein et al., 2010).↩︎