3 Methodology
3.1 Machine Learning Indicator
As mentioned in Section 1, SVM is a supervised machine learning technique that reveals unknown relationships between input characteristic and an outcome (Gründler & Krieger, 2021; Steinwart & Christmann, 2008), and one of its key features is that the function that maps input onto output does not have to be specified beforehand, as opposed to conventional statistical methods (Gründler & Krieger, 2021). These characteristics assist in the objective of alleviating the issue of simplistic and arbitrary aggregation methods (Gründler & Krieger, 2021).
The idea behind SVM regression is as follows. Given a certain dataset F = {(x1, y1); … ; (xn, yn)}, the objective is to find a functional relationship f ̂(xi) = yi that approximates the “true” function f(xi) = yi for all i = 1, … , n. In that effort, said functional relationship should have at most ε deviation from yi in the training data and at the same time the shape of that functional relationship should be as flat as possible (Gründler & Krieger, 2016; Smola & Schölkopf, 2004; Vapnik, 1995). What this means is that the basic rationale of SVM is, given a plane onto which the data is plotted, the algorithm creates a boundary around a line, i.e. a hyperplane, that tries to encompass the highest amount of data without violating the margins of said boundary. The distance of the boundaries of the hyperplane from its center line are defined by ε (Gründler & Krieger, 2016; Smola & Schölkopf, 2004; Vapnik, 1995). This ensures, in a hypothetical scenario, that one would “want to be sure not to lose more than ε money when dealing with exchange rates, for instance”(Smola & Schölkopf, 2004, p. 200). The support vectors are the points on the plane that fall outside of the boundary of the hyperplane. The objective of SVM regression is to reduce the error to a minimum, by keeping as many data points inside the hyperplane as possible while keeping the distance to the support vectors to a minimum. That is where the cost parameter C comes into play. It determines to what extent are violations of the boundaries, namely the points outside the hyperplane, penalized. If C = 0 then there is no cost to these violations. However, a large C can have a substantial effect on the functional relationship (Gründler & Krieger, 2016; Smola & Schölkopf, 2004).
Though sometimes the functional relationship might be linear, sometimes this relationship can also be non-linear. To deal with this issue Boser et al. (1992) suggest using a higher dimensional space instead of the plane used previously (Gründler & Krieger, 2016; Smola & Schölkopf, 2004). However, how to map the data onto a higher dimensional space is usually unknown as well as computationally infeasible. Boser et al. (1992) propose a method to overcome these issues by using a so-called kernel trick. The kernel, at its simplest, is a function that takes that non-linear problem and transforms into a linear one within a space of higher dimensionality and that is then handled by the algorithm (Gründler & Krieger, 2016; Smola & Schölkopf, 2004). Though there are many kernels, this contribution used the Gaussian Radial Basis Function (RBF) kernel, as it is broadly accepted, recommended by Guenther & Schonlau (2016) and it has, in previous research, shown promising results in robustness checks (Gründler & Krieger, 2016, 2021). This is a very simplified explanation of the process by which SVM regression is conducted, as the mathematical background does not fall under the scope of this contribution. By applying said methods and parameters the function f ̂(xi) that approximates the true function is calculated (Gründler & Krieger, 2016).
The SVM regression, for the purposes of this contribution, produced a measure of democracy by taking a sample Z that included n ≫ 0 country-year combinations of observations (i,t), for which the regime characteristics z1, … , zr were available (Gründler & Krieger, 2021). What this means is that the SVM takes a sample that contains a substantial amount of country-year observations, all of which need to have available data for the selected characteristics. The SVM regression was developed and the data wasworked on using the programming language R and the platform RStudio, as well as several packages available for it, e.g. dplyr, ggplot2, e1071 and caTools. Furthermore, a specific R package with specialized functions was created for the purposes of this contribution9. The functions created and the code used for this contribution can be found in the Appendix.
3.1.1 Priming Data
To train the model, there was a necessity for a set of cases whose level of democracy was directly observable. This was needed so that unknown functional relationship between the regime characteristics and the level of democracy could be learned (Gründler & Krieger, 2021). Regimes that laid on either side of the autocracy-democracy-spectrum were an appropriate choice to this end. This is because it is well-established that their classification is rather uncontroversial (Cheibub et al., 2010; Gründler & Krieger, 2021; Lindberg et al., 2014). For an instance, there is little debate on whether Nazi Germany in 1941 was an authoritarian state, much alike there isn’t much controversy on whether Sweden in 2020 should be considered a democratic country. As Gründler and Krieger (2021) argue, the low level of disagreement about these cases implies that their degree of democracy is directly observable.
The selection of regimes that belong was based on two indices, i.e. the EDI and UDS. These indicators were chosen mainly for two reasons. First, they have a relatively wide time coverage. Second, they both are continuous indices, which provided a finer level of detail. Regimes that were in either the upper or lower decile of either indicator were labeled as democratic or autocratic, respectively. The reason that two indices were used is that previous research has shown that this increases the quality and heterogeneity of the priming data and hence of the final index (Gründler & Krieger, 2021). The EDI was also used as target variable for the priming data.
Consensus on the priming data
It was paramount to correctly label the priming data as that determined how characteristics influence the overall index. The first step was to check whether there is a sizeable discrepancy between the priming data and other indicators. For this purpose, five other indicators were used: the DD, the BMR, the Polity V Index, the FH Index and the LIED (Gründler & Krieger, 2021). First, the priming data was compared to other established indices, to check for any noticeable incongruencies (Gründler & Krieger, 2016, 2021). Second, previous research was used to confirm, in cases of incongruencies, that there weren’t any regimes in the priming data that were outwardly unreasonable.
Representativeness of regime types
The data was also checked for representativeness in both ends of the spectrum. Autocratic regimes types can differ substantially from one another (Cheibub et al., 2010; Gründler & Krieger, 2021). The same goes for democratic types (Gründler & Krieger, 2021; Pérez-Liñán, 2020). To produce plausible democracy indices, the priming data had to account for these different types of regimes. Following the work of Gründler and Krieger (2021), the autocratic regimes in the priming data were tested thrice.
First, following the definitions of Cheibub et al. (2010), autocratic regimes were distinguished by different types of dictatorships, i.e. civil, military, communist and royal. This was done using the data provided by Bjørnskov & Rode (2020) and Geddes et al. (2014). Second, the regimes were checked for the presence of an electoral regime. Lastly, they were checked for heterogeneity in durability (Gründler & Krieger, 2021).
Democracies were controlled for institutional heterogeneity (Gründler & Krieger, 2021). Much like the autocratic regimes the steps were threefold. First, they were checked for form of government, i.e. presidential, semi-presidential and parliamentarian (Gründler & Krieger, 2021). Second, they were controlled for unicameral and bicameral parliamentary chambers (Gründler & Krieger, 2021). Finally, they were controlled for proportional and majoritarian voting systems (Gründler & Krieger, 2021).
3.1.2 Aggregation Procedure
Having selected the priming data, the next step was to transform the characteristics of individual regimes into a measure of democracy. The scale of the final index was continuous as that provides a higher degree of discrimination between regimes and it is a better scale to observe changes over time. The next step was to randomly extract a training set T out of the priming data P (Gründler & Krieger, 2021). A uniformly distributed random number generator determined the total number of observations in the training set, which was no smaller than 20% and no greater than 50% of the total number of observations in P (Gründler & Krieger, 2016, 2021).
This training set was in turn used to estimate the aggregation function, for which purpose the SVM regression tool was applied. However, there are a few methodological choices made beforehand. The penalization parameter was set to one as proposed by Mattera & Haykin (1999) (Gründler & Krieger, 2021) and the kernel to be employed was the RBF, for the reasons mentioned in Section 2 (Gründler & Krieger, 2021). The margin parameter 𝜀 was set to 0.025 (Gründler & Krieger, 2021). The data was also centered and scaled, as the variables had varying scales.
This returned a model containing the aggregation function fs, which in turn could be used to calculate the democracy index for all other country-year observations in sample Z. Lastly, these steps were repeated for 2000 iterations (Gründler & Krieger, 2021). This ensured that the aggregation procedure converged and hence reduced the error to a minimum (Gründler & Krieger, 2021).
The aggregation output consisted of three categories (Gründler & Krieger, 2021). The first was the continuous index of democracy of each country-year combination in the sample Z. This was done by taking the median of the 2000 iterations that were produced by the aggregation procedure (Gründler & Krieger, 2021). The second output was the standard deviation of said iterations for each observation (Gründler & Krieger, 2021). The third was all percentiles of all iterations for each observation (Gründler & Krieger, 2021). This in the end was a dataset with the identification information for each iteration, e.g. country and year, the resulting SVM index, the standard deviation of all iterations, 100 columns representing each percentile of the aggregation results for all 2000 iterations. The second and third categories are useful to reflect on the extent of measurement uncertainty and can be used to calculate a confidence interval for each observation (Gründler & Krieger, 2021).
3.2 Data Selection
The primary source of data was V-Dem’s database (Coppedge, Gerring, Knutsen, Lindberg, Teorell, Alizada, et al., 2021), which includes a plethora of information for a variety of countries and a relatively long timeframe, as alluded to previously. Following Munck’s and Verkuilen’s (2002) guidelines and Gründler’s and Krieger’s (2021) previous work, the set of variables included both objective and subjective/expert-based data. Naturally, not all of the over 4000 variables in V-Dem’s dataset were used. The variables were chosen as they related to the constitutive aspects of Embedded Democracy (Merkel, 2004) as outlined in Section 2. Most of them related to mid-level indices in V-Dem’s dataset. In such cases, all corresponding low-level indices were used, so as not use mid-level indices aggregated using BFAM, and hence isolate the aggregation function. The only two mid-level indices employed, were either manually coded or were objective values. Furthermore, this contribution focused on the timespan from 1919 to 2020. The reason for this was twofold. The simplest one was data availability, particularly when checking for consensus with other indices and representativeness of the data (Bjørnskov & Rode, 2020; Coppedge, Gerring, Knutsen, Lindberg, Teorell, Alizada, et al., 2021; Geddes et al., 2014). Furthermore, it facilitates comparison with the SVDMI index in Section 5.
3.3 Analysis of Results
The data was used to create four datasets. One took the same variables as an existing index, namely the EDI. This was done to isolate the aggregation functions and assess their distinct effects on the individual indices. In that effort the SVM index was compared to the EDI, the reasons for which are described further in Section 5. The observations with deltas larger than 0.01 were separately analyzed using Ordinary Least Squares (OLS) regression to assess which variables had a higher impact on each individual index. The results are further exemplified with select cases from this subset. This was described in further detail in Section 5.
The other three encompassed variables that relate to individual democratic regimes and contexts as established by Merkel (2004) and discussed in Section 2. There was no overlap between regimes/contexts in terms of variables, meaning that a specific characteristic of democracy was only related to one regime/context of an Embedded Democracy. The result was three datasets. One including only the electoral regime, a second including all internally embedded regimes and a third including the externally embedded regimes as well. This was later applied to assess whether a broader definition of democracy helps identify gradations in democratic quality. Three select cases exemplify how conceptualization, regardless of aggregation function, might lead to a decreased ability of an index to identify variations in the quality of a democracy. This was described in further detail in Section 5.
3.4 Summary
In this section the concept of SVM was shortly explained, to contextualize in simple terms how the regression calculated the score of democratic quality for each observation. It was explained how that took place in technical terms and what parameters are important in that effort, based on recommendations established by the current literature (Gründler & Krieger, 2021; Guenther & Schonlau, 2016; Mattera & Haykin, 1999). Paramount in this case was the selection of the training data. In this section it was outlined how this data, i.e. the priming data containing reasonable cases of democracy or autocracy, was selected. This priming data was controlled for consensus among democracy indices and its representativeness in terms of specific regime characteristics. Furthermore, the overall aggregation procedure was described as well as the output of the SVM regression. Lastly it was described how the data that was used for the analysis, how it was selected, as well as how the analysis was to be conducted.
It is available at https://github.com/jlhilgert/demMA and can be installed onto R using the devtools package and the install_github function that comes with it using the command install_github(“jlhilgert/demMA”).↩︎