2017 Ranking Adjusted for Economic Development
Methodology of adjusting for levels of economic development.
In our main rankings, the performance of a country is measured against world- best practice. But comparisons of performance should also be made with that of countries at similar levels of economic development. More precisely, how well does a country perform on each of our criteria relative to its level of per capita income? In order to adjust for national levels of income we regress the values for each variable, in original units, on GDP per capita using data for all 50 countries. The GDP we use is for 2014 in US dollars measured in Purchasing Power Parity (PPP) terms. Both linear and quadratic relationships are used. Logarithmic models performed less well. Given the tenfold range in GDP per capita across our 50 countries, values for countries at the very top and bottom ends of the income range show some sensitivity to functional form. The values of all but one of our 19 variables in the Resources, Connectivity and Output modules increase significantly with GDP per head (the only exception is the unemployment variable, O9). The coefficient on the quadratic term was always negative, implying some tapering off of increases at high levels of GDP per capita.
The fitted equation gives the expected value of a variable for a nation’s level of income. The difference between the actual and expected value will be positive or negative depending on whether a country performs above or below the expected value. In the few cases where data are missing we assume that the variable takes the expected value for that country’s level of GDP per capita, that is, we assume a deviation value of zero. For the two Output variables based on the Shanghai rankings (O4 and O5) the presence of zero values limits the use of regression, so instead we rank the countries by GDP per capita and take a moving average of actual scores to derive more robust estimates of predicted values.
In aggregating over variables we first express deviations from the regression line as a percentage of the average of the actual and predicted values. To use the percentage deviations from the line would ignore the fact that the predicted values below the line are capped at 100 per cent whereas there is no limit above the line. Our method ensures symmetry in that values that are half what is expected at a given level of GDP per capita have the same influence as values that are double those expected. By construction, our calculated deviations lie in the range –200 per cent to +200 per cent. The average deviation for each module is a weighted sum of the deviations for each of the measures within the module. The method of measuring deviations needs to be borne in mind when interpreting the weighted average numerical scores for each module and for the overall ranking.
We use the same dependent variables and weights as described in section 3 with two exceptions. The exceptions are research expenditure (R4 and R5) and publication output (O1 and O2) where in each case we had a measure expressed in two different forms. This becomes unnecessary when we control for differences in income levels. We delete R5 and move the weight to R4, so that each of the four measures of Resources has a weight of 5 per cent in the overall ranking. In the output module we use as a single publication measure the number of articles divided by (total) GDP, thus combining O1 and O2 (the weights are added).
Results after adjusting for levels of economic development
Expenditure levels are best described by a linear relationship with GDP except for research expenditure where a quadratic curve fits best. The highest ranked country is Malaysia, which devotes nearly 50 per cent more to resources than what is expected given its income level. Resources devoted to higher education are over 30 per cent more than expected in Serbia, Turkey and Ukraine. Next in rank order are Sweden, Finland, Denmark, Canada, India and Saudi Arabia where scores are at least 20 per cent above those expected.
Compared with the non-adjusted rankings, the biggest improvers are Brazil (up 37 places to 12th), India (up 31 place to ninth), South Africa (up 30 places to 11th ) and China (up 24 places to 19th.) At the other end of the income range, Singapore falls from fourth to 33rd and the United States from sixth to 20th.
Turning to the four variables that are included in the Resources module, government expenditure and total expenditure on higher education show only slight increases as a share of GDP as income levels rise. For each ten thousand dollar increase in GDP per capita, government expenditure is estimated to increase by only 0.07 per cent of GDP and total expenditure by 0.08 per cent. It follows that rankings are similar to those discussed in section 3.1. The top eight countries for the level of government expenditure after adjusting for GDP per capita are Ukraine, Saudi Arabia, Finland, Malaysia, Austria, Denmark, India and Turkey. The highest ranked countries for total expenditure as a share of GDP are now Ukraine, Chile and Malaysia, but despite their high income levels the United States and Canada remain ranked at fourth and fifth, respectively. Expenditure (which includes research expenditure) per student increases markedly with income levels: on average by around USD361 for each USD1,000 increase in GDP per capita (adjusted R2 = 0.730). The top six countries on an income-adjusted basis are, in order, Brazil (data for public institutions only), South Africa, Malaysia, the United Kingdom, India and the United States.
Research expenditure in higher education as a share of GDP increases with GDP per capita, but at a declining rate. The quadratic regression estimates imply that at GDP per capita of USD25,000 the expected expenditure on R&D is 0.32 per cent of GDP whereas the corresponding figure at GDP per capita of USD50,000 is 0.58. The top eight countries for research expenditure as a share of GDP are now Serbia, Denmark, South Africa, Malaysia, Turkey, Portugal, Sweden and Finland.
In principle, the creation of a favourable environment is independent of income levels so we do not carry out regression analysis. Instead, we use mean values for expected and calculate the percentage deviation from expected as was done in other modules. The rankings are necessarily very similar to those for the unadjusted data.
The scores for the top three countries (the United States, New Zealand and Australia) are around 20 per cent above expected values.
The connectivity measures are quite strongly positively related to levels of GDP per head, with the adjusted R2 values lying in the range 0.4 to 0.6. The top eight countries, in rank order, are South Africa, Ukraine, Serbia, the United Kingdom, New Zealand, Switzerland and Denmark.
The equation for international co-authorship (C2) implies that for each USD10,000 increase in GDP per capita the percentage of articles that have an international co-author increase by 5.3 percentage points. The top six countries, in rank order, are Chile, Saudi Arabia, South Africa, Bulgaria, Belgium, Sweden and New Zealand.
Relationships with industry reveal different emphases on more informal links through knowledge transfer (C5) versus ‘basis research links’ as exhibited through joint publications (C6). Knowledge transfer is the more important in Israel, Malaysia and China: Israel is first on knowledge transfer but 32nd on joint publications; the corresponding rankings for Malaysia are third and 50th, and for China, seventh and 21st. Conversely, in Ukraine and Indonesia basic research links are more important: Ukraine is first on joint publications but 31st on knowledge transfer; Indonesia third on publications and 28th on knowledge transfer. India, on the other hand, is in the top five for both measures.
China and the United States are the highest ranked countries for Web impact (C4).
All but one of the Output measures (unemployment, O9) show a significant increase with levels of GDP per capita but the increase flattens out at high income levels. Two Output measures show a particularly strong relationship with GDP per capita (adjusted R2 > 0.6): impact as measured by citations (O3) and researchers per head of population (O8). The impact measure picks up not only the quality of research but its nature: applied research in developing countries is unlikely to be highly referenced despite its relevance for economic development.
The top five ranked countries for Output are Serbia, Portugal, Israel, China and the United Kingdom. For these countries Output is between 20 and 40 per cent above expected values for their levels of income. Turning to the components, the top six countries for publications relative to (total) GDP are Serbia, India, Portugal, Australia, Slovenia and Singapore. After adjusting for differences in income levels, the impact of publications (O3) is highest for India, South Africa, the United Kingdom, Italy and Denmark. China and the United States are ranked at the top for the quality of the best three universities; the next three ranked countries are well above their expected values: Brazil, Russia and the United Kingdom.
After allowing for income levels, Ukraine is ranked first on participation rates (O6), followed by Greece, Chile, Argentina and Turkey. Ukraine also comes first on tertiary qualifications of the workforce (O7), followed in rank order by Russia, Israel, Canada and Japan. Israel is first for researchers per head of population, next in rank are Serbia, Korea and Finland.
The overall score is calculated by weighting the percentage deviations for each module using the weights as for the unadjusted data: Resources (20%), Environment (20%), Connectivity (20%) and Output (40%). The median aggregate score is minus eight per cent so that a score above this level can be interpreted as above average for the 50 countries we consider.
The top three countries in rank order are Serbia, the United Kingdom and South Africa. These are followed by three of the Nordic countries: Denmark, Sweden and Finland. Compared with the original rankings in Section 3, nine countries improve their ranking by at least twelve places. These countries, in order of the ranking improvement, are Serbia, South Africa, India, Portugal, China, Brazil, Ukraine, Croatia and Greece. In several of these countries real income growth has been low or negative in which case some stickiness in higher education performance will translate into an improvement in the GDP-adjusted rankings.
The largest fall in rank compared with the Section 3 results is that of Saudi Arabia. While the United States and Singapore are still judged to be above average in performance, they each fall 14 places compared with the Section 3 results.