We are your data science workshop.

Modelling Eurovision voting

This is a follow-up to our previous blog on Eurovision voting, where we’ll explain how we modelled the objective quality of songs and voting biases between countries in Eurovision, and how we grouped the countries into blocks based on their biases. The source code can be found here. We’ve taken the data from a Kaggle competition, and sourced data for any missing years from Wikipedia.

The hierarchical model

The idea is that the voting outcome is dependent of both the inherent quality of the entry, and the biases countries have for voting for each other. There are lots of possible ways of doing this, but ours is fairly simple and works quite well.

Let \(r_{c_ic_jy_k}\) denote the fraction of people in country \(c_i\) that voted for country \(c_j\) in the year \(y_k\). Note that \(\sum_{j=1}^Nr_{c_ic_jy_k} = 1\), so it is reasonable to model the vector \(\mathbf{r}_{c_iy_k} = (r_{c_ic_1y_k}, \dots, r_{c_ic_Ny_k})\) as following a Dirichlet distribution:

\[\mathbf{r}_{c_iy_k} \sim \operatorname{Dir}(\beta_{c_ic_1y_k}, \dots, \beta_{c_ic_Ny_k}).\]

We choose a model where the parameters \(\beta_{c_ic_jy_k}\) decompose as

\[\beta_{c_ic_jy_k} = \operatorname{Exp}\bigl(\theta_{c_jy_k} + \phi_{c_ic_j}\bigr),\]

where \(\theta_{c_jy_k}\) captures the objective quality of the song from country \(c_j\) in the year \(y_k\), and \(\phi_{c_ic_j}\) captures the bias country \(c_i\) has in voting (or not voting) for country \(c_j\). Furthermore, we assume that the \(\theta_{c_jy_k}\)’s and \(\phi_{c_ic_j}\)’s are drawn from an (unknown) normal distribution:

\phi_{c_ic_j}, \theta_{c_jy_k}\sim N(\mu, \sigma).

Note that we don’t actually have access to \(r_{c_ic_jy_k}\), we only have data on the number of points each country was awarded. But we make do with what we have and approximate \(r_{c_ic_jy_k}\) by

\frac{\text{(points awarded to country \(c_j\) by country \(c_i\) in the year \(y_k\)}) + \alpha}{(\text{total points awarded by country \(c_i\) in the year \(y_k\)}) + N\alpha},

where \(\alpha\) is a constant that we set to 0.1.

It’s hard to say for definite whether this is a reasonable approximation without being able to actually see the voting data, but preferences often follow power laws, and the decreasing sequence of points 12, 10, 8, 7, 6, 5, 4, 3, 2, 1, 0, 0, 0, … at least follow a similar shape:

Voting/Powerlaw fit

It’s not perfect, but hopefully good enough. Note that we do completely miss out on any information about the tail, but we assume that this is mostly noise that don’t contribute much anyway.

Fitting the model

We fit the model using using Stan, a programming language that is great for making Bayesian inferences. It uses Markov chain Monte Carlo methods to find the distribution of the parameters which best explain the responses. Stan is very powerful, all we really need to do is to specify the model, then pass in our data and Stan does the rest!

As Stan uses Bayesian methods, it returns a sample of the distribution of your parameters, in our case consisting of 16 000 (paired) values for each parameter. In our previous analysis we simply took the means as point estimates for our parameters, but having the distribution lets us talk about the uncertainty of these estimates. For example, the point estimate for objective quality \(\theta\) of the winner in 2015 (Sweden) is 1.63, and 1.41 for the runner up (Russia). This however, doesn’t tell us the full picture. Here’s a plot for the joint distribution of \(\theta\) for these entries:

Distribution of theta for Sweden and Russia in 2015

From this joint distribution we can calculate the probability that Sweden’s entry was objectively better than Russia’s entry as the proportion of samples above the blue line, and this turns out to be about 93%.

Finding the blocks

Looking at the bias terms \(\phi_{c_ic_j}\), we can attempt to group the countries into groups that are tightly connected, i.e. where there’s positive biases within a group and neutral or negative biases between groups. We use a method based on Information Theoretic Co-Clustering, where we choose a clustering where we loose as little mutual information as possible.

The basic idea can be described as follows: For each vote from country A to country B, take a marble and label it ‘from: country A, to: country B’. Now put all the marbles in a jar, and pick one at random. How much does knowing from which country the vote was from tell you who it was for? For example, Romania and Moldova almost always trade 12 points, so knowing that the vote was from Romania tells me there is a high probability that the vote was for Moldova. Mutual information gives us a quantitative value of this knowledge for the whole jar.

Now if we have clustered the countries into blocks, we can instead label the marbles ‘from: block A, to: block B’. We generally loose information by doing this. As we don’t know which country the vote was actually from it’s harder to predict which block the vote was for. By finding the clustering that looses the least amount of information, we get the clustering that best represents the biases.

Below is a heatmap showing the probabilities of countries voting for each other, with our identified blocks separated by lines. We do see that the blocks certainly capture voting behaviour fairly well, voting within the blocks is far more likely than between the blocks (with Lithuania and Georgia being a notable exception). Also, we can identify an “ex-Soviet block” within the “Eastern Europe” block, and a “Northern Europe” block with in the “Western Europe” block, both highlighted in gray.

Voting blocks in Eurovision

Block voting in The Eurovision Song Contest

The Eurovision Song Contest started out as a technological experiment for live tv broadcast across Europe in 1956, but has grown to become an institution, a mass spectacle of kitsch and glam with hundreds of millions of viewers across the globe. But there is one thing that is consistently blamed for ruining the competition: block voting. But exactly what are the blocks, and how big influence do they have? Let’s try and find out!

Overview of recent competitions

Before identifying any blocks, we’ll have a look at past winners to get a sense for what is going on. Below are maps showing, for the years 2001-2018, which countries the points for the winner and runner-up came from. Although block voting is clearly happening in some cases (Serbia’s win in 2007 and Azerbaijan’s win in 2011 stand out in particular), winners tend to receive points from all over Europe, and a block is rarely able to swing the competition without supporting votes from countries outside the block.

Votes for the winner (left) and the runner up (right) in the Eurovision finals 2001-2018. The darkest shade of blue means 12 points, lightest shade 1 point, dark grey 0 points and light grey means didn’t participate.

One reason why a block usually can’t determine the winner is that while they vote for each other to a higher degree, they don’t necessarily agree on one best act who scores highly from all members of the block.
Sometimes, although this doesn’t happen very frequently, a block does ‘works together’ and all vote for the same country, and with support from outside the block they can decide the winner. This happened most clearly in 2007, when all former Yugoslavian countries gave Serbia 12 points. However, Serbia wouldn’t have won without also getting consistently high scores from mainly central and northern Europe.

So which blocks are there?

While some patterns are easy to spot (Moldova and Romania almost always exchange 12 points, as do Greece and Cyprus), bigger groups can be harder to identify. Since not all countries participate in all stages (the two semifinals and the final), the data is sparse which further complicates analysis.
To tackle this, we fit a statistical model that untangles the inherent preferences that exist between countries from the overall popularity of a song. This lets us combine results from different stages (and years) in a consistent way. As the voting system has changed quite a few times over the years, we limited ourselves to the time period between 2010—2015 when the votes were determined by a combination of televotes and jury. This lets us:

  • analyse the biases countries show for each other and identify blocks, and
  • eliminate these biases to find the ‘true’ rankings each year.

We’ll explain exactly how this model works in a future post, but for now let’s have a look at the results. Once we have extracted the biases, we can use cluster analysis to help us identify the blocks (we use a method based on information theoretic co-clustering). Doing this, we find three blocks: a tightly knit block with the ex-Yugoslavian countries including Albania; an eastern block, centred around Ukraine, Georgia and Azerbaijan; and a western block centred around the Nordic countries. This is in line with previous analyses. The blocks are shown on the map below, where a darker shade indicates a stronger association with the block.

Map showing the three clusters
The three blocks we identified. The darker the colour, the stronger the association with the block.

Another way to visualise this is to look at the individual biases, shown in the graph below. Here we can clearly see the strong connections within the ex-Yugoslav block, as well as parts of the eastern block. However, the western block is much less tightly knit.
The close relationships between Romania and Moldova, and Greece and Cyprus also stand out.

Graph showing which countries favour each other
This graph shows how strong the preferences to vote for a particular country is. Each line indicates that the country on the left is very likely to vote for the country on the right (the darker the colour, the stronger the bias). The colours indicate which block the country is in.

Did the best song win?

In the process of building the model, we also get the unbiased ‘quality’ of each entry. So how do these compare to the final results? In fact we found that the ‘best’ song won in all of the years we covered. However there are some changes further down the ranks, where our model favours a third place act over a runner-up, especially when the vote is close. This is consistent with our assertion earlier that to win you need broad support, just getting votes from your block is not enough.

Year 2010 2010 unbiased
Winner Germany (246) Germany
Runner-up Turkey (170) Romania
Third Romania (162) Turkey
Fourth Denmark (149) Belgium
Year 2011 2011 unbiased
Winner Azerbaijan (221) Azerbaijan
Runner-up Italy (189) Sweden
Third Sweden (185) Italy
Fourth Ukraine (159) Denmark
Year 2012 2012 unbiased
Winner Sweden (372) Sweden
Runner-up Russia (259) Russia
Third Serbia (214) Serbia
Fourth Azerbaijan (150) Albania
Year 2013 2013 unbiased
Winner Denmark (281) Denmark
Runner-up Azerbaijan (234) Azerbaijan
Third Ukraine (214) Norway
Fourth Norway (191) Ukraine
Year 2014 2014 unbiased
Winner Austria (290) Austria
Runner-up The Netherlands (238) The Netherlands
Third Sweden (218) Sweden
Fourth Armenia (174) Armenia
Year 2015 2015 unbiased
Winner Sweden (365) Sweden
Runner-up Russia (303) Italy
Third Italy (292) Russia
Fourth Belgium (217) Australia


We have found that, although block voting happens to a high degree, it doesn’t in general determine who ultimately wins the competition. To win you need broad support, and to do that you need to put on a good show. Which doesn’t necessarily mean a good song…

_data journey

Your data can tell you a lot about your customer's journey. Our services can provide you with the information and tools that you need to match your services to customers.