What went wrong and what went right with our latest prediction?

The results of the Croatian election came as a surprise to many. All the pollsters (in total only 4, in addition to our BASON survey) gave the SDP-led coalition an advantage from 3 to 7 seats over HDZ. In fact, the election result for the first two parties was the exact opposite of what the polls were saying – it was SDP that should have gotten 59 seats, and HDZ 54 seats. The reason why this hasn’t happened is the crucial question on the minds of domestic political pundits, pollsters, some voters, and most political enthusiasts.

It is difficult to pin down the answer to this question on a specific factor. There are a number of hypotheses being discussed on social networks, with low turnout being the most popular one. According to this hypothesis the SDP coalition lost more voters due to low turnout than did HDZ (105,000 compared to 75,000 votes). The third party, MOST, lost even more due to low turnout (they got 180,000, down from 300,000 last year), however still not enough for them to fall below the 5% electoral threshold. On the other hand, the anti-establishment Živi zid got 20,000 more votes than last time (when they got 95,000), which were distributed all across the country – just enough for them to pass the 5% threshold in 8 out of 10 electoral districts and hence get 8 seats in parliament. The second popular hypothesis is that turnout had nothing to do with it since it was evenly distributed across the electoral districts (in other words, it fell equally across the board). Rather the result was led by the swing from the voters of MOST to the HDZ, and from SDP to Živi zid (so across a similar ideological spectrum), as depicted by the graphs below.

It is possible that both hypotheses are true. The ill-conceived campaign by the SDP leader and former PM Zoran Milanović who flirted with nationalism could have turned some left-wing SDP voters either towards the hard-left Živi zid, or towards abstinence. In a similar fashion, the dissatisfied MOST voters did not go to either SDP nor the alternative third option party Živi zid, but also abstained from voting or to a lesser extent supported HDZ. Neither of these hypotheses is possible to prove without a direct analysis of voter preferences with specific questions like: “who did you vote for on last year’s elections, and who did you vote for now?”


None of this however answers the question of why the polls got it so wrong (again). Even the exit polls, which are usually quite precise, suggested a tie between the two top parties, with the final result being a +5 advantage for HDZ. This is still within the 5% margin of error, however most voters don’t react too kindly to the pollsters citing their success within the error margin. Citing the margin of error is important precisely in order to tell the voters that the results of the polls should be taken with a pinch of salt. Polls may suggest an outcome, they may be very good in showing a clear trend, but to be perfectly precise, even on election day, is impossible. After all, if the polls were perfectly precise, if polling was an exact science, there would be no point in having elections at all. Nonetheless, we won’t touch upon why other pollsters failed. We will only analyze the hits and misses of our own BASON survey.


Our BASON survey

In our survey, apart from asking our respondents who they will vote for, we also asked them to give us a prediction of how much each party would get (in percentages) in each electoral district. And while our method has built-in mechanisms to control for group bias for each survey respondent, the problem was obviously the collective illusion of all respondents that the SDP-led coalition would win. An illusion that was further enhanced by other polls. After all, the perception that this election will be a shoe-in for SDP has been perpetuated since May when there was first speculation that the HDZ-MOST government was going to fall. This certainly affected the expectations of the survey respondents which firmly believed in a relative victory of the SDP. In other words, the method in its current form failed to break the collective bias of its respondents.

In order to recognize the potential problem, in our post-election analysis of the data we looked at the individual predictions of the voters for each party separately. For each group of voters there was an obvious overestimation of their preferred party’s chances, and an underestimation for all others, especially those ideologically distant. The interesting thing is that by looking solely at HDZ voters on the level of the entire country, on average they produced expectations that HDZ would get 65 seats, while SDP 53. A clear overestimation, but an interesting one nonetheless. This doesn’t make them good predictors (last year they gave their party an advantage of 15 seats, while the result was a tie), but it does suggest that they had a felling, even if it was overly optimistic, that their party would defeat SDP. On the other hand, those same respondents underestimated their own party in the four electoral districts around Zagreb (three of which are swing districts, and one is an SDP stronghold). In these districts their estimates of HDZ’s vote share where lower than its actual vote share. So even the optimistic HDZ voters in these districts did not expect such a strong result.

This could imply that we would need to assemble a better sample for our survey. In other words, we should aim to have around the same number of right-wing and left-wing voters (about a third of each, plus a third of all other voters), in order to more accurately capture the aggregate feelings of our respondents, even if they suffer from confirmation bias. If the survey had the same number of HDZ and SDP voters this would have most likely increased HDZ’s seat projection in our prediction.

As far as the smaller parties are concerned, the method proved its resilience and gave incredibly accurate results. We could again notice the overestimation of one’s own party and the underestimation of others, but taken together and adjusted for the biases, the predictions for the smaller parties were excellent. The BASON survey got a correct prediction on all the smaller parties within their marginal seats, where each outcome came within the highest or the second highest probability implied by the Monte Carlo simulation. This is shown on the graph below where the red line depicts the result of this election, while the black line represents the results of last year’s election. Within we have the probability distribution of the seats for each party generated by the simulation from our prediction (a longer line implies a higher probability).


Finally, we show the relative change in votes for HDZ, the People’s coalition, and for MOST on the map of Croatian cities and municipalities. It is clear how the SDP coalition lost the majority of their votes in their strongholds (15% in Zagreb, almost 20% in Rijeka and Istria). This is highly likely due to their party leader’s attempt to move too far to the right during the campaign, forcing the party to lose its traditional left-wing supporters. The only increase came in the small cities and municipalities outside of Zagreb which testifies to the positive influence of including the centrist farmer’s party HSS in their coalition (although the increase was rather small; the dark red colour in the map represents any increase in votes).  The HDZ, on the other hand, increased its number of votes in Zagreb and across Istria by roughly a few thousand votes, while also experiencing a decline of support in its traditional strongholds – Slavonia, Lika, and Dalmatia. Their decline was albeit smaller than that of the SDP in its strongholds. The third party MOST lost more than 50% of its votes in Zagreb, Rijeka and their surroundings, and even more in Lika and Mid-Dalmatia. The reason why they were able to keep their 5 seats from Dalmatia (2 in the 9th district, and 3 in the 10th) was due to increased support in other parts of Dalmatia, particularly in the South.


hdz_pad mostpad

Share This