Sampling checks in GE 2015
The Elections Department released early indications of polling results for the first time during the 2015 General Elections by publicly announcing the result of sampling checks within 2 hours of the close of polls. Sampling checks have been carried out since 2001 or earlier, but the results were not made known to all candidates or the public until this year. The sampling check is carried out by Counting Assistants drawing a sample of 100 ballots from each counting table after opening the ballot boxes and mixing the ballot papers together. As each counting table corresponds to one polling district and there are about 2,000-3,000 voters per polling station, the 100-ballot sample per polling district corresponds to 3-4% of votes cast and is a large enough sample to make a good estimate of the final polling result
As shown in Table 1, the margin of error varies from ±1.2% points for a large GRC, Pasir Ris-Punggol with 66 polling districts, to ±4.4 % points for Potong Pasir, the smallest Single Member Constituency (SMC) with only 5 polling districts. For simplicity, no adjustments are made to account for the variation in number of voters in each polling district, and the error margin is calculated for a 50:50 vote split. The Straits Times has reported that the error margin for sampling counts is ±4% but this is a worst case and only applies to Potong Pasir which is the smallest SMC by far. On average, the uncertainty in the sampling check is within 1.3% points for GRCs and 3.3% points for SMCs.
|Table 1 – Error margins for selected constituencies and SMC and GRC averages|
|Number of polling districts||Sample size||Margin of error at 50% vote share|
|Pasir Ris-Punggol GRC||66||6,600||1.2%|
|East Coast GRC||32||3,200||1.7%|
|Bukit Panjang SMC||11||1,100||3.0%|
|Potong Pasir SMC||5||500||4.4%|
|Error margin is calculated as half the width of the two-sided 95% confidence interval (Normal approximation) at 50% vote share. All polling districts within a constituency are assumed to have an equal number of voters.|
In a first-past-the-post election, however, what counts is not the actual vote share but crossing the 50% threshold. If we ignore three-cornered fights and make use of the normal approximation again, we can calculate a “victory threshold” or minimum sampling check result for which a candidate can be 95% certain of victory. This is shown in Table 2 for selected constituencies. Note that the victory threshold is slightly lower than would be obtained if we simply added the error margin from Table 1 to 50% because the victory threshold is a one-sided rather than a two-sided test. If you’re not statistically-inclined, don’t worry about it – that effect is small in this case. In Aljunied, despite the appearance of a nail-biting finish, the 52:48 sampling check result was above the victory threshold for the Workers’ Party and the PAP in fact had only a 0.2% chance of winning in that constituency once the sampling check result was known [See note 1]. In Punggol East, however, the sampling check result of 51% for PAP was lower than the victory threshold of 52.4% and the PAP could only be 75% certain that they would win there. Put another way, Lee Li Lian still had a 25% chance of winning even after seeing the sampling check results. In all the other constituencies, the winning party’s sampling check result significantly exceeded the victory threshold so the final result was not in doubt once the sampling checks were completed.
|Table 2 – Victory threshold for selected constituencies.|
|No. of polling districts||Victory threshold||Actual PAP share in sampling check||Probability of PAP win given sampling check result|
|Pasir Ris-Punggol (6-member GRC)||66||51.0%||73%||100%|
|Aljunied (5-member GRC)||50||51.2%||48%||0.2%|
|East Coast (4-member GRC)||32||51.5%||61%||100%|
|Sengkang West (SMC)||13||52.3%||63%||100%|
|Punggol East (SMC)||12||52.4%||51%||75%|
|Potong Pasir (SMC)||5||53.7%||68%||100%|
|1. Victory threshold is defined as the minimum sample count result for which a candidate can be 95% certain of receiving over 50% of the actual vote (one-sided test).|
|2. Probability of PAP win is the probability that the actual PAP vote share is > 50%, given the observed sampling check result.|
The error between the sampling check and the actual result gets smaller as the sample size increases. Hence GRCs will have smaller error margins than SMCs and 6-member GRCs will have smaller error margins than 3-member GRCs. This is seen in Figure 1 where the difference between the sampling check and the actual result was only 0.1% points in the 6-member Pasir Ris-Punggol GRC while the largest difference of 2.6% points was observed in MacPherson SMC. The observed differences between sampling check and actual vote counts were all within the expected 95% error margins except for one constituency, but one out of 29 is about right, statistically. We also did not take rounding errors into account, which slightly widen the margin of error.
The sample check is a form of a quick count, which is used in developing democracies where there are concerns with regards to the compilation of electoral results by the central government. In Singapore’s case, there is no obvious need for a quick count as the entire counting process can be observed by candidates’ counting agents and elections results have always been announced within a few hours. Nonetheless, given that the Elections Department has chosen to conduct sampling checks, the decision to publicly reveal sampling check results is a welcome one.
 Some additional uncertainty is caused by the sampling check results only being reported as whole number percentages. The true PAP sampling check result from 5,000 samples could have been as high as 48.5% rather than the reported 48%. However, this would still have given the PAP only a 1.8% chance of victory once the sampling check results were known.
Sampling check results and actual vote counts are tabulated in Sample check vs actual v3 (Excel format).
A version of this note was previously published on The Online Citizen, http://www.theonlinecitizen.com/2015/09/how-accurate-is-the-ge-sample-vote-count/.
Filed under: Elections, General Elections, Sampling check, Validity check | Leave a Comment