RAJAR - what's the margin for error?
Follow @mediaukdiscussWe’re regularly told, with opinion polls and other pieces of research, what the margin of error is. But we’ve never, apparently, been told it with RAJAR. Doesn’t RAJAR have any margin of error?
Interestingly, the South African version of RAJAR, a survey called RAMS, has a margin of error of 5%, so it appears
This appears to show that if a station moves from a 20% share to a 20.8% share, that actually means nothing: since that’s within the margin of error. Similarly, if a radio station with 1,000,000 listeners loses 50,000 listeners, that, too, is a trick of statistics: it could be that they’ve actually gained 50,000 listeners too.
Does anyone know what the margin of error actually is for RAJAR? Why doesn’t it publish it? And why are we keen to examine even a 0.1% increase as something that is statistically accurate?
I’m no longer in the industry – and it was JICRAR in my day – but I’d love to know. Anyone?
That’s also my understanding – a margin of error of +/- 3% appears to be the norm for most surveys, including political etc. polls carried out by IpsosMori (Rajar’s research contractor).
As Richard rightly points out, with a +/- 3% margin of error, a station reporting a 20% reach, may actually have achieved a 17% or 23% reach. So the next set of results which show a 22% reach, may either be a two to five percentage points increase or even a one percentage point decrease in performance.
Which is why – as I’ve often said – in order to get a better picture you need to compare figures year-on-year and look at trends rather than take just one quarter in isolation. This irons out any seasonal swings.
Similarly it’s important to remember that most stations are on ‘half-yearly’ and ‘yearly’ survey periods. i.e. the figures published for Q4/2011 for a ‘half-yearly’ actually cover the period 27th June – 18th December. Therefore a good or bad showing in a single quarter will also affect the following quarter’s figures.
If you’re a ‘yearly’ station then any programming changes will take at least a year to work their way through.
Finally (here follows a blatant plug) my usual hourly breakdown of London’s commercial stations is available on my blog
As for JICRAR. Different methodology means you cannot directly compare any figures from that era (pre-1992) with Rajar – even though some have been seen to try.
As a side note, Media UK’s graphs also clearly mention the period of the station’s RAJAR survey. You can find them by simply searching for the station and then hitting the “listening figures” button: here is a nice example of a station that has an almost perfect growth.
The general rule of thumb is about 3% I understand. Not sure if that is a RAJAR official line but it is what the UK Radio industry work on… I think.