Because some of us "commentators" have the sense to look at our databases of the previous 1000+ polls in the country, track true underlying opinion, and point out the statistically obvious.
A 6.4% topline change in the TPP including a 7.8% change on the TPP among women in 7 days? Without a leadership change? When 2 published and 3 unpublished polls (combined: n=~4200) picked up changes substantially less than half of that over the same 7 day period?
Yet somehow people that pay attention to those inconvenient details are dismissed, as you've done, as some sort of third rate wannabe psychics or [deleted for abuse]?
[Deleted for abuse]
2
Sunday, 01 August 2010 11:15
graham
Oh dear. Is the possum stirred? Language.
Your 1000 polls have no bearing on anything as they weren't taken last week. You summarise the polls that were thus:
"Morgan got a 53% TPP to the ALP over the 27th and 28th, Galaxy got a 50% TPP on the 28th (and maybe on the 27th as well – I can’t find if Galaxy was in the field for one night or two) and Nielsen got a 48% TPP over the 27th, 28th and 29th. So Nielsen is the odd one out here, suggesting a bit of overshoot – and probably combined with the ALP week getting worse as it went on."
Your bias is showing. They're all within sampling error. Nielsen isn't the "odd one out" at all, it would appear you are.
3
Sunday, 01 August 2010 12:30
Scott Steel (aka Possum Comitatus)
The 1000+ polls taken previously have an enormous bearing because they show, empirically, the reality of the inter-temporal sensitivity of political public opinion. We can see the historical pattern of the maximum limits of public opinion change over any given period and compare it to current polling outcomes on the one hand (the raw context), and use robust statistical processes to look through the noise of public opinion produced by multiple, *quality* polling results over some given arbitrary period on the other hand, to derive an ultimate estimation of not only the true underlying state of public opinion at a given time, but how it has changed through time.
During the same period that Galaxy and Morgan phone poll were in the field (2 of those 3 days happening to overlap with Nielsen being in the field), there were two other polls in the field during the exact same period, and a further poll being in the field on the 29th & 30th - all three of which were consistent in the change in their topline results with Galaxy and Morgan, and none of which showed a nearly 8% change among female voters.
First derivatives matter.
Nielsen was the odd one out, not because of any inherent problem they had (far from it), but because of a mix of time and chance alone - they came in among a cluster of polls that were in the field between the 20th-23rd of July when the ALP had a peak in support (Morgan phone poll also picked that up among the published polls, having an ALP TPP of 55.5% on 20/21 July while an unpublished poll during the same 4 day period had the ALP TPP on 53% and where their subsequent polling had that number falling by 2 points).
Nielsen had it on 54% even though the trend estimate at the time was 53% - which meant that a group of other polls were below the 54 and higher topline result during the same 5 day period.
Nielsen then came in with a low 48 - a vote estimate that *no other* pollster picked up during that same period, and a magnitude of change which no other pollster picked up *during that period* - even though all pollsters during that period showed a decline in the same direction.
This isn't 'bias' - its the statistics of variance. If you want to tell me all about sampling error, particularly the uncertainty as it applies to time series, it would help your cause if you understood what it actually means!
I can remember previously, for instance, when you have stated that because a subsequent poll result was outside of the margin of error of the previous poll result, then it is a significant movement - overlooking the fact that there is two probability distributions involved (the first poll *AND* the second poll) rather than just one (let alone design effect issues - but that's for another day), a consequence of which is that it blows out the interval that determines the difference between two means. A common enough mistake, yet one which made me bite my tongue at the time, coming as it did from a bloke that runs polls and should be across that.
But if you are going to lecture me about how a lack of grasping sampling error shows my bias (rather than just an actual understanding of the numbers involved), it would help your cause immensely if you were across the field of knowledge you are talking about.
4
Sunday, 01 August 2010 12:48
graham
So now you're claiming that there is some maximum limit to changes in public opinion? You really ought to quit while you're ahead. Have you caught up with the latest Newspoll? It shows it 50/50 too. So Morgan appears to be out on his own if you want to pick on one of a group of polls that have all given an answer within samping error.
5
Sunday, 01 August 2010 13:19
Scott Steel (aka Possum Comitatus)
What we can see is the maximum size of movements during modern polling history over a 7 day period, and that the movement Nielsen suggested over its last two polls was way outside of what we have seen in the modern historical polling record.
And yes, not only have I seen Newspoll's 50/50 result - I predicted it based on the *very* methodology that you are accusing me of bias for utilising:
That actually makes your bias worse. So you expected a fourth poll to come in 50/50 meaning three out of four polls made it even or 52/48 to the Coalition while one (with a small sample) made it 53/47 percent to Labor, but you say the 52/48 one is out of line.
That you fluked predicting the Newspoll result doesn't make you precise. That you ignored what you thought would happen means your bias was showing.
It's nonsense to suggest that there is some sort of maximum movement in polling. Collapses in vote can happen in very short periods of time. Just because you haven't seen it doesn't mean it doesn't exist. People aren't constrained by mathematical formulae - they make up their own minds. I've watched tracking polling whip around much more than this in the past.
And looking at your first post on this thread in the clear light of morning I think you owe me an apology.
7
Monday, 02 August 2010 00:51
Scott Steel (aka Possum Comitatus)
If I fluked at predicting the Newspoll I must have rainbows coming out of my bum, considering the regularity with which it occurs. It's not a fluke, it's not psychic -it's just maths.
There could well be record movements at any time - but there wasn't here, the data didn't suggest it. It didn't suggest it at the time, and subsequent polling results have demonstrated it.
When you get one poll showing an extraordinary movement you need to take a serious look at it, as such extraordinary movements in the true underlying value of public opinion are rare at the national aggregate level... it's why they are, er, "extraordinary". When the magnitude of that movement is not replicated by *any other* polling series in the field during the same period, it speaks for itself.
People aren't constrained by maths, but tracking true underlying opinion without getting sidetracked by random variation cannot be done without substantial dollops of it. It also cannot be done unless time itself is utilised as a variable.
The proof is in the pudding. The data heavily suggested the Nielsen had overshot at the time, I stated that at the time, subsequent data demonstrates it to be true. The only person surprised at this appears to be you - calling it a fluke.
Which sort of gets back to the original point - polling analysis is an exercise in tracking the density of certainty through a time series of overlapping probability distributions. So when you said "So why do these commentators bother with polls if they know what happens in the real world better than the polls?" - simple, the real world involves more than any single poll taken in isolation. To treat it any other way, such you did, is an invitation to get sidetracked by random variation.
As for an apology - I'm sorry I was right, apparently it annoyed you.
8
Monday, 02 August 2010 02:38
graham
All this verbiage and aggression just to avoid admitting that you were wrong and that you made a basic error. You can put as many arguments into my mouth as you like but it doesn't change the fact that Nielsen was more likely to be correct than Morgan. For the record, and if you bothered to read my article you'd know, my guess is that neither Morgan nor Nielsen are correct and that Galaxy and Newspoll are probably about right.
Of course it is only a guess and no-one will ever know who is correct because probability does not equal certainty. It could be that Nielsen is correct, just less probable than that Newspoll or Galaxy are.
There is certainly no doubt that the type of moves they describe can happen in the real world.
Anyway, I've taken some of the abuse out of your posts above and suggest you restrain yourself in future.
Oh, and of course it was a fluke. There's no law of probability that says the last poll taken has to be about the average of the other polls. It could just as easily have been 52/48 or 48/53 give or take a fraction of a percent.
9
Monday, 02 August 2010 04:37
Scott Steel (aka Possum Comitatus)
The statement "Nielsen was more likely to be correct than Morgan" is a complete non-sequitur - you're treating those polls as two numbers and not what they actually are, two probability functions among many that partially overlap in time. There were 6 probability distributions involved over 4 days in total (and another 7 in the 7 days leading up to the period of the 27th to 30th,which was our starting point for the true underlying level of public opinion). Over that 4 day period,only one of those distributions - Nielsen - had a mean that was 1.5 standard deviations or more away from the trend line that tracked the maximum probability density through all of those functions through time.
Hence, Nielsen had a very high probability of being out. How much? Well the alternative, that 3 other phone polls were all overcooked to the point that would provide for Nielsen's point estimate being correct within 1% of the true value of public opinion, had a probability of less than 6%.
This isn't verbiage as you conveniently call it, its the underlying maths involved to reach the conclusion I did, the one which caused you to quip "why bother" and linking into my site, without having the faintest clue about how it was derived.
As for the fluke - assuming that my trend line (that I use for this - not my Pollytrend line which is a different thing altogether) was a correct estimate of the true underlying level of public opinion to within 0.5% (what it is designed to be) and forecasting it forward by two days to get the Newspoll result, there was a 74% implied probability that "it should come in around the 50/50 mark give or take a point" - the result I predicted on Sunday - and a slightly greater than 30% implied probability that it would hit 50/50 exactly.
This has nothing to do with simplistic "averages" as you suggest, it is an exercise in modelling maximum certainty through time as the best estimate of the true underlying value of public opinion from the probability distributions that we have as our raw and relatively uncertain estimates.
As for what the rest of your article said - I care not. I'm only concerned about your swipe at Pollytics.
The "basic error" here isn't mine, the error here was assuming that I was doing something basic to begin with.This isn't stats or econometrics 101 here - that isn't good enough to track opinion properly.Something I assumed you would know.
10
Monday, 02 August 2010 10:29
graham
What you can assume I know is that a poll, or series of polls, taken before last week are not predictive of a poll taken last week. You can dress it up as much as you like, but that is the fact of the matter. Your trend line is unscientific. It is not even stats 101.
There were four polls taken at roughly the same time, all with different sample sizes and Morgan was the outlier, not Nielsen. You can only come to your conclusion by putting an artificial constraint on the mathematics so it has to conform to your trend line.
As for your skill in forecasting, if I accept your logic, which I don't, you accept that there was a 70% chance you would be wrong. Sounds like on your logic it is a fluke.
You're similar to the charlatans who take money from stock market investors on the basis of charting to explain what is essentially a random walk. Or the PhDs who design the computerised trading programs that claim to be able to predict the future of the same random walk. Been doing wel lately haven't they?
11
Monday, 02 August 2010 22:49
Scott Steel (aka Possum Comitatus)
Time series analysis of probability functions is unscientific!? Holy smokes batman - its the cult of stochastic denialism!
I thought you lot died out with the abacus.
I'm quite happy to side with every national bureau of statistics agency and fed reserves,let alone core sectors of the capital markets like the reinsurance industry on this one thanks.
The fact that you are still sitting here banging on mercilessly about *either* Morgan or Nielsen having to be an outlier goes to the heart of the problem - a large failure to understand the statistics involved.And you have the hide to call other people charlatans when you're obsessed by a mathematical non-sequitur!
Let's say, hypothetically, we had 5 polls taken at the same period by 5 separate organisations (which we now actually do in reality), where house effects were zero and the population was equally accessible, creating zero design effect on the confidence interval.
Let's say four of those 5 polls produced very similar results, (where those 4 overlapped on a tight range of given values with a respective high probability density) and one poll - the 5th poll - did not. With this 5th poll, only the right hand tail of its distribution overlapped this key range, but out beyond 1.5 standard deviations from its mean.
To really show the silliness here:
- I would be saying one of these polls is undercooked - even though that poll in isolation was still within 2 standard deviations of the mean of the true underlying value of public opinion derived from all those 5 polls (not an outlier in any true sense of the word). I could also say the size of the probability involved.
-you, on the other hand, would be arguing that the 5th poll is actually really good because some other single poll taken a day earlier was an actual outlier in the opposite direction! To make it worse, you then say that any analysis of the combined probabilities of those 5 distributions that says anything to the contrary is simply the work of a charlatan!
Are you trying to be Christopher Monckton of the polling world or something - because you're doing a damn good job of it at the moment?
12
Tuesday, 03 August 2010 11:05
graham
Look Scott, people who have a strong hand don't stoop to abuse. You're politically motivated as your last par shows, and your understanding of the polling process is woeful.
When you throw a dice each time there is a one in six chance that a six will come up, no matter how many times you have thrown six in a row before. So polls last week have no bearing on polls this week.
When you conduct a poll this week with a particular sample size there is a probability that it will be correct each time. The fact that two polls come up with the same answer slightly increases the probability that they are correct because you increase your sample size.
If you had done the calculations on this one properly by combining the samples you would find that Morgan is the outlier. End of story. It doesn't make any of them right.
You can claim to have the weight of the insurance industry behind you, but as anyone who reads financial statements knows insurers regularly make under-writing losses, and more often than not only make profits from their investments, because the models aren't the real world, they are just best guesses.
What you are doing is substituting best guesses for reality, and then claiming they are real. As there aren't elections every week you get away with this because most people have no idea what you are talking about.
You also seem to have no idea how slippery the concepts are that you are measuring. When someone takes a poll mid-term respondents aren't thinking in terms of alternatives. They are referenda on the government, by and large. When you get into an election period things change significantly.
The mathematics you are applying to polling is just an elegant parlour game - it has no bearing on anything. It tells no-one anything they can use, and it's not properly conceived. I guess the first follows from the last.
13
Wednesday, 04 August 2010 00:10
Scott Steel (aka Possum Comitatus)
Abuse?
You reckon you're a pollster and on your site you had a sarcastic and derogatory swipe at Pollytics where:
a) You didn't have the faintest clue about what I was doing, its methodology nor its robustness - anything about it at all actually. Just some ho hum assumption that you made up in a drive by smear.
b) When pinged on it, you've fallen back on simplistic nonsense (and a raging non-sequitur) like declaring that one poll must have been an outlier in order to justify another result (when there is not actually a *single* *piece* *of evidence* to suggest any outlier here by any pollster over the entire campaign so far). Worse, you've resorted to using ridiculous high school levels of mathematics in an attempt to justify it e.g your latest:dice! As if discrete uniform probability distributions are even remotely relevant here, when this whole thing is *about* probability density (which, for others - and maybe Graham - a dice has an equal probability density across the full range of its 6 possible values, unlike survey results which, for good pollsters, are distributed approximately normally, exhibiting a higher density towards their center than they do towards their tails). It doesn't even make for an accurate analogy.
The pooled sample example is another - pooling requires specific and necessarily arbitrary definitions of the time period that polls can be pooled within. From what date do we start and end the pooling? Why those dates? (no - convenience isn't an answer Graham). Do we start on the 26th? The 28th? - Why?. When do we end - the 29th? The 31st? The 2nd? Why? What makes those dates more worthy than a day or two either side? (Again, no - convenience isn't an answer here Graham) The arbitrary choice will determine your ultimate, and as a consequence, ultimately arbitrary outcome.
It is a third best option when we already have the data we need to work it out - the probability density and uncertainty of each poll. A standard but slightly more complicated method which doesn't destroy the time-dependency of information (time dependency such as public opinion moving across the entire period in question here - which we know occurred from the 25th to the 2nd with p
14
Wednesday, 04 August 2010 00:14
Scott Steel (aka Possum Comitatus)
Since there's apparently a word limit here - the chopped remainder went:
It is a third best option when we already have the data we need to work it out - the probability density and uncertainty of each poll. A standard but slightly more complicated method which doesn't destroy the time-dependency of information (time dependency such as public opinion moving across the entire period in question here - which we know occurred from the 25th to the 2nd with p keep complaining about abuse while calling people (me) charlatans.
I have nothing further to add - it's already been said.
I'll leave it to readers to make up their own mind here on just which one of us needs to get our act together.
15
Wednesday, 04 August 2010 00:19
Scott Steel (aka Possum Comitatus)
Try again - less than signs seem to be html problematic here.
It is a third best option when we already have the data we need to work it out - the probability density and uncertainty of each poll. A standard but slightly more complicated method which doesn't destroy the time-dependency of information (time dependency such as public opinion moving across the entire period in question here - which we know occurred from the 25th to the 2nd with p less than 0.01, and we knew at the time of Nielsen with p less than 0.04 ). Pooling destroys information we want in this case, without letting us use any of the benefits that pooling can otherwise provide.
c) Then, after all this nonsense, from the initial drive-by smear through to the half-baked justifications, from the the simplistic, almost high-school statistics examples not up to the job through to the naivety and arrogance (nice mix by the way) - you have the hide to keep complaining about abuse while calling people (me) charlatans.
I have nothing further to add - it's already been said.
I'll leave it to readers to make up their own mind here on just which one of us needs to get our act together.
16
Friday, 06 August 2010 14:24
graham
I understand what you are doing, and it is nonsense.
Presumably your business depends to some extent on your analysis on your blog so I can understand you wanting to defend it, but I can't understand your defensive aggression which would deter me from ever recommending you to anyone, and I would have thought would have the same effect on anyone reasonable person reading this exchange.
When you make a mistake, the smart thing to do is to admit it.
What time period would I recommend we use? Well, looks like the polls were all done during the same three days, so let's use that time period. Is this convenient? Yes it is. Is it poor methodology? No it isn't. You have no idea when the various polling organisations did most of their interviews in that time period so your high falutin maths may well be dealing with the wrong things if you think you are taking some sort of time dependency into account.
But of course you don't think this because you note in your comments you don't know how many days Galaxy was in the field. So much for the time period counting! If it did you wouldn't have been able to do your calculation missing this bit of information.
Typically fewer successful interviews occur as a poll progresses, because you are hunting for the more elusive representatives in the community, such as young men under 30. They're sometimes so hard to find that their responses are faked by weighting a smaller sub-sample so it looks like a larger one.
And that's just one of the imprecisions.
Being good at maths is one thing. Understanding the weaknesses in the mathematical approach, and the problems inherent in the data you are trying to measure, is another.
Then knowing what to do with the real world implications is another thing again.
You fail on all of these. And you fail on the basic mathematics. In all the polling that has occurred since this exchange started Nielsen is an outlier, but not by as large a margin as Morgan.
Your trend is probably catching up with this fact. But that is the problem with trends. They're not leaders, they are followers. Pay more attention to the actual polls and less to your preconceptions (mathematical or political) and you'll do a lot better.
A 6.4% topline change in the TPP including a 7.8% change on the TPP among women in 7 days? Without a leadership change? When 2 published and 3 unpublished polls (combined: n=~4200) picked up changes substantially less than half of that over the same 7 day period?
Yet somehow people that pay attention to those inconvenient details are dismissed, as you've done, as some sort of third rate wannabe psychics or [deleted for abuse]?
[Deleted for abuse]
Your 1000 polls have no bearing on anything as they weren't taken last week. You summarise the polls that were thus:
"Morgan got a 53% TPP to the ALP over the 27th and 28th, Galaxy got a 50% TPP on the 28th (and maybe on the 27th as well – I can’t find if Galaxy was in the field for one night or two) and Nielsen got a 48% TPP over the 27th, 28th and 29th. So Nielsen is the odd one out here, suggesting a bit of overshoot – and probably combined with the ALP week getting worse as it went on."
Your bias is showing. They're all within sampling error. Nielsen isn't the "odd one out" at all, it would appear you are.
During the same period that Galaxy and Morgan phone poll were in the field (2 of those 3 days happening to overlap with Nielsen being in the field), there were two other polls in the field during the exact same period, and a further poll being in the field on the 29th & 30th - all three of which were consistent in the change in their topline results with Galaxy and Morgan, and none of which showed a nearly 8% change among female voters.
First derivatives matter.
Nielsen was the odd one out, not because of any inherent problem they had (far from it), but because of a mix of time and chance alone - they came in among a cluster of polls that were in the field between the 20th-23rd of July when the ALP had a peak in support (Morgan phone poll also picked that up among the published polls, having an ALP TPP of 55.5% on 20/21 July while an unpublished poll during the same 4 day period had the ALP TPP on 53% and where their subsequent polling had that number falling by 2 points).
Nielsen had it on 54% even though the trend estimate at the time was 53% - which meant that a group of other polls were below the 54 and higher topline result during the same 5 day period.
Nielsen then came in with a low 48 - a vote estimate that *no other* pollster picked up during that same period, and a magnitude of change which no other pollster picked up *during that period* - even though all pollsters during that period showed a decline in the same direction.
This isn't 'bias' - its the statistics of variance. If you want to tell me all about sampling error, particularly the uncertainty as it applies to time series, it would help your cause if you understood what it actually means!
I can remember previously, for instance, when you have stated that because a subsequent poll result was outside of the margin of error of the previous poll result, then it is a significant movement - overlooking the fact that there is two probability distributions involved (the first poll *AND* the second poll) rather than just one (let alone design effect issues - but that's for another day), a consequence of which is that it blows out the interval that determines the difference between two means. A common enough mistake, yet one which made me bite my tongue at the time, coming as it did from a bloke that runs polls and should be across that.
But if you are going to lecture me about how a lack of grasping sampling error shows my bias (rather than just an actual understanding of the numbers involved), it would help your cause immensely if you were across the field of knowledge you are talking about.
And yes, not only have I seen Newspoll's 50/50 result - I predicted it based on the *very* methodology that you are accusing me of bias for utilising:
http://blogs.crikey.com.au/pollytics/2010/08/01/nielsen-poll-the-other-stuff/
(above the charts at the bottom of the post)
"Bias"? No - it's called precision.
That you fluked predicting the Newspoll result doesn't make you precise. That you ignored what you thought would happen means your bias was showing.
It's nonsense to suggest that there is some sort of maximum movement in polling. Collapses in vote can happen in very short periods of time. Just because you haven't seen it doesn't mean it doesn't exist. People aren't constrained by mathematical formulae - they make up their own minds. I've watched tracking polling whip around much more than this in the past.
And looking at your first post on this thread in the clear light of morning I think you owe me an apology.
There could well be record movements at any time - but there wasn't here, the data didn't suggest it. It didn't suggest it at the time, and subsequent polling results have demonstrated it.
When you get one poll showing an extraordinary movement you need to take a serious look at it, as such extraordinary movements in the true underlying value of public opinion are rare at the national aggregate level... it's why they are, er, "extraordinary". When the magnitude of that movement is not replicated by *any other* polling series in the field during the same period, it speaks for itself.
People aren't constrained by maths, but tracking true underlying opinion without getting sidetracked by random variation cannot be done without substantial dollops of it. It also cannot be done unless time itself is utilised as a variable.
The proof is in the pudding. The data heavily suggested the Nielsen had overshot at the time, I stated that at the time, subsequent data demonstrates it to be true. The only person surprised at this appears to be you - calling it a fluke.
Which sort of gets back to the original point - polling analysis is an exercise in tracking the density of certainty through a time series of overlapping probability distributions. So when you said "So why do these commentators bother with polls if they know what happens in the real world better than the polls?" - simple, the real world involves more than any single poll taken in isolation. To treat it any other way, such you did, is an invitation to get sidetracked by random variation.
As for an apology - I'm sorry I was right, apparently it annoyed you.
Of course it is only a guess and no-one will ever know who is correct because probability does not equal certainty. It could be that Nielsen is correct, just less probable than that Newspoll or Galaxy are.
There is certainly no doubt that the type of moves they describe can happen in the real world.
Anyway, I've taken some of the abuse out of your posts above and suggest you restrain yourself in future.
Oh, and of course it was a fluke. There's no law of probability that says the last poll taken has to be about the average of the other polls. It could just as easily have been 52/48 or 48/53 give or take a fraction of a percent.
Hence, Nielsen had a very high probability of being out. How much? Well the alternative, that 3 other phone polls were all overcooked to the point that would provide for Nielsen's point estimate being correct within 1% of the true value of public opinion, had a probability of less than 6%.
This isn't verbiage as you conveniently call it, its the underlying maths involved to reach the conclusion I did, the one which caused you to quip "why bother" and linking into my site, without having the faintest clue about how it was derived.
As for the fluke - assuming that my trend line (that I use for this - not my Pollytrend line which is a different thing altogether) was a correct estimate of the true underlying level of public opinion to within 0.5% (what it is designed to be) and forecasting it forward by two days to get the Newspoll result, there was a 74% implied probability that "it should come in around the 50/50 mark give or take a point" - the result I predicted on Sunday - and a slightly greater than 30% implied probability that it would hit 50/50 exactly.
This has nothing to do with simplistic "averages" as you suggest, it is an exercise in modelling maximum certainty through time as the best estimate of the true underlying value of public opinion from the probability distributions that we have as our raw and relatively uncertain estimates.
As for what the rest of your article said - I care not. I'm only concerned about your swipe at Pollytics.
The "basic error" here isn't mine, the error here was assuming that I was doing something basic to begin with.This isn't stats or econometrics 101 here - that isn't good enough to track opinion properly.Something I assumed you would know.
There were four polls taken at roughly the same time, all with different sample sizes and Morgan was the outlier, not Nielsen. You can only come to your conclusion by putting an artificial constraint on the mathematics so it has to conform to your trend line.
As for your skill in forecasting, if I accept your logic, which I don't, you accept that there was a 70% chance you would be wrong. Sounds like on your logic it is a fluke.
You're similar to the charlatans who take money from stock market investors on the basis of charting to explain what is essentially a random walk. Or the PhDs who design the computerised trading programs that claim to be able to predict the future of the same random walk. Been doing wel lately haven't they?
I thought you lot died out with the abacus.
I'm quite happy to side with every national bureau of statistics agency and fed reserves,let alone core sectors of the capital markets like the reinsurance industry on this one thanks.
The fact that you are still sitting here banging on mercilessly about *either* Morgan or Nielsen having to be an outlier goes to the heart of the problem - a large failure to understand the statistics involved.And you have the hide to call other people charlatans when you're obsessed by a mathematical non-sequitur!
Let's say, hypothetically, we had 5 polls taken at the same period by 5 separate organisations (which we now actually do in reality), where house effects were zero and the population was equally accessible, creating zero design effect on the confidence interval.
Let's say four of those 5 polls produced very similar results, (where those 4 overlapped on a tight range of given values with a respective high probability density) and one poll - the 5th poll - did not. With this 5th poll, only the right hand tail of its distribution overlapped this key range, but out beyond 1.5 standard deviations from its mean.
To really show the silliness here:
- I would be saying one of these polls is undercooked - even though that poll in isolation was still within 2 standard deviations of the mean of the true underlying value of public opinion derived from all those 5 polls (not an outlier in any true sense of the word). I could also say the size of the probability involved.
-you, on the other hand, would be arguing that the 5th poll is actually really good because some other single poll taken a day earlier was an actual outlier in the opposite direction! To make it worse, you then say that any analysis of the combined probabilities of those 5 distributions that says anything to the contrary is simply the work of a charlatan!
Are you trying to be Christopher Monckton of the polling world or something - because you're doing a damn good job of it at the moment?
When you throw a dice each time there is a one in six chance that a six will come up, no matter how many times you have thrown six in a row before. So polls last week have no bearing on polls this week.
When you conduct a poll this week with a particular sample size there is a probability that it will be correct each time. The fact that two polls come up with the same answer slightly increases the probability that they are correct because you increase your sample size.
If you had done the calculations on this one properly by combining the samples you would find that Morgan is the outlier. End of story. It doesn't make any of them right.
You can claim to have the weight of the insurance industry behind you, but as anyone who reads financial statements knows insurers regularly make under-writing losses, and more often than not only make profits from their investments, because the models aren't the real world, they are just best guesses.
What you are doing is substituting best guesses for reality, and then claiming they are real. As there aren't elections every week you get away with this because most people have no idea what you are talking about.
You also seem to have no idea how slippery the concepts are that you are measuring. When someone takes a poll mid-term respondents aren't thinking in terms of alternatives. They are referenda on the government, by and large. When you get into an election period things change significantly.
The mathematics you are applying to polling is just an elegant parlour game - it has no bearing on anything. It tells no-one anything they can use, and it's not properly conceived. I guess the first follows from the last.
You reckon you're a pollster and on your site you had a sarcastic and derogatory swipe at Pollytics where:
a) You didn't have the faintest clue about what I was doing, its methodology nor its robustness - anything about it at all actually. Just some ho hum assumption that you made up in a drive by smear.
b) When pinged on it, you've fallen back on simplistic nonsense (and a raging non-sequitur) like declaring that one poll must have been an outlier in order to justify another result (when there is not actually a *single* *piece* *of evidence* to suggest any outlier here by any pollster over the entire campaign so far). Worse, you've resorted to using ridiculous high school levels of mathematics in an attempt to justify it e.g your latest:dice! As if discrete uniform probability distributions are even remotely relevant here, when this whole thing is *about* probability density (which, for others - and maybe Graham - a dice has an equal probability density across the full range of its 6 possible values, unlike survey results which, for good pollsters, are distributed approximately normally, exhibiting a higher density towards their center than they do towards their tails). It doesn't even make for an accurate analogy.
The pooled sample example is another - pooling requires specific and necessarily arbitrary definitions of the time period that polls can be pooled within. From what date do we start and end the pooling? Why those dates? (no - convenience isn't an answer Graham). Do we start on the 26th? The 28th? - Why?. When do we end - the 29th? The 31st? The 2nd? Why? What makes those dates more worthy than a day or two either side? (Again, no - convenience isn't an answer here Graham) The arbitrary choice will determine your ultimate, and as a consequence, ultimately arbitrary outcome.
It is a third best option when we already have the data we need to work it out - the probability density and uncertainty of each poll. A standard but slightly more complicated method which doesn't destroy the time-dependency of information (time dependency such as public opinion moving across the entire period in question here - which we know occurred from the 25th to the 2nd with p
It is a third best option when we already have the data we need to work it out - the probability density and uncertainty of each poll. A standard but slightly more complicated method which doesn't destroy the time-dependency of information (time dependency such as public opinion moving across the entire period in question here - which we know occurred from the 25th to the 2nd with p keep complaining about abuse while calling people (me) charlatans.
I have nothing further to add - it's already been said.
I'll leave it to readers to make up their own mind here on just which one of us needs to get our act together.
It is a third best option when we already have the data we need to work it out - the probability density and uncertainty of each poll. A standard but slightly more complicated method which doesn't destroy the time-dependency of information (time dependency such as public opinion moving across the entire period in question here - which we know occurred from the 25th to the 2nd with p less than 0.01, and we knew at the time of Nielsen with p less than 0.04 ). Pooling destroys information we want in this case, without letting us use any of the benefits that pooling can otherwise provide.
c) Then, after all this nonsense, from the initial drive-by smear through to the half-baked justifications, from the the simplistic, almost high-school statistics examples not up to the job through to the naivety and arrogance (nice mix by the way) - you have the hide to keep complaining about abuse while calling people (me) charlatans.
I have nothing further to add - it's already been said.
I'll leave it to readers to make up their own mind here on just which one of us needs to get our act together.
Presumably your business depends to some extent on your analysis on your blog so I can understand you wanting to defend it, but I can't understand your defensive aggression which would deter me from ever recommending you to anyone, and I would have thought would have the same effect on anyone reasonable person reading this exchange.
When you make a mistake, the smart thing to do is to admit it.
What time period would I recommend we use? Well, looks like the polls were all done during the same three days, so let's use that time period. Is this convenient? Yes it is. Is it poor methodology? No it isn't. You have no idea when the various polling organisations did most of their interviews in that time period so your high falutin maths may well be dealing with the wrong things if you think you are taking some sort of time dependency into account.
But of course you don't think this because you note in your comments you don't know how many days Galaxy was in the field. So much for the time period counting! If it did you wouldn't have been able to do your calculation missing this bit of information.
Typically fewer successful interviews occur as a poll progresses, because you are hunting for the more elusive representatives in the community, such as young men under 30. They're sometimes so hard to find that their responses are faked by weighting a smaller sub-sample so it looks like a larger one.
And that's just one of the imprecisions.
Being good at maths is one thing. Understanding the weaknesses in the mathematical approach, and the problems inherent in the data you are trying to measure, is another.
Then knowing what to do with the real world implications is another thing again.
You fail on all of these. And you fail on the basic mathematics. In all the polling that has occurred since this exchange started Nielsen is an outlier, but not by as large a margin as Morgan.
Your trend is probably catching up with this fact. But that is the problem with trends. They're not leaders, they are followers. Pay more attention to the actual polls and less to your preconceptions (mathematical or political) and you'll do a lot better.