|
|
Faking that NASA faked the moon landing
Data
integrity is a central issue in all research, and internet-based data
collection poses a unique set of challenges. Much attention has been
devoted to that issue and procedures have been developed to safeguard
against abuse. There have been numerous demonstrations that internet
platforms offers a reliable and replicable means of data collection, and
the practice is now widely accepted.
Nonetheless, each data set must be examined for outliers and “unusual” responses, and our recent paper on conspiracist ideation and the motivated rejection of science is no exception.
Perhaps unsurprisingly, after various unfounded accusations against us have collapsed into smithereens,
critics of our work have now set their sights on the data. It has been
alleged that the responses to our survey were somehow “scammed,” thereby
compromising our conclusions.
Unlike the earlier baseless accusations, there is some merit in casting a critical eye on our data. Science is skepticism and our data must not be exempt from scrutiny.
As it turns out, our results withstand skeptical scrutiny. We will
explain why in a series of posts that take up substantive issues that
have been raised in the blogosphere in turn.
This first post deals with the identification of outliers; that is,
observations that are unusual and deserve to be considered carefully.
Outlier detection and identification
Let’s begin by examining the variable of greatest interest in our
paper, namely the indicators for “conspiracist ideation,” which is the
propensity to endorse various theories about the world that are, to
varying extents, demonstrably unfounded and absurd (there are some
reasonably good criteria for what exactly constitutes a conspiracy
theory but that’s not at issue here).
The full distribution of our conspiracy score (summed across the
various items using a 4-point scale ) is shown in the figure below.
Ignore the vertical red line for now.
For simplicity we are ignoring the space aliens for now (which formed
a different indicator variable on their own), so the observations below
represent the sum across 10 conspiracies (remember that the
"convenience" theories involving AIDS and climate science are omitted
from this indicator variable for the reasons noted in the paper).
Thus, a person who strongly disagreed with all conspiracies would
score a 10, and someone who strongly agreed with them all would score a
40.
The figure invites several observations. First, the distribution is
asymmetrical, with a longish upper tail. That is, most people tend to
more or less reject conspiracies; their score falls towards the lower
end of the scale.
Second there are several observations at the very top that may—repeat may—represent aberrant observations. It is those extreme scores that critiques of our data have focused on, for example the very thorough analysis
by Tom Curtis. The top two extreme data points are indeed unusual. But
then again, one might (just) expect a few such extreme scores in a
sample of more than 1,000 people given the shape of the distribution.
So how does one deal with this situation?
The first, and most important step is the recognition that once the
data have been obtained, any identification of an observation as an
"outlier", and any decision to remove a subset of observations from
analysis, almost inevitably involve a subjective decision. Thus, a
valuable default stance is that all data should be retained for
analysis. (There may be some clear-cut exceptions but the data in the
above figure do not fall within that category).
There are two ways in which data analysts can deal with outliers: One
is to remove them from consideration based on some criterion. There are
many candidate criteria in the literature, which we do not review here
because most retain an element of subjectivity. For our analysis, we
therefore elected not to remove outliers by fiat, but we instead ensured
that the inclusion or exclusion of any potential outliers has no
substantive effect on the results.
That is, we examined the extent to which the removal of outliers made
a difference to the principal result. In the case of our study, one
principal result of interest involved the negative correlation between
conspiracist ideation and acceptance of science. That is, our data
showed that greater endorsement of conspiracy theories is associated
with a greater tendency to deny the link between HIV and AIDS, lung
cancer and tobacco, or CO2 emissions and global warming.
How resilient is this result to the removal of possible outliers?
The red line in the above figure answers that question. If all
observations above that line (i.e., scores 25 or greater; there were 31
of those) are removed from the analysis, the link between the latent
constructs for conspiracist ideation and rejection of climate science
remains highly significant (specifically, the p-value is < .001),
which means that the association is highly unlikely (less than 1 in
1000) to have arisen by chance alone.
In other words, if we discard the top 3% of the data, that is those
part of the data which for conceptual reasons should arouse the greatest
suspicion, our conclusions remain qualitatively unchanged.
Why discard anything above 25? Why not 29 or 30 or 18?
Because a score of 25 represents a person who disagrees with half of
the theories and agrees with the other half (or some equivalent mix of
strongly-agreeing and strongly-disagreeing responses). Lowering the
cutoff further, thereby eliminating more observations, would eventually
eliminate anyone who endorsed any of the theories—and guess what, that
would defeat the whole point of the study. Conversely, there is no point
in raising the cutoff because we already know what happens when all
data are included.
We conclude with considerable confidence that when a highly
conservative criterion (scores 25 or above) for outliers is used, the
principal result remains qualitatively unchanged. Conspiracist ideation
predicts rejection of science—not just climate science, but also the (even stronger) and even more strongly, rejection of the link between HIV and AIDS and the link between tobacco and lung cancer. [13/9: rephrased to clarify, the 'more strongly' refers to magnitude of regression weights.]
How does the elimination of outliers relate to the notion of “scamming”, which has stimulated so much interest in our data?
The answer is both obvious and also quite subtle: The obvious part is
that the two folks at the very top of the above distribution strongly
endorsed virtually all conspiracy theories. If they then also strongly
rejected climate science, that would arguably constitute a profile of
scamming—that is, those folks may have generated responses to create the
appearance that “deniers” are “conspiracy nuts” (note the quotation
marks: this discussion is almost impossible to write succinctly without
labels, even if they are caricatures).
Now, we have dealt with the obvious bit about the "scamming" problem
by throwing out not just those two people of greatest concern, but the
top 3% of the distribution—that is, anyone who might remotely look and
act like a “scammer” based on their responses to 10 conspiracy theories.
Remember—none of the significant correlations in our data disappear when those people are removed from consideration.
But now to the more subtle part: How would we know that
someone who endorses all conspiracy theories but none of the science is
actually a scammer? We have tacitly assumed that this somehow is
evidence of scamming. But on what basis? Is there more to this than
intuition?
This brings us to the fascinating issue of mental models of people's behavior, which we will address in a future post.
106 Comments
1 2 3 Next
Comments 1 to 50 out of 105:
-
Most interesting. I appreciate being informed of the actual
processes behind the data analysis rather than having people try to
guess what went on.
-
Hear, hear. I look forward to reading more.
-
Now, we have dealt with the obvious bit about the
"scamming" problem by throwing out not just those two people of greatest
concern, but the top 3% of the distribution—that is, anyone who might
remotely look and act like a “scammer” based on their responses to 10
conspiracy theories.
Isn't it correct that the "two people of greatest concern" were found in both the groups - moon landing and climate deniers?
If so, with a total number moon landing deniers being 10 (of more than
1,000) , then taking out these two "scammers" would make the
moon-landing deniers come out more equal between climate deniers and
pro-science wouldn't it?
Was this considered when deciding the preface of the title of the paper "NASA faked the moon landing - Therefore (Climate) Science is a Hoax"?
-
Okay tlitb1, we get it. You didn't like the title of the paper.
Would you kindly step back a moment and re-read the full title of the
paper - at its own face value, not the title you want to see, please?
Here it is:
NASA faked the moon landing - Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science
Perhaps it would be a good idea to consider the almost 60 references?
E.g. if dodgy titles are a problem, I'm sure many skeptics must have taken a close interest in this one:
Nattrass, N. (2010). "Still crazy after all these years: The challenge
of AIDS denialism for science." AIDS and Behavior , 14 , 248{251).
And this one:
"Cool dudes: The denial of climate change among conservative white males
in the United States." Global Environmental Change, 21 , 1163{1172.
And this one:
Douglas, K. M., & Sutton, R. M. (2011). "Does it take one to know
one? Endorsement of conspiracy theories is influenced by personal
willingness to conspire." British Journal of Social Psychology, 50 ,
544{552).
And this one:
Boyko , M. T. (2007). "Flogging a dead norm? Newspaper coverage of
anthropogenic climate change in the United States and United Kingdom
from 2003 to 2006." Area,
39 , 470{481).
I think it's called whimsy, and that it involves a sense of humour.
-
@4. Bluebottle at 21:25 PM on 12 September, 2012
Thanks for iterating the most whimsically titled references.
To impress upon me the normality of what I "should see" in the title
then I think you need further separate the whimsical part of those
titles and then show me that they contained a statement of fact that was
not supported by the contents of the papers themselves.
If you can then I will accept this is normal practice in pyschological science.
And please, don't tell me the Cool dudes whimsy clearly wasn't dealing wth body temperature and so wasn't shown in the paper ;)
-
@3 tlitib1,
I suspect you may have a really good point, but it's not coming across
yet, because you're talking about "climate deniers" (who don't exist)
and "pro science" (which is everyone). Please say what you mean. You
might or might not be on to something.
-
@3 tlitib1,
on closer reading, I see what you were doing there—sorry.
-
@7. Brad Keyes at 22:12 PM on 12 September, 2012
Yes you are right I was letting myself become sloppy, in fact I fully agree with statement in the piece above.
(note the quotation marks: this discussion is amost impossible to write succinctly without labels, even if they are caricatures)
I do consider the "pro-science" and "deniers" labels as caricatures
really, and I use them here only as identifiers inferred from the
context of the paper being discussed. I should be more consistent with
the quotes.
The expansive debate about how people can be defined as "pro" science,
and a "denier" of climate is so huge that, at least on this topic, I
think it should be set aside by all. ;)
-
I think Lewandowsky is perfectly justified in retaining his
outliers. Imagine someone who, when asked by an interviewer to explain
his apparently eccentric responses, replies: “I make a point of
believing any conspiracy theory I come across”. You might find his
attitude eccentric, and a nuisance for social scientists exploring views
held by a tiny minority of the population, but I see no a priori reason
for deleting it. What right has one to bar somone from the sample
simply because he’s an awkward cuss?
The problem comes from the nature of the online survey. Surveys based
on face-to-face interviews derive their validity from nothing more solid
than their resemblance to the common situation in everyday life of
being stopped in the street by a stranger who asks you the way. It is
assumed that you would no more lie to an interviewer than you would to
someone who asks the way to the post office. (Though I suppose you might
be tempted, if he added “Is it to the left or to the right? You are not
allowed to answer ‘don’t know’”).
The online survey is not like that. It’s more like filling in your tax
form, but with no danger of being found out and punished for giving a
false answer.
-
OK, you've got my attention.
The fundamental outstanding question is of course how your analysis and
Tom's give such different results when excluding the two extreme
outliers. I presume the difference comes from the method you are using.
If you can explain that to a non-specialist, then you will have
dramatically strengthened your case in my eyes. I am willing to learn.
Moderator Response: Good question. Stand by
for more. There are a few more posts to come on this issue. It's
non-trivial and deserves to be considered carefully. SL
-
Oddly enough, throughout this whole kerfuffle over the paper, the
fact that there is a demonstrable correlation between acceptance of
certain conspiracy theories and denial of AGW is only the *secondary*
conclusion of the paper. The primary conclusion of the paper is, and I
quote from the paper's abstract (highlight is mine):
"Paralleling previous work, we find that endorsement of a
laissez-faire conception of free-market economics predicts rejection of
climate science (r ~ .80 between latent constructs). Endorsement of
the free market also predicted the rejection of other established
scienti c ndings, such as the facts that HIV causes AIDS and that
smoking causes lung cancer. We additionally show that endorsement of a
cluster of conspiracy theories (e.g., that the CIA killed Martin-Luther
King or that NASA faked the moon landing) predicts rejection of climate
science as well as the rejection of other scienti c fi ndings, above and
beyond endorsement of laissez-faire free markets. This provides
empirical con rmation of previous suggestions that conspiracist ideation
contributes to the rejection of science. Acceptance of science, by
contrast, was strongly associated with the perception of a consensus
among scientists."
But all the contrarians have latched on to the conspiracy theory part of
it, when the elephant in the room is that it's political/economic
ideology that is at the heart of the matter. I wonder why that is? Oh,
look, there goes a squirrel! :-)
-
IEHO you have yet to confront the TISSIWNE problem (hint the start
is there is some and the end is I will not eat) problem that
conspiracies differ in the degree to which they diverge from reality,
and someone trying to game the survey would have limits.
To give a NASA example there is a cohort of climate science deniers who
are rocket jocks, and work for, have worked for, or want to work for
NASA and are deeply invested in the Apollo success. Perhaps the pattern of rejectionism would be interesting.
-
@- Steve Metzler
"Oddly enough, throughout this whole kerfuffle over the paper, the fact
that there is a demonstrable correlation between acceptance of certain
conspiracy theories and denial of AGW is only the *secondary* conclusion
of the paper. ...
But all the contrarians have latched on to the conspiracy theory part of
it, when the elephant in the room is that it's political/economic
ideology that is at the heart of the matter. I wonder why that is? "
Perhaps it is because the many who reject the mainstream findings of
climate science do so for reasons linked to their political/economic
ideology and resent the implication that their refusal to accept AGW as a
significant scientific finding is linked to a conspiracy belief they
also regard as foolish or mistaken.
Foolish and mistaken in a way that they would never consider that their belief in free market economics to be.
However the problem is that acceptance or rejection of scientific
findings can never be validated by an ideological or political dogma,
the credibility of scientific knowledge is not correlated with its
political implications.
Although those who clearly judge science by such political criteria have to re-frame the science as a political program.
-
izen@13
You make an incontestable point about scientific findings.
But if those who reject the mainstream findings of climate science
resent the implication that their refusal to accept AGW is linked to a
conspiracy belief that they also regard as dopey or wrong, how come they
are so often silent - or perhaps tolerant - about the company they
keep? You must have noticed that, say Monckton, is self-evidently a
conspiracist/birther/one-world Government. I single him out only for his
prominence - my observation is based on the obviously widespread and
persistent claims of AGW being a fraud, a hoax, etc. Regardless of
ideology, I fully accept that there are many rational, intelligent,
well-informed people with serious misgivings about some of the policy
responses being suggested; what I can't accept is that so few speak out
strongly and consistently against their many fellow-travellers who, it
seems, really do think there's a global conspiracy.
-
SM@ 11: "But all the contrarians have latched on to the conspiracy
theory part of it, when the elephant in the room is that it's
political/economic ideology that is at the heart of the matter.
That could only be true if the possibility of 'ideology' being a
substantial factor in the holding with consensus science had been tested
for. On the assumption that the survey canvassed the beliefs of a lay
audience, it is necessarily true that 'ideology' is a component of
holding with consensus science; the mechanism of such a connection being
'trust'.
Moreover, a distinction needs to be made between 'science' and
'consensus science'. A rejection of the latter is not necessarily a
rejection of the former. For instance, the survey does not test for a
rejection of the scientific method -- this much is presupposed by the
analysis and subsequent commentary.
Returning to the subject of 'ideology' and assent/dissent to consensus
science, the putative consensus is broad and not exclusively
'scientific'. The material scientific basis and the policy imperatives
are routinely confused, and are interchangeable in many of wider debates
about policy -- especially economic policy.
Therefore, it is not unreasonable to again suggest that there is an
'ideological' component to both assent and dissent to consensus science,
but that the survey doesn't adequately take account of this, and
reproduces the confusion about the object of 'consensus science' ---
science vs policy. A corollary of the apparent correspondence of
laissez-faire economics and the phenomenon of climate change scepticism
might be an equivalent connection between an 'ideological' preference
for economic and political institutions that *seem* to be the
consequence of consensus climate science, and 'consensus science', but
which might be only connected through 'ideology'.
There are precedents in the history of environmental politics which
reflect this confusion. For instance, Garett Hardin's influential 1969
Essay 'The tragedy of the Commons' makes an argument for the abolition
of the common ownership of property, on the basis that it allows 'free
riders' to over-exploit natural resources. However, the same argument is
made for the control of all resources as if they were a commons -- i.e.
that institutions of private property are not sufficient to ensure the
protection of natural resources. This is a contradiction within
environmental politics which has not been fully resolved, and has led to
disagreements with the environmental movement about the best approach
to reducing carbon emissions through policy, for instance through
carbon-trading vs rationing.
In summary, the survey's attempt to model 'ideology' is naive at best,
and inadequate to the task of meaningfully reducing the complexities of
'ideology' and its interaction with material science to draw statistical
relationships between groups of beliefs. This seems to reflect the
authors' own ideological prejudices and preferences for environmental
politics, which are a matter of record.
-
First, using a spread sheet, and taking the correlation between the
mean of CY question scores excluding CYAids and CYClimateChange and the
mean of CC questions plus CauseCO2, I find the following linear
correlations:
-0.139 using all 1145 responses;
-0.11 excluding just the two most suspect responses; and
-0.074 for all responses equal to or less than 2.0833 (25/12).
You report a pairwise correlation between the CC and CY latent
constructs of -0.197 in your paper, but do not report the pairwise
correlation after discarding all responses with a cumulative CY score
greater than 25. If it declines by 45% as do the pairwise correlations
between mean responses after excluding the equivalent responses, or by
20% as do the pairwise correlations above after removing the two most
suspect responses, then that is a significant difference in your result,
even if it does not greatly alter the statistical significance of your
result. Therefore I need to ask, what was the pairwise correlation
between the conspiracy and pairwise latent constructs in your robustness
test?
Second, excluding CYAids and CYClimateChange, you have 12 CY questions,
meaning a response which contained as many scores of 2 as of 3, and as
many scores of 1 as of 4 would sum to 30, not 25 as you have it.
Excluding all scores greater than 25 excludes all scores with a mean
response greater than 2.0833 (as noted above).
Thirdly, when all CY scores with a mean greater than 2.0833 are
excluded, I retain just 1,055 responses, meaning that 8% have been
excluded, not 3% as you claim above.
Is there any reason for these discrepancies?
Moderator Response: Tom, your analysis of
the conspiracy items differs from ours in two ways. First, we looked at
the sum total of 10 items, excluding the two items on space aliens,
which formed a separate factor in our initial factor analysis (and also
including the conspiracy theories related to the scientific
propositions). Your analysis included the two space-aliens items. Both
ways of looking for multivariate outliers are equally valid—as we noted,
outlier identification involves many subjective decisions—and
fortunately they lead to the same conclusion (see below). Second, we
analyzed the data using structural equation modeling, which enables us
to compute the correlations between latent variables that exclude
measurement error. You seem to have simply added the items to form
scales; this is the statistically less optimal procedure.
If we do the outlier analysis for the 12-item scale including the
space-aliens items, the minimum score is 12, and the maximum is 48. A
value of 30 reflects moderate rejection of half the items, and moderate
acceptance of the other half. If we eliminate all subjects with a value
of 30 or higher, the correlation between the latent factor for
conspiracist ideation and the latent factor for acceptance of climate
science drops from -.197 to -.141; both values are significant at p <
.001.
The numerical reduction of the correlation is a necessary consequence of
restricting the range of the conspiracist-ideation variable – this is a
simple statistical phenomenon. Effectively, by excluding every subject
that could, by a very generous criterion, be regarded as an outlier, we
necessarily throw away many observations that are most likely valid, and
artificially restrict the variance of the conspiracy variable. This
necessarily restricts the covariance between that variable and any other
variable, and thereby results in an underestimation of the true
correlation.
The important conclusion is that the correlation is still clearly
significant, implying that the correlation does not depend on any
observations that might be outliers. In other words, our conclusions are
not compromised in any way by outliers.
On a more general note: It is always tempting to eliminate observations
from data that stand in the way of one’s preferred result, and one can
always find justifications for such a practice: Perhaps the offending
subjects tried to make fun of the experimenter, or they did not
understand the instructions, or they did not pay attention, or they
confused the response buttons, or they completed the questionnaire after
midnight and therefore were not fully awake, or they just looked not
quite right. The practice of eliminating observations by criteria
defined after looking at the data is meanwhile seen very critically in
the social sciences. The better practice, which we followed, is to
obtain a large sample, such that potential outliers or aberrant
respondents make little difference to the results, and to analyze the
data with and without potentially aberrant observations and ensure that
the conclusions don’t differ.
K.O.
-
A quote from page 7 of your own article: "Links were posted on 8
blogs (with a pro-science science stance but with a diverse audience); a
further 5 \skeptic" (or \skeptic"-leaning) blogs were approached but
none posted the link."
Sorry, but I have learned more about today's science from skeptic (anti
science?) blog http://climateaudit.org than from any of the 7 "pro"
science blogs that you mention.
-
Could you comment please on the IP address distribution among your questionnaire respondees? Were there any duplicates?
Moderator Response: As stated clearly in the Results (p. 8), duplicates were removed.
-
Dribbling standard quality metrics out one at a time is a travesty.
You should have had this and much more in an extended methodology
section of your paper.
You should do so now.
You also need to evaluate your postulated conspiracy theories,
especially the one regarding the Iraq war, as I doubt if most of the
American Congress and English public would agree with your
characterization of their opinion as a conspiracy theory.
Finally, I'm please you brought in Klaus Oberauer, especially if he can provide some rigor to these
(-snip-). However it is more important that you release some
information about your study that requires no analysis at all, which I
have called for earlier.
Anyone trying to evaluate this paper need to know the questions
presented to respondents at each point of entry to the survey and how
they differed from one another. We need to know the frequencies.
Moderator Response: Inflammatory term snipped.
-
izen said:
"However the problem is that acceptance or rejection of scientific
findings can never be validated by an ideological or political dogma,
the credibility of scientific knowledge is not correlated with its
political implications."
Exactly. But then why does every prominent U.S. republican candidate
reject AGW outright? Maybe because if it's true, we would actually have
to do something about it, and that costs money and requires
inter-governmental cooperation? Free market types have historically had a
very bad track record dealing with externalities, and this is, like,
the biggest one there ever was.
-
You have no valid control group. How many non-climate science blogs have people that believe in conspiracy theories?
-
Explain how your off topic questions in your questionaire are not leading questions?
Here is an example of what is meant by leading questions
http://www.youtube.com/watch?v=G0ZZJXw4MTA
-
This has much more in common with Anderegg, Prall et al (PNAS 2010)
or Mashey's multi-colored screed against Wegman than it does with
Gleick.
This stuff tends to stick around, though.
-
Dr Adam Corner, cardiff uni, first wrote about this paper in the
Guardian, back in July. Adam reproduced this article on his cardiff,
nottingham uni backed blog Talking Climate, where Geoff and Adam had an
interesting discussion in the comments.
The blogs surveyed and links to Six articles with the surveys that were found.
Skeptical Science claim they hosted it and detest it after the survey period.
A concern is that some of these blogs telegraphed what was bring looked
for in the article announcing the survey. As these blogs are generally
anti climates sceptics, this should be a concern.
Links from Dr Adam Corners blog.
Actual links to the original articles.. these were the links I found:
http://scienceblogs.com/illconsidered/2010/08/counting-your-attitudes/http://profmandia.wordpress.com/2010/08/29/opinion-survey-regarding-climate-change/http://scienceblogs.com/deltoid/2010/08/29/survey-on-attitudes-towards-cl/http://hot-topic.co.nz/questionnaire/http://tamino.wordpress.com/2010/08/28/survey-says/http://bbickmore.wordpress.com/2010/08/28/take-a-survey
The comments of participants at the more high traffic blogs, also
expressed concerns about methodology, that perhaps the authors could
address.
The 2 other blogs 'surveyed' no links yet.
did the authors keep a copy if how the blog owners introduced the survfyto their readers. Ie Mandias was the best
http://www.skepticalscience.com http://www.trunity.net/uuuno/blogs/
-
"We conclude with considerable confidence that when a highly
conservative criterion (scores 25 or above) for outliers is used, the
principal result remains qualitatively unchanged."
Only because you are including "climate related" conspiracies.
Ignore climate related conspiracies, and the few outlier responses, and
you see that in actual fact skeptics were marginally LESS likely to
believe conspiracy theories.
Had you added a "big oil is conspiring to suppress climate truth"
conspiracy, you'd have found the "warmists" were suddenly much more
conspiratorial. By including only "climate denier" conspiracies in your
questionnaire your results were going to be obviously unbalanced.
It's almost beyond belief that such a poor study could ever see the light of day.
-
I would also like to point out that the study's results still hold
if you remove *all* the respondents who answered '4' to *any* of the
"conspiracy" questions.
(By "results still hold" I mean that you still get a highly significant
correlation between mean conspiracy score - excluding Climate - and the
CauseCO2 variable, and a highly significant partial correlation after
controlling for average "free market" score. And, yes, a very small but
still significant correlation with the Moon landing question.)
-
Interesting that it comes down to surveys to prove that catastrophic anthropogenic global warming (CAGW) is a fact. (-snip-)
Knowledge may be a commodity (whether the knowledge is correct or not),
but expertise will never be. As a paleohydrologist currently working on
a Ph.D. in a physics field and writing a dissertation on an alternate
conceptual model to CAGW, I don't seem to have a place in your survey.
That's also funny to me, because from professional experience interacting with CAGW proponents, (-snip-)
Moderator Response: Inflammatory tone snipped.
-
Perhaps I'm not alone in wishing folks would keep their remarks at a substantive level here.
Mistaking a different survey for the one under discussion, referring to
irrelevant and stale academic scandals of the past, bringing up Gleick
as a ritual whipping-boy, all these things are useless compared to the
very few remarks following the standard set by Tom Curtis.
If you're outraged but don't have the means to explain why, surely it
would better to pipe down and let people with better skills articulate
objections, without getting under their feet and making an incoherent
mess of the discussion?
If you do feel you have something to contribute, why sully your points with florid adjectives?
Maybe it's a little late to change the moderation rules but a more
ruthless noise filter would surely help this conversation, now that it's
entered the actual meat of the matter.
-
Dr Adam Corners blog and Guardian article is here
http://talkingclimate.org/are-climate-sceptics-more-likely-to-be-conspiracy-theorists/
-
Mr. Bostrom, we have not entered the actual meat of the matter. We
cannot, because Professor Lewandowsky has not published adequate
documentation of how the survey was fielded.
-
"Interesting that it comes down to surveys to prove that catastrophic anthropogenic global warming (CAGW) is a fact."
I heard Arctic sea ice and the likes of Brown et al (2012), amongst others, were doing a fine job of that.
-
Typos !! I'm writing on a smartphone in sunlight.. detest = deleted in my earlier comment
Do the authors have a copy of the wording that introduced the articles
to blog readers, at the 2 blogs, as we do not have available the article
to see now.
Scott Mandias was suitable neutral, others wee much less so.
-
Thomas, if history is a guide there will -never- be adequate
documentation of survey sufficient to satisfy people who are not
concerned with the machinery of the survey itself but instead do not
like what it says. There are two threads of communication here; one is
about anger, the other curiosity. One impulse can't be satisfied by
elucidation, the other can.
Lewandowsky and Oberauer have promised to lead us through a detailed
description of their methods. My fear is that their level of effort
devoted to this interesting explanation will be diminished to the extent
that people can't stop venting anger and thus making irrelevant remarks
and dishing out insults.
Emulate Curtis. Listen, think, reply with reason.
-
Many thanks for the response on duplicates. I am not asking about
duplicate responses, I am asking about duplicate IP addresses.
The paper says (on page 8) "duplicate responses from any IP number were eliminated (N = 71)."
Hence it appears you allowed non-duplicated responses from the same IP
address (as might occur from a shared computer in an academic lab in a
university setting).
Is each point in the survey from a distinct IP address or are multiple responses from the same IP address part of the survey?
-
To Bowers.
(-snip-) Among other skeptics, I have no issues with the facts of
summer Arctic ice loss or with warming even on an averaged global scale.
I take serious issue with the anthropogenic and catastrophic parts of any claims by experts or those who pose as experts.
Moderator Response: Inflammatory tone snipped.
-
zt - refer page 101 of the paper referred to on page 8:
http://homepage.psy.utexas.edu/homepage/faculty/Gosling/reprints/AmPsy04Shouldwetrustwebstudies.pdf
-
Yes
Hanich made that reply to Pielke, who did nor post the survey, because he was concerned by the methodology.
Ie
Duplicate IP addresses, with variation in responses, were allowed..
How do the authors know that these were not also trying to game the surveys
How many duplicate IP addresses were allowed
What effect does this have onthe survey
Will the raw data, with all the responses, and all the questions answers
be made available. Until now just a subset has been released after,
responses removed.
Additionally a break down of number of responses from refering blogs send essential. As somebligmuch higher traffic than others.
If the responses come from predominantly a few high traffic blogs. Just a narrow finder if blogs have been captured.
-
@Sou. The paper that you cite implies that multiple responses from
the same computer (IP address) are included in this study.
Gosling says: 'To avoid eliminating responses from different individuals
using the same computer (e.g., roommates or people using public
computer labs), we matched consecutive responses from the same IP
address on several key demographic characteristics (e.g., gender, age,
ethnicity) and when such a match was detected, we retained only the ?rst
response.'
Could one of the authors clearly state whether multiple responses from
the same computer/IP address (albeit with changes to some responses)
were included in the responses to this survey?
-
Or just provide all the raw data, so we can all look at what was removed, and agree/discuss the reasons made.
-
Professor Lewandowsky,
Do you have access to the IP addresses of respondents, or are they held by kwiksurveys?
In the latter case, are they still available, following the recent hack?
Will you be releasing the demographics and the answers to the questions not analysed?
Will you be releasing information allowing us to know from which blog
each respondent was contacted? The latter information would be most
useful for assessing whether the survey was gamed, since the possibility
was discussed by commenters at certain blogs, and the suspect responses
seem to cluster at certain points in the list.
-
urls formatting error in 24#:
these should all work.
http://scienceblogs.com/illconsidered/2010/08/counting-your-attitudes/
http://profmandia.wordpress.com/2010/08/29/opinion-survey-regarding-climate-change/
http://scienceblogs.com/deltoid/2010/08/29/survey-on-attitudes-towards-cl/
http://hottopic.co.nz/questionnaire/
http://tamino.wordpress.com/2010/08/28/survey-says/
http://bbickmore.wordpress.com/2010/08/28/take-a-survey
-
tlitb,
Thanks for a great reply. Unfortunately you're being too nuanced, charitable, thoughtful and circumspect.
Calling either "side" in the climate change debate "anti science" is not a caricature, and it's not debatable.
It's a fantasy and has no justification.
People have different beliefs about the climate, but no position, however absurd, can be more or less scientific than any other.
Science is not a position. It's a process.
(-snip-)
Because most people don't understand anything about the scientific method.
If you think you understand it, that's either because:
1. you haven't studied it, and therefore don't know that it's different from what you imagine (98% of cases)
2. you have studied it, so you understand it (2% of cases)
Don't get me wrong—some people are right about the climate, some people are grossly mistaken about it.
Like everyone, I've gotten frustrated with the sheer wrongness of some
people's ideas, but I've never stooped to calling them anti-science.
That's really low.
Moderator Response: Inflammatory snipped.
-
In reference to #38, Gosling's comment regarding duplicate IP
addresses, I've always found it funny that when detecting an attempt to
submit multiple responses to a survey, researchers occasionally decide
to keep the first one. Best practice is to throw all of them out.
It's like getting two $20 bills with the same serial number and hoping that one of them is real. It doesn't work that way.
Mr. Bostrom, based on the scanty information we have about the
conduction of the survey, would it or would it not be an equally valid
headline to write "70% of survey respondents who believe the moon
landing was a conspiracy shared Professor Lewandowsky's views on climate
change"?
You and I both know such a statement would be inaccurate. So is
Professor Lewandowsky's headline. And we don't know enough to write an
accurate headline at this time.
I'm sure there are climate skeptics that will go to town on whatever is
released. (They would have finished by now and moved on already had
sufficient data been released initially.) You might note that they're
going to town right now on Professor Lewandowsky's leisurely refusal to
release data. Doesn't that just sort of double your pain?
I don't care what they do. I want to know if there's usable data in the survey results. And I don't.
-
"...most people don't understand anything about the scientific method."
And we're -all- above average in our skills. :-)
-
@- Bluebottle
"You make an incontestable point about scientific findings.
But if those who reject the mainstream findings of climate science
resent the implication that their refusal to accept AGW is linked to a
conspiracy belief that they also regard as dopey or wrong, how come they
are so often silent - or perhaps tolerant - about the company they
keep? "
Given that the conspiracy theorists are almost invariably also endorsing
a laissez-faire conception of free-market economics it may be tribal
loyalty. Work by Altmeyer indicates that holding a 'free market'
ideology strongly correlates with an authoritarian mindset. That
includes deference to traditional sources of authority which preach a
static set of truths as absolute certainties. It also encourages group
unity and avoiding any display of internal dissension. As another poster
noted the GOP does not welcome any divergence from the conformity of
rejecting AGW.
There seems to be an implication floated by recent posters that if only
the website, date and IP address of all the survey answers could be
examined perhaps it would be possible to detect whether the poll was
'gamed' by a colluding group to link AGW rejection and acceptance of
conspiracy fantasies. I see no suggestion that it could have been
'gamed' to generate the link between the endorsement of a laissez-faire
conception of free-market economics and rejection of the scientific view
of AGW.
It is difficult to see this link as evidence that holding those
ideological/political beliefs would enhance the ability of the
individual to detect statistical errors in temperature reconstructions
from proxy data or discover the flaws in the present understanding of
climate sensitivity.
-
Can you clarify your handling of multiple responses from a single
IP address, as neither your article nor the above response does so.
If you had multiple (but non-duplicate) responses from the same IP
address, did you keep all of them? If not, how did you decide which one
to keep?
If you had duplicate responses from the same IP address, did you eliminate ALL of the responses or did you keep one?
-
Absent a better understanding of how the survey was fielded, it would be as legitimate to suggest a survey headline like,
"64% of the survey respondents who believe in the Roswell conspiracy
theory also believe Professor Lewandowsky’s theory of climate change…"
-
@44 Doug Bostrom
I'm not sure what your irony was getting at, but I'm pro irony so I'll assume that irony was pro-me.
But the idea that we're all above-average is a kind of delusion. I'm not
suggesting that, and I didn't mean to impute a Dunning-Kruger effect to
98% of the population.
I'm talking about a lack of information, that's all.
People who can't speak Icelandic *know* they can't speak it, don't they?
People who've never studied fluid mechanics *know* they don't understand it, don't they?
People who've never studied genetics *know* they're not going to win an
argument with a geneticist about genetics, so they don't even try.
The scientific method is different.
Everybody knows:
You have to learn Icelandic to understand it.
You have to learn physics to understand it.
Nobody seems to know:
You have to learn TSM to understand it.
-
(I'm not sure I have the raw numbers right, but you'll get the picture, I'm sure)
Respondents were asked their level of agreement with a number of
hypothesized explanations for dramatic events in recent history,
commonly characterized as conspiracy theories.
One such theory is the bombing of the Alfred P. Murrah building in Oklahoma City was the work of a conspiracy.
'Our' survey found that 269 respondents agreed or strongly agreed that
this incident was part of a conspiracy. Of that number, 223 (83%) also
share Professor Lewandowsky's opinions about climate change. The
remaining 17%, 46 respondents, are classified as either 'skeptic' or
'skydragon'. The first term is meant to capture those with mildly
skeptical views of the climate change issue while 'skydragon' is for
those more extreme in their opinions.
Not so hard, is it?
-
Brad: I had a few thousand citations under my belt before I
actually took some time out to read up on philosophy of science and the
scientific method. We weren't taught it. Just as you don't need to have
studied grammar to be able to speak your native language fluently, you
don't need to have studied philosophy of science to do it. (Caveat: This
does depend somewhat upon the field however. I'm guessing that the
closer you get to the boundaries of science, the more important method
becomes.)
The issue gets messier the more philosophy of science you read. I
presume from your terminology that you are thinking of the scientific
method in terms of Popper? Are you aware of Feyerabend's "Beyond
Method"? If Feyerabend is right, then the scientific method may even
hinder progress. I actually don't agree with him: I find Lakatos much
more plausible. However even Lakatos is in my view inadequate - science
is in practice clearly a social endeavour, therefore to understand how
science has functioned in the past it seems to me that a sociology of
science is also required. The field exists, but I have yet to find time
to familiarise myself with it.
1 2 3 Next
Post a CommentYou need to be logged in to post a comment. Login via the left margin or register a new account. |
|
Relevant events (mainly in Australia) will be announced here as they become available.
|
|
About Us 
