Our analysis in our two recent articles, in GRL and E&E, has moved well beyond the points made in MM03, while building on them. Nevertheless, Mann has recently claimed of MM03:
All of their original claims have now been fully discredited (see e.g. this previous post] as well as this discussion of a paper 'in press' in the Journal of Climate by Rutherford et al).
So I thought it would be interesting to look back at the claims in MM03 and see how they’ve stood up. MM03 examined specific matters of the MBH98 data set and then described our emulation of the effect of these matters, which will be described in turn.
Data Set Defects
MM03 provided a listing of 9 specific defects in MBH98:
a)
unjustified truncation of 3 series;
Confirmed. The unreported truncation of the Central England and Central Europe series was acknowledged in the Corrigendum (Nature, July 1 2004). Duplicate versions of these series can be observed at Mann’s UVA web site. It contains, for each of these three series, the full records and the truncated versions in separate files. The rostering file multiproxy.inf at UVA confirms the use of the truncated versions. The use of truncated data is thereby confirmed as being intentional and not accidental. Although the 3rd series was not mentioned in the Corrigendum, it also exists in duplicate versions at Mann’s website. The truncation was supposedly justified by an attribution of the truncated version to Bradley and Jones [1993], but this is hardly a valid justification since the truncation was not annotated in that article either. Given that Bradley and Jones [1993] was an influential critique of the concept of the LIA, its unannotated deletion of the very cold late 17th century period in England would seem to be very problematic in its own right. Questions about the quality of the earliest portions of these series do not justify their arbitrary truncation, since they are being used in MBH98 not as instrumental records but as proxies, and as such surely have as much plausibility, even in their earliest portions, as many of the tree ring chronologies.
b) copying 1980 values from one series onto other series, resulting values in at least 13 series;
Data set withdrawn by Mann et al. This observation was true of the data set originally provided to us. After MM03, this data set was withdrawn and a new directory identified. Since the new directory is not collated, these collation errors do not exist in the new directory.
It is quite possible that this criticism applies only to the now withdrawn data set. However, the only way of verifying that the collation errors observed in this data set were not also present in MBH98 is to inspect source code, which Mann has refused to provide. Thus, on the present record, while Mann have denied that these errors occurred in MBH98, it is impossible to confirm this one way or another.
c)
displacement of 18 series to one year earlier than apparently intended;
Data set withdrawn by Mann et al. Same as (b) above.
d)
unjustified extrapolations or interpolations to cover missing entries
Confirmed. The Corrigendum reported the extrapolation of the Gaspé series, the start date of which was misrepresented in MBH98 (thereby not disclosing the unique extrapolation.) We disagree with the unsubstantiated claim that this extrapolation does not “matter” (see demonstration of the effects in our E&E05 paper).
Although the extrapolation of closing values of many series had been reported in the original SI, it appeared to us that these extrapolations were statistically unnecessary. This position has been adopted by Mann et al., who have done subsequent calculations without these extrapolations. We agree that not much turns on this aspect of the issue.
e)
geographical mislocations and missing identifiers of location;
Not rebutted, obviously correct and more pervasive than indicated in MM03. The Corrigendum acknowledged that the citation of instrumental data was inaccurate, but still failed to provide a verifiable citation of their instrumental data. Merely citing “NOAA” as the source of the data without further particulars is not a data citation within AGU data citation guidelines for example.
The geographical mislocations were not acknowledged in the Corrigendum or Corrigendum SI, which perpetuate the incorrect information of the original SI. In fact, the actual geographic mislocations are more pervasive than reported in MM03. MM03 noted that a Paris precipitation series had been located in New England; it now appears that nearly every MBH98 precipitation series is mislocated (but none have been corrected in the new SI).
f)
inconsistent use of seasonal temperature data where annual data is
available;
Confirmed. This was acknowledged in the Corrigendum.
See explanation in (a).
g)
obsolete data in at least 24 series, some of which may have been at the time of
the MBH98 calculations
Not rebutted, obviously correct and more pervasive than indicated in MM03. Mann et al. have attempted to argue that the obsolete “grey” versions that they used for certain series were preferable to the official versions. I’ve not seen any third parties who have bought into Mann’s arguments (and they are not made in Rutherford et al. [2005]). There are some interesting further developments on this front. There is an unreported and unarchived update of the Gaspé series, which does not have a hockey stick shape. Jacoby and d’Arrigo have refused to either archive or provide this data on the basis that the old version is more “temperature sensitive”. Mann has specifically contested our use of the updated (1992) version of the TTHH series. Now there is a further update to 2001, which shows a decline in ring width with rising temperatures. This is mentioned in our E&E article.
h)
listing of unused proxies;
Confirmed and more pervasive than indicated in MM03. When the Mann FTP site became public, this observation turned out to be much more pervasive than indicated in MM03. MM03 reported 5 listed proxies as not being used, but the FTP site showed that over 35 listed proxies had not been used (and that 2 unlisted proxies had been used). These errors are acknowledged in the Corrigendum. The explanation of the errors in the Corrigendum is false (which I’ll discuss some time). The 2 unlisted proxies appear to be errors of some sort, as their first 120-125 years are identical with other listed versions.
i)
incorrect calculation of all 28 tree ring principal components;
Confirmed. MBH98 had described the calculation of “conventional” principal components. At the time of MM03, we were unable to precisely diagnose why their principal components calculations were wrong. Subsequently, we have shown that they de-centered the columns and used an uncentered calculation, resulting in erroneous calculations. There are several issues involved in the MBH98 principal component series, so I’ll recapitulate our specific observation in MM03 here:
Figure 5 shows the MBH98 and re-calculated Australian PC1. The Australian PC1 is one of relatively few MBH98 series that shows anomalous 20th century behaviour and which closes on a dramatic “uptick”. The correct computation shows that this feature of this particular MBH98 series is entirely an artefact of incorrect calculation.
Figure 5 from McIntyre and McKitrick [2003]
Emulation
MM03 also undertook an emulation of MBH98 and reported that the following highly controversial finding, listed as a tenth finding in the above lettering:
j) 15th century values exceeded 20th century values, when the above defects were corrected.
Not discredited by Rutherford et al. [2005] (or Mann et al. [2003]). Mann et al. [2003] acknowledged that high early 15th century values resulted from calculating PCs over the maximum available period, but argued that our emulations were in error due to our failure to implement a previously undisclosed stepwise procedure. Mann et al. [2003] argued that this “error” resulted in the omission of the “leading pattern of variance” as follows:
The leading pattern of variance in this data set [North American tree ring network] exhibits conditions from1400-1800 that are dramatically colder than the mid and late 20th century, and a very prominent cooling in the 15th century in particular.
This claim is repeated in Rutherford et al. [2005]. We already refuted Mann et al. [2003] (and thus Rutherford et al [2005]) by showing that:
15th century values exceeded 20th century values using (correct) centered PC calculations and otherwise copying the methods of MBH98 regarding stepwise rosters and the exact inclusion of PC series used in MBH98, including 2 PCs in the AD1400 North American network (as in MBH98);
the “pattern of variance” in the MBH98 PC1 is not the “leading” or “dominant” pattern of variance when the PCs are correctly calculated, but merely a PC4 (a finding which Mann et al. have acquiesced in).
Their new argument is based on a claim that application of Preisendorfer’s Rule N, described as a “standard selection rule”, requires the use of 5 PCs in the AD1400 North American network under centered calculations; and that, if this selection rule were used, that a hockey stick shaped reconstruction would be obtained. I will discuss this in detail in another post to be made soon. However, there is evidence that the PC selections in MBH98 were not made according to this rule. Furthermore,
the new reconstruction is still not robust to the presence/absence of bristlecone pines and is merely a carrier for these controversial and problematic proxies (see our E&E paper) and
the new reconstruction still lacks statistical significance (see our GRL paper).
Since Rutherford et al [2005] does not discuss any of these issues, it obviously does not stand for the proposition that this claim is “discredited”.
Scorecard
Of the 10 claims in MM03, my scorecard indicates that:
7 have either been explicitly confirmed or are obviously correct and unrebutted. In 3 cases, subsequent information has shown the problem to more pervasive than indicated in MM03.
MBH have withdrawn one data set. 2 claims may only apply to the withdrawn data set, but other than mere assertion, there is no proof that the problems did not infect other calculations;
1 claim has led to the creation of new Supplemental Information at Nature, and the making public of the MBH98 directory at Mann’s FTP site. This tenth claim is not discredited in Rutherford et al. [2005], which fails to deal with responses already on the record, and, more importantly, this tenth claim has led to more advanced (but not inconsistent) considerations of MBH98 through robustness, proxy validity and statistical significance, which have led to 2 new peer-reviewed articles.