From: Steve McIntyre

Sent: March 17, 2004 12:08 AM

To: Ashman, Dinah

Subject: Fw: Nature corrigendum Mann02478

Dear Dinah, 

I have attached the enclosed email as a word document, which may be easier to read. 

Regards, Steve McIntyre

I apologize for the length of this email, but sometimes it’s hard to write short. Regardless of the disposition of the comments below, we appreciate the steps which Nature has already taken and the ongoing consideration. The following observations are sent to you on the basis that it’s easier to deal with all pertinent issues while the file is open. To expedite matters, I have taken the liberty of attaching a list of confirmations and clarifications which I believe to be required, which you may use as you see fit.

Listing of Series

1.      The original SI lists 61 Vaganov series as being used, but, in fact, only 60 series were used, with series 55 being excluded. This is not mentioned in the correction, but should be. This presumably has a different explanation than the exclusion of the ITRDB series. The SI does not identify the Vaganov series.

2.      The original SI identified 10 Stahle SWM sites; the site for the two “unpublished” Stahle SWM series is not identified in the Corrigendum. It needs to be identified, especially since Item 6 in the response letter by Prof. Mann forwarded to me by Heike Langenburg on Feb. 27, 2004 (the Response Letter) states that there is substantial duplication of values in several SWM series, indicating the possibility of duplicated series.

3.      The Response Letter stated that one of the excluded series, “arge030”, was excluded because a more recent and more conservatively standardized replacement series was available “directly” from a colleague. In the South American network involved, it is the more recent series that have been excluded and the older series which have been used. We are unable to identify the “more recent” series used to replace this series.

4.      We strongly urge that you ensure that the SI attached to the correction of these discrepancies shows the application of the criteria listed in the asterisked explanation for the discrepancies. In particular, these are the criteria listed by Professor Mann in ref. 5 or his previous response:

(1)    the availability of “reliable information” on the methods used to remove biological trend;

(2)    the median length of the individual segments used to build the chronology, showing that this exceeds 150 years;

(3)    The mean correlation of these individual segments with the site chronology, showing that these exceed 0.5;

(4)    The first year of the chronology, showing that this was before AD 1626,

(5)    The number of segments by 1680, showing at least 8;

(6)    The last year of the chronology, showing that this was after 1970;

(7)    The number of segments after 1960, showing that this was at least 8 segments;

(8)    Whether the series was deleted because a replacement series was available “directly” from a colleague. If this criterion is used, full particulars on the replacement series should be provided including all of the information listed in 1-7 above, together with evidence showing that the replacement series was more “conservatively” detrended.

We tested out these rules on Professor Mann’s data, and found numerous examples where they break down. For instance, site cana153, a Jacoby treeline series, exhibited extremely low correlations (<0.045) between individual segments and the site chronology, violating criterion (3). We made an inquiry about the low correlations to the originating author, Professor Rosanne d’Arrigo. She discovered thereupon that the wrong site chronology had been published and promptly notified WDCP to withdraw the series. It is hard to see how the above quality control rules could have been applied as claimed by Professor Mann, as this problem would surely have come to light sooner.  Based on our investigation, regarding the MBH data archive, relative to the criteria set out above:

·         39 series had not commenced by 1626;

·         22 sites did not have 8 trees by 1680;

·         171 sites had less than 0.5 mean correlation of the individual trees with the site chronology.

 Additional minor corrections - comments

 1.      The original SI attributed the long instrumental series referred to in minor correction (2) to Bradley and Jones (1993). Without reference to the original citation, the nature of the correction is not clear. Also, the citation as it stands does not identify the NOAA Climate Data centre series used – is it GHCN v1, GHCN v2 or some other iteration. The MBH response letter states “We have updated this reference with a description of the specific source of the long instrumental data series and means by which they were gridded, in the revised supplementary information.” A little more detail in the line item in the Corrigendum would be helpful.

2.      The language in minor correction (3) is very unclear. The original SI states that the “earliest available” year for the Central Europe series is 1550, while the earliest available year in the cited publication is 1525. The first year actually used in MBH98 is 1550 – which is at odds with the language of this item. I would suggest language like: “The start year for the Central Europe series of ref. 1 was 1550, applying the version used in ref. 2, while the start year for the Central Europe series in the reference cited in the original SI was 1525.”

3.      This has several errors in a very short space. The original SI listed earlywood widths and latewood widths (not maximum density). Ref. 4, now provided as the supposedly correct citation for the SWM series, does not list any latewood width or maximum density series, but does list earlywood width and total width series. The sites listed in Ref 4. do not coincide with the sites listed in the original SI, with some of the sites listed in the original SI not listed in Ref. 4 and some sites listed in Ref. 4 not listed in the original SI. This needs to be dealt with properly. Finally, as discussed in prior correspondence, several series in this dataset have identical values in the early years. The sites for these series need to be clearly identified and any duplication corrected.

4.      Jacoby et al, use 11 northern treeline sites, not 12 as referred to here.

Other unlisted data corrigenda

1.      Although (3) above discusses the correction for the “earliest available” start year for Central Europe, there is a similar issue for Central England, not referred to. The original SI stated that the “earliest available” year was 1730, while the earliest year in the citation was 1659. This should also be clarified along the lines of the language suggested above.

2.      There is a similar issue for the series chin004, in which the earliest year in the citation is 2 years earlier than the first year used in MBH98 and a similar clarification should be made.

3.      There are many geographical errors in the original SI in respect to the grid cell locations of the long instrumental series, especially in the precipitation series. The Paris France precipitation series is incorrectly located in a New England grid cell. These are known to Mann et al., are not listed in the errata, but should be identified and dealt with at this time.

4.      The geographical locations of the Central Greenland stack in the original SI is incorrect.

5.      The geographical locations of the “npatagonia” and “cpatagonia” series are reversed in the versions used at the FTP site.

6.      We believe that that there are errors in the listing of the total number of indicators stated in Nature to have been used on an interval-by-interval basis. For example, from the FTP site information, it appears that 3 non-principal component series start in the AD1450 interval; a Vaganov PC1 is also stated in the SI to have commenced in this interval, making 4 in total, but the total number of indicators is shown as increasing only by 3. Also, MBH have stated that 159 series were used; we are unable to replicate this number with the number of indicators stated in Nature to have been used on an interval-by-interval basis. I suggest that you ask Prof. Mann to check his calculations on the number of indicators used and to include any required corrections at this time.

Methodological corrigenda

In his prior response and in other recent comments, Prof. Mann has referred to a number of corrections to the methodology description, which have not been listed in the corrigendum (and which would seem to require an explicit reference.)  These include the following corrections to existing methodological descriptions:

1.      MBH’s response letter stated: “Small gaps in the instrumental record we used (an older CRU surface temperature dataset which goes back to 1856) were filled by linear interpolation, as described in our revised supplementary information.” This is actually a pretty material issue. The original methodological description in MBH98 stated that “conventional” principal components were used – which require no missing data. There is actually a lot of missing data in the CRU dataset. The gaps are not necessarily small. For example, in the current CRU version, 4 grid cells used in MBH98 are completely empty. While the versions are different and we do not presently have access to the CRU version in MBH98, we would be very surprised if there were only “small gaps” in these data cells in the older version and no data at all in the current version. Thus, MBH98 has to have some methodology to deal with large gaps. Since the original description was not fully accurate, we think that this deserves at least a line item in the corrigendum and hopefully comprehensive explanation in the revised SI, as indicated in the MBH response letter.

2.      MBH98 stated that for the temperature data  “the mean was removed, and the series was normalized by its standard deviation”. Recently, Mann et al. stated that they used  “de-trended gridpoint standard deviations” to normalize temperature data. Again, in view of the inaccurate prior description, a line item in the Corrigendum would appear to be warranted.

3.      Mann et al. recently stated that 159 series were used in MBH98 calculations. The figure of 159 series is not itself stated in Nature or the SI thereto, where a total of 112 indicators is mentioned. This is an acutely important point, as the failure to identify the correct number of series means that, before now, researchers could not have replicated the MBH98 results. This surely merits a separate line item in the corrigendum.

4.      Related to 3 above, MBH changed rosters of tree rings in certain calculation intervals. The SI needs to show the exact number of principal components retained in each network in each period. There is no such information in MBH98. Despite careful study of MBH98 and specific requests to Prof. Mann, we were unable to discern this methodology within the MBH98 descriptions and strongly believe that it warrants a line item in the Corrigendum. (Also Prof. Mann has made statements recently that the principal components were re-calculated for every index calculation interval – which, on the basis of the FTP site, was not the case. While these recent statements are not the responsibility of Nature, clarification is much needed on this matter.)

5.      Mann et al. state in their response letter that “Precise details regarding how the [tree ring data] data were standardized are provided in the revised supplementary information”. This acknowledges that such information was not provided in MBH98 or the SI thereto. Since the “standardization” procedures carried out for the tree ring principal component calculations are far from conventional and were undisclosed, a separate line item is likewise indicated.

Supplementary Information

We originally requested the following information in connection with MBH98:

(1)   the identity of the 159 series now said to be used in MBH98 calculations;

(2)   identification of the PC proxy rosters (and sub-directory calls) on a period-by-period basis.;

(3)   residual series, together with programs used in the derivation of residuals and confidence intervals in MBH98;

(4)   programs used in MBH98 calculations;

Items (1) and (2) were referred to above, but (3) is important for the full consideration of the stepwise methods of MBH98, especially for the AD1400 interval. We believe that the following information should be included in the new SI:

1.      the results of the application of the quality control criteria set out in ref. 5, showing the compliance of the various series with each listed criterion;

2.      the standardization information used by the authors in 1997 to distinguish between retained and excluded series;

3.      the identity of the 159 series now said to be used in MBH98 calculations;

4.      identification of the number of principal component series used in each interval for each of the 6 tree ring networks;

5.      the results of each of the stepwise calculations – in particular, the AD1400 step, including residuals;

6.      the original CRU dataset used in MBH98 – both for the entire grid and for the NH average in the verification period;

7.      the correct geographic locations of the various instrumental temperature and precipitation series;

These are all required to carry out replication of the procedures in MBH98. Rather than continuing ongoing requests and disputes over the provision of this data, it makes far more sense to provide the data in the revised SI in a systematic basis.

We have also requested that the MBH98 programs be included in the SI. A reviewer of an article which we submitted on this topic to Nature stated that “there is a wide margin of uncertainty that could be resolved only by looking in detail into the whole data set and the whole software used by the authors.” In our revision, we plan to include all the software which we used in out reconstruction in order to facilitate this reconciliation. It would make a lot of sense for Mann et al. to do the same.

Conclusion

Under the circumstances, I suggest that you ask Prof. Mann to warrant that, to the best of his knowledge, the corrigenda are complete. While we have examined his data set diligently and have identified a number of discrepancies, the issuance of a correction, for an erratum known to the authors, should not be dependent on whether we have brought it to your attention. 

Again, we appreciate your consideration and apologise for the length of these comments, but we could see no obvious way of shortening them. For some of the methodological descriptions which we have referred to, we require them for our study of MBH98.

I have taken the liberty of re-stating the above comments in the form of a draft letter to Professor Mann, which you may use or disregard as you see fit.

List of Confirmations for Prof. Mann

1.      Confirm whether 61 Vaganov series were used in your calculations as stated in the original SI. (We have been advised that series 55 was not used.) If not, please amend the list of (34) series and the asterisked explanation.

2.      Please include identification of the 61 Vaganov series.

3.      Please identify the “more recent” replacement series provided directly from a colleague used as a replacement series for arge030, as described in your Response Letter.

4.      Confirm that the asterisked explanation for exclusion of listed series is a complete explanation, including application to series arge030.

5.      Provide an identification of the location of the two unpublished Stahle/SWM series, which were used but not listed. In this connection, please confirm that the duplication of early values, acknowledged in Item 7 of your Response Letter, are not errata in the unpublished data.

6.      Please ensure that the SI attached to the correction of the listing discrepancies completely shows the application of the criteria listed in the asterisked explanation for the discrepancies, including, in particular, the following criteria listed in the Response Letter (as stated in ref. 5) or his previous response:

1)      the availability of “reliable information” on the methods used to remove biological trend;

2)      the median length of the individual segments used to build the chronology, showing that this exceeds 150 years;

3)      The mean correlation of these individual segments with the site chronology, showing that these exceed 0.5;

4)      The first year of the chronology, showing that this was before AD 1626,

5)      The number of segments by 1680, showing at least 8;

6)      The last year of the chronology, showing that this was after 1970;

7)      The number of segments after 1960, showing that this was at least 8 segments;

8)      Whether the series was deleted because a replacement series was available “directly” from a colleague. If this criterion is used, full particulars on the replacement series should be provided including all of the information listed in 1-7 above, together with evidence showing that the replacement series was more “conservatively” detrended.

7.      In the Corrigendum minor correction (2), please identify the exact NOAA data set used, together with the data set stated to have been used in the original citation.

8.      Please clarify Corrigendum minor correction (3) along the following lines: “The start year for the Central Europe series of ref. 1 was 1550 (not 1525 as stated in the Corrigendum), applying the version used in ref. 2, while the start year for the Central Europe series in the reference cited in the original SI was 1525.”

9.      Please provide a similar correction in respect to the Central England series and for the chin004 series, if applicable.

10.  Please confirm that SWM series used in MBH98 included maximum density and latewood width series, as stated in the Corrigendum, rather than earlywood width and latewood width series as stated in the original SI. If the SWM sites listed in the revised SI differ from the sites listed in the original SI, then please include a comment to this effect in the Corrigendum. If this is not the case, please confirm that all the sites listed in the SI are listed in the amended citation (ref. 4) and conversely, that all the series listed in ref. 4 are included in the SI. We are advised that, notwithstanding the Corrigendum, ref. 4 does not contain any references to maximum density or latewood widths. Would you please confirm that ref. 4 does in fact report on maximum density and latewood widths, as stated in the Corrigendum.

11.  Would you please confirm that there are no geographical errors in the original listing of instrumental or other series. If there are any such errors, would you please ensure that there is a reference in the Corrigendum to these errata.

12.  Would you please confirm that the total number of indicators in the original article and SI on an interval-by-interval basis are correct. If not, would you please include an item in the Corrigendum.

13.  For methodological description items discussed in the revised SI, would you please confirm that none of the procedures described are inconsistent with the description in MBH98 and that MBH98 contained no material omissions in its methodological description. If there were any inaccuracies or material omissions in MBH98, would you please insert a line item in the Corrigendum for each such item.  Without limiting this question, we ask for specific confirmation below on several methodological issues brought to our attention.

14.  For temperature data, would you please confirm that in your calculations “the mean was removed, and the series was normalized by its standard deviation”. If the series was in fact normalized by its detrended standard deviation, would you please include an appropriate line item in the Corrigendum.

15.  For the calculation of temperature principal components, would you please define what is a “small gap” as mentioned in your Response Letter. Would you please confirm the comment in your Response Letter that only “small gaps” as so defined have been interpolated. We suggest a line item in the Corrigendum disclosing the interpolation procedure.

16.  Would you please advise us whether 159 series were used in MBH98 calculations. If this is the case, would you please include a line item in the Corrigendum referring to this and sufficient information in the SI that the 159 series can be completely identified.

17.  Would you please confirm that the revised methodological description in the new SI includes an explanation of how the number of principal component series retained in each network for each interval are determined.

18.  Would you please confirm that the new SI enables the determination of the 251 series mentioned in ref. 4 and also the 415 series referred to in question 6 of your Response Letter.

19.  Would you please confirm that the new SI describes the procedure used to standardize tree ring data. If this procedure is material, please include a reference in a line item pursuant to (12) above.

20.  Would you please confirm that, (a) to your knowledge, there are no remaining errata in data or data citations in MBH98 which have not been disclosed; (b) to your knowledge, the methodological description contained in the new SI is sufficient to enable replication of your results, using the data provided in the new SI or cited from public archives.