Cooling turned into warming
On following Willis E’s Darwin thread on WUWT, I’ve noticed that a number of commentators have indicated that perhaps Darwin is just a special case? In order to see if this is the case, I have fitted a trend line (a first order polynomial i.e. linear regression) to all the available raw and adjusted combined data series for each WMO station in the NOAA GHCN dataset. By examining the relative magnitudes and signs of these raw/adjusted fitted slopes for each WMO station, it has been possible to identify the worst cases in which a “cooling” trend in the raw data has been converted into a “warming” trend in the adjusted data by virtue of the adjustments applied to the raw data.
In total, I have found 194 instances of WMO stations where “cooling” has been turned into “warming” by virtue of the adjustments made by NOAA to the raw data. As can be seen from the following “Cooling turned into warming” table (Table 1) below, which lists the Top 30 WMO station on the “cooling to warming” list, Darwin is ranked in only 26th place! The list is sorted by the absolute difference in the magnitude of the raw to adjusted slopes i.e. the list is ranked so that the worst case of “cooling” converted to significant “warming” comes first, followed by the next worse etc.
It’s clear from looking at the list that Darwin is certainly not “just a special case” and that in fact there are many other cases of WMO stations where (as with Darwin) NOAA have performed physically unjustifiable adjustments to the raw data. As can been seen from Table 1 many of these adjustments result in trend slopes which are greater than the IPCC’s claimed 0.6 deg. C/century warming during the 20th century said by the IPCC to be caused by man’s emissions of CO2 through the burning of fossil fuels.
Are we to believe that physically unjustifable adjustments of this size and number are necessary to allow for time of observation bias (TOB) and Station History Adjustment Programs (SHAP) i.e. station movements (and other adjustements for MMTS, FILNET etc) in the raw data? If not, then surely there should be some doubt as to the validity of the IPCC’s claim that post 1970s ‘unprecedented’ warming trend can only be explained by man’s emissions of CO2? Given the size and number of these adjustments made by NOAA, isn’t it also in fact equally possible that the claimed late 20th Century warming trend could well be (at least in part, if not in total) result as a consequence (i.e an artefact) of the adjustments made to the raw data?
Table 1 – Cooling turned into warming list – Top 30
The Top 30 ‘Cooling turned to warming’ list can be downloaded by clicking here
From Table 1 above the 13th worst case is Edson, Alberta, Canada which starts with a raw cooling slope of -1.7 deg. C/century and is converted to a very significant warming slope of +5.1 deg. C/century. According to the NOAA’s GHCN station inventory file Edson is located at latitude 53′ 58” , longitude -116′ 47” at an elevation 927 meters and has a airport code of ‘A’ which means it is an airport station. Google Maps shows Edson airport here.
I’ve combined the different raw and adjusted data series for Edson into a single combined series as was done for Darwin in my earlier thread on reproducing Willis E’s analysis. The following charts (Figure 6 and 7) show the combined raw and adjusted series and the differences between them (adjusted minus raw) for each year of the combined series.
Figure 6 – Edson raw/adj. series Figure 7 – Edson adjusted – raw differences
Figures 6 and 7 clearly show that as a consequence of the physically unjustifiable adjustments made by NOAA a clear negative “cooling” slope in the raw data for Edson of -1.7 deg. C/century has been converted into a significant “warming” slope in the adjusted data of +5.1 deg. C/century. Once again as with Darwin, “Cooling” has been converted into significant “warming”.
Warming turned into cooling
Now of course this analysis of trends would not be complete without performing a similar analysis in which we look at ‘warming turned into cooling’ as well as ‘cooling turned into warming’ as we have just done above. In order to produce a similar list as done above in Table 1 but this time for ‘warming turned into cooling’, I have carried out a very similar analysis of the raw and adjusted slopes except in this second case looking for where a signifiant positive raw slope (i.e. warming) has been converted into a significant negative adjusted slope (i.e cooling).
In total I found 216 instances of WMO stations where ‘warming has been turned into cooling’ by virtue of the physically unjustifiable adjustments made to the raw data by NOAA. The following ‘Warming turned into cooling’ table (Table 2) below lists the top 30 WMO station on this ‘warming to cooling’ list. The list is sorted by the absolute difference in the magnitude of the raw to adjusted slopes i.e. the worst case of warming converted to significant cooling is listed first, the next worse second etc.
Table 2 – Warming turned into cooling list – Top 30
The Top 30 ‘Warming turned to cooling’ list can be downloaded by clicking here
The 28th worst case is Mayo Airport, Yukon, Canada which starts with a raw warming slope of +1.3 deg. C/century and is converted to a very significant cooling slope of -4.5 deg. C/century. According to the GHCN station inventory file it is located at latitude 63′ 62” , longitude -135′ 87” at an elevation 504 meters. Google Maps shows Mayo Airport here.
I’ve combined the different raw and adjusted data series for Mayo Airport into a single combined series as was done for Edson above Darwin on a previous thread. The following charts (Figures 8 and 9) show the combined raw and adjusted series and the differences between them (adjusted minus raw) for each year of the combined series.
Figure 8 – Mayo raw/adj. series Figure 9 – Mayo adjusted – raw differences
Figures 8 and 9 clearly show that as a consequence of the physically unjustifiable adjustments made by NOAA a clearly positive “warming” slope in the raw data for Mayo Airport of +1.3 deg. C/century has been converted into a significant “cooling” slope in the adjusted data of -4.5 deg. C/century. In contrast to Darwin and Edson, in this case Mayo airport’s “warming” has been converted into significant “cooling”.
Conclusions
The following conclusions my be drawn from my analysis of the NOAA GHCN
adjustments to the raw station data.
1. It has been possible to perform an analysis to show that, far from being a special case, Darwin is in fact only one of many cases where physically unjustifiable adjustments have been made to the raw data by NOAA. NOAA’s adjustments have resulted in many cases were a previously negative slope in the raw data for a WMO station has been converted to a significantly large warming trend in the adjusted data. In effect ‘cooling’ in many WMO stations has been turned into significant ‘warming’ by virtue of the physically unjustifiable adjustments made to the raw data by NOAA.
2. It has also been possible to perform an analysis to show that further physically unjustifiable adjustments have been made to the raw data for many WMO stations by NOAA which have resulted in a previously positive slope in the raw data being converted to a significantly large cooling trend in the adjusted data. In effect ‘warming’ in many WMO stations has been turned into significant ‘cooling’ by virtue of the physically unjustifiable adjustments made to the raw data by NOAA.
3. Evidence has been presented which clearly shows that NOAA have carried out many physically unjustifiable adjustments to the raw data for many WMO stations. In the light of this evidence and given the no. of stations adjusted and the size of many of these adjustments then surely there should be some doubt as to the validity of the IPCC’s claim that post 1970s ‘unprecedented’ warming period is due to man’s emissions of CO2 from the burning of fossil fuels. Is it not equally possible given the size of many of these adjustments that the late 20th century warming trend could well have resulted in part if not in whole from these adjustements i.e that it could just be an artefact of the adjustments made to the raw data by NOAA?
Digging in the Clay 
A valuable analysis, but :-
The “cooling” adjustments appear to be just as large as the “warming” adjustments, so there does not appear to be any bias.
You assert that the adjustments are “physically unjustifiable” but you provide no evidence of this.
I agree that your findings suggest that the “global temperature” is unreliable, as are IPCC findings based on it.
I agree with Jonas’ exceptions to your assertions, especially if the adjustments cancel out statistically even when broken down temporally. But is it possible that the data offers an orchard of ripe cherries, begging to be picked by researchers with an ax to grind?
Nice work. What needs to be done is to relate these figures to the wieghting each station is given in global averages to see if it introduces a bias.
Since NOAA is the “warmest” dataset these adjustments maybe part of the reason.
One point please get the facts straight about what IPCC AR4 says.
“greater than the IPCC’s claimed 0.6 deg. C/century warming during the 20th century said by the IPCC to be caused by man’s emissions of CO2 through the burning of fossil fuels.”
In fact AR4 says “majority” , ie >50% due to man. You imply , as is often reported in the press, they say it is [all] due to man. Niether do they attribute all anthropogenic change to fossil fuels. Much of it is land use changes, deforestation etc.
MJ,
“You assert that the adjustments are “physically unjustifiable” but you provide no evidence of this.
I agree that your findings suggest that the “global temperature” is unreliable, as are IPCC findings based on it.
“
Hopefully you’ve bene following Willis E’ darwin Before and After thread on WUWT? If so then you’ll see that Willis has attempted and failed so far to replicate NOAA’s SHAP adjustments for Darwin. Also Ryan Stepehenson at (02:35:00), provided a lengthy replyto temp 18:44 statement that
“No homogenization method is going to be perfect or “natural”, but the fact of the matter is that the data does need to be homogenized”
As you’ve hopefully seen from my thread I’ve done a similar analysis for Edson and Mayo Airport, bot atthe extremes of these clearly non-physical adjustments that are not even consistent with NOAA’s own documentation. Willis has now sent an email to NOAA requested an explanation as to why darwin’s adjustment do not tie up with NOAA’s own documentation. Willis will no doubt keep us all posted on developments.
I think that’s enough for now. Do you now agree that adjustments of the size and number that I and other have demonstrated that NOAA have carried out to over 400 WMO stations can not be physically justified?
KevinUK
neil,
“But is it possible that the data offers an orchard of ripe cherries, begging to be picked by researchers with an ax to grind?
“
To quote Steve M, I think the NOAA GHCN adjustments and GISS and CRU adjustments ‘will be a gift that keeps on giving’. At the moment I’m just picking some rather low hanging fruit. Now if you could tell me where exactly i’ve been picking cherries that would help. Do I personally have an agenda? Of course I do, it’s called ensuring that the scientific method is properly applied to climate science.
Unlike Eric Steig, I subscribe to the school of scientific method where you attempt to falsify your own or someone else’s analysis and don’t just give up at the point where the results of your own or anyone else’s analysis agree with your pre-conceived conclusions.
KevinUK
Anonymous
“Anonymous said…
Nice work. What needs to be done is to relate these figures to the wieghting each station is given in global averages to see if it introduces a bias.”
I’m working on that just now, but its going to be a little while before the jury is out. Atthis stage there is still plenty to do just looking at the adjustements to individual stations. Griddingthe data then has a tendnecy to spread any possible bias, so atthis stag eits best to avoid the additional complication that gridding the data brings.
“In fact AR4 says “majority” , ie >50% due to man. You imply , as is often reported in the press, they say it is [all] due to man. Niether do they attribute all anthropogenic change to fossil fuels. Much of it is land use changes, deforestation etc.
“
Thank you for qualifying/correcting my statement on the IPCC’s claims forlate 20th century warming. Its never good to exaggerate is it? The IPCC would never exaggerate their claims would they? What do you think of Lord Monckton’s and Steve Fielding’s letter to Pachauri?
KevinUK
All,
I think the point Kevin is trying to make, is that Mayo and Edson are both small rural airports for which there has been no ‘site move’; any move of the monitoring equipment around the sites or changes of equipment would have small effects on the data. However, both have adjustments applied of up to 4 deg Celsius, which seems excessive in the circumstances and it it this magnitude that he is refering to as ‘physically unjustifiable’.
I have GIStemp data for Edmonton, about 100 miles from Edson (although I need to check the relative altitudes). Even this largish city is warming at a fraction of the rate ascribed to Edson – small rural airport. I will try to post the data shortly.
KevinUK,
“neil,
“But is it possible that the data offers an orchard of ripe cherries, begging to be picked by researchers with an ax to grind?”
…
Now if you could tell me where exactly i’ve been picking cherries that would help.”
Noooo. It’s the CRU/GISS researchers I had in mind, not you. Obviously you have fairly laid out what you’re doing.
…
“Do you now agree that adjustments of the size and number that I and other have demonstrated that NOAA have carried out to over 400 WMO stations can not be physically justified?”
I haven’t yet seen the breadth and depth of analysis that would scientifically justify such a bald and categorical statement. If such analysis has occurred I would be very interested in a link. Transparency by the originators of the data is what we really need, of course. Don’t see that happening.
Cheers.
“”You assert that the adjustments are “physically unjustifiable” but you provide no evidence of this.”
Well I agree with Kevin. Looking at the Mayo site I notice that in 1945ish there appears to be a small discontinuity in the means before and after this year. This might suggest the site suffered a change, like a relocation or re-equipping with new thermometers. So we would expect this algorithm to detect this discontinuity and correct for it, producing a new data set which blends out the discontinuity producing a smoother trendline.
Only it doesn’t. The discontinuity in 1945 in the adjusted data is much bigger than in the raw data. It has shifted the pre-1945 data up by 1.5Celsius. The adjustment is clearly not correct.
Looking at the graphs I would have said it should have shifted the data before 1945 DOWN by 1.5Celsius, which may indicate that the algorithm has a plain old-fashioned bug in it.
In the case of Edson it is a little bit of a different story. The adjustments are being made every ten years or even more frequently. This means the site is not stable for more than 10 years. Some of the adjustments being made in the first few years are massive – despite the fact that these are the years where most of the readings are present! What has happened here is that the algorithm hasn’t detected that the readings after 1960 are sporadic nor has it detected that the required adjustments are so frequent that achieving a genuine correlation with other sites is impossible and that the Edson data should be thrown out.
So now we seem to have found three bugs in the algorithms used to make these adjustments. One bug where the algorithm has made changes to the raw data that didn’t need to be adjusted (i.e. Darwin where it seems the discontinuity detection algorithm has detected a need for adjsutments where no such need exists), one bug where the algorithm may have correctly detected that the raw data needed an adjustment but where it made the adjustment in the wrong direction (i.e. Mayo) and finally a bug where the algorithm tried to do an adjustment on a dataset that was so unreliable that it couldn’t possibly correlate properly to other sites.
Now you might well say “oh but all these things cancel out in the end” but it is a bit of an article of faith. Imagine scientists told Neil Armstrong that the lunar module software was full of bugs but they thought that these bugs would all cancel out in the end and the moon landing would go OK?
Ryan,
Kevin and I have been looking at the Edson (and surroundings) data this evening. Others in the area seem to suffer the same fate of ‘too much’ adjustment – worth a closer look we think. The Canadian Stations seem to suffer a lot of short series and may suffer from this more than most.
The “oh but all these things cancel out in the end” arguement is held up for Global Average Temperature (which is a flawed concept anyway), but these ajustments affect regional temperatures and then the models based on them. No wonder the whole climate prediction thing is so wrong! 😉
Yes, the algorithm used is clearly full of bugs which are conveniently exposed by these extreme cases. I looked again at the Darwin plots from Willis and bringing all the detected bugs together I found the following flaws in the algorithm:-
1] Incorrectly detecting data for adjustment that does not require adjustment
2] Correctly detected adjustments which are then made in the wrong direction.
3] Adjustments of unreliable data or data that is too short in duration to be used reliably.
4] Adjustments of single years which are then not applied to all following years
5] Failure to detect some obvious (and known)discontinuities that actually require adjustment
6] Applying adjustments up to two years too early or up to two years too late based on incorrect detection of the date at which the discontinuity occured.
7]Errors in the magnitude of the correction applied.
I don’t think you can assume that these will always cancel out over a large number of datasets. Especially [5] seems troublesome (if it didn’t then you wouldn’t need to do the adjustment!). [2] Could also be a big issue if it is biased in one direction – the two examples I have seen have both led to increased temperatures. If you put an increased temperature in the earlier years it will tend to be ignored, because people are only really interested in the rate of change in the later years – so if you put a increased temperature in the later years then it will contribute to a perceived warming trend.
I think it is worth Kevin posting his Mayo example at WUWT to reach a wider audience. It is such a clear example of a daft bug in the algorithm that these climate trends are relying on that its laughable. It’s really clear that the algorithm has made the adjustment in the wrong direction. The shift is actually 2Celsius in the wrong direction between 1937 and 1950 (not 1.5Celsius as I had said earlier).
Ok, this bug came to me in my sleep (it’s a bed bug?). Looking at the Darwin data and the Edson data we can see that the adjusted value for the most recent years shows the data to have been adjusted. Since we are looking for the absolute temps as well as the trends in order to perform calibration against satellite and tree ring data, this seems perverse. The most recent year should be postulated as being an “absolute” temperature reading especially if it is in the last twenty years or so. So that gives me bug [8]
[8] The algorithm does not shift the adjusted data such that the most recent years are shown as coincident in temperature.
Do we have a record of when physical readings of Mercury thermometers at a station was changed to remote readings of digital equipment? This might make a difference in observed temperatures which may or may not be corrected by a realistic level.
Also are we yet able to say WHICH stations are adjusted for UHI, if so from WHEN and by how MUCH?
Tonyb
Tonyb, this takes time, looking at stations and groups of stations individually.
Same too for UHI. Unlike GIStemp where there are supposed criteria for adjustment, based on satellite detection of light pollution at night and population size. As you know even with that there are wrong adjustments (such as outlined in “Climate Fast Food” this blog, 13 Nov). As a rule GISS doesn’t adhust rural data, but there are exceptions. Here (NOAA/GHCN) the algorithms are also adjusting rural data.
If you can look for and provide population data by town/city, the database can help with your question on UHI quickly.
Update: With Tom Peterson’s (GHCN’s) reply to Willis Eschenback here: http://wattsupwiththat.com/2009/12/20/darwin-zero-before-and-after/
(comment from Willis Eschenbach (21:06:35)on 24th Dec)we are peomised publication of the GHCN algorithms and new ones as it seem GHCN are unhappy with errors in the present ones.
The thing that seems to be overlooked too often is this:
A scientific result should be reproducible.
Any scientist that doesn’t disclose their data and their methods to allow their results to be replicated is not practising science.
steve