Inconvenient GW emails?

[Dr Mordrid]

Other sites are picking up on the hacker-stolen records from the Hadley Climate Research Unit. If not forgeries this is evidence of selective "science" in the GW community.

They seem to be fairly represented here even if the end of the Telegraph article goes a bit hyperbolic. Much more to go. I won't publish them here, but you can read some of the data on these links. You can also search bittorrent or even Google for FOI2009.zip to get the whole package.

TGIF story....(PDF)

DailyTech story....

Telegraph link....

[Technoid]

Anyone even slightly surprised?

[Brian Ellis]

Surprised? No. It is obvious that if they chose to publish 1079 e-mails, this must be just a tiny, tiny fraction of those exchanged, no doubt selectively chosen, if not edited, and certainly taken out of context.

As a retiree, my e-mail traffic is small, yet I've received 1918 in just 2 months, of which 278 were spam.



I can't believe that the Hadley Unit received/sent only 1079 over the years. If these guys want to be credible, they should publish all the traffic.

Apart from that, there are unfortunately always scientists (?) who fudge results, always have been, always will be. However, this works both ways and many of the anti-climate change brigade have been known to do likewise. Furthermore, part of the workings of the Hadley unit, as in all scientific teams is for one person to communicate to others new results to his colleagues who have to play devil's advocate. It is therefore normal that contradictory views may be expressed in internal communications.

And the least surprising thing of all is the Daily Telegraph jumping on the bandwagon. This rag is a disreputable right-wing paper, known for its distortions of truth and has been anti-climate change for years. Many of its articles on the subject have been so laughable that even the more serious deniers have been embarrassed.

I'll be interested to see what happens after the dust has settled.

[Evildead666]

Might be why the others are holding off for the moment.

I've a had a quick look, and the emails go all the way back to 99 and before....
Thats a piddling amount of emails for 9 years or so...

Theres a lot of .pro files also, which are stats and lots of numbers i expect.....

[cjolley]

And the quotes that Doc originally included sounded mostly like casual shop talk to me.

Why don't anti GW activist study GW science instead of GW scientists?
You could substitute the word Evolution for Global Warming in just about any anti-GW argument in current use and it would be indistinguishable from current anti-evolution rhetoric.

Quote mining for example.

[Jammrock]

Knowing the British authorities this will probably spark some sort of official investigation. My money goes on the "hacker" being a former student or colleague that is either trying to whistle blow or was ousted for being a skeptic or some other reason.

Either way, this will very likely damage the public reputation of the pro-Global Warming movement. Which is sad because if they are right it could delay putting in sound counter tactics for quite some time. Especially with the economic downturn piling on top of everything.

[Dr Mordrid]

At the very least trying to shoehorn or otherwise falsify your data to fit your desired results is a no-no. It's already hit CBS, the Washington Post and other major media outlets on this side of the pond.

Infosecurity had this to say;

>
According to Mark Fullbrook, a director with Cyber-Ark, the IT security specialist, whilst details of the hack are still emerging, it is looking likely that it is a major one, and will act as a classic case study on the need for secure collaborative working practices.

"It appears that the data stolen includes more than 1000 emails and 70-odd documents that are highly contentious as regards the issue of global warming - something that various groups have alleged the governments of the world have kept a lid on for years", he added.

According to Fullbrook, it remains to be seen how explosive the data stolen by the hackers is, but unconfirmed reports suggest that the information is potentially embarrassing to several of the leading academics in the field of climate research in the UK and US.

What's interesting about the story, Fullbrook went on to say, is that the FTP link is on a Russian server which the hackers have chosen carefully - possible because of worries that the data might be taken down when the server owners realise the political dynamite it contains.
>

[cjolley]

At the very least trying to shoehorn or otherwise falsify your data to fit your desired results is a no-no.

Maybe the most ironic statement I've ever read.

[Dr Mordrid]

Just taking them at their own words.

[Dr Mordrid]

Washington (DC) Times...

EDITORIAL: Hiding evidence of global cooling

Scientific progress depends on accurate and complete data. It also relies on replication. The past couple of days have uncovered some shocking revelations about the baloney practices that pass as sound science about climate change.

It was announced Thursday afternoon that computer hackers had obtained 160 megabytes of e-mails from the Climate Research Unit (CRU) at the University of East Anglia (UEA) in England. Those e-mails involved communication among many scientific researchers and policy advocates with similar ideological positions all across the world. Those purported authorities were brazenly discussing the destruction and hiding of data that did not support global-warming claims.

Professor Phil Jones, the head of the Climate Research Unit, and professor Michael E. Mann at Pennsylvania State University, who has been an important scientist in the climate debate, have come under particular scrutiny. Among his e-mails, Mr. Jones talked to Mr. Mann about the "trick of adding in the real temps to each series ... to hide the decline [in temperature]."

Mr. Mann admitted that he was party to this conversation and lamely explained to the New York Times that "scientists often used the word 'trick' to refer to a good way to solve a problem 'and not something secret.' " Though the liberal New York newspaper apparently buys this explanation, we have seen no benign explanation that justifies efforts by researchers to skew data on so-called global-warming "to hide the decline." Given the controversies over the accuracy of Mr. Mann's past research, it is surprising his current explanations are accepted so readily.

There is a lot of damning evidence about these researchers concealing information that counters their bias. In another exchange, Mr. Jones told Mr. Mann: "If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone" and, "We also have a data protection act, which I will hide behind." Mr. Jones further urged Mr. Mann to join him in deleting e-mail exchanges about the Intergovernmental Panel on Climate Change's (IPCC) controversial assessment report (ARA): "Can you delete any emails you may have had with Keith re [the IPCC's Fourth Assessment Report]?"

In another e-mail, Mr. Jones told Mr. Mann, professor Malcolm K. Hughes of the University of Arizona and professor Raymond S. Bradley of the University of Massachusetts at Amherst: "I'm getting hassled by a couple of people to release the CRU station temperature data. Don't any of you three tell anybody that the UK has a Freedom of Information Act!"

At one point, Mr. Jones complained to another academic, "I did get an email from the [Freedom of Information] person here early yesterday to tell me I shouldn't be deleting emails." He also offered up more dubious tricks of his trade, specifically that "IPCC is an international organization, so is above any national FOI. Even if UEA holds anything about IPCC, we are not obliged to pass it on." Another professor at the Climate Research Unit, Tim Osborn, discussed in e-mails how truncating a data series can hide a cooling trend that otherwise would be seen in the results. Mr. Mann sent Mr. Osborn an e-mail saying that the results he was sending shouldn't be shown to others because the data support critics of global warming.

Repeatedly throughout the e-mails that have been made public, proponents of global-warming theories refer to data that has been hidden or destroyed. Only e-mails from Mr. Jones' institution have been made public, and with his obvious approach to deleting sensitive files, it's difficult to determine exactly how much more information has been lost that could be damaging to the global-warming theocracy and its doomsday forecasts.

We don't condone e-mail theft by hackers, though these e-mails were covered by Britain's Freedom of Information Act and should have been released. The content of these e-mails raises extremely serious questions that could end the academic careers of many prominent professors. Academics who have purposely hidden data, destroyed information and doctored their results have committed scientific fraud. We can only hope respected academic institutions such as Pennsylvania State University, the University of Arizona and the University of Massachusetts at Amherst conduct proper investigative inquiries.

Most important, however, these revelations of fudged science should have a cooling effect on global-warming hysteria and the panicked policies that are being pushed forward to address the unproven theory.

[Evildead666]

Sadly this might sound the bell for moving to alternative fuels....

We SHOULD be looking to be more carbon neutral, this just states that we may not be having a collective GLOBAL threat to the environment, but sh*t me, anyone can see we're screwing with local environment in a big way.

Oh and Carbon is not the only gas we should be checking.

Some of the incinerators are legally bound to not produce certain gases, so they transform the output into a non-controlled gas. or particle. Polluting, but legally..../tin hat off

[KRSESQ]

As was discussed in this thread, the Earth may have a far greater capacity for keeping itself carbon-neutral than anyone suspected.

This is going to be damning as hell and will cause a major uproar if it's proven that these scientists withheld evidence in order to bolster their research agenda, especially if the evidence withheld ultimately blows AGW theory out of the water.

I'm still reserving judgement until I've seen all the evidence.

Kevin

[Jammrock]

We SHOULD be looking to be more carbon neutral, this just states that we may not be having a collective GLOBAL threat to the environment, but sh*t me, anyone can see we're screwing with local environment in a big way.

Amen, Mr. Evil.

[Dr Mordrid]

Sadly this might sound the bell for moving to alternative fuels....

There are other reasons for moving to alternative fuels, national security for one. We (meaning the west as a whole) have been depending on unstable and at times unfriendly sources for energy too long and need multiple reliable, and preferably renewable, domestic sources.

Cellulosic alcohols (made from most anything containing cellulose - paper, plant waste etc.) will be a big part of this especially given recent work on the biotech front, and it'll also help reduce what goes into landfills. So will be natural gas, for which we already have a massive infrastructure...perhaps the best way to transport hydrogen....and in the US hundreds if not thousands of years worth of reserves. Not to say we have to burn it, there are other ways to skin a cat.

First one to find a cheap, mass production method to remove the carbon atom and not make CO2 is a billionaire, and the leading candidate is the Norwegian Kværner process; using a commercial plasma-arc waste disposal system to break LNG and other hydrocarbons down into carbon, heat and hydrogen. This plasma variant of Kværner was just patented this year.

Article....

We'll see how well it scales up.....

[Dr Mordrid]

CBS News has a report, and part of it is a tome by one of the CRU programmers. Sounds like the data, and their software, is a ball of confusion.

>
In addition to e-mail messages, the roughly 3,600 leaked documents posted on sites including Wikileaks.org and EastAngliaEmails.com include computer code and a description of how an unfortunate programmer named "Harry" -- possibly the CRU's Ian "Harry" Harris -- was tasked with resuscitating and updating a key temperature database that proved to be problematic. Some excerpts from what appear to be his notes, emphasis added:

I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation - apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective - since we're using an off-the-shelf product that isn't documented sufficiently to say that. Why this wasn't coded up in Fortran I don't know - time pressures perhaps? Was too much effort expended on homogenisation, that there wasn't enough time to write a gridding procedure? Of course, it's too late for me to fix it too. Meh.

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!

One thing that's unsettling is that many of the assigned WMo codes for Canadian stations do not return any hits with a web search. Usually the country's met office, or at least the Weather Underground, show up – but for these stations, nothing at all. Makes me wonder if these are long-discontinued, or were even invented somewhere other than Canada!

Knowing how long it takes to debug this suite - the experiment endeth here. The option (like all the anomdtb options) is totally undocumented so we'll never know what we lost. 22. Right, time to stop pussyfooting around the niceties of Tim's labyrinthine software suites - let's have a go at producing CRU TS 3.0! since failing to do that will be the definitive failure of the entire project.

Ulp! I am seriously close to giving up, again. The history of this is so complex that I can't get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections - to lat/lons, to WMOs (yes!), and more. So what the hell can I do about all these duplicate stations?...

>
One programmer highlighted the error of relying on computer code that, if it generates an error message, continues as if nothing untoward ever occurred. Another debugged the code by pointing out why the output of a calculation that should always generate a positive number was incorrectly generating a negative one. A third concluded: "I feel for this guy. He's obviously spent years trying to get data from undocumented and completely messy sources."

Programmer-written comments inserted into CRU's Fortran code have drawn fire as well. The file briffa_sep98_d.pro says: "Apply a VERY ARTIFICAL correction for decline!!" and "APPLY ARTIFICIAL CORRECTION." Another, quantify_tsdcal.pro, says: "Low pass filtering at century and longer time scales never gets rid of the trend - so eventually I start to scale down the 120-yr low pass time series to mimic the effect of removing/adding longer time scales!"

It's not clear how the files were leaked. One theory says that a malicious hacker slipped into East Anglia's network and snatched thousands of documents. Another says that the files had already been assembled in response to a Freedom of Information request and, immediately after it was denied, a whistleblower decided to disclose them. (Lending credence to that theory is the fact that no personal e-mail messages unrelated to climate change appear to have been leaked.)
>
ScienceMag.org published an article noting that deleting e-mail messages to hide them from a FOI request is a crime in the United Kingdom. George Monbiot, a U.K. activist and journalist who previously called for dramatic action to deal with global warming, wrote: "It's no use pretending that this isn't a major blow. The emails extracted by a hacker from the climatic research unit at the University of East Anglia could scarcely be more damaging."
>