In God we trust, all others must bring data – unless they are energy efficiency people

We often talk about the performance gap in energy efficiency and there is no doubt that it, or at least perceived performance risk, is one of the barriers to greater investment into energy efficiency. I was reminded recently, however, that all energy sources have performance risk. kWh Analytics published their Solar Risk Assessment: 2020 report which looks at actual data on 20% of the US operating solar projects with the aim of informing investors. It makes for interesting reading and contains lessons for the energy efficiency industry.

The first conclusion is that in solar P90 isn’t P90 anymore. P90 production events are occurring more than 3x more frequently than the P90 definition implies and P90 downside events occur so often they have nearly become P50s. Furthermore extreme downside (‘P99’) events are occurring 1 in 6 years, an increase from 1 in 20 observed last year and far from the 1 in 100 per definition.  Given its position in the capital structure, equity capital suffers most when solar assets underperform. When a typical solar performs at the official P90, equity cash yield drops by 50%.

Other conclusions worth noting:

• Optimistic irradiance assumptions contribute to 5% under performance. Some of this is due to “irradiance shopping” i.e. purposely using an assumed irradiance higher than the long-term average at the site
• Sub-hourly Variability of the solar resource impacts actual production by between 1 and 4%
• US regional irradiance is down 5 to 7% from average
• The true cost of O&M can be 28% higher than planned
• ‘Weather adjustment bias’ is responsible for up to 8% bias in measured underperformance. This is because the industry often measures weather impact in two different ways; relying on pyranometers for actual insolation but using Independent Engineer satellite data for expected insolation.

None of this is intended to criticize the solar industry, only to reflect that output of solar, which is often incorrectly thought to be very predictable, (‘the sun always shines’), is more error prone than is usually thought. It also shows the importance of looking at actual data and really understanding where the data going into a financial model comes from.

Also we should never forget that every type of energy project has performance risk, whether it be solar, wind, coal, nuclear or gas generation or even an oil well.  They don’t all produce exactly what was planned at the investment decision! Energy efficiency is not alone in having performance risk. Back in Enron Energy Services we started work in the area of risk assessment of energy efficiency projects and Paul Matthew, Steve Kromer, Osman Sezgen and Steve Myers, who were the pioneers, wrote about the approach in a paper that was way ahead of its time; ‘Actuarial pricing of energy efficiency projects: lessons foul and fair’[1].  It needs to be studied again.

The lesson here for energy efficiency is that we need to collect the data on performance of real projects and start to carry out these kinds of analysis to generate portfolio performance curves. Typically this is not done, even within organisations making internal investments. The EEFIG Derisking Energy Efficiency Platform (DEEP) project is a good starting point but to be really useful it needs real data from actual project performance, something that is being included in the latest EEFIG project. The work of Recurve in the USA and the use of a standardised and industry agreed ‘weights and measures’ approach to measuring a unit of energy efficiency, is the way forward as it can access smart meter data across all sites where interventions occur automatically. The old way of doing M&V cannot scale. Better data can enable project financing and insurance solutions.

Collecting real performance data in the way that kWh Analytics do for solar is the way to get better understanding, drive better performance and drive more investment into efficiency.

In God we trust, all others must bring data

W. Edwards Deming

[1] Energy Policy 33 (2005) 1319-1328