NESO forecasting errors: the silent threat to energy security

My blog post on the threat to energy security on 8 January went viral. It’s the single most widely read thing I have ever written, and it has been widely re-shared on social media and quoted in the press. The National Energy System Operator (“NESO”) is very unhappy with me, and has disputed my claims, telling journalists:

“NESO operates Great Britain’s electricity network to one of the highest levels of safety and reliability anywhere in the world. Yesterday our control room engineers used our standard operational tools to manage the electricity network and ensure that we maintained enough electricity for our standard operating contingency. At no point were electricity supplies less than anticipated demand and our engineers were able to rebalance the system without the need to consider emergency measures. One of the standard operating reserves held by NESO at all times is for the largest power generator on the system, which last night was 1400 MW, not the 580MW that has been quoted online,”
– Craig Dyke, Director of System Operations, NESO

OK, then, so what units exactly were held in reserve and how quickly could they have come on? Generating assets do not sit around idle on days when there’s a lot of money to be made – if they could run on 8 January, they did run. Elsewhere in comments to journalists, NESO pointed to the fact that the Capacity Market was not activated, which could have yielded further supply, but again, my question is, which units could run in the Capacity Market but were not running anyway on the day? The other problem is that one reason there was not a Capacity Market activation was that NESO got its demand forecast wrong, something which is actually quite common.

The demand forecast is a critical component of the determination of the spare margin – if you don’t know what the demand will be, you cannot be confident that sufficient margin has been secured.

So in this post I will look at what NESO does to forecast demand and the system margin, why it is difficult, and why it is actually getting harder as the energy transition progresses. I will demonstrate that these forecast errors can be so large that it is simply not possible for NESO to claim with any confidence that it always has the required level of reserves in hand, and that on days when the market is tight, this creates additional risks to security of supply.

How is the system margin and day ahead energy requirement calculated?

The Grid Code sets out the basis for NESO’s forecasts (OC 1.6.1) – see box. It is interesting to see what is missing from this list which is a feature of the age of the Grid Code. Yes, the website is regularly updated but this section of the Code has clearly not been revised in a long time – for example, why are batteries only included for demand and not supply? And why do embedded assets not have to provide information to NESO? The answer is that they did not exist when the list was written.

Another question is why has NESO not proposed any Code amendments to bring the forecasting requirements up to date (I suspect the answer is that this would lead to an obligation to update its models and processes which takes time and money – a bit like turkeys voting for Christmas!)

There are five main components to this calculation of system margin and the day ahead energy requirement that create significant levels of uncertainty:

• Expected contribution from renewables: this is difficult to forecast since it depends on the notoriously hard to predict weather. The error attributable to this component is increasing as the amount of renewables on the system grows.
• Expected contribution from embedded generation: NESO has no direct visibility of the amount of embedded generation so this is simply an estimate. Again, as the amount of embedded generation, particularly renewables, grows, this becomes harder to predict.
• Amount of potential demand flexibility (consumer demand management): this does not relate to the Demand Flexibility Service, it is the growing trend of suppliers and aggregators helping consumers to optimise their energy costs by changing their usage profiles. For example, where large energy consumers have hourly pricing, they could save money if they reduced consumption during peak hours. While these data are required under the Grid Code to be submitted to NESO by Suppliers, they are not published anywhere. It is therefore unclear how significant this element is, but anecdotally, this is a growing trend, so there are questions about how changes in the behaviour of demand affects NESO’s modelling.
• Expected grid constraint limits and volumes of generation that may be constrained: because of the difficulties in modelling renewable and embedded generation, constraint modelling becomes much harder. In addition, reserve is procured nationally, and it is not always clear to what extent constraints may interfere with the delivery of the reserve. A further difficulty is that Network Management Systems (“NMS”) on the distribution networks are increasing in complexity, and their impact on the transmission system is increasing, but, as with embedded generation, NESO has limited visibility of them. The impact of this is that the control room might instruct say 50 MW of reserve to activate, but because a DNO activated its NMS nearby as a result of an overloaded line, 50 MW of nearby generation disappears negating the actions of the control room.
• The likely interconnector position including the possibility of short-notice counter trading: the difficulties with this were described in my previous post – interconnectors can be re-traded during the day, sometimes at short notice, and this is next to impossible for NESO to model ahead of time.

Forecasting errors are growing but there is little transparency over them

In my previous post, I described the very limited forecast error data published by NESO – demand forecast performance analysis contains half-hourly forecast versus out-turn data from April 2021 and November 2024. According to this, the forecast error can be from 0 MW to 4,686 MW with an average of 609 MW. This magnitude of error could be critical on a tight day like 8 January.

However, these data are (a) not published very often, and (b) not sufficiently granular. Half-hour intervals are very long in the context of system margin management – averaging over half an hour potentially disguises the size of the error over the shorter timescales that are relevant to margin and frequency management. And the most recent data are only up to the end of November. NESO should publish its forecast error in 1-minute intervals, and it should be available much faster, so that market participants can understand the reliability of system margin and demand data.

I recently wrote about frequency jumps, demonstrating that the frequency is moving outside the operational limits thousands of times a year. A 1 GW loss of supply would typically drop the frequency by 0.2 Hz. However, unlike generation losses, which are instant, forecasting errors can build up, so will not necessarily be visible as a jump on the frequency. The accumulation of these errors could be an explanation for the regularity with which the frequency operating limits are breached, but without transparency on the size and nature of the errors, it is difficult to be certain.

Forecasting errors will lead to greater frequency variability, and if frequency moves away from 50 Hz ie outside the operational limits without a corresponding loss of generation, it is highly likely that a forecasting error was to blame. This chart shows the number of 5-second intervals during which grid frequency was more than 0.05 Hz away from the 50 Hz target. These are increasing over time, other than a hiatus during covid when they trended sideways).

In addition to its existing day-ahead accuracy disclosures, NESO should publish its forecasting error at 4 and 8 hours ahead of delivery because the forecasts are not static – as a general rule, forecasts should become more accurate the closer to real time they are produced, but with so many variables over which NESO has limited visibility, this may not be the case. The market should not only know what the margin and load expectations are at different time intervals before delivery, but what forecast errors apply over these time intervals.

It is particularly critical at short lead times because of the relationship between demand, supply and grid frequency. The Security and Quality of Supply Standard (“SQSS”) requires NESO to secure the single largest infeed loss, but this is likely to be inadequate if there are material forecasting errors. It is essential that over 1-5 minute intervals there is enough reserve to cover the largest infeed loss, but this assumes that the full amount of demand is already covered – if the full amount of demand is not covered because of errors in forecasting either demand or generation, a larger reserve would be needed. To decide whether this is the case, these data need to be known and understood.

NESO should also publish error data on each component of its forecasts using the categories set out in OC 1.6.1 of the Grid Code. There’s a well-known truth that if something is not measured it is never fixed, and this is the case here. We know that NESO’s demand forecasts can be very wrong. We also know that on a tight day like 8 January these errors could prove to be the difference between the lights being on or off.

And since anecdotally (and logically) the causes of the forecasting errors are becoming more important to the system overall, over time these errors are likely to grow, so without measurement and transparency over the measurements, there will be no incentive to improve, which could prove disastrous as we move towards the Clean Power 2030 goals. And the Grid Code forecasting requirements should be updated to take account of the way the system is changing.

The SQSS should require NESO to secure the single largest infeed loss plus an amount which would cover forecasting errors. Of course, to be able to do this, the forecasting errors must be calculated, and for transparency reasons, published to the market. Not only do these errors lead to an elevated risk of blackouts in tight conditions, they could also be one of the drivers of the controversy around battery skip rates – the current under-use of batteries when there is excess generation on the network.

.

NESO has been quick to refute my claims that the system margin was extremely tight on 8 January, but it has not provided the data to support its position (other than the fact there was neither demand control or a blackout). However, if its demand forecast can be out by 4.7 GW over a half-hourly interval (on a day-ahead basis), what sort of errors is it making within-day? And what is the error over 1-minute intervals which are relevant for frequency control? How confident can we really be that NESO actually does hold sufficient reserves at all times if there is so much uncertainty over the actual system margin?

So I am calling for an audit of NESO’s margin and demand forecasting to determine the size of the day ahead and within day total error compared to the reserve holding policy for the past year.

Days like 8 January make the answers to these questions critical, otherwise blackout risks will only grow as the energy transition progresses, and the public will not be quick to forgive blackouts caused by modelling failures.