AI for WAR — Weather and Reliability
In the age of climate crisis, it’s essential to understand and prepare for the effects of extreme weather conditions. Frequent climate disasters are costing the world billions of dollars — according to the National Centers for Environmental Information (NCEI), the U.S. alone was impacted by 18 weather and climate disaster events with losses exceeding $1 billion each in 2022.
As the weather continues to increase in volatility, having the knowledge and tools in place to mitigate hazardous conditions will be crucial to people, businesses, and governments across the world. Luckily, there have been radical advancements in technology that are allowing us to pave the way toward better weather insights for business operations, most recently with generative AI.
What is Weather AI?
High-impact weather events, such as severe thunderstorms, tornadoes, and hurricanes, cause significant disruptions to infrastructure, property loss, and even fatalities. High-impact events can also positively impact society, such as the impact on savings through renewable energy. Prediction of these events has improved substantially with greater observational capabilities, increased computing power, and better model physics, but there is still significant room for improvement. Artificial intelligence (AI) and data science technologies, specifically machine learning and data mining, bridge the gap between numerical model prediction and real-time guidance by improving accuracy. Weather AI refers to the use of artificial intelligence technology to improve weather forecasting, prediction, and output. With the ability to synthesize millions of data points into concise insights, it offers a unique edge in the ever-changing world of weather by providing a simple daily summary of weather events and expected impact enabling them to make more informed operations decisions. Organizations can use these daily summaries to optimize operations, improve safety and efficiency, create new opportunities, and even protect communities from extreme weather events. AI techniques also extract otherwise unavailable information from forecast models by fusing model output with observations to provide additional decision support for forecasters and users. In this work, we demonstrate that applying AI techniques along with a physical understanding of the environment can significantly improve the prediction skill for multiple types of high-impact weather.
Decision Tree for Weather Predictions
Decision-tree-based methods are popular in data science for handling big data. They are able to identify and learn with only the most relevant variables, enabling users to provide many possible predictive features without worrying whether extraneous variables will overwhelm the training process. Decision trees are also human readable, which can provide insight into what relationships the model has identified related to the event being forecasted. Decision-tree-based methods have proven quite powerful in a wide variety of weather applications.
Subjective (human derived) decision trees have been used in meteorology since at least the mid-1960s. A decision tree splits data recursively by identifying the most relevant question at each level of the data.
This tree is developed to predict whether hail will occur. At the root node, the data are split with the question “Is the mean radar reflectivity ≤ 43.4 dBZ?” The data are further refined down each of the yes and no branches until a prediction is made at a leaf node, which may contain a class label (e.g., hail: yes), probability p [e.g., p(hail) = 0.8; scalar prediction [hail size = 3.1 in. (∼7.9 cm)], or a linear predictive function. To develop a decision support system for the effective integration of variable generations, we have leveraged proven forecasting methodologies for each temporal, as well as spatial, scale. Disparate sources of data, including power generation data, as well as local and regional weather observations, are combined using artificial intelligence methods with the information about physics and dynamics of the atmosphere to predict power output
Numerical Weather Prediction Models
Numerical Weather Prediction (NWP) data are the most familiar form of weather model data. NWP computer models process current weather observations to forecast future weather. Output is based on current weather observations, which are assimilated into the model’s framework and used to produce predictions for temperature, precipitation, and hundreds of other meteorological elements from the oceans to the top of the atmosphere. Many physical processes in the atmosphere or at the surface like the formation of clouds or the interaction between solar radiation and cloud droplets take place on very small spatial scales which cannot be resolved explicitly by the NWP models. The impact of these unresolved processes on the model variables has to be included approximately via so-called parameterization schemes.
To solve the complex set of model equations on computers different numerical methods can be employed. In grid point models the temporal evolution of the model variables are calculated in a three-dimensional spatial grid which covers the atmosphere from the surface up to a given model top, e.g. at 75 km above the ground.
Finite Difference Equations (FDEs)
The analytical solution for the aforementioned set of equations must be solved using the discrete form with numerical methods. Therefore, finite difference equations (FDEs) can be used to find approximate solutions of the PDEs.
∂u/∂t + c ∂u/∂x = 0
we take discrete values for x and t: xj = jΔx and tn = nΔt, where Δxis the grid space and Δt is the time step of integration. The solution of the FDE is defined at the discrete points (xj, tn) = ( jΔx, nΔt):
Since we employ an FDE to approximate a PDE, two fundamental conditions should be satisfied:
(i) The FDE should be consistent with the PDE.
(ii) For a given time t > 0, the solution of the FDE should converge to that of the PDE as Δx ! 0 and Δt ! 0.
Grid Staggering Methods
Once the continuous PDEs are discrete in the grid mesh, all model variables are defined in the grids. Even in spectral models, since the transformations of spectral space to grids and from grids to spectral space are necessary and commonly used, model variables are defined in the grid space to some extent. The arrangement of model variables on different grid points becomes one of the considerations when designing numerical schemes for an NWP model. Instead of arranging all variables at the same grid point, many numerical models adopt a staggered grid approach. The staggered grid combines several types of nodal points located in different geometrical positions and looks rather complex. However, the staggered grid allows for a natural and more accurate formulation of several crucial PDEs with finite differences; thus it is widely used in numerical models.
A very important characteristic of the model grid is the grid spacing, i.e. the horizontal distance of neighbouring grid points. The smaller the grid spacing, the more detailed atmospheric structures can be resolved by the numerical prediction model.
In operational applications of NWP models we distinguish between deterministic and probabilistic prediction. For the deterministic prediction the NWP model calculates the future weather based on a single initial state (called analysis). For a probabilistic prediction, also called ensemble prediction system (EPS), an ensemble of forecast runs is performed simultaneously based on several, slightly different initial states and model characteristics. The ensemble of future weather states allows predicting different possible developments, e.g. different tracks of storm systems, and quantifying the current uncertainty of the forecast. Based on EPS products the probability for severe weather events like heavy precipitation or gale force winds can be estimated well in advance.
The Aspect of Renewable energy
Forecasting for renewable energy resources is another example of high-impact weather forecasts. In this case, forecasting enables using clean, locally available, but highly variable renewable resources to produce energy in place of fossil fuel energy sources. Because the wind, water, and solar resources are highly variable, forecasting is needed to blend renewable power with other energy sources to assure reliable, efficient, and economic deployment. One of the promising practices in scheduling smart grids is to forecast the energy production of the resources that lead to energy and cost efficient replacement of current methods. However, deterministic forecast methods are not suitable to be applied to smart grids with renewable energy resources since they depend on unique sets of inputs and outputs. Therefore, stochastic forecast models are favored for this purposeUtilities require forecasts on various scales. Here, we describe two shorter-range scales: the nowcast, for the next 3–6 h, and the day-ahead forecast (which can extend to 72 h to cover weekends). The nowcast is necessary to blend renewable energy into the grid in order to meet the electric load in real time. The day-ahead forecast is used for planning unit allocation and trading energy with other utilities.
Solar models typically predict the clearness index, the ratio of the global horizontal irradiance (GHI) that reaches the surface of Earth to that at the top of the atmosphere. The clearness index ranges between 0 and 1 and depicts the depletion of solar energy via absorption and scattering by clouds and aerosols on its path through the atmosphere. It also removes the effects of the seasonal cycles and partially accounts for diurnal effects. One can explicitly compute the GHI at the top of the atmosphere given the solar angle and location information.
Some recent work has sought to identify regimes and forecast solar irradiance changes specific to those regimes through both implicit and explicit methods. The implicit method employs a regression tree approach with an embedded nearest neighbor scheme to forecast both deterministic irradiance and its variability. Explicit regime identification using k-means clustering and training ANNs for each cluster was shown to improve over training a single ANN on the entire training dataset. These approaches to statistical forecasting outperformed a “smart persistence” approach that includes the change in solar angle. For true decision support, utilities and grid operators do not want only wind speed or GHI forecasts; they actually require power predictions. Although manufacturers of wind turbines and solar panels provide average power curves, these are not perfectly representative of actual power produced at a site because of variation in terrain elevation, turbulence, and other factors. Thus, training an AI method to convert from wind or GHI to power can produce better power predictions for a specific site and does not require the detailed metadata needed to apply alternative methods for solar irradiance. The National Center for Atmospheric Research (NCAR) has successfully applied the cubist regression tree approach to both wind and solar.
Neural Networks
In eletric systems, the NNs have been applied to deal with various problems such as load forecasting, component and system fault diagnosis, security assessment, unit commitment, etc. In solar PV power forecasting applications, the main function of NN is to predict PV power for the next half-hour, hour, or day(s). In general, the main variables that drive the solar PV are solar radiation and temperature. The most commonly used NN to alleviate forecasting problems is BPNN. The BPNN consists of fully interconnected layers of processing units. The RBFNN is more effective when compared to BPNN, as the RBFNN takes less computation time for learning and shows more satisfied performance.
Finally, many utilities request probabilistic predictions to estimate the forecast uncertainty and to plan their reserve requirements. Although NWP model ensembles traditionally provide probabilistic forecasts, the analog ensemble approach has successfully produced probabilistic forecasts based on a single high-quality forecast from a consistent prediction system. The AnEn searches through historical forecasts for those most similar to the current forecast. Observations associated with each historical forecast form a probability density function that defines the forecast uncertainty. Application of modern AI techniques to high-impact weather forecasting is improving our ability to sift through the deluge of big data to extract insights and accurate, timely guidance for human weather forecasters and decision-makers. AI techniques build on traditional methods, such as MOS, by providing more flexible and powerful models capable of identifying complex relationships between a huge number of modeled and observed weather features or derived quantities. In addition, AI methods extend easily to directly predicting impacts of high-impact weather, such as power generated by variable sources such as solar or wind, energy consumption in an area, or airport arrival capacity.
