Enhancing weather forecast accuracy through variational data assimilation and deep learning

By Ólafur Rögnvaldsson

This was done via a close collaboration between Belgingur, Reykjavík UniversitySamsýn, and Tempo OK Ltd. in Brazil. The project was partially funded by the Icelandic Technology Development Fund through grant nr. 1910034-0611.

What is Data Assimilation
Data Assimilation is the technique by which observations are combined with output from a numerical weather prediction system (the first guess or background forecast) and their respective error statistics to provide an improved estimate (the analysis) of the atmospheric state. Variational Data Assimilation (3D/4D-VAR DA) achieves this through the iterative minimization of a prescribed cost function. Differences between the analysis and observations/first guess are damped according to their perceived error.
DA is used by governmental meteorological institutions, such as the National Oceanic and Atmospheric Administration in the USA, to initialize global forecast models. These global models provide the initial and boundary conditions for the regional models.

Fleet Management Systems
Fleet management- and vehicle/asset tracking systems (FMS) have been used for many years to monitor location and status of vehicles and personnel. With increased coverage of mobile data and lower costs, the trend has been to develop FMS towards being more situational aware. Our forecasting system is being integrated with the SiteWatch fleet management system, developed by Samsýn. Users of SiteWatch include nearly all public safety organizations in Iceland in addition to several organizations and companies in utilities, public transport and travel services.

Deep Learning
In recent years, deep learning techniques have advanced the state-of-the-art for many important tasks, including computer vision, natural-language processing, and building of predictive models. This success is in part a result of ever-increasing computing power and abundance of data, but also because of a better algorithmic understanding of how to build and train deep architectures. Over the last decade there have been many advances in computing-based natural-language processing (NLP). Today’s advanced systems for NLP use deep learning algorithms, in particular the so-called convolutional neural networks (CNNs) and recurrent neural networks (RNNs), which are a particularly good match as they can naturally exploit the sequential nature of speech and text processing. Such models, especially when augmented with memory-based strategies, attention mechanisms, reinforcement-learning (for unsupervised learning), and deep generative models represent the current state-of-the-art in NLP. In the last few years there has been increased interest in applying deep learning in the context of weather forecasts (e.g., IBM’s Deep Thunder). The ability of such methods to handle large amounts of data and learn complex non-linear models make them an appealing framework for learning to improve the complex high-resolution spatial-temporal models required for accurate weather forecasting. Further expanding on these ideas to create a renewable energy power production model is, to our best of knowledge, a completely new application of these techniques.

More articles

Comparisons of observed and simulated weather

This study compares three reanalysis datasets – CARRA, Icebox, and RÁV2 – against observations of temperature, wind, precipitation, and radiation fluxes over a twenty-year period (1 September 1999 – 31 August 2019).
Read more →

GenCast – AI meets ensemble forecasting

New machine learning weather model from Google explained with cats and old TVs.
Recently released GenCast model has made its way into traditional media. While it's not the first successful AI model to emerge in the recent years, it stands out for its extraordinary accuracy and its innovative approach to ensemble ...
Read more →

High resolution ML weather forecast for Iceland

In 2023 we started the adventure of building our own machine learning weather models. The goal of our first experiment was to create a high-resolution weather model for Iceland. As the backbone architecture we chose the ClimaX weather model proposed by Microsoft.
Read more →
Scroll to Top