Enhancing weather forecast accuracy through variational data assimilation and deep learning

By Karolina Stanislawska

The overall goal was to enhance Belgingur’s weather forecasting system by integrating it with a 3D/4D-variational data-assimilation module and to develop tools to disseminate forecasts, from a plethora of providers, in a unified manner as well as make the forecasts available to end-users through a novel fleet management system. We also used deep learning methods to develop a novel renewable energy production model and a tool to create multilingual forecasts using direct weather model output.

This was done via a close collaboration between Belgingur, Reykjavík UniversitySamsýn, and Tempo OK Ltd. in Brazil. The project was partially funded by the Icelandic Technology Development Fund through grant nr. 1910034-0611.

What is Data Assimilation
Data Assimilation is the technique by which observations are combined with output from a numerical weather prediction system (the first guess or background forecast) and their respective error statistics to provide an improved estimate (the analysis) of the atmospheric state. Variational Data Assimilation (3D/4D-VAR DA) achieves this through the iterative minimization of a prescribed cost function. Differences between the analysis and observations/first guess are damped according to their perceived error.
DA is used by governmental meteorological institutions, such as the National Oceanic and Atmospheric Administration in the USA, to initialize global forecast models. These global models provide the initial and boundary conditions for the regional models.

Fleet Management Systems
Fleet management- and vehicle/asset tracking systems (FMS) have been used for many years to monitor location and status of vehicles and personnel. With increased coverage of mobile data and lower costs, the trend has been to develop FMS towards being more situational aware. Our forecasting system is being integrated with the SiteWatch fleet management system, developed by Samsýn. Users of SiteWatch include nearly all public safety organizations in Iceland in addition to several organizations and companies in utilities, public transport and travel services.

Deep Learning
In recent years, deep learning techniques have advanced the state-of-the-art for many important tasks, including computer vision, natural-language processing, and building of predictive models. This success is in part a result of ever-increasing computing power and abundance of data, but also because of a better algorithmic understanding of how to build and train deep architectures. Over the last decade there have been many advances in computing-based natural-language processing (NLP). Today’s advanced systems for NLP use deep learning algorithms, in particular the so-called convolutional neural networks (CNNs) and recurrent neural networks (RNNs), which are a particularly good match as they can naturally exploit the sequential nature of speech and text processing. Such models, especially when augmented with memory-based strategies, attention mechanisms, reinforcement-learning (for unsupervised learning), and deep generative models represent the current state-of-the-art in NLP. In the last few years there has been increased interest in applying deep learning in the context of weather forecasts (e.g., IBM’s Deep Thunder). The ability of such methods to handle large amounts of data and learn complex non-linear models make them an appealing framework for learning to improve the complex high-resolution spatial-temporal models required for accurate weather forecasting. Further expanding on these ideas to create a renewable energy power production model is, to our best of knowledge, a completely new application of these techniques.

More articles

High resolution ML weather forecast for Iceland

In 2023 we started the adventure of building our own machine learning weather models. The goal of our first experiment was to create a high-resolution weather model for Iceland. As the backbone architecture we chose the ClimaX weather model proposed by Microsoft.
Read More →

Wind Resources in Iceland

The weather in Iceland 1994-2011 has been downscaled at a horizontal resolution of 3 km with the ARW-WRF atmospheric model with boundary data taken from the atmospheric analysis of the ECMWF.
Read More →

Enhancing weather forecast accuracy through variational data assimilation and deep learning

The overall goal was to enhance Belgingur’s weather forecasting system by integrating it with a 3D/4D-variational data-assimilation module and to develop tools to disseminate forecasts, from a plethora of providers, in a unified manner as well as make the forecasts available to end-users through a novel fleet management system. We ...
Read More →
Scroll to Top