Achievements in atmospheric sciences by the large-ensemble and high-resolution forecasting studies using the supercomputer Fugaku

This article reviews the outcomes of a three-year project utilizing "Fugaku," Japan's flagship supercomputer, to conduct high-resolution ensemble simulations using atmosphere or atmosphere–ocean coupled models for both the Japan region and the entire globe. The project name was "Large Ensemble Atmospheric and Environmental Prediction for Disaster Prevention and Mitigation." The primary objective is to enhance the accuracy of numerical weather forecasting and provide probabilistic prediction information. To address the increasing severity of extreme weather events associated with global warming, such as torrential rainfall and tropical cyclones, high-resolution large-number ensemble atmospheric forecasting experiments have been conducted across timescales ranging from a few minutes to several weeks, extending to seasonal scales. This project aims to investigate advanced methodology using high-performance computing that provides probabilistic forecasts with sufficient lead time for effective disaster prevention and mitigation. Three sub-themes are explored: 1. meso-scale and regional modeling studies; 2. global and seasonal to sub-seasonal studies; and 3. innovative approaches to environmental studies. Central to this effort are high-resolution simulations that accurately represent cumulonimbus clouds and meso-scale systems, which are crucial for predicting severe weather phenomena alongside improved initial conditions derived from observational big data. These advancements are essential for predicting meteorological disasters caused by extreme events. Furthermore, the integration of probability information with improved accuracy significantly enhances disaster risk management, thereby increasing the practical utility of forecasts. This research also aims to develop pioneering innovative numerical weather and atmospheric environment forecasting technologies by incorporating big data from trace gas observations in addition to conventional meteorological data.

The need for disaster prevention and mitigation in the face of increasingly severe extreme weather events, such as torrential rainfall and tropical cyclones, has been growing in recent years. To address these challenges, we need to develop forecasting technology that provides longer lead time compared to the current operational system. The necessary approach involves high-resolution large-ensemble experiments that accurately represent critical weather phenomena, such as cumulonimbus clouds, which are responsible for extreme weather events. These experiments are supported by advanced observation data collected with unprecedented accuracy, frequency, and density due to recent advances in observation technology. This combination will facilitate the development of innovative numerical weather and environment forecasting systems, integrating probability forecast information to enhance the accuracy and reliability of predictions.

In this article, we review the outcomes of the research project "Large Ensemble Atmospheric and Environmental Prediction for Disaster Prevention and Mitigation" using the Japan's flagship supercomputer, Fugaku, during fiscal years 2020 to 2022 to evaluate near-future forecasting capability using high-resolution large-number ensemble experiments. We conduct numerical experiments over timescales ranging from a few minuets to several weeks, extending to seasonal scales, with the aim of establishing a new era of forecasting technology capable of providing probability forecasts with sufficient lead time to support better preparation and response to impending disasters. For disaster prevention and mitigation of extreme weather events, we expect more accurate forecasts could be achieved by employing large-ensemble and high-resolution atmospheric and environment forecasting simulations. Furthermore, detailed probability forecast information can be provided for severe events, such as torrential rains, which occur infrequently but with potentially significant impacts, offering a longer lead time than is currently provided by operational agencies. By conducting ensemble experiments with O(1000) which is 10 to 100 times more simulations than those used in the current operational standard, we aim to determine the optimal ensemble size for future practical applications.

To achieve these objectives, we have focused on three main themes: Theme 1: Short-term regional-scale forecasting, Theme 2: Extended-range to seasonal-term global-scale forecasting, and Theme 3: Advanced large-scale data assimilation. Collectively, these themes aim to significantly advance atmospheric and environmental forecasting methods, improving both the accuracy and utility of predictions for disaster prevention and mitigation. In the following sections, we will first provide an overview of the background and historical context of the present project, followed by representative outcomes on the above themes. Subsequently, we will describe the follow-on simulations using Fugaku and future perspectives and conclude with final remarks.

2.1 Research background

The project covered the fiscal years 2020 to 2022. This project was part of "the Program for Promoting Research on the Supercomputer Fugaku" (https://www.r-ccs.riken.jp/en/fugaku/org-relations/promoting-research/2020/), established by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) to achieve early outcomes using Fugaku, which began operations in FY2021. Our project was selected as one of the four focus areas, specifically Area 2, titled "Reinforcement of Efforts on Protecting People's Life and Property". Information about the project is archived at https://cesd.aori.u-tokyo.ac.jp/fugaku/.

This project aims to establish new disaster prevention and mitigation methods in response to extreme weather events, such as torrential rains and tropical cyclones, which frequently lead to significant weather-related disasters. By leveraging the capabilities of the supercomputer Fugaku, large-ensemble atmospheric prediction experiments are conducted on scales ranging from a few minutes to seasonal scales. These experiments provide high-precision probabilistic forecasts, allowing sufficient lead times before the occurrence of hazardous weather events. This next-generation forecasting technology with large-ensemble and high-resolution calculations is expected to make significant contributions to disaster prevention and mitigation by enabling timely and accurate responses.

The project involves high-resolution large-number ensemble simulations that accurately represent precipitation systems, such as cumulonimbus clouds and meso-scale systems. These simulations are combined with observational big data, made possible by recent advancements in observation technology, which have improved the accuracy, frequency, and density of data collection. By integrating these high-quality observations, the project aims to develop advanced forecasting technologies that deliver highly accurate numerical weather predictions supplemented by probabilistic information, thereby realizing an innovative approach to numerical weather and environment forecasting to protect society from weather-related disasters.

2.2 Structure of the research project

The project consists of three main themes and one cross-cutting activity (Fig. 1).

• Theme 1: Short-term regional-scale forecasting: Develop probability forecasts for severe weather events, such as heavy rainfall, up to several days in advance.

• Theme 2: Extended-range to seasonal-term global-scale forecasting: Realize probability forecasts for extreme weather events, such as typhoons, from one week to several months ahead.

• Theme 3: Advanced large-scale data assimilation: Develop a data assimilation (DA) system that enables high-resolution, large-member ensemble DA and its application to environmental research.

The DA system in Theme 3 supports all the themes as a cross-cutting activity utilizing the supercomputer Fugaku.

2.3 History of weather and climate simulation studies using the flagship computers in Japan

The supercomputer Fugaku began operation in 2021. Our project started before the public use of Fugaku under "the Program for Promoting Research on the Supercomputer Fugaku" covering fiscal years between 2020 and 2022. This program succeeded previous decade-long initiatives by MEXT aimed at enhancing high-performance computation technology, represented by the supercomputers K and Fugaku.

The weather and climate simulation studies were conducted using K and Fugaku. The previous supercomputer K was in operation between 2012 and 2019, and to maximize its utilization of K, the project SPIRE (Strategic Programs for Innovative Research) was conducted between 2011 and 2015 (https://aics.riken.jp/jp/science/spire.html; https://www.jamstec.go.jp/hpci-sp/ [in Japanese]). Satoh et al. (2017) reviewed the research outcomes of SPIRE focusing on global high-resolution non-hydrostatic atmospheric simulations using the K computer.

Following SPIRE, a new research initiative, "FLAGSHIP 2020 Project" (https://aics.riken.jp/fs2020p/en/), was conducted from 2015 to 2020 (FLAGSHIP stands for "Future LAtency core-based General-purpose Supercomputer with HIgh Productivity"). The FLAGSHIP2020 project aimed to prepare for the utilization of Fugaku after K and was referred to as the "post-K project". The present project succeeded one of Priority Issue 4, Disaster Prevention/Environment, of FLAGSHIP2020, with the title, "Advancement of meteorological and global environmental predictions utilizing observational Big Data" (https://www.jamstec.go.jp/pi4/en/). Saito et al. (2023) review the outcomes of forecast and numerical simulation studies on meso-/micro-scale high-impact weather under SPIRE and FLAGSHIP2020.

3.1 Overview

This section presents the outcome of Theme 1, which focuses on short-term regional-scale forecasting. Theme 1 aims to realize probabilistic predictions of severe weather events up to a few days in advance. To achieve this objective, we conducted large-number ensemble experiments and super-high-resolution simulations initialized through advanced data assimilation. A Japan-wide experiment was conducted using 1000 members to study typhoons and meso-convective phenomena (Sect. 3.2). Additionally, as another experiment showing the impact of the ensemble size, a real-time experiment was conducted to assimilate observations from a phased array weather radar (PAWR) every 30 s using also 1000 members (Sect. 3.3). A super-high-resolution simulation is described in Sect. 3.4.

In addition to these major accomplishments, several notable studies have also been conducted. Below, we provide some examples: Rezuanul Islam et al. (2023) proposed a novel assessment for storm surge multiscenarios based on the large-ensemble data; Usui et al. (2022) studied the local "Karakkaze" wind that resulted in an aircraft accident; Saito et al. (2022) clarified the effect of northward ageostrophic winds on the PRE enhancement in a tropical cyclone case; Kobayashi et al. (2020) analyzed the impact of the number of ensemble members on rainfall–runoff and inundation simulations; Hirano et al. (2022) investigated the structure of deep eye clouds in a tropical cyclone using T-PARCII dropsonde observations; Fujita et al. (2022) introduced observation errors correlated in time and space into a 4D-Var assimilation system; Oishi et al. (2023) established a local ensemble transform Kalman filter-based ocean research analysis; Sekiyama et al. (2023) developed a surrogate downscaling of meso-scale wind fields using convolutional neural networks; Sato et al. (2021) examined lightning frequencies using a numerical model coupled with an explicit bulk lightning model; Momoi et al. (2023) emulated a rainfall–runoff inundation model using deep neural network; Terasaki and Miyoshi (2020) investigated the cause of a quasi-linear meso-convective system using a 1024-member NICAM-LETKF.

3.2 Probability predictions of meso-convective systems and typhoons with a 1000-member ensemble data assimilation

3.2.1 Experimental set-up

Two data assimilation experiments involving a large number of ensemble members were conducted using Fugaku. While the number of ensemble members in operational forecast centers is typically around 100 or fewer, the use of Fugaku allowed us to significantly increase this number. We demonstrate the benefits of employing a large number of ensemble members in data assimilation. When the number of ensemble members is small, an ad-hoc technique called localization is typically applied to suppress sampling noise between distant points. In contrast, a large number of ensemble members allow for the partial removal of localization in ensemble data assimilation. We selected 1000 ensemble members for the two experiments to allow vertical localization to be turned off. This removal ensures coherent vertical structures in atmospheric fields, which are crucial for forecasting extreme events such as heavy rain and tropical cyclones.

Two extreme events were targeted: heavy rain in Kyushu in 2020 and typhoon Hagibis in 2019. Two state-of-the-art data assimilation methods, the four-dimensional local ensemble transform Kalman filter (LETKF; Hunt et al. 2007) and the four-dimensional ensemble-variational assimilation (4DEnVAR), have been applied to create better initial conditions for forecasting these events. Both methods were built around the former operational limited area model JMA-NHM (Saito et al. 2006; Japan Meteorological Agency Non-Hydrostatic Model 2019), resulting in the two data assimilation systems NHM-LETKF (Duc et al. 2021) and NHM-4DEnVAR (Rezuanul Islam et al. 2023). The same domain as the former operational domain of the Japan Meteorological Agency (JMA) was adopted. Analyses were given at 5 km grid spacing, while analysis perturbations were given at 15 km grid spacing to reduce computational costs in running assimilation. At 15 km grid spacing, the domain consists of 273 ×ばつ 221 horizontal grid points and 40 vertical levels. These values increase to 817 ×ばつ 661 horizontal grid points and 50 vertical levels at 5 km grid spacing. Boundary conditions for JMA-NHM were obtained from JMA's Global Spectrum Model (GSM), while boundary uncertainties were provided by JMA's one-week global ensemble prediction system, WEPS.

The two experiments were initiated four days before the target date at a 3-h assimilation cycle to avoid model spinup and initialization imbalances. All observations from the JMA database were assimilated into NHM-LETKF and NHM-4DEnVAR. Since the analysis ensemble was not yet available at the first cycle, analysis perturbations were identified with climatological perturbations, which were randomly generated from the climatological background error covariance. The 1000-member analysis ensemble was used as the initial conditions of extended forecasts using the same model, JMA-NHM, to examine the impact of data assimilation on forecast performance. The extended forecasts were run at 5 km grid spacing with the same domain for the Hagibis case. For the Kyushu heavy rain case, the extended forecast was run at a higher resolution of 2 km with a smaller domain (819 ×ばつ 715 horizontal grid points and 60 vertical levels) centered at Kyushu to save computational cost.

3.2.2 Heavy rain in Kyushu in 2020

"Impact-based forecasts and warnings" is a concept introduced by the World Meteorological Organization (WMO) that predicts not only extreme weather conditions but also the potential damage levels of disasters. We developed an ensemble flood forecast system that predicts the probability of flooding nationwide at three damage levels (Oizumi et al. 2025). The system utilizes the operational flood forecast model of JMA and a large-ensemble weather forecast. In this study, we investigated the impacts of using a large-ensemble size (1000) on the probability of flooding for the different sizes of river basins. We applied this system to an extreme flood event in the Kuma River basin in July 2020.

For large and middle-size rivers, the system predicted a 40–60% probability of exceeding historical floods 11 h before the event. In the small river, the system predicted a 50% probability of flood occurrence, which was influenced by complex mechanisms such as backwater effects. These results demonstrate that the system predicts floods in rivers of different sizes (Fig. 2).

3.2.3 Typhoon Hagibis in 2019

Probability forecasts are critical for addressing the uncertainty in weather forecasts caused by the chaotic nature of the atmosphere. Compared to rainfall forecasts, the probability forecasts of hazardous winds are less developed and are not as commonly used in weather forecasting. This study investigated the feasibility of using 1000-member ensembles with 1 km model grid spacing to produce flow-dependent probabilistic predictions of strong winds. Focusing on Typhoon Hagibis, we downscale the initial and boundary conditions from the 5 km ensemble (Wu et al. 2025) and perform ensemble simulations using JMA-NHM.

Compared with the 5 km ones, the 1 km ensemble simulations were able to capture a finer-scale distribution of strong wind over land, which was more consistent with observation (figure not shown). In terms of the probabilistic predictions, the 1 km ensemble generally captured the likelihood of strong wind occurrences (Fig. 3). Over coastal areas, the ensemble accurately captured the high possibility of strong winds, in agreement with observations. However, simulating high winds inland remains challenging, even with a 1 km resolution, as only a few ensemble members were able to capture the strong winds, resulting in a low probability inland. Conversely, some regions showed high predicted probabilities despite no strong winds being reported. The challenge of predicting strong winds inland may arise from model resolution limits, the parameterization of the boundary layer, and inadequate orography and land surface roughness representation. In particular, since the surface winds are diagnosed quantile, we found that roughness is a crucial component (figure not shown).

Overall, the results indicate the feasibility of producing wind probability forecasts using high-resolution ensemble forecasts. However, predicting strong winds inland remains challenging. To deliver more accurate and detailed weather forecasts for high-impact, strong wind events, future works could work on improving both the NWP model and ensemble’s performance. Higher model resolution without the need for boundary layer parameterization may mitigate the underestimates of strong winds by directly resolving turbulence. Regarding ensemble simulations, a suitable ensemble spread could help better capture extreme events. In particular, perturbing model parameterizations and land-use data may be beneficial, as land-related ensemble spread tends to be underestimated (Reichle et al. 2021).

3.3 A real-time assimilation every 30 s with a 1000-member ensemble

3.3.1 Introduction to the real-time assimilation method

Using Fugaku, we explored precise real-time prediction of individual convective clouds, which typically have a horizontal scale of less than several km and evolve rapidly within minutes (e.g., Fig. 1 of Miyoshi et al. 2016a). The PAWR (Yoshikawa et al. 2013; Ushio et al. 2015) can simultaneously observe approximately 100 elevation angles using electronic beam steering, and by rotating its azimuth, it can seamlessly scan the sky within a 60 km radius in 30 s.

Miyoshi et al. (2016a; 2016b) developed what is called the Big Data Assimilation (BDA) system, assimilating PAWR data every 30 s using LETKF (Miyoshi and Yamane 2007). Honda et al. (2022) developed the first real-time BDA system with a new MP-PAWR (Multi-Parameter PAWR; Takahashi et al. 2019), which produces single deterministic 30 min forecasts updated every 30 s at a 500 m horizontal grid spacing using 50 ensemble members. Based on the experience in the 2020 experiment, the real-time BDA system was updated and ported to Fugaku for an experiment conducted in the summer of 2021 during the Tokyo Olympic and Paralympic Games. Thanks to the computational power of Fugaku (Yashiro et al. 2020), we increased the problem size to 1000 ensemble members for LETKF and 10 ensemble members for the 30-min forecasts. Miyoshi et al. (2023) reported this real-time experiment at a renowned supercomputing conference "SC’23" as one of the three finalists of the first Gordon Bell Prize for Climate Modeling.

In this section, we review two studies with the BDA system using Fugaku. The next subsection presents a detailed analysis of the real-time experiment for a local heavy rain case during the Tokyo Olympics, while the following subsection presents observing system simulation experiments (OSSEs) to evaluate the potential impact of a network of 17 PAWRs for a larger-scale heavy rain case.

3.3.2 A real-time 30 s update BDA system for high-precision rainfall prediction

For the 2021 demonstration of the real-time BDA system during the Tokyo Olympic and Paralympic Games, we cycled the large ensemble every 30 s using observations from a MP-PAWR, with extended 30 min precipitation forecasts initialized from 10 members after each update. As part of the post-analysis, we examined the performance of 10 ensemble member forecasts for a convective system that passed over Tokyo in the early hours of July 29 and brought heavy rainfall to parts of Tokyo. The forecasts accurately predicted the steady northeastward propagation of the system, as well as rapid changes in the storm's structure and rainfall intensity. Figure 4 presents rain rates (mm h⁻¹) calculated from the ensemble-mean 10, 20, and 30 min forecasts initialized at 19:20 UTC on July 29, as well as rain rates calculated from MP-PAWR observations and nowcasts generated from an advection-based model (3DTREC [Three-Dimensional Tracking Radar Echo by Correlation], Otsuka et al. 2016). The forecasts show the convective system's northwestward propagation and intensification of precipitating cells in areas further to the south. At 30 min lead time (FT = 30), the location of the heaviest rainfall is in close agreement with the observations (19:50 UTC). In comparison, the nowcasting system shows a more rapid loss in forecast skill and the location of heaviest rainfall is in much less agreement with observations at each lead time, The side-by-side comparison of forecasts performed with the real-time system with nowcasts made with an advection-based model demonstrates the advantage of using a full NWP model with an integrated physics scheme to predict changes in the convective system structure and intensity. The rapid evolution of the convective system that occurred within a matter of a few minutes meant that the nowcasting model was unable to predict the development of individual convective cells that brought heavy, localized rainfall to parts of Tokyo.

Figure 5 shows the mean threat score calculated from 120 SCALE-LETKF forecasts and nowcasts up to 30 min based on thresholds of 15 and 30 dBZ. For both thresholds, the SCALE-LETKF is shown to outperform the nowcast system, with the gap between forecast skill increasing with lead time. The result is consistent with the similar studies that showed NWP models outperforming advection models at longer lead times and is understandable given the advection models inability to predict changes in the system.

We investigated the atmospheric variables of the ensemble-mean forecasts to better understand the environmental conditions that led to the growth in convective activity and intensification of rainfall in the ensemble-mean forecast. Member forecasts that showed stronger intensification exhibited higher total precipitable water and stronger low-level convergence at the initialization of the forecasts. The combination of high moisture availability and dynamical instability was undoubtedly an important factor for convective development to occur. The importance of the 30 s cycling was also evident from the analyses, which showed a buildup of these conditions through successive updating with the radar observations. As the storm quickly developed into a more intense system, this information was disseminated to each of the forecast members through the assimilation of both the reflectivity and Doppler wind velocity observations, leading to the prediction of heavier rainfall in the forecast.

While the propagation and location of the heaviest rainfall were well predicted by the model in the 19:20 UTC forecast, Fig. 5 shows notable deviations from the observations, including the absence of rainfall in areas further south of the system. The inability of the model to predict weaker rainfall is one of its limitations that are caused not exclusively by the buildup of model instability caused by repetitive updating with limited ensemble size and use of localization. Even with a 1000-member ensemble, inaccuracies within forecast errors introduce errors in the analyses at each update, which accumulate to produce large errors in atmospheric conditions in the model that can act to increase spurious convection and damp out weak convectively. Furthermore, the assimilation of reflectivity and wind data is not adequate to provide the necessary information on moisture distribution that would aid the model in making accurate predictions of the development of convective activity. Future development of the BDA system should include the assimilation of new datasets that contain information on moisture distribution to improve forecast skills.

3.3.3 Observing system simulation experiments of a rich phased array weather radar network covering Kyushu for the July 2020 heavy rainfall event

On July 4, 2020, southern Kumamoto (Fig. 6) encountered torrential rainfall associated with an extensive monsoon front known as the "Baiu front" (Hirockawa 2020; Duc et al. 2021; Taylor et al. 2021). Many JMA gauge stations in Kumamoto recorded over 400 mm of precipitation during the rainfall event. This rainfall event led to 65 fatalities due to catastrophic flooding and landslides (Government of Japan 2020). To improve the accuracy of torrential rainfall forecasts, we developed a revolutionary NWP system featuring a 30 s update LETKF that assimilates data from a single PAWR every 30 s (Miyoshi et al. 2016a, 2016b, 2023, 2020; Lien et al. 2017; Maejima et al. 2017, 2019; Taylor et al. 2021). Following previous studies, Maejima et al. (2022) investigated the potential impact of multiple PAWRs covering the entire Kyushu on the prediction of the July 2020 heavy rainfall event using 30 s update, 1 km mesh, 50-member OSSEs. This subsection provides a brief review of Maejima et al. (2022).

This study (Maejima et al. 2022) performed perfect-model, identical-twin observing system simulation experiments (OSSEs). First, the nature runs for the OSSEs are generated by the SCALE-RM, which was initialized using the JMA Meso-Scale Analysis (JMA MANAL; Japan Meteorological Agency 2019) at 18:00 JST (Japan Standard Time, UTC + 9) on July 3, 2020, at 1 km resolution. The nature run was performed for 16 h, including the torrential rainfall period from 04:00 to 10:00 JST on July 4. Synthetic observation data of 17 PAWRs are generated from the nature run. The locations of the PAWRs are shown as red dots in Fig. 6. The observing range of each PAWR was set to 60 km, based on an actual PAWR at Osaka University. The virtual PAWR network covered the entire Kyushu (orange shaded areas in Fig. 6). The radar reflectivity and the radial velocity were synthesized using the observation operator described by Amemiya et al. (2020). Following that, a series of OSSEs (NO-DA, 5MIN, and 30SEC) is performed. The initial conditions of OSSEs are completely blind to the nature run, and the OSSEs receive information of the nature run by assimilating noisy and imperfect observations. We verify the OSSEs by comparing with the nature run.

For the OSSEs, a 30 s update, 1 km mesh, 50-member SCALE-LETKF was performed using virtual PAWR data every 30 s from 02:30 to 04:00 JST, labeled as "30SEC". For reference, the same procedure as 30SEC was followed, but without PAWR data assimilation ("NO-DA"). Additionally, to clarify the impact of the rapid update LETKF, another experiment was conducted using every 5 min, 15-layer radar data, which mimics conventional radars are performed ("5MIN"). Finally, to evaluate the predictability of heavy rainfall, we conducted 6 h forecast experiments initialized by the ensemble-mean states of 30SEC, 5MIN, and NO-DA at 04:00 JST.

Figure 7 shows the side-by-side comparisons of rain mixing ratio (QR) [g kg⁻¹] at a 1.5 km altitude from 04:00 JST (analyses) to 07:00 JST (3 h forecasts) on July 4. In the nature run, a high QR area where QR > 1 g kg⁻¹, corresponding to torrential rainfall near the disaster site in southern Kumamoto, was maintained (Fig. 7d, black circles). NO-DA at 04:00 JST also shows a high QR area, but its location was shifted approximately 80 km southward compared to the nature run (Fig. 7a0, d0). In 5MIN, although the rain pattern around the disaster area was slightly improved compared to NO-DA at 04:00 JST (Fig. 7a0, b0, black circles), this advantage rapidly disappeared after 05:00 JST (1 h forecast) (Fig. 7a1–a4, b1–b4). In contrast, 30 SEC clearly captured the torrential rains from 04:00 JST (Fig. 7c0-1, d0-1). At 05:00 JST, the improvement due to PAWR data assimilation was still evident. However, the intense rain system gradually moves eastward after 06:00 JST (2-h forecast) (Fig. 7c2, d2), likely due to the prevailing westerly wind from the western boundary gradually dominating over Kyushu. Eventually, the positive impact of PAWR data assimilation faded, and the overall rain patterns became similar to those of NO-DA (Fig. 7a3, c3, d3).

This series of OSSEs demonstrated the positive impact of rich PAWR data on the rainfall forecast in the July 2020 rainfall event. 30SEC had a significant advantage over 5MIN and NO-DA in both analyses and forecasts. Assimilating synthetic PAWR data every 30 s contributed to better forecasts of torrential rainfall, particularly up to a 1 h lead time, whereas the improvement gradually decayed by about a 3 h lead time.

3.4 Clear air turbulence resolved by numerical weather prediction model validated by onboard and virtual flight data

Yoshimura et al. (2023) investigated Clear Air Turbulence (CAT) using a super-high-resolution numerical weather simulation. Aviation turbulence refers to atmospheric turbulence that occurs at any altitude due to various causes, such as jets, internal gravity waves, cumulus convection, fronts, and surface structures (Kim and Chun 2010; Sharman et al. 2012; Trier et al. 2012, 2020; Bramberger et al. 2018; Bramberger 2019; Misaka et al. 2023). CAT, a type of turbulence occurring outside convective clouds (Federal Aviation Administration 2016), is difficult to detect using onboard radar and is often encountered by aircraft without any visual signs. Occasionally, even large passenger aircrafts experience turbulence with amplitudes exceeding 1G, posing an invisible threat to the aviation industry (Japan Transport Safety Board 2016). Many CAT events are caused by Kelvin–Helmholtz instability (KHI) induced in a stratified wind shear when the Richardson number is smaller than 0.25 (Drazin and Reid 1981). Instability waves from KHI break into turbulence, cascading down to smaller eddies that ultimately reach the scale of aircraft, which can be dangerous (Joseph et al. 2004; Sekioka 1970).

Large eddy simulation (LES) is an effective approach for investigating the behavior of aircraft-scale eddies, which are typically parameterized in relatively coarse (Δx > 1 km) numerical weather prediction models. The past LES studies (Lane et al. 2003; Kudo 2013; Lane and Sharman 2014; Zovko-Rajak and Lane 2014; Yoshimura et al. 2022) successfully reproduced turbulence eddies with large wind fluctuations at aircraft cruising altitudes. However, there have been few observation campaigns for CAT in the free atmosphere, and few studies have validated LES results for high-altitude turbulence using observation data. In this study, we attempted to reproduce a real CAT event by running a 35 m resolution LES on the supercomputer Fugaku with ASUCA operational non-hydrostatic model (Ishida et al. 2022). We also validated the LES results using the acceleration histories recorded by passenger aircraft flying in the affected airspace.

The flight data indicated that the 35 m LES successfully reproduced turbulent eddies with frequencies up to the natural frequency of aircraft motion (0.1–0.2 Hz). We implemented a flight simulation that approximates the motion of B787, the same aircraft model that recorded the flight data, and simulated the aircraft flying through turbulence generated by the LES (Fig. 8 shows the simulated aircraft responses to two large wind changes). Figure 9 shows comparison of the frequency spectra of the vertical acceleration between the flight simulation results and the flight data and revealed that: 1) The flight data exhibited larger amplitudes at frequencies above 1 Hz, while the simulation exhibited smaller amplitudes, which is due to the insufficient grid resolution of the LES to reproduce eddies to shake aircraft at larger amplitudes and frequencies greater than 1 Hz, and 2) the simulation and the flight data correlated well in the lower-frequency region (0.1–0.2 Hz) that includes the natural frequency of the vertical motion of B787-class aircraft (see Sect. 3.2 of Yoshimura et al. 2023 for detail). In future work, we plan to accumulate knowledge on CAT generation based on super-high-resolution simulations (dx ~ 5 m) leading to the development of new forecasting methods.

3.5 Summary and discussion of Theme 1

Theme 1 presents the first effort in the world to use a large ensemble of 1000 members to conduct probabilistic prediction of extreme events, such as quasi-linear meso-convective systems and typhoons, which cause extensive damage (Duc et al. 2021; Rezuanul Islam et al. 2023; Wu et al. 2025), and Duc et al. (2021) discussed how the 1000 ensemble was superior to a smaller ensemble with 100 members. Furthermore, by integrating the Japan Meteorological Agency's (JMA) flood model with a high-resolution meteorological model, we first achieved the world's first probabilistic prediction of flood timing and magnitude (Oizumi et al. 2025). A real-time demonstration experiment was conducted using the regional meteorological forecasting system SCALE-LETKF, which assimilated observations from PAWR every 30 s with 1000 members to predict rainfall up to 30 min ahead (Amemiya et al. 2020; Miyoshi et al. 2023; Maejima et al. 2022; Taylor et al. 2024).

Additionally, we conducted an LES to investigate a flight incident with a flight simulator, making the first study to verify the simulation with actual observational data (Yoshimura et al. 2022, 2023). Finally, we achieved significant advancements in data assimilation (e.g., Fujita et al. 2022; Terasaki and Miyoshi 2020; Ohishi et al. 2023), large-ensemble (e.g., Kobayashi et al. 2020), super-high-resolution modeling (e.g., Sato et al. 2021; Usui et al. 2022; Saito et al. 2022; Hirano et al. 2022), and machine learning (Sekiyama et al. 2023) for severe meso-convective systems, contributing to efforts to mitigate the damages caused by these phenomena.

4.1 Overview

Typhoons and the resulting wind- and water-related disasters, which are among the most prominent meteorological hazards in Japan, are key targets of Theme 2 of this project. Accurate simulation of typhoons requires grid spacings at the order of kilometers. However, the formation and path of typhoons are influenced by various factors, including global-scale phenomena such as El Niño (e.g., Chan 1985; Camargo and Sobel 2005), monsoons (e.g., Gray 1968, 1998; Zehr 1992), tropical cloud activities including the Madden–Julian Oscillation (e.g., Liebmann et al. 1994; Sobel and Maloney 2000) and Easterly waves (e.g., Ritchie and Holland 1999; Yoshida and Ishikawa 2013), and the jet stream in the mid-latitude (Enomoto 2004). Given the limited computational resources, there is a dilemma between prioritizing higher resolution (finer grid) and broader computational domains to capture the influence of distant factors. Since a typhoon takes time to approach Japan from a long distance, the former approach is more suitable for short-term forecasts, while the latter is better for longer-term forecasts (more than a few days). For example, the former approach can provide highly accurate predictions of typhoons already present within the computational domain but cannot detect those that form outside the domain and approach later. Thus, global-scale forecasts aim to improve the accuracy of medium- to long-term predictions, typically ranging from one week to several months. Producing useful probabilistic forecasts for a sporadic phenomenon like typhoons requires a larger number of ensemble members. Generally, the spread between ensemble members grows over time. With a small number of members, the distribution becomes too sparse for forecasts beyond one week, even with the use of a cloud-resolving configuration, resulting in an unreliable probability distribution. Here we shed light on forecasts beyond one week by increasing the number of ensemble size, with a 14 km mesh configuration which is relatively cheap in terms of computational costs but still reasonably good at representing typhoons and convective cloud clusters. We utilize the immense computational power of Fugaku to generate a large ensemble of 1600 members and evaluate its probability distribution. Furthermore, at such a timescale, predicting sea surface temperature (SST) changes becomes increasingly important. The necessity for a fully coupled ocean–atmosphere numerical model increases compared to shorter predictions of several days. However, due to the additional degrees of freedom introduced by the ocean component, ocean-coupled numerical models are known to drift away from reality toward their quasi-balanced climatological state. The drift can cause local discrepancy of SSTs that fuel typhoons and also distort large-scale features (e.g., Wills et al. 2022) that affect typhoon development. We introduced a flux adjustment method (Masunaga et al. 2023) to control the SSTs to remain close to reality while allowing the ocean to interact with the atmosphere.

In this project, Yamada et al. (2023) have conducted 1600-member ensemble simulations targeting Typhoon Faxai (TY1915). Details of the outcome by Yamada et al. (2023) are introduced in Sect. 4.2. We aim to identify probable scenarios or less likely scenarios that could cause significant damage if realized, using the probability distributions from early predictions of typhoon tracks. These scenarios will be further refined through hierarchical ensemble predictions. Since operational forecasts by JMA currently provide typhoon forecasts only up to five days in advance, this research presents a challenging goal. However, if successful, it could lead to the provision of valuable early warning information on localized phenomena such as heavy rainfall and flooding caused by typhoons.

4.2 Large-ensemble simulations of Typhoon Faxai in 2019

A long lead time forecast will facilitate the mitigation of disasters associated with typhoons. To archive the long lead time forecast of typhoons, it is necessary to accurately forecast not only typhoon formation but also the typhoon's track. We demonstrated that typhoon formation could be predicted two weeks in advance using a 14 km mesh Non-hydrostatic Icosahedral Atmospheric Model (NICAM; Tomita and Satoh 2004; Satoh et al. 2008, 2014) on K computer (Nakano et al. 2015). However, the evaluation of typhoon tracks was not included in these studies. In this project, we probabilistically evaluated a typhoon track using a large-ensemble simulation with a 14 km mesh NICAM on Fugaku (Yamada et al. 2023).

The ensemble simulation targeted Typhoon Faxai in 2019, which caused severe damage in the Tokyo metropolitan area. NICAM-LETKF Japan Aerospace Exploration Agency (JAXA) Research Analysis (NEXRA) (Kotsuki et al. 2019) was utilized as the initial atmospheric conditions for the ensemble simulation. The 100-member ensemble simulation was initialized at 18UTC each day from August 20 to September 4, 2019 (16 days in total). The total number of ensemble simulations was 1600. In this subsection, lead time was defined with reference to 18UTC on September 8, 2019, when Faxai traversed Tokyo Bay. We evaluated whether each member successfully reproduced Faxai-like vortices based on the timing of the formation and the passage of Tokyo Bay (See Sect. 2 of Yamada et al. 2023).

The ratio of the number of members that were labeled as successful in reproducing a Faxai-like vortex including its route (within 500 km of the observed track), increased from a lead time of 15–12 days ("Type-AB" members shown with red tracks and bars in Fig. 10a, b). At lead times between 12 and 8 days, the ratio was on average more than 65%, largely exceeding that of the model climatology. The increase was due to two factors. First, the number of members that included the precursor of the Faxai-like vortex increased from a lead time of 15–12 days (Fig. 11). Most of the precursor vortices were embedded in one of the two northeast–southwest oriented lines of high-relative vorticity regions that propagated westward. The two lines were formed in the equatorial eastern Pacific, likely due to an inter-tropical convergence zone (ITCZ) breakdown event (e.g., Hack et al. 1989; Ferreira and Schubert 1997) that occurred in late August (not shown). Second, the interaction between the upper tropospheric vortex and the Faxai-like vortex was more accurately simulated as the lead time became shorter. As shown in Fig. 10b, 70% of the 100 members initialized at 12-day lead time (LT) successfully produced the type-AB vortex, whereas 60% of the 100 members initialized at 16-day LT produced the type-A vortex that takes erroneous routes. The erroneous route that occurred frequently was westward toward the East China Sea, slightly shifted southward compared to the westward portion of the observed route, and without largely changing its direction (Fig. 12). Yamada et al. (2023) discussed that an upper tropospheric trough tended to be better reproduced in the members with lead times shorter than two weeks, which had an effect to steer the vortex northward in the Pacific near 170°E, thereby making the vortex stronger interact with the sub-tropical high and advance along its rim toward Japan.

Yamada et al. (2023) highlighted the role of precursor vortices that originate in the tropical eastern Pacific and the effect of an upper tropospheric trough over the western Pacific. The former implies that typhoon forecasts will improve by using relatively high horizontal resolutions sufficient to maintain and propagate the small precursor vortices from the eastern tropical Pacific into the western Pacific, where they develop into typhoons.

4.3 Model development for S2S timescale

At the S2S (Subseasonal to Seasonal) timescale, it is essential to consider interactions between the ocean and the atmosphere to achieve more accurate predictions. Therefore, the use of coupled atmosphere–ocean models is required. By employing coupled models, it is possible to appropriately account for these interactions, which is expected to enhance predictability. However, an increase in the model's degrees of freedom may also result in larger biases in prediction outcomes, particularly when SST biases affect the atmospheric component. To mitigate such SST biases, Masunaga et al. (2023) introduced flux adjustment into the atmosphere–ocean coupled model NICOCO.

In typical flux adjustments, a free-running coupled model is run for several decades, and adjusted fluxes are estimated to bring the model's climate SST closer to reality. In this study, however, since the model is still drifting toward its climatology during the S2S timescale, a large number of S2S-scale experiments were conducted to estimate flux adjustment values corresponding to the forecast lead times. This study demonstrated that SST biases in coupled models can be significantly reduced (Fig. 13). They further show that the flux adjustment does not severely distort air–sea interaction process on a shorter timescale (Fig. 14). Lead–lag profiles are known to illustrate a causal relationship between atmospheric and ocean variability (e.g., Bishop et al. 2017; von Storch 2000).

5.1 Overview

In this section, we introduce the results of Theme 3, which focuses on advanced large-scale data assimilation. In Theme 3, we concentrated on developing a data assimilation (DA) system that enables high-resolution, large-member ensemble DA. The critical question was how large an ensemble DA calculation could be realized using Fugaku. To address this, we conducted a Grand Challenge experiment, performing DA with a 3.5 km global mesh and 1024 ensemble members (Yashiro et al. 2020). This calculation is more than 500 times larger than the ensemble DA currently operated by meteorological agencies worldwide. In addition to computation time, a major bottleneck was transferring 1.3 petabytes (PB) of simulation results from 1024 meteorological simulations to the DA system. After various program optimizations and speed enhancements, we successfully utilized approximately 130,000 CPUs on Fugaku to complete the experiment—the largest meteorological calculation in history—within the target time. As a result, Yashiro et al. (2020) were nominated as a finalist for the prestigious Gordon Bell Prize in Computational Science.

DA is used not only for weather forecasting but also for climate change and atmospheric environment prediction. For example, observations combined with a DA system can inversely estimate when, where, and how much atmospheric greenhouse gases (GHGs) and air pollutants such as PM2.5 were emitted or absorbed. The Paris Agreement, which came into force in 2016, requires countries to periodically report their GHG emissions and make efforts to reduce them. Inversion estimates are highly regarded as a tool for verifying and comparing reported emissions. Additionally, it is possible to improve the global water cycle and cloud/precipitation processes by utilizing simulations and observations of atmospheric water isotope ratios. In Sect. 5.3, we introduce a newly developed model to simulate water isotopes, Non-Hydrostatic ICosahedral Atmospheric Model Equipped with Stable Water Isotopes (NICAM-WISO), and the results of our analysis of its predictive performance.

5.2 Fugaku grand challenge experiment: global 3.5 km mesh, 1024-member ensemble DA

In the development project of Fugaku, target problems from each scientific field were established to evaluate the system's performance. These experiments were expected to utilize many nodes, close to the entire system, in what is termed "capability computing" or "capacity computing". For the weather and climate research fields, we selected a problem as complex as future weather forecasting operations. Specifically, this problem involved not merely a single forward calculation of a weather model with file I/O turned off but instead a DA cycle calculation with an ensemble DA system. Considering the number of MPI (Message Passing Interface) processes and total memory capacity available on Fugaku, we conducted a global 3.5 km mesh, 1024-member ensemble DA calculation using NICAM and LETKF as the Grand Challenge experiment (Yashiro et al. 2020). This calculation was the world's largest ensemble DA benchmark experiment. A total of 1.3 PB of data were exchanged between the simulation model and the DA system simultaneously. Therefore, it was necessary to improve the software in various computational science aspects.

To realize and optimize this huge-scale computational task, we applied the following innovative technique:

1) Data-centric design of the DA system

The bottleneck for the total elapsed time of a large-ensemble DA system originated not only from the computation but also from the file I/O and global communication. To maximize the I/O throughput, we used computational nodes to read and write data in parallel as much as possible. We used local solid-state drives (SSD) as storage for the intermediate files for DA. However, the physically distributed data introduced new challenges: Inter-node data exchange was required due to differences in the data sets needed by the simulation and DA programs. For example, to execute stencil calculations, the simulation program requires a group of horizontal grid points, whereas the computation for each ensemble member can be performed independently.

In contrast, the DA system requires all ensemble data at once for each grid point. Therefore, we redesigned the process discretization, data structure, and communication group in both programs based on the "data-centric" and "throughput-aware" design policy (Yashiro et al. 2016a). This approach reduces the number of nodes involved in group communication, the total size of the exchanged data, and the number of calls to the MPI library. Yashiro et al. (2016a) showed that the LETKF part of the program based on the new design was ×ばつ3 faster when run on the K computer with 40 MPI processes per member and 256 members.

2) Kernel-level optimization

The weather simulation component, which accounts for more than half of the elapsed time in the benchmark experiment, is a memory-bound program. It is well known that the computational performance of weather and climate models exhibits a "flat profile", meaning there are no hot spots—sections of code that consume a large portion of floating-point operations and computation time (e.g., Yashiro et al. 2016b; Lawrence et al. 2018). Array allocation/deallocation, initialization, and copying are time-consuming throughout the program code. Another characteristic is the frequent occurrence of conditional branches, which hinders parallel optimization. We improved arithmetic intensity by subdividing the entire code and acquiring and analyzing detailed profiles at the kernel/loop level, eliminating unnecessary zero-filling, and copying of intermediate arrays. We refer to this approach as the "household accounting method". These optimizations improved the performance of subroutines and user functions, which were approximately 1.1–5 times slower than the elapsed time predicted based on the amount of data transfer and calculation.

3) Effective usage of the single precision

Using variables with lower floating-point precision is an effective way to speed up memory-bound programs. This method is beneficial for machines, as modern computers often have insufficient memory performance. Using low-precision variables has several advantages, such as reducing data transfer times (in memory, communication, and file I/O), increasing cache retention time, increasing the number of simultaneous operations (such as single instruction, multiple data [SIMD]), and improving data compression efficiency. However, it also has negative effects that can degrade simulation accuracy. To effectively utilize low-precision variables and calculations, we switched the precision from double to single while carefully monitoring changes in calculation results at the kernel level. We also evaluated the time evolution of calculation errors using idealized experiments (Nakano et al. 2018). The growth in the l₂ difference norm over 11 days in the idealized simulation results between double and mixed precision calculation was comparable to those between multiple models in the original experiment (Jablonowski and Williamson 2006), making it acceptable for practical use. By converting the main part of the source code to single precision, we achieved a speedup of roughly 1.6 times in simulation time.

4) Optimization of eigenvalue solver

LEKTF requires an eigensolver to find the eigenvalues and eigenvectors of real symmetric square matrices. The number of ensemble members determines the size of the target matrix, and the solver must run for each grid point. In HPC research, eigenvalue calculations are often focused on solving problems involving large matrices, with little attention given to solving small matrices as quickly as possible. Therefore, for this purpose, in collaboration with experts in numerical software libraries, we used a "Kevd" library (Kudo and Imamura 2019), which was developed for Fugaku. Kevd has also been optimized for single-precision operations. We extracted the calculation core of the LETKF part as a kernel and conducted a benchmark with varying numbers of ensemble members from 128 to 4096. Kevd outperformed Linear Algebra PACKage (LAPACK) optimized for Fugaku for all cases of ensemble members. In particular, in the case of 1024 members, Kevd was more than 10 times faster than EISPACK, which has only general optimizations, and more than twice as fast as Fugaku-optimized LAPACK.

5.3 Water isotope simulation for data assimilation

5.3.1 Background

Stable water isotopes (e.g., H₂¹⁸O and ¹H²H¹⁶O) are helpful for investigating the atmospheric hydrological cycle (Araguás-Araguás et al. 1998). Isotopic ratios of water (H₂¹⁸O/H₂¹⁶O, and ¹H²H¹⁶O/H₂¹⁶O) are sensitive to phase changes in water and record their history from the vapor source region to the vapor sink region, resulting in large spatiotemporal variations. However, interpretations of isotopic variability are complicated because many factors affect the water isotopic ratios (e.g., raindrop evaporation, isotopic exchange between falling raindrops and the surrounding vapor, shallow convective mixing, top-heaviness of the large-scale ascent, large-scale convective organization, precipitation recycling at the land surface, and vapor source regions).

The interpretation of complex variations in precipitation isotopic ratios can be enhanced using atmospheric general circulation models (GCMs) that incorporate water isotopes (e.g., Joussaume et al. 1984; Hoffmann et al. 1998; Yoshimura et al. 2008; Risi et al. 2010). However, these GCMs struggle to simulate the complex structures of meso-scale convective systems due to their coarse horizontal resolution. In contrast, global cloud-system-resolving models (GCSRMs), such as NICAM, have a greater capacity to simulate precipitation isotopic ratios more accurately. Nevertheless, previous studies have not conducted a three-dimensional global simulation using a GCSRM equipped with water isotope processes. The development of an isotope-incorporated NICAM would enable a more realistic simulation of the atmospheric hydrological cycle and isotopic ratios. Tanoue et al. (2023) developed NICAM-WISO, which is a GCSRM equipped with water isotope processes, applied it to a current climate simulation, and validated the simulated precipitation isotopic ratios using station data.

5.3.2 Experimental design

NICAM-WISO was developed by adding the variables of stable isotopes of water (i.e., the isotopic specific humidity of water vapor, cloud water, rainwater, cloud ice, snow, and graupel) into the prognostic variables of NICAM. We used single-moment bulk cloud microphysics. The relative tendency is the same between the normal and isotopic specific humidity. However, specific isotope formulations are required during evaporation, condensation, and deposition, as stable isotopes of water are sensitive to these processes.

We conducted a water isotope climate simulation (grid division level 09 [GL09]) for the period from January 1, 1979, to December 31, 1982, with a horizontal resolution of GL09 (approximately 14 km) and 78 vertical layers. To evaluate the dependency of the simulated precipitation isotopic ratios on horizontal resolution, we also conducted coarse-resolution simulations (GL07 and GL05). These simulations were identical to GL09 but were performed at horizontal resolutions of 56 km (GL07) and 223 km (GL05), respectively. These simulations were run from 1979 to 1990, and we used the last 10 years of results. The GL05, GL07, and GL09 simulations were executed using 10, 640, and 2560 nodes of Fugaku, respectively. Simulations that included the water isotope process took approximately 2.3 times longer than those without it. Simulation results will be introduced in Sect. 5.3.3.

5.3.3 Results and future direction

The simulated hydrogen isotopic ratio (δ²H) of precipitation is shown in Fig. 15. NICAM-WISO captures the general features of geophysical isotope effects, specifically the latitude and altitude effects. The latitude effect represents the major spatial pattern of precipitation δ²H, with high values at low latitudes and low values at high latitudes. An altitude effect is observed in high mountain regions, including the Himalayas and the Rocky Mountains, where precipitation δ²H is lower than in the surrounding regions. The spatial correlation coefficients between simulated and observed precipitation δ²H are 0.92.

Both GL05 and GL07 capture the latitude and altitude effects. Their spatial patterns are similar to those of GL09. The spatial correlation is higher, and the root-mean-square error is lower for GL09 compared to GL05. The fine-horizontal-resolution simulation captures the climatological distribution of precipitation isotopic ratios more accurately. For example, the GL09 simulation shows a stronger spatial correlation with observed precipitation δ²H and reduces bias in the annual mean precipitation δ²H at high altitudes (Fig. 16). Since stable water isotopes indicate the intensity of the atmospheric hydrological cycle, the simulation with the fine horizontal resolution more appropriately captures such cycles in mountain and rough terrain regions. These results indicate that a fine-horizontal-resolution simulation can more accurately reproduce climatological isotope distributions, which is consistent with the finding of Werner et al. (2011). Therefore, a fine-horizontal-resolution simulation is superior for simulating the climatological distributions of the isotope hydrological cycle.

NICAM-WISO simulates water isotope circulation at various horizontal resolutions from a few to a few hundred kilometers. One future direction is to conduct isotope simulations at several km-scale resolutions. The km-scale simulations are expected to appropriately represent meteorological isotope effects (e.g., the amount effect, a negative relationship between precipitation amount and precipitation δ²H) and to simulate isotopic variations in the complex structures of meso-scale convective systems. However, NICAM-WISO underestimates precipitation δ²H in the tropical ocean region due to insufficient isotope exchange between falling raindrops and the surrounding vapor (Tanoue et al. 2023). The insufficient isotope exchange is attributed to a limitation of the current modeling framework: the parameterization of particle size distributions of precipitating hydrometeors. Multi-moment cloud microphysics, which treats additional components of prognostic water variables (e.g., particle numbers), allows for more detailed calculations of the equilibration time between raindrops and the surrounding vapor. Since NICAM has already implemented a double-moment cloud microphysics scheme (e.g., Seiki et al. 2015), this capability may improve the simulated isotopic fields under various climate conditions, thereby enhancing our understanding of interactions between moisture and tropical convection.

Through the three-year project introduced in this paper, we found that the supercomputer Fugaku is a prospective facility for advanced atmospheric and environmental numerical simulations, particularly through high-resolution and large-number ensemble experiments. Since the completion of the present project, Fugaku has been utilized for many ongoing activities and research. The following simulations have higher resolution than those previously conducted and have now become possible using Fugaku as a result of the numerical model developments in this project.

One practical use of Fugaku is the real-time regional model forecast by JMA using the 1-km mesh ASUCA in the summer season. To accelerate the development of numerical forecast technology and improve the forecast accuracy of quasi-linear meso-convective systems, JMA has been developing numerical forecast models using Fugaku with the full cooperation of MEXT and RIKEN. The real-time forecast simulations using Fugaku were first conducted in 2022 from June 1 to October 31 and continued during the same warm period in 2023 and 2024. For cases involving quasi-linear meso-convective systems, the MSM (Meso-Scale Model; 5 km grid interval, forecast for 78 h ahead) currently operated by JMA frequently underpredicts precipitation amounts compared to actual conditions. LFM (Local Forecast Model), which is currently operated by JMA (2 km grid interval), is being developed for Fugaku with a 1 km grid interval and a forecast period of 18 h. Although the LFM forecast exhibits positional errors in predicting quasi-linear meso-convective systems, it more often predicts stronger precipitation by utilizing the higher resolution of the 1 km mesh model.

Fugaku is also used for global high-resolution simulations with horizontal grid intervals of about a few kilometers or less. Global models with a mesh size of about 1 km are called a global km-scale model or a global storm-resolving model (GSRM) (Stevens et al. 2019; World Climate Research Programme 2022). The km-scale resolution refers to a broader resolution range with a mesh size of less than about 5 km. In our view, therefore, the global model with a mesh size of less than 5 km is referred to as GSRM. Takasuka et al. (2024) conducted a one-year simulation using NICAM with a horizontal mesh size of about 3.5 km. The world's highest resolution of global simulation by NICAM, with a horizontal mesh size of about 220 m, was also conducted (Satoh and Matsugishi 2023 [in Japanese]). GSRMs are considered promising for extreme weather forecasts. The world community is now coordinating efforts to enhance the use of GSRMs for climate service (Stevens et al. 2024). If a certain portion of Fugaku's computer resources were continuously used, it could be possible to run the 3.5 km mesh NICAM for one simulation year per one day. Achieving this computational performance would enable GSRMs to be used for more practical real-time forecasting and monitoring.

Regarding the subsequent flagship supercomputer of Fugaku, the next-generation computing infrastructure or the so-called Fugaku-next is planned to be operated around 2030 (https://github.com/cs-forum/roadmap-2023/releases/download/v1/roadmap.pdf [in Japanese]). The expected performance is approximately 5–10 times more efficient than that of Fugaku. The new machine will be expected to be more capable of supporting artificial intelligence (AI) for scientific research. As a follow-up study of the present project, for example, we expect a global 3.5 km mesh 1024-member ensemble DA calculation using NICAM-LETKF will be feasible for an extended period with a more frequent cycle—specifically, a two-month duration with a one-hour cycle.

We conducted the research project "Large Ensemble Atmospheric and Environmental Prediction for Disaster Prevention and Mitigation" with the goal of achieving more accurate weather forecasting through higher resolution and greater numbers of ensemble simulations. Prior to the public use of Fugaku in FY2021, we began using Fugaku and were impressed by its power. We have developed NICAM as a high-resolution "global storm-resolving model" that covers the entire globe with a mesh of several kilometers—several years earlier than the rest of the world (Tomita and Satoh 2004; Tomita et al. 2004; see also Satoh et al. 2019)—and are now able to conduct large-ensemble experiments of more than 1000 members using this model. We demonstrated that high-resolution, large-ensemble experiments can provide highly accurate probability forecasts of quasi-linear meso-convective systems half a day in advance, which had been difficult to reproduce with previous numerical forecast models. High-resolution, large-ensemble numerical weather forecasting is our future direction.

Until now, it has been necessary to allocate limited computer resources to either resolution or the number of ensembles, but with Fugaku, it is now possible to pursue both simultaneously. By using Fugaku and conducting this research project, we have been able to clearly outline the future of numerical weather prediction, which has not yet been realized for daily weather forecasts. Currently, powerful supercomputer resources on the scale of Fugaku are not dedicated to weather forecasting at all times. With the progression of global warming, torrential rains and typhoons that cause severe disasters are expected to occur more frequently in the future. Therefore, it is crucial to apply the numerical weather prediction technology demonstrated as feasible by Fugaku to actual daily weather forecasting as soon as possible.

This paper is a review of the existing literature. All data and material described in this paper can be referred to the original papers.

3DTREC:

Three-Dimensional Tracking Radar Echo by Correlation

4DEnVAR:

Four-dimensional ensemble-variational assimilation

AI:

Artificial intelligence

ASUCA:

ASUCA is a System based on a Unified Concept for Atmosphere

BDA:

Big data assimilation

CAT:

Clear air turbulence

EDA:

Early-stage Dvorak analysis data

ERA5:

European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis v5

GCM:

General circulation model

GCSRM:

Global cloud-system-resolving models

GHG:

Greenhouse gas

GL:

Grid division level

GSM:

Japan Meteorological Agency's Global Spectrum Model

GSRM:

Global storm-resolving model

ITCZ:

Intertropical convergence zone

JMA:

Japan Meteorological Agency

JMA MANAL:

JMA Meso-Scale Analysis

JMA-NHM:

Japan Meteorological Agency Non-Hydrostatic Model

JST:

Japan standard time

KHI:

Kelvin–Helmholtz instability

LAPAC:

Linear Algebra PACKage

LES:

Large eddy simulation

LETKF:

Local ensemble transform Kalman filter

LFM:

Local Forecast Model

LT:

Lead time

MEXT:

Ministry of Education, Culture, Sports, Science and Technology

MP-PAWR:

Multi-Parameter Phased Array Weather Radar

MPI:

Message passing interface

MSM:

Meso-Scale Model

NEXRA:

NICAM-local ensemble transform Kalman filter (LETKF) Japan Aerospace Exploration Agency (JAXA) Research Analysis

NICAM:

Non-hydrostatic Icosahedral Atmospheric Model

NICAM-LETKF:

Non-hydrostatic Icosahedral Atmospheric Model-local ensemble transform Kalman filter

NICAM-WISO:

Non-hydrostatic Icosahedral Atmospheric Model Equipped With Stable Water Isotopes

NWP:

Numerical weather prediction

OSSE:

Observing system simulation experiment

PAWR:

Phased array weather radar

PB:

petabytes

RSMCBT:

Regional Specialized Meteorological Center-Tokyo best track

S2S:

Subseasonal to Seasonal

SCALE-RM:

Scalable Computing for Advanced Library and Environment-Regional Model

SIMD:

Single instruction, multiple data

SSD:

Solid-state drive

SST:

Sea surface temperature

WEPS:

JMA's one-week global ensemble prediction system

WMO:

World Meteorological Organization

Figure 8 shows snapshots from 3-D visualization of the LES results created with the support from Cybernet Systems Co. Ltd. All the authors thank Ms. Makiko Shimada for her assistance in conducting the project.

MEXT, Japan, JPMXP1020200305, Masaki Satoh, RIKEN Center for Computational Science, hp200128, Masaki Satoh, hp210166, Masaki Satoh, hp220058, Masaki Satoh.

Authors and Affiliations

1. Atmosphere and Ocean Research Institute, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba, 277-8564, Japan

   Masaki Satoh, Tomoki Miyakawa & Takao Kawasaki

2. Typhoon Science and Technology Research Center, Yokohama National University, 79-5 Tokiwadai Hodogaya-Ku, Yokohama, Kanagawa, 240-8501, Japan

   Masaki Satoh & Masuo Nakano

3. Meteorological Research Institute, 1-1 Nagamine, Tsukuba, 305-0052, Japan

   Takuya Kawabata, Pin-Ying Wu, Tsutao Oizumi, Koji Terasaki & Masahiro Tanoue

4. Japan Agency for Marine-Earth Science and Technology, 3173-25 Showa-machi,Kanazawa-Ku, Yokohama, Kanagawa, 236-0001, Japan

   Masuo Nakano, Yohei Yamada & Ryusuke Masunaga

5. National Institute of Environmental Studies, 16-2 Onogawa, Tsukuba, Ibaraki, 305-8506, Japan

   Hisashi Yashiro

6. RIKEN Center for Computational Science, 7-1-26 Minatojima-Minami-Machi, Chuo-Ku, Kobe, Hyogo, 650-0047, Japan

   Takemasa Miyoshi, Yasumitsu Maejima & James Taylor

7. Department of Civil Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-Ku, Tokyo, 113-8656, Japan

   Le Duc

8. Postdoctoral Fellow By Japan Society for the Promotion of Science, 5-3-1 Kojimachi, Chiyoda-Ku, Tokyo, 102-0083, Japan

   Pin-Ying Wu

9. Graduate School of Maritime Sciences, Kobe University, 5-1-1 Fukae-Minami-Machi, Higashinada, Kobe, Hyogo, 658-0022, Japan

   Yasumitsu Maejima

10. Institute for Space-Earth Environmental Research, Nagoya University, 2-1-1 Katahira, Aoba-Ku, Sendai, Miyagi, 980-8577, Japan

    Ryoichi Yoshimura

11. Institute of Fluid Science, Tohoku University, 2-1-1 Katahira, Aoba-Ku, Sendai, Japan

    Ryoichi Yoshimura

12. Institute for Space and Earth Environmental Research, Nagoya University, Higashiyama Campus, Furo-Cho, Chikusa-Ku, Nagoya, Aichi, 464-8601, Japan

    Ryoichi Yoshimura

Contributions

Masaki Satoh was the principal investigator of the project "Large Ensemble Atmospheric and Environmental Prediction for Disaster Prevention and Mitigation" and the corresponding author of the present Manuscript by writing Sects. 1, 2, 6, and 7 and coordinating the whole sections. Takuya Kawabata, Tomoki Miyakawa, Masuo Nakano, Hisashi Yashiro, and Takemasa Miyoshi are sub-leaders of the project and are responsible for Sects. 3, 4, and 5. Le Duc, Pin-Ying Wu, Tsutao Oizumi, Koji Terasaki, Yasumitsu Maejima, James Taylor, and Ryoichi Yoshimura are members of Theme 1, writing Sects. 3.2, 3.3 and 3.4. Yohei Yamada, Ryusuke Masunaga and Takao Kawasaki are members of Theme 2, writing Sects. 4.2 and 4.3, respectively. Masahiro Tanoue is a member of Theme 3, writing Sect. 5.3.

Corresponding author

Correspondence to Masaki Satoh.

Competing interests

The authors declare that they have no competing interests.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions