Basic Climate Analysis Using the CM SAF R Toolbox

Introduction

A map of the Sherry Triangle in southern Spain.

The Sherry Triangle, an area in the province of Cádiz in southwestern Spain that is known for its sherry wine production.

Spain is one of the world’s largest wine producers; grapes are an important commodity of the country’s agriculture industry.

Imagine you are a solar engineer consulting with a farmer about their vineyard in the Sherry Triangle, an area in southwestern Spain known for its sherry wine production. Due to the increased demand for sherry, farmers growing white grapes in the city of Jerez de la Frontera are interested in installing solar electric panels to help reduce electric costs for powering farm operations. You were sent to assess the value of installing solar panels in the area.

You understand how much land is needed to accommodate a sufficient number of solar panels to power the entire vineyard. However, the quantity of land needed is greater than the amount of land available on this particular farmer’s vineyard. In order to leverage what is available in the smaller plot of land, you must determine whether there is adequate solar radiation to power the solar panels during the grape growing season from June through September. You have analyzed a 30-year climate data record (CDR) from EUMETSAT's Satellite Application Facility on Climate Monitoring (CM SAF) of surface incoming direct radiation for the city.

Review the data plot and answer the questions in the carousel below. Click the arrows on the right and left side of the panel to navigate through the questions.

A box-and-whisker plot of the monthly mean surface incoming direct radiation for Jerez de la Frontera, Spain.


What other information can we learn from a CDR? Before we answer this question, let’s explore the various CDRs available and how we can obtain one from the CM SAF.

What is a Climate Data Record?

The CM SAF produces and archives satellite-derived climate data records (CDRs), a time series of measurements that are adequate in length, consistency, and continuity to determine climate variability. CDRs are used to support observations of the Earth system, such as studies of climate trends and variability and model verification to help improve climate models.

The CM SAF produces four types of CDRs: Environmental Data Records (EDRs), Interim Climate Data Records (ICDRs), Fundamental Climate Data Records (FCDRs), and Thematic Climate Data Records (TCDRs). Hover your cursor on the acronyms in the image below to learn more about each type of CDR.

A diagram of CM SAF’s satellite-based climate data records. Time-tagged, Earth-located geophysical parameters produced from the satellite sensor data using the latest algorithms. These records are most suitable for studying the conditions at fixed times. A regularly updated TCDR available in short-time latency with an algorithm and processing system as consistent as possible to the generation of the corresponding TCDR. Geophysical variables derived from the FCDRs.  An algorithm is applied to the FCDR to estimate the geophysical variable from the satellite observation. The production of a TCDR requires great time and computational resources, and it is usually updated every few years Re-calibrated and inter-calibrated long-term data records of satellite radiance information. The need for recalibration results from the changes in the sensitivity of a satellite sensor during its operational orbit time.  The need for inter-calibration results from technological advancements made in satellites and remote sensing sensitivity.

EDR - Time-tagged, Earth-located geophysical parameters produced from the satellite sensor data using the latest algorithms. These records are most suitable for studying the conditions at fixed times.

ICDR - A regularly updated TCDR available in short-time latency with an algorithm and processing system as consistent as possible to the generation of the corresponding TCDR.

TCDR - Geophysical variables derived from the FCDRs. An algorithm is applied to the FCDR to estimate the geophysical variable from the satellite observation. The production of a TCDR requires great time and computational resources, and it is usually updated every few years.

FCDR - Re-calibrated and inter-calibrated long-term data records of satellite radiance information. The need for recalibration results from the changes in the sensitivity of a satellite sensor during its operational orbit time. The need for inter-calibration results from technological advancements made in satellites and remote sensing sensitivity.

Question

For each task below, determine what type of climate data record (CDR) would work best to accomplish the task. Use the pull-down menu to choose the best answer.

a) Assimilate a long time series of radiance data into a climate model:
b) Study the climatological trend in latent heat flux:
c) Determine the amount of cloud fraction during the last month:
d) Investigate how much sunshine duration during the last month differs from the long-term mean:

The correct answers are highlighted above.

A FCDR is used by clients who want to independently retrieve a long time series of radiance information or who want to assimilate these data into their numerical weather prediction or climate models. FCDRs are well suited for assimilation into climate models because these data are calibrated and homogeneous in time.

A TCDR is the basis for all climate analysis or monitoring. As the TCDR holds the geophysical variables derived from a calibrated and homogeneous FCDR, it can be used to study climate trends.

In principle, an EDR should be the best product to use for geophysical parameters during a recent time period, particularly if the retrieval algorithm employs the full information of modern satellite instruments. However, EDRs are not well suited for anomaly analyses or other statistics that rely on consistent longer time series.

An ICDR may be used similarly to an EDR. However, ICDR and TCDR data can be used together for climate analysis. For example, you can create an anomaly by subtracting the long-term mean of a specific month (based on TCDR) from the recent data of that month (based on ICDR), such as October2019 minus mean(October1983to2015). Thus, the ICDR has an advantage over the EDR in that it uses the same retrieval algorithm as the corresponding TCDR and can be used for climate analysis.

Please make a selection.

The CM SAF provides CDRs for Essential Climate Variables (ECVs). ECVs are physical, chemical or biological variables (shown below) that play an important role in the Earth’s climate.

The primary variables covered by the CM SAF climate data records.

The primary variables covered by the CM SAF climate data records.

What is a Climate Data Record? » How Do I Get Data from the CM SAF?

Let's consider another scenario. Imagine you are working for the climate division in a National Meteorological and Hydrological Service (NMHS). Your main responsibility is to help gather and analyze climate information to support decision-making efforts. Where can you get climate data to help you provide decision-support services? The EUMETSAT's CM SAF is here to help!

To obtain a CDR from the CM SAF, follow the instructions below. You must click in the correct spot on the image in order to advance to the next step.

Now you know how to get a CDR, but how can you visualize these data records? Let’s review the software packages needed to visualize CDRs from the CM SAF.

What is a Climate Data Record? » What Do I Need to Visualize a CDR?

What are the steps you need to take to get a CDR from the CM SAF? Click and drag the steps below into the correct order.

  1. Submit order and wait for email that states order processing is complete
  2. Customize your CDR order if desired and add to your order cart
  3. Review email for instructions to download your CDR
  4. Sign into the CM SAF website
  5. Go to CM SAF - Product Navigator WUI and find your CDR
  6. Register an account on the CM SAF website

The CM SAF provides different tools to work with CDRs, one of which is the CM SAF R Toolbox.

The CM SAF R Toolbox can prepare, analyze, and visualize CDRs in NetCDF format using the R programming language. To install the necessary software for the Toolbox on your operating system, please follow the instructions in the carousel below. Click on the right/left arrows to navigate through the steps.



The CM SAF R Toolbox is now ready for use!

A screenshot of the CM SAF R Toolbox interface ready for use.

Now that you know where you can get climate data, and the tools you need to visualize these data, what kinds of plots can you make? In the next section, we will learn how to analyze and generate various data plots using the CM SAF R Toolbox.

Using Toolbox to Help Install Observatory

A picture of the Atacama Cosmology Telescope (ACT) in the Atacama Desert in Chile.

A picture of the Atacama Cosmology Telescope (ACT) in the Atacama Desert in Chile.

As a forecaster working in the climate division of an NMHS, you have been consulted by an astronomical society in South America interested in installing a high-altitude observatory site that would have unobstructed views of the horizon and less light pollution. In conversing with the astronomical society, you learned that cloud cover is an important factor in determining the quantity of time that is useful for an observatory. An average fractional cloud cover of 20% or less is the optimal condition needed for the installation project. You also learned that an ideal location for the site is at an elevation of at least 2000 m. At higher elevations, the Earth’s atmosphere is thinner and starlight appears less distorted in these conditions.

You want to generate a customized map that shows the mean fractional cloud cover across your domain. To begin your assessment, you want to download a CDR of monthly mean fractional cloud cover across South America based on the 34-year period from 1982 January 01 through 2015 December 01.

Let’s get the CM SAF R Toolbox up and running by opening RStudio and executing the following commands (shown in the image below):

library(cmsaf)
run_toolbox()

A screenshot of the RStudio console when starting the CM SAF R Toolbox.

The carousel below will walk you through a series of steps in the CM SAF R Toolbox to prepare, analyze, and visualize your CDR. Click in the correct spot on the image, or click on the right/left arrows to navigate through the steps.



Question

To supplement your assessment of the mean fractional cloud cover across your domain, you decide to compare that data with the topography of South America.

all time mean map s amer topo

Based on the data plots above, which location would best serve an astronomical observatory? Choose the best answer.

The correct answer is d.

Location D (near 26 S, 70 W) is in an area that has a mean fractional cloud cover of about 10%. Based on this analysis of the CDR, location D is below the minimum threshold requirements for project feasibility. Also, the topographic map shows that location D is at an elevation of around 2000 meters, which will make this location more enticing.

Please make a selection.

After describing the location you determined was most suitable for the astronomical observatory, the astronomical society is now interested in the cloud cover throughout the year at that location. It is important to build the observatory site in a place that experiences minimal cloud cover because a small percentage of annual meteor shower activity is visible to observers in the southern hemisphere. Let’s generate a graph that would give you a sense of the fractional cloud cover throughout the year at the specific location.

Using Toolbox to Help Install Observatory » Time Series

You want to generate a customized time series of the multi-year monthly mean fractional cloud cover based on the 34-year observation record. The carousel below will walk you through a series of steps to generate this time series. Click in the correct spot on the image, or click the right/left arrows to navigate through the steps.



Question

You have generated a customized graph that shows the multi-year monthly mean fractional cloud cover at the desired location for the high-altitude observatory.

A plot of multi-year monthly mean for fractional cloud cover data at a point in the CM SAF R Toolbox.

To maximize viewing conditions, the location should experience a monthly mean fractional cloud cover of no more than 20%. If the location has a history of not exceeding the monthly mean fractional cloud cover threshold for six months or more, the astronomical society will consider installing the high-altitude observatory. Review the graph above and answer the following questions. Use the pull-down menu to choose the best answer.

a) For how many months does the monthly mean fractional cloud cover not exceed the observatory’s threshold?
b) Will installing a high-altitude observatory at this location be ideal?

The correct answers are highlighted above.

In your assessment, you have determined that nine out of the twelve months at location D (near 26 S, 70 W) experienced a monthly mean fractional cloud cover of less than 20%. Based on this analysis, you have determined that this location is suitable for the installation of a high-altitude observatory.

Please make a selection.

Using Toolbox to Help Install Observatory » Analyze Time Series

The astronomical society has a few more questions before making their decision on whether to install the high-altitude observatory at location D (near 26 S, 70 W). The limited number of meteor showers that are best seen in the Southern Hemisphere occur during the winter/spring months of August, September, and October. The astronomical society is interested in the distribution of fractional cloud cover observations at that location during the winter/spring months to determine whether visibility would be optimal to observe the meteor showers during maximum activity.

Let’s generate a diagram that would give you a sense of the fractional cloud cover distribution throughout the year at the specific location. You want to generate a customized box-and-whisker diagram of the monthly mean fractional cloud cover based on the 34-year observation record. The carousel below will walk you through a series of steps to generate this diagram. You must click in the correct spot on the image in order to advance to the next step.



In this case, we will focus our attention to the box-and-whisker diagram.

Question

You have generated a customized box-and-whisker plot that shows the distribution of monthly mean fractional cloud cover at the desired location for the high-altitude observatory.

A box-and-whisker plot of the monthly mean fractional cloud cover for the location 26 S, 70 W.

To maximize viewing conditions during the winter/spring months of August, September, and October, the fractional cloud cover at location D should be less than 25%. If 75% of the historical observations are below this threshold for a majority of those months, the astronomical society will consider installing a high-altitude observatory. Review the plot above and answer the following questions. Use the pull-down menu to choose the best answer.

a) In how many of the winter/spring months are 75% of the observations below the observatory’s threshold?
b) Will installing a high-latitude observatory at this location be worthy for observation during the winter/spring months?

The correct answers are highlighted above.

At this location, 75% of the observations in August, September, and October never exceed an average monthly mean fractional cloud cover of 25%. Based on this analysis, you have determined that a high-altitude observatory at this location is likely to experience minimal cloud cover to support meteor shower observation periods during the winter and spring.

Please make a selection.

After discussing your findings about the fractional cloud cover in South America and sharing your thoughts on installing the high-altitude observatory, the astronomical society are optimistic about their plans. You wish them luck in their pursuit of the installation and offer your services for future inquiries.

Now that you are familiar with the procedures for visualizing a CDR in different forms using the CM SAF R Toolbox, you decide to leverage the tool in your case study of the 2003 heat wave across Europe.

Using Toolbox to Revisit Heat Wave Event

What is the purpose of each tool below? Use the pull-down menu to choose the best answer.

a) R software:
b) CM SAF Toolbox:
c) RStudio:

The correct answers are highlighted above.

To review the tools needed to visualize CDRs, please see the section “What Do I Need to Visualize a CDR?”

Please make a selection.

You are revisiting case study data from a 2003 heat wave across Europe to learn about the characteristics that made this heat wave such a historic event. The heat wave began in June and lasted through mid-August of 2003. High temperatures and no precipitation dominated for an extended period of time. You begin with Germany, which saw excessive mortality rates in the month of August as a result of the heat wave.

You will be using the CM SAF R Toolbox to analyze the heat wave event based on the monthly sum of sunshine duration during a 33-year data period from 1983 January 01 to 2015 December 31. Review and answer the series of six questions in the carousel below to gain insight about the 2003 heat wave in Germany. You must answer each question in order to view the next in the series.

Summary

In this lesson, we learned about the group of satellite-derived climate data records generated by EUMETSAT's Satellite Application Facility on Climate Monitoring (CM SAF): Environmental Data Records, Fundamental Climate Data Records, Thematic Climate Data Records, and Interim Climate Data Records. We reviewed the steps necessary to obtain a CDR using the CM SAF - Product Navigator Web User Interface, and gained an understanding of the software packages needed to visualize our data in the CM SAF R Toolbox: RStudio and the R Programming Language (with a couple of other packages for Mac OS X users).

A screenshot of information about the CM SAF R Toolbox on the EUMETSAT webpage.

To showcase CM SAF R Toolbox applications, the lesson provided an example of working with an astronomical society wanting to install an observatory in South America. The learner explored how to use the Toolbox to analyze a CDR of fractional cloud cover, interpreted the analysis, and shared the findings with the astronomical society. Finally, a quick overview of the 2003 European heat wave was provided to test the learner's understanding of using the CM SAF R Toolbox.

For more information on downloading and using the CM SAF R Toolbox, please see the following resources:

Contributors

COMET Sponsors

MetEd and the COMET® Program are a part of the University Corporation for Atmospheric Research's (UCAR's) Community Programs (UCP) and are sponsored by

To learn more about us, please visit the COMET website.

Project Contributors

Project Lead
  • Vanessa Vincente — UCAR/COMET
Instructional Design
  • Bryan Guarente — UCAR/COMET
  • Tony Mancus — UCAR/COMET
Science Advisors
  • Dr. Steffen Kothe — Deutscher Wetterdienst (DWD)
  • Dr. Christine Traeger Chatterjee — EUMETSAT
  • Dr. Jörg Trentmann — Deutscher Wetterdienst (DWD)
  • Bryan Guarente — UCAR/COMET
Program Oversight
  • Bryan Guarente — UCAR/COMET
Graphics/Animations
  • Steve Deyo — UCAR/COMET
Multimedia Authoring/Interface Design
  • Gary Pacheco — UCAR/COMET

COMET Staff, November 2019

Director's Office
  • Dr. Elizabeth Mulvihill Page, Director
  • Tim Alberta, Assistant Director Operations and IT
  • Dr. Paul Kucera, Assistant Director International Programs
  • Dr. Wendy Gram, Implementation Manager
Business Administration
  • Lorrie Alberta, Administrator
  • Auliya McCauley-Hartner, Administrative Assistant
  • Tara Torres, Program Coordinator
IT Services
  • Bob Bubon, Systems Administrator
  • Joshua Hepp, Student Assistant
  • Joey Rener, Software Engineer
  • Malte Winkler, Software Engineer
Instructional Services
  • Dr. Alan Bol, Instructional Designer/Scientist
  • Tony Mancus, Instructional Designer
  • Sarah Ross-Lazarov, Instructional Designer
  • Tsvetomir Ross-Lazarov, Instructional Designer
International Programs
  • David Russi, Translations Coordinator
  • Martin Steinson, Project Manager
Production and Media Services
  • Steve Deyo, Graphic and 3D Designer
  • Jordan Goodridge, Student Assistant
  • Dolores Kiessling, Software Engineer
  • Gary Pacheco, Web Designer and Developer
Science Group
  • Dr. William Bua, Meteorologist
  • Patrick Dills, Meteorologist
  • Bryan Guarente, Instructional Designer/Meteorologist
  • Matthew Kelsch, Hydrometeorologist
  • Andrea Smith, Meteorologist
  • Amy Stevermer, Scientist/Instructional Designer
  • Vanessa Vincente, Meteorologist

Back to Top