Overview

The purpose of this chapter is to introduce rgee, a non-official API for Earth Engine. You will explore the main features available in rgee and how to set up an environment that integrates rgee with third-party R and Python packages. After this chapter, you will be able to combine R, Python, and JavaScript in the same workflow.

FormalPara Learning Outcomes

  • Becoming familiar with rgee, the Earth Engine R API interface.

  • Integrating rgee with other R packages.

  • Displaying interactive maps.

  • Integrating Python and R packages using reticulate.

  • Combining Earth Engine JavaScript and Python APIs with R.

Assumes you know how to

  • Install the Python environment (Chap. 30).

  • Use the require function to load code from existing modules (Chap. 28).

  • Use the basic functions and logic of Python.

  • Configure an environment variable and use.Renviron files.

  • Create Python virtual environments.

1 Introduction to Theory

R is a popular programming language established in statistical science with large support in reproducible research, geospatial analysis, data visualization, and much more. To get started with R, you will need to satisfy some extra software requirements. First, install an up-to-date R version (at least 4.0) in your work environment. The installation procedure will vary depending on your operating system (i.e., Windows, Mac, or Linux). Hands-On Programming with R (Garrett Grolemund 2014, Appendix A) explains step by step how to proceed. We strongly recommend that Windows users install Rtools to avoid compiling external static libraries.

If you are new to R, a good starting point is the book Geocomputation with R (Lovelace et al. 2019) or Spatial Data Science with Application in R (Pebesma and Bivand 2019). In addition, we recommend using an integrated development environment (e.g., Rstudio) or a Code Editor (e.g., Visual Studio Code) to create a suitable setting to display and interact with R objects.

The following R packages must be installed (find more information in the R manual) in order to go through the practicum section.

A twelve-line algorithm to use install packages to install R packages from the C R A N respiratory. Some steps of the algorithm connect Python with R, install R packages from remote repositories, and read write, manipulate, analyze, and model spatial data.

Earth Engine officially supports client libraries only for the JavaScript and Python programming languages. While the Earth Engine Code Editor offers a convenient environment for rapid prototyping in JavaScript, the lack of a mechanism for integration with local environments makes the development of complex scripts tricky. On the other hand, the Python client library offers much versatility, enabling support for third-party packages. However, not all Earth and environmental scientists code in Python. Hence, a significant number of professionals are not members of the Earth Engine community. In the R ecosystem, rgee (Aybar et al. 2020) tries to fill this gap by wrapping the Earth Engine Python API via reticulate. The rgee package extends and supports all the Earth Engine classes, modules, and functions, working as fast as the other APIs.

Figure 31.1 illustrates how rgee bridges the Earth Engine platform with the R ecosystem. When an Earth Engine request is created in R, rgee transforms this piece of code into Python. Next, the Earth Engine Python API converts the Python code into JSON. Finally, the JSON file request is received by the server through a Web REST API. Users could get the response using the getInfo method by following the same path in reverse.

Fig. 31.1
A bidirectional flowchart for the r g e e functionality flows through the following, Python A P I client library, web REST A P Is, batch computation, on-the-fly computation. The components are R Ecosystem, I slash O module, r g e e, map $ add layer, Reticulate, and e e manage.

Simplified diagram of rgee functionality

Full size image

2 Practicum

2.1 Section 1. Installing rgee

To run, rgee needs a Python environment with two packages: NumPy and earthengine-api. Because instructions change frequently, installation is explained at the following checkpoint:

Code Checkpoint F64a. The book’s repository contains information about setting up the rgee environment.

After installing both the R and Python requirements, you can now initialize your Earth Engine account from within R. Consider that R, in contrast to JavaScript and Python, supports three distinct Google APIs: Earth Engine, Google Drive, and Google Cloud Storage (GCS).

The Google Drive and GCS APIs will allow you to transfer your Earth Engine completed task exports to a local environment automatically. In these practice sessions, we will use only the Earth Engine and Google Drive APIs. Users that are interested in GCS can look up and explore the GCS vignette. To initialize your Earth Engine account alongside Google Drive, use the following commands.

A four-line algorithm for Initializing just Earth Engine and initializing Earth Engine and G D. The commands read, e e initialize left parenthesis right parenthesis and e e initialize left parenthesis drive = true right parenthesis.

If the Google account is verified and the permission is granted, you will be directed to an authentication token. Copy and paste this token into your R console. Consider that the GCS API requires setting up credentials manually; look up and explore the rgee vignette for more information. The verification step is only required once; after that, rgee saves the credentials in your system.

Code Checkpoint F64b. The book’s repository contains information about what your code should look like at this point.

2.2 Section 2. Creating a 3D Population Density Map with rgee and rayshader

First, import the rgee, rayshader, and raster packages.

A three-line command reads, library left parenthesis ray shader right parenthesis, library left parenthesis raster right parenthesis, and library left parenthesis r g e e right parenthesis.

Initialize the Earth Engine and Google Drive APIs using ee_Initialize. Both credentials must come from the same Google account.

A one-line command reads, e e underscore initialize left parenthesis drive = true right parenthesis.

Then, we will access the WorldPop Global Project Population Data dataset. In rgee, the Earth Engine spatial classes (ee$Image, ee$ImageCollection, and ee$FeatureCollection) have a special attribute called Dataset. Users can use it along with autocompletion to quickly find the desired dataset.

A three-line command to access the classes for the following. Collections, population underscore data, and population underscore data underscore max.

If you need more information about the Dataset, use ee_utils_dataset_display to go to the official documentation in the Earth Engine Data Catalog.

A one-line command reads, population underscore data % greater than % e e underscore utils underscore dataset underscore display left parenthesis right parenthesis.

The rgee package provides various built-in functions to retrieve data from Earth Engine (Aybar et al. 2020). In this example, we use ee_as_raster, which automatically converts an ee$Image (server object) into a RasterLayer (local object).

An eleven-line algorithm to access the following built-in functions of the r g e e package. E e $ geometry $ rectangle and e e underscore a s underscore raster. The command home, p c-user 0 1 population dot t i f changes the user’s path.

Now, turn a RasterLayer into a matrix suitable for rayshader.

A one-line command reads, pop underscore matrix less than hyphen raster underscore to underscore matrix left parenthesis population underscore data underscore l y underscore local right parenthesis.

Next, modify the matrix population density values, adding:

  • Texture, based on five colors (lightcolor, shadowcolor, leftcolor, rightcolor, and centercolor; see rayshader::create_texture documentation)

  • Color and shadows (rayshader::sphere_shade)

A six-line pop matrix algorithm works with the sphere underscore shade and plot underscore 3 d classes. The values for zoom, theta, z scale, solid depth, solid color, shadow depth, shadow color, shadow width, and window size are given under the second class.

Lastly, define a title and subtitle for the plot. Use rayshader::render_snapshot to export the final results (Fig. 31.2).

Fig. 31.2
A three-dimensional spatial map of South America highlights its population density. The coastline of Brazil, Colombia, central Chile, Nicaragua, Costa Rica, and Panama are highlighted with the highest population densities.

3D population density map of South America

Full size image
A fourteen-line algorithm to work with the text less than hyphen paste 0 and render underscore snapshot classes. The attributes for file name, title text, title size, title color, and title font are given under the second class.

Code Checkpoint F64c. The book’s repository contains information about what your code should look like at this point.

2.3 Section 3. Displaying Maps Interactively

Similar to the Code Editor, rgee supports the interactive visualization of spatial Earth Engine objects by Map$addLayer. First, let’s import the rgee and cptcity packages. The cptcity R package is a wrapper to the cpt-city color gradients web archive.

A three-line command reads, library left parenthesis r g e e right parenthesis, library left parenthesis c p t city right parenthesis, e e underscore initialize left parenthesis right parenthesis.

We will select an ee$Image; in this case, the Shuttle Radar Topography Mission 90m (SRTM-90) Version 4.

A one-line command reads, d e m less than hyphen e e $ image $ dataset $ C G I A R underscore S R T M 90 underscore V 4.

Then, we will set the visualization parameters as a list with the following elements.

  • min: value(s) to map to 0

  • Max: value(s) to map to 1

  • palette: a list of CSS-style color strings.

A five-line algorithm to access the viz less than hyphen list class and work with the attributes of the min value, max value, and palette.

Then, we will create a simple display using Map$addLayer.

A one-line command reads, m 1 less than hyphen map $ add layer left parenthesis d e m, vis params = viz, name = double quotation starts S R T M double quotation ends, shown = true right parenthesis.

Optionally, you could add a custom legend using Map$addLayer (Fig. 31.3).

Fig. 31.3
A world map of Gaussian curvature highlights S R T M – 90 version 4 elevation values. The lowest value of 1000 is highlighted for places like Brazil, Argentina, the United States, half of Canada, India, Russia, and Australia. The highest value of 6000 is highlighted for China.

Interactive visualization of SRTM-90 Version 4 elevation values

Full size image
A two-line command reads, pal less than hyphen map $ add legend left parenthesis viz right parenthesis, m 1 + pal.

The procedure to display ee$Geometry, ee$Feature, and ee$FeatureCollections objects is similar to the previous example effected on an ee$Image. Users just need to change the arguments: eeObject and visParams.

First, Earth Engine geometries (Fig. 31.4).

Fig. 31.4
A partial map of Peru highlights Lima, its adjoining coastline, and the surrounding areas in a shaded circle. A scale of 20 kilometers is given.

Polygon buffer surrounding the city of Lima, Peru

Full size image
A four-line algorithm to display the following objects. E e $ geometry $ point, e e $ feature $ buffer, map $ center object, and map $ add layer. The e e object for map $ add layer is e e $ geometry $ polygon.

Next, Earth Engine feature collections (Fig. 31.5).

Fig. 31.5
A spatial map of Lagos highlights its building footprints. The footprints are clustered and denser in the northern, north-eastern, and south-eastern areas. A road that passes through the area is labeled Clinic Road.

Building footprints in Lagos, Nigeria

Full size image
A four-line algorithm to display the following objects. E e $ feature collection $ dataset $, map $ set center, and map $ add layer left parenthesis building right parenthesis. The e e object for map $ add layer left parenthesis building right parenthesis is a e e $ feature collection.

The rgee functionality also supports the display of ee$ImageCollection via Map$addLayers (note the extra “s” at the end). Map$addLayers will use the same visualization parameters for all the images (Fig. 31.6). If you need different visualization parameters per image, use a Map$addLayer within a for loop.

Fig. 31.6
A world heatmap highlights the MOD 1 6 A 2 total evapotranspiration values. The higher range of values is highlighted in places like Chile, Argentina, the United States, the Democratic Republic of Congo, South Africa, India, Kazakhstan, Mongolia, and Australia.

MOD16A2 total evapotranspiration values (kg/m2/8 day)

Full size image
A thirteen-line algorithm executes the following tasks. Defining an image collection. Setting viz params. Printing map results interactively.

Another useful rgee feature is the comparison operator (|), which creates a slider in the middle of the canvas, permitting quick comparison of two maps. For instance, load a Landsat 4 image:

A one-line command reads, land sat less than sign hyphen e e $ image left parenthesis single quotation starts land sat slash L T 0 4 slash C 0 1 slash T 1 slash L T 0 4 underscore 0 0 8 0 6 7 underscore 1 9 8 9 0 9 1 7 single quotation ends right parenthesis.

Calculate the Normalized Difference Snow Index.

A one-line command reads, n d s i less than symbol hyphen land sat $ normalized difference left parenthesis C left parenthesis single quotation starts B 3 single quotation ends, single quotation starts B 5 single quotation ends right parenthesis right parenthesis.

Define a constant value and use ee$Image$gte to return a binary image where pixels greater than or equal to that value are set as 1 and the rest are set as 0. Next, filter 0 values using ee$Image$updateMask.

A one-line command reads, n d s i masked less than symbol hyphen n d s i $ update mask left parenthesis n d s i $ g t e left parenthesis 0.4 right parenthesis right parenthesis.

Define the visualization parameters.

A seven-line algorithm works with the viz params less than hyphen list class and n d s i V i z less than hyphen list class. The attributes discussed under the first class are for the min value, max value, and gamma.

Center the map on the Huayhuash mountain range in Peru.

A one-line command reads, map $ set center left parenthesis I o n = negative 77.20, l a t = negative 9.85, zoom = 10 right parenthesis.

Finally, visualize both maps using the | operator (Fig. 31.7).

Fig. 31.7
A map of the Huayhuash mountain range overlayed with a false-color composite. Some northern, central, and southeastern areas are highlighted in cool tones. The Eastern side is overlayed with a warm shade which resembles a rocky terrain.

False-color composite over the Huayhuash mountain range, Peru

Full size image
A three-line algorithm to visualize the maps using the modulus operator between m 2 and m 1 where m 2 deals with N D S I masked and m 1 deals with false color composite.

Code Checkpoint F64d. The book’s repository contains information about what your code should look like at this point.

2.4 Section 4. Integrating rgee with Other Python Packages

As noted in Sect. 31.2.1, rgee set up a Python environment with NumPy and earthengine-api in your system. However, there is no need to limit it to just two Python packages. In this section, you will learn how to use the Python package ndvi2gif to perform a Normalized Difference Vegetation Index (NDVI) multi-seasonal analysis in the Ocoña Valley without leaving R.

Whenever you want to install a Python package, you must run the following.

A three-line command reads, library left parenthesis r g e e right parenthesis, library left parenthesis reticulate right parenthesis, e e underscore initialize left parenthesis right parenthesis.

The ee_Initialize function not only authenticates your Earth Engine account but also helps reticulate to set up a Python environment compatible with rgee. After running ee_Initialize, use reticulate::install_python to install the desired Python package.

A one-line command reads, p y underscore install left parenthesis double quotation starts n d v I 2 g i f double quotation ends right parenthesis.

The previous procedure is needed just once for each Python environment. Once installed, we simply load the package using reticulate::import.

A one-line command reads, n g i f less than symbol hyphen import left parenthesis double quotation n d v i 2 g i f double quotation ends right parenthesis.

Then, we define our study area using ee$Geometry$Rectangle (Fig. 31.8) and use the leaflet layers control to switch between basemaps.

Fig. 31.8
A partial spatial map of the coastline of Peru. The Ocona Valley, its adjoining coastline, and the surrounding areas are highlighted under a shaded rectangle.

Rectangle drawn over the Ocoña Valley, Peru

Full size image
An algorithm of the following objects. E e $ geometry rectangle left parenthesis c o l c a right parenthesis, map $ center object left parenthesis r o i right parenthesis, and map $ add layer left parenthesis r o i right parenthesis.

In ndvi2gif, there is just one class: NdviSeasonality. It has the following four public methods.

  • get_export: Exports NDVI year composites in.GeoTIFF format to your local folder.

  • get_export_single: Exports single composite as.GeoTIFF to your local folder.

  • get_year_composite: Returns the NDVI composites for each year.

  • get_gif: Exports NDVI year composites as a.gif to your local folder.

To run, the NdviSeasonality constructor needs to define the following arguments.

  • roi: the region of interest

  • start_year: the initial year to start to create yearly composites

  • end_year: the end year to look for

  • sat: the satellite sensor

  • key: the aggregation rule that will be used to generate the yearly composite.

For each year, the get_year_composite method generates an NDVI ee$Image with four bands, one band per season. Color combination between images and bands will allow you to interpret the vegetation phenology over the seasons and years. In ndvi2gif, the seasons are defined as follows.

  • winter = c(‘-01-01’, ‘-03-31’)

  • spring = c(‘-04-01’, ‘-06-30’)

  • summer = c(‘-07-01’, ‘-09-30’)

  • autumn = c(‘-10-01’, ‘-12-31’).

A 12-line my class algorithm executes the following tasks. Estimating the median of the yearly composites from 2016 to 2020 and the median of the winter season composites from 2016 to 2020.

We can display maps interactively using the Map$addLayer (Fig. 31.9) and use the leaflet layers control to switch between basemaps.

Fig. 31.9
A partial spatial map of the coastline of Peru overlayed with another map highlighting the seasons with maximum value. The northern, eastern, and southern parts of the overlayed map highlight seasons.

Comparison between the maximum historic winter NDVI and the mean historic NDVI. Colors represent the season when the maximum value occurred

Full size image
A two-line code with the map $ add layer object deals with the winter max list of minimum and maximum values and the median list of minimum and maximum values.

And we can export the results to a GIF format.

A one-line command reads, my class $ get underscore gift left parenthesis right parenthesis.

To get more information about the ndvi2gif package, visit its GitHub repository.

Code Checkpoint F64e. The book’s repository contains information about what your code should look like at this point.

2.5 Section 5. Converting JavaScript Modules to R

In recent years, the Earth Engine community has developed a lot of valuable third-party modules. Some incredible ones are geeSharp, ee-palettes, spectral (Montero 2021), and LandsatLST (Ermida et al. 2020). While some of these modules have been implemented in Python and JavaScript (e.g., geeSharp and spectral), most are available only for JavaScript. This is a critical drawback, because it divides the Earth Engine community by programming languages. For example, if an R user wants to use tagee, the user will have to first translate the entire module to R.

In order to close this breach, the ee_extra Python package has been developed to unify the Earth Engine community. The philosophy behind ee_extra is that all of its extended functions, classes, and methods must be functional for the JavaScript, Julia, R, and Python client libraries. Currently, ee_extra is the base of the rgeeExtra (Aybar et al. 2020) and eemont (Montero 2021) packages.

To demonstrate the potential of ee_extra, let’s study an example from the Landsat Land Surface Temperature (LST) JavaScript module. The Landsat LST module computes the land surface temperature for Landsat products (Ermida et al. 2020). First, we will run it in the Earth Engine Code Editor; then, we will replicate those results in R.

First, JavaScript. In a new script in the Code Editor, we must require the Landsat LST module.

A one-line v a r code with the land sat L S T module uses the require function. The code inside the function is single quote starts users slash Sofia e r m i d a slash land sat underscore s m w underscore l s t colon modules slash land sat underscore L S T dot j s single quote ends.

The Landsat LST module contains a function named collection. This function receives the following parameters.

  • The Landsat archive ID

  • The starting date of the Landsat collection

  • The ending date of the Landsat collection

  • The region of interest as geometry

  • A Boolean parameter specifying if we want to use the NDVI for computing a dynamic emissivity instead of using the emissivity from ASTER.

In the following code block, we are going to define all required parameters.

A five-line code block defines the following parameters. Geometry, satellite, data underscore start, data underscore end, and use underscore n d v i.

Now, with all our parameters defined, we can compute the land surface temperature by using the collection method from Landsat LST.

A one-line algorithm to calculate the land surface temperature reads, v a r land sat coll = land sat L S T dot collection left parenthesis satellite, date underscore start, date underscore end, geometry, use underscore n d v i right parenthesis semicolon.

The result is stored as an ImageCollection in the LandsatColl variable. Now select the first element of the collection as an example by using the first method.

A one-line code reads, v a r ex image = land sat coll dot first left parenthesis right parenthesis semicolon.

This example image is now stored in a variable named ‘exImage’. Let’s display the LST result on the Map. For visualization purposes, we will define a color palette.

A 1-line code reads, v a r c map=left bracket single quote starts blue single quote ends, single quote starts cyan single quote ends, single quote starts green single quote ends, single quote starts yellow single quote ends, single quote starts red single quote ends right bracket semicolon.

Then, we will center the map in the region of interest.

A one-line code reads, map dot center object left parenthesis geometry right parenthesis semicolon.

Finally, let’s display the LST with the cmap color palette by using the Map.addLayer method (Fig. 31.10). This method receives the image to visualize, the visualization parameters, the color palette, and the name of the layer to show in the layer control. The visualization parameters will be

Fig. 31.10
A partial map of Portugal partially overlayed with a heatmap highlights the L S T. The central coastlines are highlighted with the warmest tones.

Map illustrating LST, obtained by following the JavaScript example

Full size image

  • min: 290 (a minimum LST value of 290 K)

  • max: 320 (a maximum LST value of 320 K)

  • palette: cmap (the color palette that was created some steps before).

The name of the layer in the Map layer set will be LST.

A four-line algorithm for the map dot add layer class deals with the attributes for ex image, min value, max value, and palette.

Code Checkpoint F64f. The book’s repository contains a script that shows what your code should look like at this point.

Now, let’s use R to implement the same logic. As in the previous sections, import the R packages: rgee and rgeeExtra. Then, initialize your Earth Engine session.

A four-line code reads, library left parenthesis r g e e right parenthesis, library left parenthesis r g e e extra right parenthesis, library left parenthesis reticulate right parenthesis, e e underscore initialize left parenthesis right parenthesis.

Install rgeeExtra Python dependencies.

A one-line code reads p y underscore install left parenthesis packages = c left parenthesis double quote starts regex double quotes ends, double quote starts e e underscore extra double quotes, double quote starts j s beautifier double quotes ends right parenthesis right parenthesis.

Using the function rgeeExtra::module loads the JavaScript module.

A one-line code written using the land sat L S T module function. The code inside the function reads, double quote starts users slash Sofia a e r m i d a slash land sat underscore s m w underscore l s t colon modules slash land sat underscore L S T dot j s double quote ends right parenthesis.

The rest of the code is exactly the same as in JavaScript (Fig. 31.11).

Fig. 31.11
A screenshot of a computer screen with a command window on the left pane and a heatmap on the right pane. In the map, a part of the central area, the western coastline, and a part near the southwestern coastline are highlighted with the warmest tones.

Map illustrating LST, obtained by following the R example

Full size image
An algorithm includes the attributes of the following parameters and the land sat collection function. Geometry, satellite, date underscore start, date underscore end, use underscore n d v i, ex image, c map, and I mod.

Code Checkpoint F64g. The book’s repository contains information about what your code should look like at this point.

Question 1. When and why might users prefer to use R instead of Python to connect to Earth Engine?

Question 2. What are the advantages and disadvantages of using rgee instead of the Earth Engine JavaScript Code Editor?

3 Synthesis

Assignment 1. Estimate the Gaussian curvature map from a digital elevation model using rgee and rgeeExtra. Hint: Use the module 'users/joselucassafanelli/TAGEE:TAGEE-functions'.

4 Conclusion

In this chapter, you learned how to use Earth Engine and R in the same workflow. Since rgee uses reticulate, rgee also permits integration with third-party Earth Engine Python packages. You also learned how to use Map$addLayer, which works similarly to the Earth Engine User Interface API (Code Editor). Finally, we also introduced rgeeExtra, a new R package that extends the Earth Engine API and supports JavaScript module execution.