City Plus 2023
Interactive cloud computing with Google Earth Engine and geemap
- Notebook: https://geemap.org/workshops/City_Plus_2023
- Earth Engine: https://earthengine.google.com
- Geemap: https://geemap.org
Introduction¶
This notebook is for the short course presented at the City+2023@Perth International Conference.
Abstract¶
This short course provides an introduction to cloud-based geospatial analysis using the Earth Engine Python API. Attendees will learn the basics of Earth Engine data types and how to visualize and analyze Earth Engine data interactively in a Jupyter environment with geemap. Through practical examples and hands-on exercises, attendees will enhance their learning experience.
Prerequisites¶
To use geemap and the Earth Engine Python API, you must register for an Earth Engine account and follow the instructions here to create a Cloud Project. Earth Engine is free for noncommercial and research use. To test whether you can use authenticate the Earth Engine Python API, please run this notebook on Google Colab.
It is recommended that attendees have a basic understanding of Python and Jupyter Notebook.
Familiarity with the Earth Engine JavaScript API is not required but will be helpful.
Attendees can use Google Colab to follow this workshop without installing anything on their computer.
Introduction to Earth Engine and geemap¶
Earth Engine is free for noncommercial and research use. For more than a decade, Earth Engine has enabled planetary-scale Earth data science and analysis by nonprofit organizations, research scientists, and other impact users.
With the launch of Earth Engine for commercial use, commercial customers will be charged for Earth Engine services. However, Earth Engine will remain free of charge for noncommercial use and research projects. Nonprofit organizations, academic institutions, educators, news media, Indigenous governments, and government researchers are eligible to use Earth Engine free of charge, just as they have done for over a decade.
The geemap Python package is built upon the Earth Engine Python API and open-source mapping libraries. It allows Earth Engine users to interactively manipulate, analyze, and visualize geospatial big data in a Jupyter environment. Since its creation in April 2020, geemap has received over 2,800 GitHub stars and is being used by over 1,000 projects on GitHub.
Google Colab and Earth Engine Python API authentication¶
Install geemap¶
Uncomment the following line to install geemap if you are running this notebook in Google Colab.
# %pip install geemap[workshop]
Import libraries¶
import ee
import geemap
Authenticate and initialize Earth Engine¶
You will need to create a Google Cloud Project and enable the Earth Engine API for the project. You can find detailed instructions here.
geemap.ee_initialize()
Creating interactive maps¶
Let's create an interactive map using the ipyleaflet plotting backend. The geemap.Map class inherits the ipyleaflet.Map class. Therefore, you can use the same syntax to create an interactive map as you would with ipyleaflet.Map.
Map = geemap.Map()
To display it in a Jupyter notebook, simply ask for the object representation:
Map
To customize the map, you can specify various keyword arguments, such as center ([lat, lon]), zoom, width, and height. The default width is 100%, which takes up the entire cell width of the Jupyter notebook. The height argument accepts a number or a string. If a number is provided, it represents the height of the map in pixels. If a string is provided, the string must be in the format of a number followed by px, e.g., 600px.
Map = geemap.Map(center=[40, -100], zoom=4, height=600)
Map
To hide a control, set control_name to False, e.g., draw_ctrl=False.
Map = geemap.Map(data_ctrl=False, toolbar_ctrl=False, draw_ctrl=False)
Map
Adding basemaps¶
There are several ways to add basemaps to a map. You can specify the basemap to use in the basemap keyword argument when creating the map. Alternatively, you can add basemap layers to the map using the add_basemap method. Geemap has hundreds of built-in basemaps available that can be easily added to the map with only one line of code.
Create a map by specifying the basemap to use as follows. For example, the Esri.WorldImagery basemap represents the Esri world imagery basemap.
Map = geemap.Map(basemap='Esri.WorldImagery')
Map
You can add as many basemaps as you like to the map. For example, the following code adds the OpenTopoMap basemap to the map above:
Map.add_basemap('OpenTopoMap')
Using Earth Engine data¶
Earth Engine data types (Image, ImageCollection, Geometry, Feature, FeatureCollection)¶
Earth Engine objects are server-side objects rather than client-side objects, which means that they are not stored locally on your computer. Similar to video streaming services (e.g., YouTube, Netflix, and Hulu), which store videos/movies on their servers, Earth Engine data are stored on the Earth Engine servers. We can stream geospatial data from Earth Engine on-the-fly without having to download the data just like we can watch videos from streaming services using a web browser without having to download the entire video to your computer.
- Image: the fundamental raster data type in Earth Engine.
- ImageCollection: a stack or time-series of images.
- Geometry: the fundamental vector data type in Earth Engine.
- Feature: a Geometry with attributes.
- FeatureCollection: a set of features.
Image¶
Raster data in Earth Engine are represented as Image objects. Images are composed of one or more bands and each band has its own name, data type, scale, mask and projection. Each image has metadata stored as a set of properties.
Loading Earth Engine images¶
image = ee.Image('USGS/SRTMGL1_003')
image
Visualizing Earth Engine images¶
Map = geemap.Map(center=[21.79, 70.87], zoom=3)
image = ee.Image('USGS/SRTMGL1_003')
vis_params = {
'min': 0,
'max': 6000,
'palette': ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5'],
}
Map.addLayer(image, vis_params, 'SRTM')
Map
ImageCollection¶
An ImageCollection is a stack or sequence of images. An ImageCollection can be loaded by passing an Earth Engine asset ID into the ImageCollection constructor. You can find ImageCollection IDs in the Earth Engine Data Catalog.
Loading image collections¶
For example, to load the image collection of the Sentinel-2 surface reflectance:
collection = ee.ImageCollection('COPERNICUS/S2_SR')
Visualizing image collections¶
To visualize an Earth Engine ImageCollection, we need to convert an ImageCollection to an Image by compositing all the images in the collection to a single image representing, for example, the min, max, median, mean or standard deviation of the images. For example, to create a median value image from a collection, use the collection.median() method. Let's create a median image from the Sentinel-2 surface reflectance collection:
Map = geemap.Map()
collection = ee.ImageCollection('COPERNICUS/S2_SR')
image = collection.median()
vis = {
'min': 0.0,
'max': 3000,
'bands': ['B4', 'B3', 'B2'],
}
Map.setCenter(83.277, 17.7009, 12)
Map.addLayer(image, vis, 'Sentinel-2')
Map
Filtering image collections¶
Map = geemap.Map()
collection = (
ee.ImageCollection('COPERNICUS/S2_SR')
.filterDate('2021-01-01', '2022-01-01')
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 5))
)
image = collection.median()
vis = {
'min': 0.0,
'max': 3000,
'bands': ['B4', 'B3', 'B2'],
}
Map.setCenter(83.277, 17.7009, 12)
Map.addLayer(image, vis, 'Sentinel-2')
Map
FeatureCollection¶
A FeatureCollection is a collection of Features. A FeatureCollection is analogous to a GeoJSON FeatureCollection object, i.e., a collection of features with associated properties/attributes. Data contained in a shapefile can be represented as a FeatureCollection.
Loading feature collections¶
The Earth Engine Data Catalog hosts a variety of vector datasets (e.g,, US Census data, country boundaries, and more) as feature collections. You can find feature collection IDs by searching the data catalog. For example, to load the TIGER roads data by the U.S. Census Bureau:
Map = geemap.Map()
fc = ee.FeatureCollection('TIGER/2016/Roads')
Map.setCenter(-73.9596, 40.7688, 12)
Map.addLayer(fc, {}, 'Census roads')
Map
Filtering feature collections¶
Map = geemap.Map()
states = ee.FeatureCollection('TIGER/2018/States')
fc = states.filter(ee.Filter.eq('NAME', 'Louisiana'))
Map.addLayer(fc, {}, 'Louisiana')
Map.centerObject(fc, 7)
Map
feat = fc.first()
feat.toDictionary()
Map = geemap.Map()
states = ee.FeatureCollection('TIGER/2018/States')
fc = states.filter(ee.Filter.inList('NAME', ['California', 'Oregon', 'Washington']))
Map.addLayer(fc, {}, 'West Coast')
Map.centerObject(fc, 5)
Map
region = Map.user_roi
if region is None:
region = ee.Geometry.BBox(-88.40, 29.88, -77.90, 35.39)
fc = ee.FeatureCollection('TIGER/2018/States').filterBounds(region)
Map.addLayer(fc, {}, 'Southeastern U.S.')
Map.centerObject(fc, 6)
Visualizing feature collections¶
Map = geemap.Map(center=[40, -100], zoom=4)
states = ee.FeatureCollection("TIGER/2018/States")
Map.addLayer(states, {}, "US States")
Map
Map = geemap.Map(center=[40, -100], zoom=4)
states = ee.FeatureCollection("TIGER/2018/States")
style = {'color': '0000ffff', 'width': 2, 'lineType': 'solid', 'fillColor': 'FF000080'}
Map.addLayer(states.style(**style), {}, "US States")
Map
Map = geemap.Map(center=[40, -100], zoom=4)
states = ee.FeatureCollection("TIGER/2018/States")
vis_params = {
'color': '000000',
'colorOpacity': 1,
'pointSize': 3,
'pointShape': 'circle',
'width': 2,
'lineType': 'solid',
'fillColorOpacity': 0.66,
}
palette = ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5']
Map.add_styled_vector(
states, column="NAME", palette=palette, layer_name="Styled vector", **vis_params
)
Map
Earth Engine Data Catalog¶
The Earth Engine Data Catalog hosts a variety of geospatial datasets. As of March 2023, the catalog contains over 1,000 datasets with a total size of over 80 petabytes. Some notable datasets include: Landsat, Sentinel, MODIS, NAIP, etc. For a complete list of datasets in CSV or JSON formats, see the Earth Engine Datasets List.
Searching for datasets¶
The Earth Engine Data Catalog is searchable. You can search datasets by name, keyword, or tag. For example, enter "elevation" in the search box will filter the catalog to show only datasets containing "elevation" in their name, description, or tags. 52 datasets are returned for this search query. Scroll down the list to find the NASA SRTM Digital Elevation 30m dataset. On each dataset page, you can find the following information, including Dataset Availability, Dataset Provider, Earth Engine Snippet, Tags, Description, Code Example, and more (see {numref}ch03_gee_srtm). One important piece of information is the Image/ImageCollection/FeatureCollection ID of each dataset, which is essential for accessing the dataset through the Earth Engine JavaScript or Python APIs.
Map = geemap.Map()
Map
dataset_xyz = ee.Image('USGS/SRTMGL1_003')
Map.addLayer(dataset_xyz, {}, "USGS/SRTMGL1_003")
Map = geemap.Map()
dem = ee.Image('USGS/SRTMGL1_003')
vis_params = {
'min': 0,
'max': 4000,
'palette': ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5'],
}
Map.addLayer(dem, vis_params, 'SRTM DEM')
Map
Converting Earth Engine JavaScripts to Python¶
Find some Earth Engine JavaScript code that you want to convert to Python. For example, you can grab some sample code from the Earth Engine Documentation.
Map = geemap.Map()
Map
# Load an image.
image = ee.Image('LANDSAT/LC08/C02/T1_TOA/LC08_044034_20140318')
# Define the visualization parameters.
vizParams = {'bands': ['B5', 'B4', 'B3'], 'min': 0, 'max': 0.5, 'gamma': [0.95, 1.1, 1]}
# Center the map and display the image.
Map.setCenter(-122.1899, 37.5010, 10)
# San Francisco Bay
Map.addLayer(image, vizParams, 'False color composite')
Map = geemap.Map(center=(40, -100), zoom=4)
dem = ee.Image('USGS/SRTMGL1_003')
landsat7 = ee.Image('LANDSAT/LE7_TOA_5YEAR/1999_2003').select(
['B1', 'B2', 'B3', 'B4', 'B5', 'B7']
)
states = ee.FeatureCollection("TIGER/2018/States")
vis_params = {
'min': 0,
'max': 4000,
'palette': ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5'],
}
Map.addLayer(dem, vis_params, 'SRTM DEM')
Map.addLayer(
landsat7,
{'bands': ['B4', 'B3', 'B2'], 'min': 20, 'max': 200, 'gamma': 2.0},
'Landsat 7',
)
Map.addLayer(states, {}, "US States")
Map.add_inspector()
Map
Using the plotting tool¶
Map = geemap.Map(center=[40, -100], zoom=4)
landsat7 = ee.Image('LANDSAT/LE7_TOA_5YEAR/1999_2003').select(
['B1', 'B2', 'B3', 'B4', 'B5', 'B7']
)
landsat_vis = {'bands': ['B4', 'B3', 'B2'], 'gamma': 1.4}
Map.addLayer(landsat7, landsat_vis, "Landsat")
hyperion = ee.ImageCollection('EO1/HYPERION').filter(
ee.Filter.date('2016-01-01', '2017-03-01')
)
hyperion_vis = {
'min': 1000.0,
'max': 14000.0,
'gamma': 2.5,
}
Map.addLayer(hyperion, hyperion_vis, 'Hyperion')
Map.add_plot_gui()
Map
Map.set_plot_options(add_marker_cluster=True, overlay=True)
Add NLCD WMS layer and legend to the map.
Add NLCD Earth Engine layer and legend to the map.
Map = geemap.Map(center=[40, -100], zoom=4)
Map.add_basemap('HYBRID')
nlcd = ee.Image('USGS/NLCD_RELEASES/2019_REL/NLCD/2019')
landcover = nlcd.select('landcover')
Map.addLayer(landcover, {}, 'NLCD Land Cover 2019')
Map.add_legend(
title="NLCD Land Cover Classification", builtin_legend='NLCD', height='460px'
)
Map
Custom legends¶
Add a custom legend by specifying a dictionary of colors and labels.
Map = geemap.Map(center=[40, -100], zoom=4)
Map.add_basemap('Google Hybrid')
legend_dict = {
'11 Open Water': '466b9f',
'12 Perennial Ice/Snow': 'd1def8',
'21 Developed, Open Space': 'dec5c5',
'22 Developed, Low Intensity': 'd99282',
'23 Developed, Medium Intensity': 'eb0000',
'24 Developed High Intensity': 'ab0000',
'31 Barren Land (Rock/Sand/Clay)': 'b3ac9f',
'41 Deciduous Forest': '68ab5f',
'42 Evergreen Forest': '1c5f2c',
'43 Mixed Forest': 'b5c58f',
'51 Dwarf Scrub': 'af963c',
'52 Shrub/Scrub': 'ccb879',
'71 Grassland/Herbaceous': 'dfdfc2',
'72 Sedge/Herbaceous': 'd1d182',
'73 Lichens': 'a3cc51',
'74 Moss': '82ba9e',
'81 Pasture/Hay': 'dcd939',
'82 Cultivated Crops': 'ab6c28',
'90 Woody Wetlands': 'b8d9eb',
'95 Emergent Herbaceous Wetlands': '6c9fb8',
}
nlcd = ee.Image('USGS/NLCD_RELEASES/2019_REL/NLCD/2019')
landcover = nlcd.select('landcover')
Map.addLayer(landcover, {}, 'NLCD Land Cover 2019')
Map.add_legend(title="NLCD Land Cover Classification", legend_dict=legend_dict)
Map
Creating color bars¶
Add a horizontal color bar.
Map = geemap.Map()
dem = ee.Image('USGS/SRTMGL1_003')
vis_params = {
'min': 0,
'max': 4000,
'palette': ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5'],
}
Map.addLayer(dem, vis_params, 'SRTM DEM')
Map.add_colorbar(vis_params, label="Elevation (m)", layer_name="SRTM DEM")
Map
Add a vertical color bar.
Map.add_colorbar(
vis_params,
label="Elevation (m)",
layer_name="SRTM DEM",
orientation="vertical",
max_width="100px"
)
Map = geemap.Map()
Map.split_map(left_layer='Esri.WorldTopoMap', right_layer='OpenTopoMap')
Map
Create a split map with Earth Engine layers.
Map = geemap.Map(center=(40, -100), zoom=4, height=600)
nlcd_2001 = ee.Image('USGS/NLCD_RELEASES/2019_REL/NLCD/2001').select('landcover')
nlcd_2019 = ee.Image('USGS/NLCD_RELEASES/2019_REL/NLCD/2019').select('landcover')
left_layer = geemap.ee_tile_layer(nlcd_2001, {}, 'NLCD 2001')
right_layer = geemap.ee_tile_layer(nlcd_2019, {}, 'NLCD 2019')
Map.split_map(left_layer, right_layer)
Map
Linked maps¶
Create a 2x2 linked map for visualizing Sentinel-2 imagery with different band combinations.
image = (
ee.ImageCollection('COPERNICUS/S2')
.filterDate('2018-09-01', '2018-09-30')
.map(lambda img: img.divide(10000))
.median()
)
vis_params = [
{'bands': ['B4', 'B3', 'B2'], 'min': 0, 'max': 0.3, 'gamma': 1.3},
{'bands': ['B8', 'B11', 'B4'], 'min': 0, 'max': 0.3, 'gamma': 1.3},
{'bands': ['B8', 'B4', 'B3'], 'min': 0, 'max': 0.3, 'gamma': 1.3},
{'bands': ['B12', 'B12', 'B4'], 'min': 0, 'max': 0.3, 'gamma': 1.3},
]
labels = [
'Natural Color (B4/B3/B2)',
'Land/Water (B8/B11/B4)',
'Color Infrared (B8/B4/B3)',
'Vegetation (B12/B11/B4)',
]
geemap.linked_maps(
rows=2,
cols=2,
height="300px",
center=[38.4151, 21.2712],
zoom=12,
ee_objects=[image],
vis_params=vis_params,
labels=labels,
label_position="topright",
)
Map = geemap.Map(center=[40, -100], zoom=4)
collection = ee.ImageCollection('USGS/NLCD_RELEASES/2019_REL/NLCD').select('landcover')
vis_params = {'bands': ['landcover']}
years = collection.aggregate_array('system:index').getInfo()
years
Create a timeseries inspector for NLCD.
Map.ts_inspector(
left_ts=collection,
right_ts=collection,
left_names=years,
right_names=years,
left_vis=vis_params,
right_vis=vis_params,
width='80px',
)
Map
Time slider¶
Create a map for visualizing MODIS vegetation data.
Map = geemap.Map()
collection = (
ee.ImageCollection('MODIS/MCD43A4_006_NDVI')
.filter(ee.Filter.date('2018-06-01', '2018-07-01'))
.select("NDVI")
)
vis_params = {
'min': 0.0,
'max': 1.0,
'palette': 'ndvi',
}
Map.add_time_slider(collection, vis_params, time_interval=2)
Map
Create a map for visualizing weather data.
Map = geemap.Map()
collection = (
ee.ImageCollection('NOAA/GFS0P25')
.filterDate('2018-12-22', '2018-12-23')
.limit(24)
.select('temperature_2m_above_ground')
)
vis_params = {
'min': -40.0,
'max': 35.0,
'palette': ['blue', 'purple', 'cyan', 'green', 'yellow', 'red'],
}
labels = [str(n).zfill(2) + ":00" for n in range(0, 24)]
Map.add_time_slider(collection, vis_params, labels=labels, time_interval=1, opacity=0.8)
Map
Visualizing Sentinel-2 imagery
Map = geemap.Map(center=[37.75, -122.45], zoom=12)
collection = (
ee.ImageCollection('COPERNICUS/S2_SR')
.filterBounds(ee.Geometry.Point([-122.45, 37.75]))
.filterMetadata('CLOUDY_PIXEL_PERCENTAGE', 'less_than', 10)
)
vis_params = {"min": 0, "max": 4000, "bands": ["B8", "B4", "B3"]}
Map.add_time_slider(collection, vis_params)
Map
Exercise - Creating land cover maps with a legend¶
Create a split map for visualizing NLCD land cover change in Texas between 2001 and 2019. Add the NLCD legend to the map. Relevant Earth Engine assets:
Created: 2023-09-02