Earlier this year we described openEO as one of the processing approaches being explored for the future evolution of GEP. We have now started beta-testing what this can look like in practice, using an existing Sentinel-1 interferometric processing workflow and a relatively large collection of derived products already produced on GEP.
The test is deliberately different from starting with a simple optical index or executing an individual algorithm through openEO. Instead, we wanted to see how openEO could complement an existing operational Earth Observation processing service and provide an additional analytical layer over the products it generates.
The full experiment takes place within the GEP cloud environment. Sentinel-1 data are processed on GEP, the resulting interferometric products remain in cloud storage, openEO accesses those products close to where they are stored, and the analytical result can subsequently be inspected in QGIS running within a GEP Workspace.
The user therefore does not need to download the complete EO product collection to a local computer simply to move from one stage of the analysis to another.
Starting from operational InSAR processing
For the test we used a Sentinel-1 stack over Nayarit, Mexico.
The starting dataset contains 44 Sentinel-1 SLC acquisitions, representing approximately 350 GB of input data. These data are processed using the SNAPPING service available on GEP, producing a stack of 43 interferograms.
Each complete interferometric product is up to around 5 GB, resulting in a collection of approximately 200 GB.
This first processing step remains based on the existing GEP processing model. SNAPPING is packaged as an EO Application Package using CWL, allowing a complex Sentinel-1 processing chain, its software dependencies and its execution logic to be deployed and operated as a cloud service.
This remains an appropriate model for operational EO production workflows of this type. The objective of the beta test is therefore not to rewrite the interferometric processing chain as an openEO process graph. Instead, openEO is introduced after this production step, when the resulting collection becomes the input to further analysis.
GEP workflow: Sentinel-1 SLCs → SNAPPING processing → interferometric collection → openEO analytics → QGIS in a GEP Workspace
Turning processing outputs into an analytical collection
The interferograms generated by SNAPPING are produced as cloud-optimised geospatial products and remain available within the GEP cloud environment.
For the beta test, the 43 products are exposed as a collection that can be accessed through openEO. The COGs are described through STAC metadata, providing the bridge between the SNAPPING outputs held in cloud storage and the collection exposed through openEO. This changes how we can work with what would otherwise simply be a directory containing several hundred gigabytes of processing results.
Instead of downloading the interferograms and creating a separate local analysis environment, the collection can be accessed directly by an openEO process graph.
The test calculation is intentionally straightforward: calculate a temporal mean of the coherence information contained in the interferometric stack for a selected area of interest.
The interesting part is therefore less the mathematical operation itself than what happens underneath it.
metadata for the SNAPPING interferogram collection exposed to the openEO environment
43 interferograms, reading only what is needed
When the user selects a smaller area of interest, the openEO workflow operates across the coherence information from all 43 interferograms, without requiring the complete interferometric collection to be downloaded or read.
The full interferometric products represent around 200 GB in cloud storage. The calculation uses only the coherence layer from each product, which is less than 1 GB, and the products are stored as Cloud Optimised GeoTIFFs so that the processing environment can further access only the portions corresponding to the requested spatial extent.
Xarray and Dask can therefore build the multidimensional data structure required for the calculation while reading the relevant parts of the COGs from cloud storage. Dask then distributes the computation across the available processing resources.
All 43 interferograms contribute to the temporal calculation, while data access remains focused on the required layer and selected area.
This is an important property when moving from individual EO products towards collections. The user works with the logical dataset and the processing environment handles how the required portions of the underlying products are retrieved and computed.
GEP openEO Editor showing the SNAPPING collection, reduce_dimension with a temporal mean, completed batch jobs and the resulting mean-coherence raster.
An alternative openEO processing path
Underneath the openEO process model, this implementation follows an Xarray/Dask processing path.
This is a different implementation approach from the GeoTrellis and Apache Spark stack used by other operational openEO environments, including the implementation used in the Copernicus Data Space Ecosystem.
Here, the openEO process graph is parsed and the corresponding processes are implemented using Python, Xarray and Dask. The Xarray/Dask integration used here was developed by EURAC Research in the context of the EOEPCA+ collaboration with Terradue and is now being tested on GEP.
The distinction is useful because openEO defines an API and a process model rather than imposing one particular processing engine underneath it. Different infrastructures can therefore implement the same openEO concepts using different computational approaches.
For this integration test, this provides the processing layer behind the openEO interface exposed on GEP.
Moving beyond the usual first examples
Many first examples of openEO naturally focus on relatively compact analytical operations, such as filtering a Sentinel-2 collection or calculating a vegetation index. Here we are testing something further downstream in an operational EO workflow.
The products accessed through openEO have already been generated by a specialised Sentinel-1 interferometric processing chain running as a service on GEP. SNAPPING remains packaged as an EO Application Package, preserving the software, dependencies and processing logic required for repeatable production. Its outputs are then made available as a cloud-hosted collection, described through STAC metadata and stored as Cloud Optimised GeoTIFFs.
openEO provides an additional analytical layer over that collection. Instead of replacing the production workflow, it can be used to filter, reduce and analyse the products it generates, using the openEO process model together with Xarray and Dask for the underlying multidimensional computation.
This allows the two processing approaches to be used where they fit best. The EO Application Package handles the specialised interferometric production, while openEO provides collection-level analytics over the resulting products. They remain connected through cloud storage, STAC metadata and cloud-optimised formats, with GEP providing the common environment around both stages.
openEO Editor showing the completed job and resulting mean-coherence layer
From processing to analytics and then to QGIS
The beta calculation produces a GeoTIFF that can be visualised directly from the openEO Editor.
The same result can also be opened in QGIS running within a GEP Workspace, where it can be inspected together with OpenStreetMap, other EO products or additional geospatial information.
QGIS in this example is therefore also part of the GEP cloud environment and not a local QGIS installation to which the result has first been downloaded.
This fits with the broader workspace model being introduced through GEP Drive.
Outputs generated by GEP processing services can remain in their associated object-storage areas and be made accessible from the user’s Drive environment. Working files, intermediate products and analytical results can then be used from cloud-hosted tools such as QGIS and notebooks alongside the user’s own project material.
The storage layer and the analysis tool do not have to be the same component. What matters for the user is that the result of one stage can become the input of the next without repeatedly downloading, reorganising and uploading the data between disconnected environments.
For this experiment the path is therefore continuous:
Sentinel-1 SLC data → SNAPPING processing → cloud-hosted interferogram collection → openEO/Xarray/Dask analytics → GeoTIFF result → QGIS in a GEP Workspace
All of these stages remain within the cloud-based GEP environment.
openEO result opened in QGIS within a GEP Workspace together with OpenStreetMap contextual information
Different tools inside the same environment
SNAPPING, openEO, QGIS and notebooks address different parts of the workflow, from operational EO production and collection-level analytics to expert inspection and further analysis.
GEP provides the common environment around these capabilities through data access, cloud storage, catalogues and workspaces, allowing the different stages to work together while keeping the data close to the processing infrastructure.
A beta test for the GEP roadmap
This remains a beta integration.
It should not be interpreted as the announcement of a generally available production openEO service on GEP today. The objective is to test how openEO can be integrated with existing GEP processing and storage capabilities and where it provides practical value for the geohazards community.
The Nayarit experiment already demonstrates an important aspect of that direction: an operational processing service can generate hundreds of gigabytes of derived EO products, those products can remain in cloud storage and become an analytical collection, an openEO workflow can operate across that collection using distributed processing, and the result can then be inspected in a familiar GIS environment without leaving the GEP cloud workspace.
That is the broader direction we want to continue exploring: connecting production, storage, analytics and expert interpretation so that users can work with increasingly large EO datasets without making data movement itself the central part of the workflow.