We are hiring Marco Duiker SkyGeo Delivering high resolution - - PowerPoint PPT Presentation

Marco Duiker – SkyGeo Delivering high resolution deformation maps with high performance and extensive processing

• rt

We are hiring

The InSAR concept

Acquisition 1 Acquisition 2

The InSAR concept

Acquisition 1 Acquisition 2

Centimeters Phase difference

Resulting data properties

• Movements measured with mm precision
• Measurement interval 4-35 days
• Data available from 1992
• Needs hard consistently reflecting surfaces
• Measurements are relative (in space and time)
• Pixel size from 3x3m to 7x26m
• Positional accuracy (X,Y,Z) of reflector 1-2m
• Measurement in direction of satellite

Injection monitoring

Gas pipeline monitoring

Municipality maintenance planning

The customer portal Travel Guide

• We deliver
• Mapping services: WMS(-C)
• Data services: WFS
• Processing services: WPS
• Data: csv-files
• We provide viewers with the
• Data
• Meaningful tools
• How do the OGC standards keep up with our needs?

Interesting features Explore data by fjltering Travel Guide

Hidden Gem: WMS-dimensions Trip Report

• WMS-dimensions: not only for time or elevation
• Is advertised in the Capabilities document
• Provides a way for subselecting data in the GetMap request
• (not restricted to the attribute which is used for classification)
• Eg: &DIM_PNT_QUALITY=0.76/1
• Gives all features with 0.76 <= pnt_quality <= 1

Interesting features Explore data by time slider Travel Guide

Time slider Trip Report

• Simpl subtraction of cumulative deformation on two dates
• Easily done with runtime substitution
• WMS-support:
• Vendor Specific Parameters
• Just add the start date and the end date to the request

Interesting features: multiple styles Trip Report

• Customer needs that to gain good insight in the data
• WMS-standard allows for defining multiple styles on a layer
• Different styles to:
• Stretch the colors over the values
• Select a point size
• Proves to be cumbersome:
• Lots of styles → very long mapfile, slower performance
• Not very user friendly
• Alternative
• Setting point size and stretching via sliders
• WMS support: vendor specific parameters
• Cumbersome to do in Mapserver
• (requires rewriting the mapfile or using mapscript)

Interesting features Multiple styles alternative Travel Guide

Interesting features Explore data by query (spatial or attribute) Travel Guide

Querying & downloading data Trip Report

• WFS-services provide quite a few ways to select and download

data

• Nice to do queries & display results on map
• Slow in mapserver 6, much better in mapserver 7
• Still slow on large datasets (compared to a database query)
• Have a download file (csv) of the entire dataset handy

Interesting features data processing Travel Guide

Processing data Trip Report

• WPS standard provides:
• A way to send a request for processing
• (send in data/ service url as well!)
• A way to send the response when done
• PyWPS provides an implementation for the WPS-standard
• The processing:
• (Grass plugin for generic geo-spatial processing)
• Program in Python
• Not confined to the data served via WMS/WFS
• Queuing & killing tasks: write yourself (or use pyWPS 4.0)!
• Difficult (impossible?) to get fully functional without Apache &

cgi

• Relatively easy to extend and/ or hack
• No proper support for sub-domain specific access
• (submitted a pull request which has lots of issues!)

Architecture: the requirements

• “Normal” mapping functionality + more unusual ones
• Fast (while doing 'live rendering')
• Reliable (24/7/365 & fool proof)
• Secure
• Scaleable
• Flexible with short time to market for new features
• maps data and analysis
• viewer and services (standards compliant !)

KISS-proof Architecture Trip Report

KISS-proof Architecture Trip Report

Almost Everything is file based & tied to viewer context Customers & viewer contexts

Deployment Trip Report

• Every application is in its own Docker container
• Providing its own execution environment
• Every application can be upgraded independently
• Multiple versions of same application
• Containers are managed (orchestrated)
• Docker-compose
• New servers are deployed by Fabric
• A python script sending all necessary commands over ssh

Conclusions Trip Report

• OGC-standards come a long way in providing the necessary

standards for a rich web application

• Live rendering (assisted by some smart caching) comes a long

way in giving a high-performance experience

• Docker really helps in keeping things manageable and

upgradeable