Earth Science Oriented Data Processing Tools and Applications for - - PowerPoint PPT Presentation

▶
earth science oriented data processing tools and
SMART_READER_LITE
LIVE PREVIEW

Earth Science Oriented Data Processing Tools and Applications for - - PowerPoint PPT Presentation

Jan 11, 2023 12 likes •387 views

Earth Science Oriented Data Processing Tools and Applications for Black Sea Catchment Basin Dorian Gorgan Computer Science Department Technical University of Cluj-Napoca dorian.gorgan@cs.utcluj.ro Authors Contribution Dorian Gorgan 1 ,

slide-1
SLIDE 1

Earth Science Oriented Data Processing Tools and Applications for Black Sea Catchment Basin

Dorian Gorgan

Computer Science Department Technical University of Cluj-Napoca dorian.gorgan@cs.utcluj.ro

slide-2
SLIDE 2

Authors’ Contribution

Dorian Gorgan1, Gregory Giuliani2, Karim Abbaspour3, Ann van Griensven4, Lukasz Kokoszkiewicz5, Pierluigi Cau6, Anthony Lehmann2, Nicolas Ray2

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 2

1Technical University of Cluj-Napoca,

dorian.gorgan@cs.utcluj.ro,

2University of Geneva, (giuliani@grid.unep.ch,

anthony.lehmann@unige.ch, nicolas.ray@unige.ch),

3EAWAG, Swiss Federal Institute for Aquatic Science

and Technology, karim.abbaspour@eawag.ch,

4UNESCO-IHE Institute for Water Education, Delft,

a.vangriensven@unesco-ihe.org,

5CERN - European Organization for Nuclear Research,

Geneva, lukasz.kokoszkiewicz@cern.ch,

6Center for Advanced Studies, Research and

Development in Sardinia, pierluigi.cau@gmail.com

slide-3
SLIDE 3

enviroGRIDS Project

enviroGRIDS - Gridifying the Black Sea catchment to support its sustainable development (http://www.envirogrids.net)

Founded by the European Commission FP7 framework (Theme 6: environment), April 2009 – March 2013, 27 partners, 7.9 mil EUR.

Coordinator

University of Geneva, Switzerland

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 3

Objectives:

Develop a SDI (Spatial Data Infrastructure) targeting the Black Sea catchment region

Use new international standards to store, analyze, process, and visualize important information regarding this area

Perform distributed spatially-explicit simulations of environmental changes

slide-4
SLIDE 4

BSC-OS Portal

Single way of the user to get into the enviroGRIDS system

Exposes personalized tools for different category of users: data manager, earth science specialist, decision maker, citizen, and system administrator.

Provide applications for:

data management

hydrologic models calibration and execution

satellite image processing

report generation and visualization

virtual training center

Support interoperability between the Geospatial and Grid infrastructures

  • n security, heterogeneous data access, distributed data processing

EnviroGRIDS functionality gathers services provided by various technologies such as SWAT related modules, Collaborative Working Environment (CWE), Uniform Resource Management (URM), gProcess, ESIP, and eGLE platforms

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 4

slide-5
SLIDE 5

Portal Architecture

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 5

slide-6
SLIDE 6

Data Flow Throughout the Portal

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 6

slide-7
SLIDE 7

Specific Objectives

Link, gather, store, manage and distribute key environmental data concerning the Black Sea Catchment Basin

Large scale and high resolution distributed hydrological models

Gridification of tools and applications

Model and process huge spatial data over the Grid (e.g. hydrological models, satellite images, and maps)

Develop early warning and decision support tools at regional, national and local levels

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 7

slide-8
SLIDE 8

Hydrological Models - SWAT

SWAT (Soil Water Assessment Tool)

hydrological model

  • perates on a daily time step

used for predicting the water resources, sediment, and chemical yields in a specific watershed

Input data: weather, soil properties, topography, vegetation, and land management practices of the watershed

SWAT estimates the impact of land management practices on water quantity and quality in complex watersheds

The SWAT model must pass through a careful calibration and uncertainty analysis

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 8

slide-9
SLIDE 9

Black Sea Catchment Basin

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 9

slide-10
SLIDE 10

Grid Based Processing

Many input data required by the SWAT model -> store the data

  • n Storage Elements

The calibration process requires running a high number of iterations, each iteration consisting in a high number of simulations -> execution on different Grid Worker Nodes

Allows the calibration of the SWAT models and the execution of different scenarios based on a calibrated SWAT model on GRID infrastructures

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 10

slide-11
SLIDE 11

gSWAT Application

Support the SWAT model development in the BSC-OS Portal

Model calibration and execution over the Grid infrastructure

Project management of the hydrological model

SWAT data visualization

Project upload and download

Support interaction with visualization tools

Support interaction with scenarios development tools

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 11

slide-12
SLIDE 12

Processing Steps

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 12

slide-13
SLIDE 13

Calibration Process

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 13

GRID

Iteration (1..n simulation steps)

slide-14
SLIDE 14

gSWAT Application - Architecture

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 14

Web Portal

SWAT Calibration SWAT Scenarios

SWAT Services

Management and Execution Data Management Monitoring Grid Infrastructure (gLite Middleware) Model repository Calibration outputs repository

slide-15
SLIDE 15

gSWAT Execution Flow

Create the job script – the executable that will run on the Grid;

Create the DIANE script –specifies the input data for each task (simulation). Each task will execute one simulation. DIANE will copy the input data on the worker node prior the start of the task executable;

Start the DIANE master – for each iteration we start a new DIANE master;

Start the Grid workers using GANGA – the workers will connect to DIANE master to receive the tasks to be processed;

Monitor the execution of the tasks (simulations) – the monitoring data is retrieved and stored on the gSWAT database from where it can be visualized by the user;

Download the output data – after all the tasks (simulations) are performed the output data is retrieved and stored locally at the gSWAT server side.

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 15

slide-16
SLIDE 16

gSWAT Control Flow

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 16

gSWAT Services gSWAT GUI User gSWAT DB DIANE master Ganga Worker Worker Storage Element Monitoring component GRID gSWAT system GRID oriented Tools

slide-17
SLIDE 17

gSWAT User Interface

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 17

Project list and detailed information on the selected project

slide-18
SLIDE 18

Modify SUFI2 Calibration Parameters

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 18

slide-19
SLIDE 19

Calibration Output Visualization

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 19

slide-20
SLIDE 20

Spatial Data Visualization by BASHYT

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 20

slide-21
SLIDE 21

Graphical User Interface Development

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 21

The output is shown on the portal using widgets like maps, charts or

  • tables. They can be organized using

HTML and javascript

slide-22
SLIDE 22

Spatial Data Visualization

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 22

slide-23
SLIDE 23

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 23

Satellite Image Processing

Explore images to identify objects and give them a significance

Analysis: detect, identify, classify, measure and evaluate the significance of physical and cultural objects, their patterns and spatial relationships

Applicability in different Earth Science domains

Land cover

Air pollution

Hydrology

Ecology

etc

slide-24
SLIDE 24

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 24

Satellite Image Processing

Satellite image types used for different purposes

Landsat

MODIS

Aster

Quickbird, etc

Size: depending on the geographical region size and on the containing information, satellite images could reach a couple of Gb in size

slide-25
SLIDE 25

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 25

GreenLand Vegetation Indices

Vegetation index based image classification

NDVI (Normalized Vegetation Index)

EVI (Enhanced Vegetation Index)

IPVI (Infrared Percentage Vegetation Index)

GEMI (Global Environment Monitoring Index)

SAVI (Soil-Adjusted Vegetation Index)

Classification process - combines different bands in the Landsat satellite images in order to correctly make the classification

slide-26
SLIDE 26

GreenLand App. - Satellite Image Proc.

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 26

slide-27
SLIDE 27

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 27

GreenLand Application

Classification output (right image) based on input satellite image (left image)

slide-28
SLIDE 28

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 28

Virtual Training Center -Lessons

Videos Documents 3D Objects Sounds Other multimedia content Images 28

slide-29
SLIDE 29

Teaching Material Description

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 29

Lesson Service Grid Infrastructure Teaching materials, tutorials Process description User Data Spatial Data EO Data Operation Service Operation Complex Services Lesson patterns Lesson resources Services, operations provided through Web Service description, search, discovery and composition. Workflow based process description. Lesson description, execution, and management. Teaching materials of various topics and subjects.

slide-30
SLIDE 30

Grid Based Execution of Env. Scenarios

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 30 This lesson exemplifies interactive scenario for teacher and students.

workflow Output

Input Input

Input Executor Viewer Scheduler Editor Manager

ESIP platform

Lesson

move data at authoring time

slide-31
SLIDE 31

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 31

eGLE Tools

31

slide-32
SLIDE 32

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 32

Lesson Samples

32

slide-33
SLIDE 33

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 33

Lesson Samples

33

Authoring (WYSIWYG editor) Runtime

slide-34
SLIDE 34

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 34

Lesson Samples – GRID Oriented Tools

slide-35
SLIDE 35

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 35

Lesson Execution Monitoring

35

slide-36
SLIDE 36

Future Works

Improve the functionality and the performance of the applications

Tools and applications for the interactive development of hydrological scenarios

Interoperability between different technologies involved in the enviroGRIDS project

Include and develop new tools and applications required by the partners (e.g. RIONI River hydrological model, Vegetation development in Istanbul, etc)

Develop training materials in the domain of Earth Sciences

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 36

slide-37
SLIDE 37

EGI - European Grid Infrastructure User Forum 2011, Vilnius, Lithuania, 11-14 April 2011 37

Thanks, … Questions?

Dorian Gorgan

Computer Science Department Technical University of Cluj-Napoca dorian.gorgan@cs.utcluj.ro