Experimental Federated Geoprocessing Cloud Services for Earth - - PowerPoint PPT Presentation

Experimental Federated Geoprocessing Cloud Services for Earth Science Community

• H. Astsatryan, A. Hayrapetyan, W. Narsisian

Institute for Informatics and Automation Problems, National Academy of Sciences of the Republic of Armenia

Institute for Informatics and Automation Problems

• f the National Academy of Sciences of Armenia (IIAP)

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

ü Founded in 1957 as the

• nly state supported

structure in Armenia for software, hardware and brainware technologies ü Operates as a State Non- Profit Organization ü Staff >150 ü More at http://iap.sci.am IIAP in a national level coordinates the e-infrastructures of the Academic Scientific Research Computer Network of Armenia (ASNET-AM National Research and Education network) a and the Armenian National Grid Initiative (ArmNGI). ü ASNET-AM provides network connectivity and services to more than 60

• rganizations in 6 cities of Armenia

ü ArmNGI is a common resource (more than 500 cores) of the scientific and academic community and represents the advanced infrastructure for the scientific research, the application of new technologies

Earth Science Community in Armenia

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Due to its intensive data processing, the multidisciplinary Earth Science communities in Armenia needs large-scale resources for different calculations, and an efficient storage to keep their data and meta-data, such as Five observational instruments are installed in the Byurakan Astrophysical Observatory on the picturesque southern slope

• f the mountain Aragatz. At the observatory the larger ones

being 2.6 m Cassegrain telescope and 1m Schmidt telescope. Implementation of a mesoscale Weather Research and Forecasting model for the territory of Armenia with fine spatial resolution, which is a big and complex application that requires lot of different input and output data National spatial data infrastructure is a common framework enabling the efficient aquisition, distribution and use of geospatial information.

Earth Science Community in Armenia: Needs and Challenges

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Challenges ü To implement the the international standards (such as Open Geospatial Consortium) that cater that allow interoperability between communities that previously were not able to easily communicate. ü To develop cloud computing based workflows in geospatial sciences consist

• f data storage and processing, as well as simulation and modeling.

ü Effective Exploitation of HPC resources for earth Science community. Needs ü To support big geospatial data analysis by building a cloud-enabled scientific workflow platform, such as processing of high-resolution time series satellite images typically requires a large amount of computational resources and time. ü To support Earth science data tackling by providing Cloud Storage: ü Single Sign-On for the community enabling to gain access to multiple resources by authenticating only once

Experimental Federated Geoprocessing Cloud Services (a)

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Processing of satellite imagery is a science of bringing measurements, units to the satellite image pixel, which are commonly found in terms of indices (vegetation indices /VI/ especially), some more physically based information can be derived and units are then starting to appear. The suggested environment used for the calculation of a) Single vegetation among 12 vegetation indices.

Experimental Federated Geoprocessing Cloud Services (a)

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

a) Single vegetation from12 vegetation indices.

Experimental Federated Geoprocessing Cloud Services (b)

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

b) Workflow as a combination of vegetation indices.

Experimental Federated Geoprocessing Cloud Services (c)

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

c) Times Series analyzes - In the front end, the user may choose an appropriate scene from the available local data repository by indicating the season and the required time period

!

Experimental Federated Geoprocessing Cloud Services

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Geoprocessing Services

Single index Combination of indices Time series analyzes

Experimental Federated Geoprocessing Cloud Services

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Local Cloud

The Web Processing Service standard is used to handle the requests from and the responses to the services. The service trader module receives the requested information from the website and decides the target platform (local, cloud)

Geoprocessing Services

Single index Combination of indices Time series analyzes

Cloud Storage Implementations and Studies

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

A Federated Cloud Platform in the BSEC Region based on OpenNebula Zone (oZone), which is a group of physical hosts under the control a single OpenNebula instance

• H. Astsatryan, A. Hayrapetyan, W. Narsisian, V. Sahakyan, Yu. Shoukourian, G.

Neagu, and A. Stanciu, Environmental Science Federated Cloud Platform in the BSEC Region, International Journal of Scientific & Engineering Research, vol. 1, no.1, pp. 1130-1133, January 2014, ISSN 2229-5518.

Federated and interoperable OpenStack clouds Synnefo: A Complete Cloud Stack over Ganeti

Experimental Federated Geoprocessing Cloud Services

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

GRASS GIS

Local Cloud

Geoprocessing Services

Single index Combination of indices Time series analyzes

An effective algorithm for parallelization within the single multicore node has been developed in order to speed up image processing.

• H. Astsatryan, W. Narsisian, A. Hayrapetyan, A. Saribekyan, Sh. Asmaryan, V. Muradyan, Y. Guigoz,
• G. Giuliani, N. Ray, An Interoperable Web Portal for Parallel Geoprocessing of Satellite Image

Indices, Springer Earth Science Informatics, 2014, DOI: 10.1007/s12145-014-0165-3.

Cloud Storage Service

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

A cloud storage, based on Synnefo/Pithos and uses a CEPH OSD storage system. ü Omit a big download/upload time, because it is in the same network. ü Data security with an encryption mechanism. ü Prevent data loose, multiple copy for the data. ü Data sharing.

Single sign on for Cloud Storage & Geoprocessing Services

EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Centralized place to have an access to all services. The unified service keeps the identity information of the community members in one centralized place and provides all these different components to the Earth Science Community. Single Sign in Cloud storage Geoprocessing services Shibboleth is among the world's most widely deployed federated identity solutions, connecting users to applications both within and between organizations.