Correction Service: Design and Implementation Issues Donato Summa, - - PowerPoint PPT Presentation

On the Development of a CSPA Error Correction Service: Design and Implementation Issues

Donato Summa, Monica Scannapieco, Marco Silipo, Diego Zardetto, Mauro Bruno Istat, Italy Istituto Nazionale di Statistica – ISTAT

The CSPA concept

• CSPA (Common Statistical Production Architecture)

is a project promoted by UNECE HLG

• provides a template architecture

for official statistics

• Final goal: standardized & reusable services
• Begun in 2013, now CSPA 1.1

(http://www1.unece.org/stat/platform/display/CSPA/Common+Statistical+ Production+Architecture+Home)

2

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

The CSPA concept

CSPA is based on Service Oriented Architecture (SOA) and includes some specifications intended to define interfaces of services in a standard way with a focus on service inputs and outputs. Statistical services are self-contained and can be reused by a number of business processes (either within or across statistical organizations).

3

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

Our work in CSPA 2014

In CSPA 2014 project there were 10 CSPA compliant services developed. Istat in particular developed the Error correction service by :

– Defining the CSPA service definition – Defining the CSPA service specification – Implementing the service

4

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service definition

It is a conceptual-level definition of the service where are described:

• the principal business

functionalities

• the inputs and outputs

according to GSIM

5

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

Name Level GSBPM Business Function Outcomes Restrictions GSIM Inputs GSIM Outputs Service dependencies Process Method

CSPA Error Correction service specification

6

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

It is a logical-level specification

• f the service in which:
• some design issues are

addressed with respect to the methods of the service (e.g. how to invoke such methods)

• GSIM inputs and outputs

implementation is specified according to a defined “logical” model (e.g. SDMX

• r DDI)

Name Protocol for invoking the Service Service location: Parameters passing mode: List of parameters Input Messages Output Messages Adopted methodology

CSPA Error Correction issues & solutions

The development of this service was much more than a technological exercise. Issues emerged on :

• the format/model of I/O data
• JSON Table schema
• the protocol and on the interface to implement
• REST Web Service

The final output is compatible with the solutions provided by the AWG to such issues

7

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service implementation

• Once a formal Service Specification is carried
• ut, then you can produce several Service

Implementations of it in order to provide the business function to the community.

• Normally it is possible to use different ICT

technologies to wrap different statistical techniques.

• We decided to use the “rspa” R package as a

tool able to implement the Error Correction task although any other could have been used.

8

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service implementation

• We have wrapped the R script into a RestFul

web service running on a node.js infrastructure, powered by the Restify library.

• Such a wrapper is specified by a file named

service.yaml, in which you can find all service input and output parameters with their types and all other associated metadata, and the syntax for running the wrapped tool.

9

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service how it works

In order to invoke the service a client must make an HTTP POST request to the server, by using any kind of client- side technology such as curl or a web browser but also including another CSPA compliant service in a chain fashion.

10

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service how it works

The content of the POST request consists of a JSON structure which contains all the actual paths or links to the files

• n which the tool will

work, namely all input and output parameters.

11

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service how it works

Once the request is made, a process is spawn on the server to handle asynchronously the execution in a non- blocking manner and a job identifier is returned to the client.

12

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service how it works

At any time a client can poll the status of the job by sending additional GET requests for the returned job id.

13

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA Error Correction service how it works

When the job status is ‘finished’ the results are stored on the server and made available to the client, which can

• btain them by making a

GET request for each produced output resource.

14

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

CSPA architecture and technologies

15

Each NSI or organization that implements a CSPA compliant architecture can be both a service consumer or provider

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015

Conclusion

• Future perspective

– Integration with NSIs environment – Services wrapping “standard” functionalities in order to maximize reuse – Off-the-shelf CSPA services

• CSPA 2015: Develop and implement CSPA

services in full production during 2015

– Istat: Coding service

16

Donato Summa, NTTS 2015, Session 6C - Enterprise architecture and integration, Brussels, 10-12/03/2015