Data Gove r nanc e and Manage me nt of Cr owd Sour c e d Data - - PowerPoint PPT Presentation

Adve nture s in Cro wdso urc ing : Da ta Ma na g e me nt a nd Go ve rna nc e fo r Be tte r Cro wdso urc e d Da ta Applic a tio ns (E DC-5 We b ina r Se rie s) Ja nua ry 28, 2020

Subr at Mahapatr a

MDOT SHA Offic e o f CHART & I T S De ve lo pme nt

Data Gove r nanc e and Manage me nt of Cr

• wd Sour

c e d Data Applic ations for Be tte r T r anspor tation De c isions

2

PRE SE NT AT I ON OUT L I NE

MDOT wide Da ta Go ve rna nc e E ffo rts MDOT SHA T SMO Da ta Go ve rna nc e a nd Ma na g e me nt MDOT SHA GI S Da ta Go ve rna nc e Pla nne d Wa ze Da ta Ma na g e me nt

• MDOT

c omprise s of six busine ss units a nd a n Authority

• T

he Se c re ta ry’ s Offic e

• Sta te Hig hwa y Administra tio n
• Ma ryla nd T

ra nsit Administra tio n

• Mo to r Ve hic le Administra tio n
• Ma ryla nd Po rt Administra tio n
• Ma ryla nd Avia tio n Administra tio n
• Maryland T

ranspo rtatio n Autho rity

• Multi-mo da l o rg a niza tio na l fra me wo rk fo r

inte g ra te d tra nspo rta tio n so lutio ns

• MDOT

E xc e lle ra to rdrive s the a g e nc y with te n ta ng ib le re sults a nd pe rfo rma nc e me a sure s

ABOUTMARYL AND DOT (MDOT )

3

4

DAT A…DAT A…DAT A…

Str ate gic , T ac tic al, Ope r ational Ac tivitie s a t the

T BUs g e ne ra te s a hig hly c o mple x se t o f da ta stre a ms…

Many ways to looks at the Data Str e ams… By MODE – Hig hwa y, T

ra nsit, Air, Wa te r, Othe r…

By F UNCT ION - Pla nning , E

ng ine e ring , Ope ra tio ns, Ma inte na nc e , Pe rfo rma nc e Ma na g e me nt

By PE RF ORMANCE ARE A – Sa fe ty, Mo b ility, Asse t

Ma na g e me nt, Administra tive , Busine ss T ra nsa c tio ns

By SOURCE

• I

nte rna l vs E xte rna l Da ta

MDOT DAT A GOVE RNANCE & DAT A HUB

• Data Gove rnanc e & Data Hub Ove rvie w
• Whe re Do We Want to Be ?
• Visio n, Go a ls, a nd Ob je c tive s
• Princ iple s to Guide I

mple me nta tio n

• What Did We L

e arn?

• MDOT

’ s Curre nt Sta te

• Ga ps in Da ta Go ve rna nc e
• Re vie w o f No ta b le Pra c tic e s
• What Do We Ne e d to Do?
• De ve lo p a n MDOT

Da ta Hub

• T

a ke Ac tio n to Addre ss Ga ps in Da ta Go ve rna nc e

• How Can We Monitor and E

valuate Data Gove rnanc e Imple me ntation?

5

PRINCIPL E S & DIME NSIONS T

O GUI DE DAT A GOVE RNANCE I MPL E ME NT AT I ON

• 1. VAL

UABL E

• 2. AVAIL

ABL E

• 3. VAL

IDAT E D

• 4. SE

CURE D

• 5. CL

E AR

• 6. E

F F ICIE NT

• 7. ACCOUNT

ABL E

AASHT O Re fe re nc e : https://data.transpo rtatio n.o rg / 6

MDOTDAT A HUB – CONCE PT UAL F RAME WORK

Data Source Validation*

T SMO DAT A GOVE RNANCE & MANAGE ME NT

T r anspor tation Manage me nt Age nc ie s/ Companie s

T r ave le r s

OPE RAT IONS & DAT A COMMUNICAT IONS F RAME WORK – SYST E M OF SYST E MS

9

AT MS/ DSS

RI T I S, I MAP, T BUs Da ta Hub (s)

T r anspor tation Ne twor k

MDOT SHA T SMO DAT A GOVE RNANCE

So urc e : 2017 F HWA Da ta Busine ss Pla n – MD Pilot

11

CROWD SOURCE D DAT A APPL I CAT I ONS

• Re al time applic ations
• Syste m Mo nito ring
• I

nc ide nt Mg mt.

• AT

M/ I CM

• Ar

c hive d data applic ations

• Pe rf. E

va lua tio n

• T

SMO De c isio n-ma king

HOW MUCH T RANSPORT AT I ON DAT A I N RI T I S?

• T

ra ffic e ve nts: 690,000 re c o rds pe r da y: 0.006 Gb / da y

• T

ra ffic de te c to rs: 65,000,000 re c o rds pe r da y: 5 Gb / da y

• Pro b e ve hic le da ta : 8,700,000,000 re c o rds pe r da y: 550 Gb / da y
• CCT

V, we a the r, ra dio , e tc : No Sta ts K e pt: ? ? ? T b / da y

• UMD CAT

T se rve s a s the MDOT SHA Da ta Wa re ho use .

• RI

T I S pro vide s the da ta a na lytic s a nd visua liza tio n e ng ine s fo r MDOT SHA Ops, Pla nning a nd Pe rfo rma nc e Ma na g e me nt e ffo rts.

So ur c e : UMD RIT IS

13

E L E ME NT S OF AN E F F E CT I VE ARCHI VE

Da ta T

• ols

Doma in E xpe rtise Insig hts

F usion, Sta tistic s,

& I nte g ra tio n Ana lysis & Visua liza tio n

+ + =

So ur c e : UMD RIT IS

Ar c hiving & Distr ibute d Computing Appr

• ac he s

Cor e Module s:

• Ha do o p-c o mmo n
• HDF

S

• Ma pRe duc e
• YARN

Othe r Module s:

• Hb a se
• Zo o K

e e pe r

• I

mpa la

• Sto rm

DIST RIBUT E D COMPUT ING T r aditional Re lational Database s

MySQL Ora c le MsSQL DB2 SyBa se Po stg re SQL

“NoSQL ” and Ne wSQL

• Ha do o p a rc hite c ture s
• Mo ng o DB
• Co uc hb a se
• Ria k
• Me mc a c he d
• Re dis
• Co uc hDB
• Ha ze lc a st
• Hb a se
• Apa c he Ca ssa ndra

ARCHIVING

So ur c e : UMD RIT IS

MD & MDOT SHA GE OSPAT IAL DAT A GOVE RNANCE & MANAGE ME NT

GIS Da ta L a ye rs fo r T ra nspo rta tio n De c isio ns Sa fe ty E xisting Asse ts T ra ffic T SMO Stra te g ie s

MDOT SHA GI S Da ta Go ve rna nc e / Po lic ie s

MDOT SHA GIS E ffo r ts ar e le d by Offic e o f Planning and Pr e liminar y E ngine e r ing with suppo r t fr

• m Offic e o f Info r

matio n T e c hno lo gy

Po lic ie s, standar ds and pr

• c e dur

e s to c o lle c t, manage , disse minate , utilize , and ar c hive e nte r pr ise data and r e late d applic atio ns.

• Data De ve lopme nt
• MDOT

SHA Ge o spa tia l Da ta I nte g rity-Da ta Sub missio n Po lic y & Pro c e dure

• Ma ryla nd iMa p Da ta Sub missio n Po lic y
• Hig hwa y Pe rfo rma nc e Mo nito ring Syste m (HPMS)
• Data Shar

ing & Se c ur ity

• Do I

TAc c o unt Ma na g e me nt Po lic y

• Applic ation De ve lopme nt
• MDOT

SHA Sta nda rd We b Ma p Co nfig ura tio n in AGOL

• MDOT

SHA We b App Do c ume nta tio n in AGOL

• E

nte r pr ise GIS Data Inve ntor y

• Do c ume nte d in sta nda rdize d te mpla te

Appro a c h to Da ta Go ve rna nc e

• Guiding princ iple s
• I

de ntify whe re suc c e ss c a n q uic kly b e a c hie ve d

• I

de ntify o ppo rtunitie s fo r da ta c le a nup, pro c e ss imple me nta tio n/ do c ume nta tio n

• Org a nize by Da ta Cla ss
• Ge o spa tia l da ta
• T

a b ula r da ta

• T

a b ula r with lo c a tio n da ta

• Da ta fe e ds (e .g . CHART

inc ide nts)

• Da ta stre a ms (e .g . CAV da ta )
• Doc ume nt polic ie s, sta nda rds a nd proc e dure s for
• Da ta De ve lo pme nt
• Da ta Ma na g e me nt & Sto ra g e
• Da ta Sha ring & Se c urity
• Applic a tio n De ve lo pme nt
• Ma inta in da ta inve ntory using sta nda rdize d te mpla te

NE XT ST E PS: Da ta Go ve rna nc e fo r Wa ze type Cro wd So urc e d Da ta

• Wa ze pro vide s MDOT

va rio us da ta stre a ms thro ug h the CCP Pa rtne rship

• Ale rt da ta
• Ja ms da ta
• Munic ipa l use r indic a tor

19

Cro wd So urc e d Da ta thro ug h the Wa ze CCP Pa rtne rship

• Pa rtne rship Ag re e me nt o n Oc t 1, 2019 a nd MDOT

SHA jo ine d o ve r 100 o the r CCP pa rtne rs.

T r affic Inc ide nt De te c tion

E xpa nding g e o g ra phic c o ve ra g e , re so lutio n, a nd time line ss

• f inc ide nt de te c tio n a nd c le a ra nc e

Ac tive Cor r idor Manage me nt

• T

ra ffic Sig na ls T iming

• Ro a d We a the r Ma na g e me nt
• Wo rk Zo ne s/ Ma inte na nc e Ac tivitie s

Communic ations & Outr e ac h

• Ma jo r Pro je c ts
• Spe c ia l E

ve nts/ E me rg e nc ie s/ E va c ua tio n

Mainte nanc e / Custome r Se r vic e Re que sts

• Po tho le s, Gua rdra ils, Sig npo sts
• Ma lfunc tio ning sig na ls, lig hts a nd o the r I

T S a sse ts

20

USE OF WAZE CCP DAT A

21

HI GH-L E VE L F RAME WORK

T hr u’ E DC-5, F HWA and MDOT SHA ar e c o llabo r ating to de ve lo p a pr

• o f o f c o nc e pt

platfo r m to mainstr e am Waze data fo r

• Ops. and pe r

fo r manc e manage me nt. E stimate d c o mple tion – e nd 2020.

CONT ACT INF ORMAT ION SUBRAT MAHAPAT RA

De puty Dire c to r, T SMO & CAT S Offic e o f CHART a nd I T S De ve lo pme nt MDOT Sta te Hig hwa y Administra tio n (410) 582-5613 sma ha pa tra @ mdo t.ma ryla nd.g o v

QUE ST IONS? T HANK YOU!

10