APDM 6.0 ArcGIS Pipeline Data Model Peter Veenstra APDM Standing - - PowerPoint PPT Presentation

APDM 6.0

ArcGIS Pipeline Data Model

Peter Veenstra

APDM Standing Committee

Abstract

• The purpose of this workshop is to review the final release of the ArcGIS

Pipeline Data Model (APDM) version 6.0.

• Changes include;

– simplification of the data model structure, – a focus on supporting gathering systems – full support for both geometric/feature-based and event-table based implementation of the model.

• An overview of the resources available for the APDM 6.0 implementation

will be provided including a preview of the latest APDM.net website.

• An overview of how the model has been implemented in Sparx System

Enterprise Architect UML Modeling software will be provided.

• Lastly, the current state of data models and best practices available to

pipeline operators and GIS practitioners will be outlined as an aid to helping pipeline operators understand the options available for managing pipeline data within a GIS.

Introductions

• APDM Standing Committee Co-Chairs

– Tom Coolidge – ESRI Pipeline Industry Manager – Peter Veenstra – Willbros Engineering

• APDM Standing Committee

– Justin Anderson – Enbridge Houston – Jeff Allen – Coler and Colantonio – Patrick Baes – Global Information Systems – Ron Brush – New Century Software – Eric James – Colonial Pipeline – John Linehan – JP Kenny – Tracy Thorleifson – Eagle Information Management – 1 open position

• www.apdm.net
• www.esri.com/industries/pipeline/community/datamodel
• https://www.linkedin.com/groups/APDM-ArcGIS-Pipeline-Data-Model-

155824/about

Overview

• Part 1 - Changes in APDM 6.0
• Part 2 - ArcGIS Pipeline Data Model (APDM)
• Part 3 – APDM 6.0 in Enterprise Architect
• Part 4 - State of Data Models

– ArcGIS Pipeline Data Model (APDM) – PODS Relational – PODS ESRI Spatial

• Part 5 - Thoughts on Pipeline Data Models

Part 1 - Changes in APDM

• Why do we do this?
• High level changes (Change Log)(Logical)(Physical)

– Metadata Tables – Corrections – New Abstract Classes (more refined) – Relationship to LineLoop for ‘online’ features – Less ‘example’ classes – Activity and Document CrossRef – Better Site Location Tables – EventOffset Attribute

Part 1 1 of 11

Changes in APDM 6.0

• APDM is a Template – it has always been a starting point

from which more comprehensive data models can be developed

• Contrary to popular belief APDM was never intended to be

a ‘be-all-end-all’ repository of pipeline data

• It has always been a design specification for how pipeline

data is created, edited, and how that data responds to alterations/modifications to the pipeline centerline

• Optional classes have been reduced to keep the model in-

line with being template

• Changes to the core have been additive rather than

deleting, merging, splitting classes and elements

Part 1 2 of 11

APDM 6.0 – Why change?

• Represents changes to the core elements of the

data model

• Maintain compatibility with changes in core ESRI

technology – guiding principle

• Stay in sync with PODS ESRI Spatial Data Model
• Get in-step with ESRI message towards data

models and GIS as a service

– location, services, data exchange, integration – simpler template to start with – helps smaller operators and gathering companies

Part 1 3 of 11

Part 1 4 of 10

MetaData Tables

• ReferenceMode, ClassMetaData,

OnlineLocationClass – remain the same

• Add RelationshipMetaData, DomainList,

DomainMetaData, DomainClass are new

Part 1 5 of 11

• APDMFeature and APDMObject
• Event-based Abstract Classes – Tables not

Feature Classes

Abstract Classes

GlobalID EventID (pk) PipelineID TagID OBJECTID CreatedBy CreatedDate EffectiveFromDate EffectiveToDate HistoricalState <d> LastModified ModifiedBy OriginEventID ProcessFlag Remarks

[Object] APDMObject [ObjectArchive]

BeginMeasure EndMeasure BeginSeriesEventID BeginStation EndSeriesEventID EndStation Status <d> BeginOffsetAngle BeginOffsetDistance EndOffsetAngle EndOffsetDistance StationLocated <d>

CLEditResponse <d> CLValidityTolerance <d> EventOffset LineLoopEventID (fk) RouteEventID (fk)

OnlineEvent

InServiceDate InstallationDate OperationalStatus <d> SiteEventID (fk)

OnlineFacilityEvent

Measure SeriesEventID Station Point_X Point_Y Point_Z SymbolRotation

DateManufactured Grade <d> InletConnectionType <d> InletDiameter <d> InletWallThickness <d> Manufacturer <d> Material <d> PressureRating <d>

FittingEvent

BeginMeasure EndMeasure BeginSeriesEventID BeginStation EndSeriesEventID EndStation

Measure SeriesEventID Station Poiint_X Point_Y Point_Z Status <d> SymbolRotation

OnlinePointEvent

OffsetAngle OffsetDistance StationLocated <d>

OnlinePolylineEvent

Part 1 6 of 11

Relate to Lineloop

• Direct relationship from online features to

LineLoop

• Designed to support feature-based gathering

systems with no underlying stationing – have pipes related directly to a ‘lineloop’ grouping

• Keeps hierarchy without need for stationing
• If you have geometric features representing your

pipes and not stationing then you can place these features, relate them to a line and be on-your- way.

• Can implement Geometric-Networks OOTB

Part 1 7 of 11

Less Example Classes

• APDM is really a ‘design standard’
• Meets the minimum requirements for handling

transmission and also gathering systems

• Cleaner starting point
• Less documentation to maintain
• It can always be grown
• Simpler and more technology focused
• Example classes are still available in version 4.0

and 5.0 documentation via www.apdm.net

Part 1 8 of 11

Activity CrossRef

• Why do (did) we have audit tables?
• Remove all audit tables from database
• Each feature class and table relates to

ActivityCrossRef

• Intersection or M-N table between Activity and

rest of database

• Less tables, less relationship classes,
• Single source for tying/grouping/querying what

are stored in separate tables or were implicitly joined by geographic location

ParentActivityEventID (fk) ChildActivityEventID (fk)

ActivityHierarchy APDMObject

ActivityDate ActivityDescription ActivityName ActivityType <d>

Activity NonFacilityObject

ExternalDocumentEventID (fk) ActivityDocumentCrossRef

AuditObject

ClassName (fk) <d> FeatureEventID Comments CommentBy CommentDate ActivityCrossRef

Etc...

Part 1 9 of 11

Document CrossRef

• Same construct as activities
• Remove all M-N relationships between

ExternalDocument and audit tables from database

• Each feature class and table relates to

DocumentCrossRef

• Less tables, less relationship classes,
• Single source for tying/grouping/querying what

are stored in separate tables or were implicitly joined by document

ActivityDate ActivityDescription ActivityName ActivityType <d>

Activity NonFacilityObject

DocumentDescription DocumentType <d> FileName FilePath FileServer Hyperlink AltHyperlink

ExternalDocument

ExternalDocumentEventID (fk) ActivityDocumentCrossRef

AuditObject

ClassName (fk) <d> FeatureEventID FeatureAttributeName ExternalDocumentEventID (fk) ExternalDocumentReference DocumentCrossRef

Etc ...

Part 1 10 of 11

Site Location/EventOffset

• Added SitePoint, SiteLayout and SitePolygon

Tables

• Added EventOffset field for all online event

types

– Allows OOTB ArcMap ‘Add Route Event’ tool to add events but offset from pipeline centerline – Useful for showing results of inspections over time

• r multiple inspections

Part 1 11 of 11

Part II - APDM

• ArcGIS Pipeline Data Model
• Built for ESRI Geodatabase

– Leverage ESR

• Useful starting place
• Describes in detail how pipeline data respond to

centerline edits, location placement, and how editing can be performed on them

• Standing Committee of 10 operators and vendors
• Meet at PUG, GITA O&G and ESRI UC
• Website – www.apdm.net and www.esri.com

Part 2 1 of 1

Part III - APDM and Case Tools

• Visio and UML and APDM
• Alternative in Enterprise Architect Software

(www.sparxsystems.com)

– Is a Case Tool – Is ESRI Business Partner – Is ESRI’s preferred choice – Uses XML Workspace Import to build schema – not additive (need XML schema diff) – Scripts

• for validation,
• for organization
• to import an existing Visio and XML Workspace into EA

Part 3 1 of 2

Part III – APDM 6.0 in Enterprise Architect

• Base Functionality - toolbar, diagrams, packages,

hyperlinks, documentation

• Import from Visio UML and from ArcGIS XML

WorkspaceOrganization of Model

• Validation of output XML Workspace before

import into ArcCatalog

• Modularization
• Base Line (Documentation, Delta)
• Multiple Inheritance (Abstract Class Hierarchy)
• Script and Query Engine

Part 3 2 of 2

Part IV – State of Pipeline Data Models

• ArcGIS Pipeline Data Model
• PODS Relational
• PODS ESRI Spatial
• Others (GDI, PODS Open Spatial)

APDM and PODS Committee members talk and discuss and collaborate often. Each model serves a purpose and has a place in the industry. It is not a competition!!!

Part 4 1 of 12

The Players

• PODS (Pipeline Open Data Standard)

– Relational database model. – Tabular and spatial data are managed as two systems. – SQL driven. – Requires GIS software although by design GIS agnostic, but optimized for ESRI. – Standards organization. Active User Community.

• APDM (ArcGIS Pipeline Data Model)

– ESRI Geodatabase model. – Tabular and spatial data are managed by one system. – Geodatabase provides built in versioning (long transactions), replication, archiving. – Requires ESRI ArcGIS software. Works with desktop, server, web and mobile software OOTB. – Template Model (build as needed). Quasi-active user community.

• PODS ESRI Spatial (Geodatabase version of PODS Relational – optimized

for Geodatabase)

Part 4 2 of 12

RDBMS PODS Relational GIS Data Spatial Enabled PODS Relational

Part 4 3 of 12

RDBMS

Part 4 4 of 12

To decide on a model, ask some questions

• What GIS are you using?
• Is your organization standards driven?
• Focused on the Model or focused on the

business?

• How do you implement your centerline and/or

pipeline hierarchy?

• Do you have any GIS software in place?
• What are your business processes?

Part 4 5 of 12

What GIS are you using?

• If using ESRI technologies then choice might be

weighted toward a Geodatabase type model …

– PODS ESRI Spatial – ArcGIS Pipeline Data Model

• If not particular about GIS software then choice

might be …

– PODS Relational – PODS Open Spatial – Geodatabase-type model

Part 4 6 of 12

Organization Standards Driven

• If business unit is standards-driven then

choice would lean towards PODS ….

– PODS Relational – PODS ESRI Spatial

• If business unit demands flexibility and agility
• ver standards OR do not want weight of the

320+ PODS tables then lean towards …

– APDM – Modified PODS ESRI Spatial

Part 4 7 of 12

Data Model or Business Focus

• If the choice of a data model drives the business

then choose …

– PODS Relational – PODS ESRI Spatial – APDM

• If the need for the data model is driven by the

business … (Integrity Management, Operations, Engineering)

– APDM – Modified PODS ESRI Spatial

Part 4 8 of 12

Centerline Hierarchy

• If the pipeline is modeled by line-route-series

and requires continuous measure and engineering stationing then choose …

– PODS Relational – PODS ESRI Spatial

• If the pipeline is modeled by only one

‘reference’ mode or no reference mode then choose …

– APDM

Part 4 9 of 12

Software

• If the company has specific software for PODS

relational or has a team of excellent SQL developers or excellent DBA support then choose …

– PODS Relational

• If the company requires OOTB tools to work with

GIS software or as a part of GIS software then choose …

– APDM – PODS ESRI Spatial

Part 4 10 of 12

What are your business processes?

• GIS is the system of record for location

Part 4 11 of 12

Decision Matrix

Criteria/Data Model PODS Relational PODS ESRI Spatial ArcGIS Pipeline Data Model

Geographic Information System (ESRI?) SQL Only. Requires integration w/ GIS. Yes Yes Require Standard Yes (Standard out of the box) Yes (Standard out of the box) No (Template Model – Framework) Data Model driven by Business Maybe (Is a standard, can be modified in additive fashion) Maybe (Is a standard, can be modified in additive fashion) Yes (Customizable – requires work but provides flexibility) Business driven by Data Model Yes (Data model requires specific workflows to implement and support.) Yes (Data model requires specific workflows to implement and support.) No (Template. Data model is created to suit business) Require measure and stationing Yes Yes No (Does not require but can support both) Only one measurement system No No Yes Customized Software Required Sort of … No No

Part 4 12 of 12

Part V – Data Models

• Who needs them?
• How should we use them?
• Is it time to re-think how they are used?

Part 5 1 of 8

The need for data models

• Why do we have them?
• Why has so much energy been put to them?
• Relational databases are great for OTP
• What are noSQL databases bringing to the table?
• How do exchange mechanisms affect us?

– JSON, GeoJSON, XML

• How can we incorporate the use of Tags?
• Is everything going to become coordinate-driven?
• Distributed Services
• Information Exchange Standards

Part 5 2 of 8

Choosing a Data Model

• The business drives the model, not the other

way around.

• Systems are going to be connected
• ESRI platform is focusing in the cloud
• ArcGIS Online is about services and the

consumption of them

• GIS is the system of record for location and

the hub that brings in all the other systems

Part 5 3 of 8

Part 5 4 of 8

DMS

Documents Data

WMS

Work Inspections Tracking

Conditions

Anomalies Tracking Analysis Prediction

Engineering

Project LifeCycle Design Installation Fabrication Life-of-Asset Records

Inspections

Planning Routing Tracking Auditing

Location Data Model

LOCATION is everything!

APDM – last bits and next steps …

• Publish the web site to APDM.NET
• Finish the core document
• Watch and see what ESRI is going to do next

Part 5 5 of 8

APDM Committee Volunteers and Contributors Through the Years

Part 5 6 of 8

Jeff Allen, Justin Anderson, Doug Asay, John Alsup, Patrick Baes, Rob Brook, Ron Brush, Brian Boulmay, Lynn Crouse, Chris Elmer, Dave Frye, Tom Gilmour, Ken Greer, Benny Guo, Scott Hills, Luke Hutmacher, Eric James, Janette Jenson, Mike Kallas, Mike King, Theo Lawrence, John Linehan, Maggie Mabrey, Tom Marcotte, Greg McCool, Rob McElroy, Bill Meehan, Carl Meinke, Todd Murphy, Mary Muse, Buddy Nagel, David Nemeth, Ted Peay, Lane Powell, Jeff Puuri, Debra Rohrer, Andrew Saje, Cindy Salas, Rex Shrunk, Colby Smith, Jay Smith, John Spangler, Fred Spickler, Tracy Thorleifson, Peter Veenstra, Troy Walda, Mark Warner, Pamela West, Ed Wiegele, Craig Wilder, Danika Yeager, Chad Zamarin, Andrew Zolnai

Cloud

A New Application for Desktop

Pro

ArcGIS

• Improved User Experience
• Very Fast
• Combined 2D and 3D
• Powerful Analysis
• Multiple Layouts

. . . Tightly Integrated with Web GIS

ArcGIS Pro Desktop ArcMap Desktop

Files DBMS Server

Part 5 7 of 8

2014 Esri UC Technical Workshop

Part 5 8 of 8

Thank you!

Questions?

tom.coolidge@esri.com – www.esri.com peter.veenstra@willbros.com – www.apdm.net

https://www.linkedin.com/groups/APDM-ArcGIS-Pipeline-Data-Model-155824/about

That’s all folks!