Building a Collaborative Process for Address Point Data for - - PowerPoint PPT Presentation

Building a Collaborative Process for Address Point Data for Minnesota

Dan Ross | Chief Geospatial Officer 1/26/2017

Information Technology for Minnesota Government | mn.gov/mnit

Build an Understanding …

• Define the stakeholders
• Who are the authoritative sources
• Counties, cities, vendors
• What state is the data in?
• Who are the users?
• Are there standards that already exist and

are being used?

• How can we share?
• What data activity is already occurring?
• Multiple efforts going on

5/22/2017 Information Technology for Minnesota Government | mn.gov/mnit 2

Stakeholders

Multiple

• State Agencies
• Counties
• Cities
• Private companies

Multiple needs to cover

• Come together to create once and use many times
• Build a common standard for the data – meets the needs of multiple agencies
• Roles and responsibilities

Use Authoritative data source wherever possible

• Initial data
• Data cleanup
• Data maintenance

5/22/2017 3

Combining multiple efforts ….

• MnDOT – Roads and Highways Deployment
• Desire to collect directly from authoritative sources
• Metropolitan Council
• Regional effort - many data sets (parcels, address

points, centerlines)

• NG9-1-1
• Statewide effort – 104 PSAP
• Moving faster than DOT so became the driver for the

local data collect

• Statewide standards

5/22/2017 4

Create a plan…

NG9-1-1 Project

• Create a plan for:
• Data intake, development, validation, normalization, aggregation,

sharing, maintenance

• Short and long term
• Consider:
• Roles and responsibility for each stage of the process
• Technology
• Architecture is important
• Make sure to align the technology with what you need to support

(e.g. applications, versus web services versus cached basemaps)

• Don’t forget security
• Resources to build/support
• Budget, human resources, in-house, vendor supported
• Don’t forget the tails
• Cost Recovery?

5/22/2017 5

Building the process

Basic Steps

• Data intake and access – multiple formats,

projections, coordinate systems

• Portal to bring in data and share back data from

stakeholders

• Validate the data
• Report back to the authoritative source
• Standardize
• Aggregate
• Share back to the community

5/22/2017 6

Recognizing the Needs of Stakeholders

• Multiple ways to

provide data

• Validated,

standardized and aggregated in a single place

• Error an potential

issues reports back to the authoritative source

• Shared back to

stakeholders and

• ther users in

multiple formats to meet a variety

• f needs
• Vision is that

some form of the data is open to all users

FAIL

GIS MSAG

NextGen 9-1-1 Database (Production Ready) ALI/MSAG Database Vendors MN Geospatial Commons Level 2 QA/QC To accommodate different systems Level 1 QA/QC Aggregator Address or street data and other statewide administrative Layers Aggregator brings statewide information together Pass Address or Street data A d d r e s s

• r

S t r e e t d a t a A d d r e s s

• r

S t r e e t d a t a Authoritative or Local Data Source Method 3 FAIL Errors Web Editor Hosted System Authoritative or Local Data Source Method 2 Authoritative or Local Data Source Method 1 Local GIS ETL

Change Request

Change Request System MnGeo or other State Agencies and makes changes or creates data as needed

MN Geospatial Commons

Possible Data Flow for Obtaining Addresses Points and Street Centerline Information

Method 1 – Requires regular extracts from authoritative data sources and incorporation of extract, transform and load processes (ETL). This will require a GIS resource or an automated process to mosaic local data together into a stands based data form Method 2 – Authoritative data sources would be provided a web editing platform that would allow editing of data in a hosted system with a standards based data model. Editing could occur either via a sketchup/redline layer or direct editing of the database with a standards based data model Method 3 – A process where the authoritative source submits change request to a change management system. This could be through a web based system and could use a sketch up/redline type of system The aggregator or some other GIS resource would be required to make the change to the GIS data Pass ETL Extract Extracts back to local systems or beyond Extract G e

• s

p a t i a l D a t a L a y e r s

MnDOT LRS

More Detailed Process

5/22/2017 Optional Tagline Goes Here | mn.gov/websiteurl 8

Data intake

5/22/2017 9

Secure Portal

• Each organization has a provided a single

IP for access

• Multiple secure logins per organization
• Each has their own organization
• Incoming and outgoing
• All geospatial data, validation reports,

scripts, standards, user guides

• Open Source

Authoritative Data and Challenges

5/22/2017 10

Standards

Standards

• Started with what others had done
• NENA, FGDC, Other States, Metropolitan Council
• Compared fields from each
• Site Structure Address Points, Street

Centerlines, ESZ, Authoritative Boundaries

• Schema definitions, examples, roles and

responsibilities

5/22/2017 11

Name Field M/C/O Type Width Definition Examples Local Address Unique Identifier ADD_ID_LOC O Text 50 The unique identification number assigned to an address by the addressing authority. 21453700, 77E45619 Subaddress Type 1 SUB_TYPE1 O Text 12 The primary type of subaddress to which the associated Subaddress Identifier applies. APARTMENT 17C, BUILDING 6, TOWER B, FLOOR 2, SUITE 1040 (subaddress type in bold) Subaddress Identifier 1 SUB_ID1 O Text 12 The primary identifier used to distinguish different subaddresses of the same type when several occur within the same structure. APARTMENT 17C, BUILDING 6, TOWER B, FLOOR 2, SUITE 1040 (subaddress identifier in bold) Subaddress Type 2 SUB_TYPE2 O Text 12 The secondary type of subaddress to which the associated Subaddress Identifier applies. APARTMENT 17C, BUILDING 6, TOWER B, FLOOR 2, SUITE 1040 (subaddress type in bold) Subaddress Identifier 2 SUB_ID2 O Text 12 The secondary identifier used to distinguish different subaddresses of the same type when several occur within the same structure. APARTMENT 17C, BUILDING 6, TOWER B, FLOOR 2, SUITE 1040 (subaddress identifier in bold) Parcel Unique Identifier PIN O Text 17 Unique state-wide parcel ID comprised of the COUNTY_CODE and Parcel

• r Property Identification

Number (PIN). 27123-7524136698 GNIS ID GNIS_ID O Text 8 The GNIS ID of the civil feature in which the address point is located. 02394269 for City of Hallock, 00659096 for Leech Lake Reservation County Code CO_CODE O Text 5 The state and county FIPS codes for the county in which the address point feature resides. 27001 for Aitkin County, 27123 for Ramsey County Residence RESIDENCE O Text 8 Address point feature has a residence or living quarters. Yes (but unsure what type), Multiple (multiple family residences or living quarters) Mailable Address MAILABLE O Text 10 Address point feature receives USPS mail delivery. Y, N, U Status STATUS O Text 10 The current operational condition of the feature. ACTIVE, PLANNED Source of Data SOURCE O Text 75 Source from whom the data provider obtained the address. Planning & Zoning, City of Alexandria Address Authority AAUTHORITY O Text 40 The name of the authority that has jurisdiction over the address of the address point feature. City of Anoka, Mdewakanton Sioux Community Editing Organization EDIT_ORG O Text 40 The organization that made the last change to the data record. Beltrami County, City of Apple Valley Comments COMMENTS C Text 254 Miscellaneous information about the feature. "House to be moved to new site in January, 2017.", "Point correctly is in more than one ESZ (exception)"

Data Validation

• Currently using Python code
• Test driving some new tools
• Validation reports for each PSAP for each

data set

• In person meeting with each PSAP to go over

reports and data issues and inconsistencies

• Authoritative Source cleans up and

updates data

• Rerun each time new data is shared
• Shooting for 98%

5/22/2017 12

Olmsted County - Address Validation Geocoding Results derived from Address Point data vs Centerline data # of NULL addresses # Unique Address Pt Geocoding Errors (unmatched) % Address Pt Geocoding Errors (unmatched) # Address Pt Geocoding Ties (Tied) % Address Pt Geocoding Ties (Tied) % Address Point Match Rate 3673 6.00% 780 1.00% 93.00% Geocoding Results derived from ALI data vs Address Point data # ALI Address Geocoding Errors (Tied) # ALI Address Geocoding Errors (Unmatched) % ALI Address Geocoding Errors % Address Point Match Rate 364 1919 7.00% 91.00% Geocoding Results derived from ALI data vs Centerline data # ALI Address Geocoding Errors (Tied) # ALI Address Geocoding Errors (Unmatched) % ALI Address Geocoding Errors % Centerline Match Rate 391 2285 10.00% 90.00%

Data normalization and Aggregation

• Individual data sets schema mapped to standard
• Data transformed into the standard
• Use of domains for common elements (e.g. City Name, Street name, prefix, etc.)
• Standardized data is aggregated (just starting this process now)
• Shared back to authoritative source and others who need it
• Data by region, web services, data download for individual county
• New tools being rolled out for online editing (started with ESZ, address point next)

5/22/2017 13

Sharing it back….

• All data to go back to the 9-1-1

community

• NG9-1-1 specific data will stay within the

9-1-1 community

• Some data – parcels, centerlines,

address points – will be opened up to all

• Data by region, web services, data

download for individual county

5/22/2017 14

Dan Ross

dan.ross@state.mn.us 651-201-2460

5/22/2017 15

Questions?