Building a Collaborative Process for Address Point Data for - PowerPoint PPT Presentation

Building a Collaborative Process for Address Point Data for Minnesota Dan Ross | Chief Geospatial Officer 1/26/2017 Information Technology for Minnesota Government | mn.gov/mnit

Build an Understanding … • Define the stakeholders • Who are the authoritative sources • Counties, cities, vendors • What state is the data in? • Who are the users? • Are there standards that already exist and are being used? • How can we share? • What data activity is already occurring? • Multiple efforts going on 5/22/2017 Information Technology for Minnesota Government | mn.gov/mnit 2

Stakeholders Multiple • State Agencies • Counties • Cities • Private companies Multiple needs to cover • Come together to create once and use many times • Build a common standard for the data – meets the needs of multiple agencies • Roles and responsibilities Use Authoritative data source wherever possible • Initial data • Data cleanup • Data maintenance 5/22/2017 3

Combining multiple efforts …. • MnDOT – Roads and Highways Deployment • Desire to collect directly from authoritative sources • Metropolitan Council • Regional effort - many data sets (parcels, address points, centerlines) • NG9-1-1 • Statewide effort – 104 PSAP • Moving faster than DOT so became the driver for the local data collect • Statewide standards 5/22/2017 4

Create a plan… NG9-1-1 Project • Create a plan for: • Data intake, development, validation, normalization, aggregation, sharing, maintenance • Short and long term • Consider: • Roles and responsibility for each stage of the process • Technology • Architecture is important • Make sure to align the technology with what you need to support (e.g. applications, versus web services versus cached basemaps) • Don’t forget security • Resources to build/support • Budget, human resources, in-house, vendor supported • Don’t forget the tails • Cost Recovery? 5/22/2017 5

Building the process Basic Steps • Data intake and access – multiple formats, projections, coordinate systems • Portal to bring in data and share back data from stakeholders • Validate the data • Report back to the authoritative source • Standardize • Aggregate • Share back to the community 5/22/2017 6

Recognizing Possible Data Flow for Obtaining Addresses Points and Street Centerline Information the Needs of Stakeholders Address or street data and other statewide administrative Layers Extracts back to local • Multiple ways to MnGeo or other State systems or beyond Agencies s r e provide data a y L t a a D l a t i a p o s e G • Validated, MN Geospatial Commons ETL standardized and Extract Extract ETL aggregated in a A d d single place Local GIS r e s s Authoritative or Local o r Pass S Data Source t r e • Error an potential Method 1 e t d a t a issues reports Aggregator brings statewide information back to the together ALI/MSAG MSAG Database Vendors authoritative Pass MnDOT LRS Address or Street data source Authoritative or Local Web Editor Level 2 QA/QC GIS and makes Level 1 QA/QC • Shared back to Data Source Hosted System To accommodate Aggregator changes or creates Method 2 different systems a t data as needed NextGen 9-1-1 a stakeholders and d Database t e e r (Production Ready) t other users in S r o MN Geospatial s s Commons multiple formats e r d d A to meet a variety Method 1 – Requires regular extracts from authoritative data sources and Change Request incorporation of extract, transform and load processes (ETL). This will require of needs a GIS resource or an automated process to mosaic local data together into a FAIL FAIL stands based data form Change Authoritative or Local • Vision is that Request Method 2 – Authoritative data sources would be provided a web editing Data Source System Method 3 platform that would allow editing of data in a hosted system with a standards some form of the based data model. Editing could occur either via a sketchup/redline layer or direct editing of the database with a standards based data model data is open to Method 3 – A process where the authoritative source submits change all users request to a change management system. This could be through a web based system and could use a sketch up/redline type of system The aggregator or some other GIS resource would be required to make the change to the GIS data Errors

More Detailed Process 5/22/2017 Optional Tagline Goes Here | mn.gov/websiteurl 8

Data intake Secure Portal • Each organization has a provided a single IP for access • Multiple secure logins per organization • Each has their own organization • Incoming and outgoing • All geospatial data, validation reports, scripts, standards, user guides • Open Source 5/22/2017 9

Authoritative Data and Challenges 5/22/2017 10

Standards Name Field M/C/O Type Width Definition Examples Local ADD_ID_LOC O Text 50 The unique identification 21453700, 77E45619 Address number assigned to an Unique address by the addressing Standards Identifier authority. Subaddress SUB_TYPE1 O Text 12 The primary type of APARTMENT 17C, BUILDING Type 1 subaddress to which the 6, TOWER B, FLOOR 2, associated Subaddress SUITE 1040 (subaddress type Identifier applies. in bold) Subaddress SUB_ID1 O Text 12 The primary identifier used APARTMENT 17C, BUILDING Identifier 1 to distinguish different 6, TOWER B, FLOOR 2, • Started with what others had done subaddresses of the same SUITE 1040 (subaddress type when several occur identifier in bold) within the same structure. Subaddress SUB_TYPE2 O Text 12 The secondary type of APARTMENT 17C, BUILDING Type 2 subaddress to which the 6, TOWER B, FLOOR 2, associated Subaddress SUITE 1040 (subaddress type • NENA, FGDC, Other States, Metropolitan Council Identifier applies. in bold) Subaddress SUB_ID2 O Text 12 The secondary identifier APARTMENT 17C, BUILDING Identifier 2 used to distinguish 6, TOWER B, FLOOR 2, different subaddresses of SUITE 1040 (subaddress the same type when identifier in bold) several occur within the same structure. • Compared fields from each Parcel PIN O Text 17 Unique state-wide parcel ID 27123-7524136698 Unique comprised of the Identifier COUNTY_CODE and Parcel or Property Identification Number (PIN). GNIS ID GNIS_ID O Text 8 The GNIS ID of the civil 02394269 for City of feature in which the Hallock, 00659096 for Leech • Site Structure Address Points, Street address point is located. Lake Reservation County Code CO_CODE O Text 5 The state and county FIPS 27001 for Aitkin County, codes for the county in 27123 for Ramsey County which the address point feature resides. Centerlines, ESZ, Authoritative Boundaries Residence RESIDENCE O Text 8 Address point feature has a Yes (but unsure what type), residence or living quarters. Multiple (multiple family residences or living quarters) Mailable MAILABLE O Text 10 Address point feature Y, N, U Address receives USPS mail delivery. Status STATUS O Text 10 The current operational ACTIVE, PLANNED condition of the feature. • Schema definitions, examples, roles and Source of SOURCE O Text 75 Source from whom the Planning & Zoning, City of Data data provider obtained the Alexandria address. Address AAUTHORITY O Text 40 The name of the authority City of Anoka, Mdewakanton Authority that has jurisdiction over Sioux Community responsibilities the address of the address point feature. Editing EDIT_ORG O Text 40 The organization that made Beltrami County, City of Organization the last change to the data Apple Valley record. Comments COMMENTS C Text 254 Miscellaneous information "House to be moved to new about the feature. site in January, 2017.", "Point correctly is in more than one ESZ (exception)" 5/22/2017 11

Data Validation • Currently using Python code Olmsted County - Address Validation Geocoding Results derived from Address Point data vs Centerline data • Test driving some new tools # of NULL # Unique % Address Pt # Address Pt % Address Pt % Address addresses Address Pt Geocoding Geocoding Ties Geocoding Ties Point Match Geocoding Errors (Tied) (Tied) Rate • Validation reports for each PSAP for each Errors (unmatched) (unmatched) data set 0 3673 6.00% 780 1.00% 93.00% • In person meeting with each PSAP to go over Geocoding Results derived from ALI data vs Address Point data # ALI Address # ALI Address % ALI Address % Address reports and data issues and inconsistencies Geocoding Geocoding Geocoding Point Match Errors (Tied) Errors Errors Rate (Unmatched) • Authoritative Source cleans up and 364 1919 7.00% 91.00% updates data Geocoding Results derived from ALI data vs Centerline data # ALI Address # ALI Address % ALI Address % Centerline Geocoding Geocoding Geocoding Match Rate • Rerun each time new data is shared Errors (Tied) Errors Errors (Unmatched) 391 2285 10.00% 90.00% • Shooting for 98% 5/22/2017 12