Towards Categorical Metadata Alternative 3 Tasks for Unreduced - PowerPoint PPT Presentation

� Towards Categorical Metadata C. Grainger slides Status Quo Towards Categorical Metadata Alternative 3 Tasks for Unreduced Climate Observations Functions Interfaces References Colton Grainger University of Colorado Boulder July 31, 2019

Towards Categorical Metadata unreduced data from ∼ 100 Tb collection of images (ACRE) C. Grainger marine logbooks 1870–1950 Status Quo Alternative 3 Tasks Functions Interfaces References land stations 1870–1930 Indian Daily Weather Reports, Todd Folios, etc.

Towards Categorical Example (reduction of 6Mb image to a 2Kb time series) Metadata C. Grainger Status Quo Alternative 3 Tasks Functions Interfaces References Figure 1: 24 hours on the USS Jeannette (P. Brohan)

Towards Categorical Metadata url C. Grainger twitter.com/NCAR_RDA/status/1144058111654711296 Status Quo qrcode Alternative Example 3 Tasks Functions Interfaces • 2 6 bytes ∼ 70 ASCII characters References • 2 12 bytes ∼ 441 × 441 pixels • a qrcode reduces to a url $ exiftool brohan_qrcode.png File Size : 2.7 kB Image Size : 441x441 MIME Type : image/png Figure 2: CISL Seminar , P. Brohan (2019)

Towards Categorical metadata Metadata C. Grainger {’key:value’ for key in <schema>} Status Quo what available metadata has low noise-to-signal ratio? Alternative 3 Tasks Functions Interfaces $ exiftool Idaho-BB-0034.JPG References File Size : 4.6 MB Image Size : 3744x5616 MIME Type : image/jpeg arc archive doc document img image obs observation plt platform Figure 3: May 1944, Idaho (BB-42)

� � � � � � Towards Categorical • Roughly, an image in the category img approximates a Metadata C. Grainger time-series σ : [ t 0 , t 1 ] → S , where S is the state space of meteorological variables. Status Quo • An observation is ρ ( t ) evaluated at a time t . Alternative 3 Tasks • A platform in the category plt approximates a time-series Functions Interfaces λ : [ t 0 , t 1 ] → M , where M is a differentiable manifold. References arc archive dependencies doc document arc plt img image ∃ ! obs observation � � plt platform doc update � � obs � � img

Towards Categorical Metadata C. Grainger Status Quo Alternative 3 Tasks Functions 1 Gather images into (at least) one repository . Interfaces References 2 Establish a common description framework for image metadata. 3 Provide bulk, programmatic access to image subsets.

Towards Categorical Metadata C. Grainger Status Quo Alternative 3 Tasks Functions Interfaces repo MySQL schema with Makefile. assign_uuid() . References framework Agnostic ingest scripts. Bundle metadata. access rda.ucar.edu/i/<uuid>

Towards Categorical Metadata C. Grainger Status Quo Alternative 3 Tasks Functions Interfaces repo Conflict resolution for assign-uuid() . References framework RELAX NG validator. access rda.ucar.edu/i/<query>

Towards Categorical Metadata C. Grainger Status Quo Alternative 3 Tasks Functions Interfaces References Figure 4: git repo