Tech Tour Winning Technology Roadmaps Sig Knapstad Cutting Edge sk@ceag.com 1
2
Avid Nexis E5 3
Tech Tour Spectra BlackPearl 4
Simplified Overview T950 5
6
Tech Tour Winning Technology Roadmaps • What does your storage landscape look like? • “Do not delete anything” is the new mantra • 30% more storage companies than last year 7
ZETA-WHAT?
If you ask Brian or Zeke… • Requirements: • Storage Considerations-speed-cost-capacity • Data Considerations-business value-recall probability-retention • Need to match the storage requirements to the storage resource • This has become more complex as new forms of storage have been introduced 9
Tech Tour Storage Architectures Available Today When to choose Object over SAN or NAS? 10
Tech Tour Block Storage • Files are split into evenly sized blocks of data, each with its own address • No additional information (metadata) • The client is loaded with specialized software that is responsible for presenting the local file-system • Everything is handled and controlled by the SAN software • SAN file-system • Ideal for high performance • The file system is managed by the client • Limited scalability 11
Tech Tour File Storage (NAS) • Based on a traditional file system comprising files organized in hierarchical directories and subdirectories • File level access to storage, as opposed to block level • Storage exposed as a network file system • NFS or CIFS/SMB • Breaks down at large scale due to the overhead of the underlying file system that must be maintained • The file system is managed by the storage server not the client • NAS system has to manage user privileges 12
Tech Tour Object Storage • Object storage, by contrast, doesn’t split files up into raw blocks of data. • Units of storage are called objects • Objects can be grouped in buckets, but are otherwise stored in a flat address space • Every object contains three things: • The data itself • Metadata • A globally unique identifier (Applications identify the object via this ID) 13
Tech Tour Object Storage • Good for unstructured data • No real file system is needed here • Unlimited scalability • Geographic reach (access anywhere) • Direct integration (via rest api) • Gateway (allows for authentication) • Single namespace • Built-in data resilience • Extensible and flexible metadata tagging • High throughput but doesn’t perform well for low-latency, I/O intensive workloads 14
Out with the old, in with the new? Not so fast- • Both Block and NAS methods for storing data worked fine for years. • So why is there a need for another concept, Object? • Both Block and NAS need to implement functionality for user’s access rights allowing them to make changes to the data. • What we now see is that much of the data being archived is immured unstructured data. Content or materiel that will never be changed again. This is where Object storage shines.
Does it ever rain in the cloud? What if something goes wrong?
Protection: losing data is like death and taxes • Parity-error correction calculated via an algorithm/RAID-overhead as much as 50% • Replication-creating multiple copies in your storage system • Erasure Coding—data is broken into fragments that are expanded and encoded with a configurable number of redundant pieces of data and stored across different locations. Efficient, usually around 20% overhead • Recovery: • With EC, the bigger the faster-the whole cluster is involved in the recovery and only recovers the data it needs • RAID or NAS/SAN-bottlenecked through a single controller
Nine lives… • Drive fail probability 1/1000 • EC is described as m:n, with m data and n parity • So an EC of 10:2, 12 drives yields a probability of losing data (losing 3 drives) as one in a billion, so we have a durability of… • “nine nines” • 99.9999999% • Amazon’s durability is “eleven nines”, or 99.999999999%
ACTIVE ARCHIVE GET THE RIGHT DATA IN THE RIGHT PLACE • Lots of storage choices: Tape, Cloud, Object Store, Moving data from one type of storage to another while maintaining visibility • Automatically optimizes where data is stored to reduce infrastructure costs and management overhead, minimizing silos and maximizing utilization • Enhance business continuance through integrated backups, data replication, and multiple disaster recovery strategies • Optimizes storage infrastructure and makes informed choices about expansion
Tech Tour “One Ring to Rule Them All” Asset Management Layer 20
Industry Trends • Your data may be here, over there, up there or down here • In addition to having a MAM or PAM control the movement of data, there will be new tools emerging this Fall from several manufacturers that are sophisticated Global Data Movers. • DMs will cover everything from Ingest to Tiering, Migration and Archiving to any storage-DISK- TAPE-WAN-CLOUD • Index where the data is on any storage • Smart Analytics to clarify storage spending, utilization and duplication • Ability to search for assets • POLICY BASED (not always a good thing), ability to quarantine files from migration
Tech Tour “One Ring to Rule Them All” DATA MANAGEMENT FABRIC-GDM 22
Industry Trends-Smarter Data Management • Eliminate storage silos: globally manage all files across otherwise incompatible storage silos. Global Namespace creates a unified view across the entire storage environment simultaneously connecting all files regardless of vendor, platform and protocol. • File migrations are automated across all storage with no disruption to users or applications. Intelligent data movement accelerates applications by ensuring files are always where they need to be, when they need to be there. User-defined policies ensure the hot and warm files are always on the right storage tier and cold data is on the lowest cost storage, on-premises or in the cloud. • Data Insights delivers actionable intelligence for predictive, real-time analytics, and verification & reporting that facilitate storage planning and optimize application performance.
Industry Trends-Smarter Data Management Extract, auto-classification, tag, organize & manage Global queries across all storage Lifecycle management, QoS, & capacity optimization Automated policies based on schedule and triggers Data Insights & reporting Migrate, I/O balancing, tiering DYNAMIC ANALYTICS METADATA WORKFLOW VISUALIZATION QUERY ENGINE DATA MOVER ENGINE ENGINE ENGINE ENGINE ENGINE TAPE WAN CLOUD DISK
Metadata Driven Archives Image courtesy of Strongbox 25
Brand Convergence 26
REpresentational State Transfer 27
How do we move data into Object Storage?
30
San Francisco 49ers REST API Asset Management
• Now deploying APIs for Google Cloud and Azure • Video, Speech-to-text, Photo recognition, AI • Rest API at AT&T Park for OnPrem Object Storage • Game changer for Cutting Edge and our clients
Data over the WAN
Multi-Site Media Project Spring 2018 37
Project Overview • Goal: Increase collaboration between existing Stadium location and new office facility located 5 miles away in Oakland • New site is linked to the Stadium by a high-bandwidth data circuit enabling multi-site workflows where editors can access media at either location • Upgrade to Stadium Infrastructure included: • New versions of Media Composer and Interplay • Replace ISIS with new 240TB Avid Nexis Storage • Add Transcoding (Content Agent) • Improve Browsing (Avid MediaCentral UX) • Upgraded Archive • Add Jack London Square Systems: • Add Edit Workstations and Browse/Edit Laptops • Add More Avid Nexis Storage(240TB) • Add Transcoding (Content Agent) • Enable “bi-directional” workflow between Jack London Square & Stadium 38
10Gb Fibre 39
Workflow Overview • Media Ingest from either location • Interplay Check-In from either location • Transcode to DNxHD 145 • Browsing using Interplay or MediaCentral UX • Editing • Using media stored in either location • Ability to begin edits at one site, finish at the other • Archiving to XenData and Spectralogic T120 • Ingest and transcode RED 4K media to Avid DNxHD 40
10Gb Fibre 41
Denver Broncos Multi-Site Solution Architecture Summer 2018
Deployment Plan • Create an identical storage solution for both locations • 2 x 96TB TerraBlock SAN Online Editorial Storage Pools • Hub Server • 192TB TerraBlock SAN Data Replication & Disaster Recovery Storage Repository • Leverage existing technology and IT infrastructure at both locations • Establish mirrored media transfers from location to location • Use existing (non-dedicated) IT-supplied 10Gb circuit between the facilities • 10Gb pipe will support mirroring of assets at night and weekends (avoiding IT bandwidth impacts) • End Result: • Editors at either location can access all media assets • Automatic Disaster Recovery solution for both facilities 43
Recommend
More recommend
