B e l n e t U p d a t e o n Storage systems Jean-Philippe Evrard - - PowerPoint PPT Presentation

Belnet Update on Storage systems

Jean-Philippe Evrard & Mario Vandaele 22/09/2014

2 TF-Storage: Belnet Update

Belnet Objectives

21/09/2014

Please our customers

– However, it’s possible they don’t know what they technically want

Reduce our costs

– Enters the “low cost solutions”...

3 TF-Storage: Belnet Update

Low cost storage: What’s in it for…

dd.mm.yyyy

The customers:

– Lower prices – New services possibilities – Better SLAs

The management:

– Cost reductions – No black boxes (better internal knowledge) – Additional resiliency (new virtualization storage backends…)

4 TF-Storage: Belnet Update

3 storage families

dd.mm.yyyy

Legacy storage systems

– Proprietary SW & HW – Netapp, EMC, Dell…

“Low cost storage” vendors

– « commodity » hardware and proprietary/free SW – Storiant, Nexenta

BYO

– « commodity » hardware and whatever you’ll pick for the SW

5 TF-Storage: Belnet Update

For each storage type…

dd.mm.yyyy

Technology available/used by Belnet Costs for Belnet (see also our Excel file)

– Not including any of our « overhead » (PM & other internal costs) – Not including DC costs (energy, rackspace) – Not including connectivity costs

Impact on our Ops team (Day to Day impact)

6 TF-Storage: Belnet Update

Legacy storage systems

dd.mm.yyyy

Technology:

– Black boxes – iSCSI – Sometimes NFS/CIFS/other systems « drivers » Belnet: (Dell Equalogic +) NetApp SANs

7 TF-Storage: Belnet Update

Legacy storage systems

dd.mm.yyyy

Belnet Costs:

– CAPEX: 25k€ per 50TB – OPEX: 3k€ per year per 50TB

Impact Day to Day:

– Almost 0. If problems, we have our supplier

8 TF-Storage: Belnet Update

Low cost storage vendors

dd.mm.yyyy

Technology:

– “commodity” hardware – Variable software stack – Data available often as block or file Belnet example: Storiant

9 TF-Storage: Belnet Update

Low cost storage vendors

dd.mm.yyyy

Storiant Technology:

– “commodity” hardware – Based on DAS boxes on storage servers (2 per rack) with SSDs – Max 4 boxes of 60 drives per storage server – ZFS for the filesystem – Proprietary drive management and web frontend – Intended for “cold storage” (glacier) – OOTB monitoring (based on Nagios) – Data available as block or file

10 TF-Storage: Belnet Update

Low cost storage vendors

dd.mm.yyyy

Belnet Costs:

– CAPEX: ~300k€ min investment (1.4PB) – OPEX: ~35k€ for the min investment – CAPEX: ~60±15k€ for later disks (per 480TB) – OPEX: 8.5±3k€ for later disks (per 480TB)

Impact Day to Day:

– Depends on the integrator. Here we have:

• HW support (outsourced, but you could change the model).
• SW support (you can just “use the solution”)

11 TF-Storage: Belnet Update

Build your own

dd.mm.yyyy

Phase1: Belnet PoC Technology:

– 1U Pizza boxes (2.5k€) with 12 drives – Hot swappable disks (per 3 disk, not disk per disk :/ ) – PSU are not redundant – Multiple 10Gbps Ethernet – No SSDs – GlusterFS for first PoC – Ceph for second PoC – Custom scripts for HDD power management and XFS mgmt – Integration of our own software

• Zabbix auto-discovery for monitoring
• Saltstack for de/provisionning

12 TF-Storage: Belnet Update

Build your own

dd.mm.yyyy

Phase2: Belnet possible future:

– Larger multi-host boxes (4U) with redundant PSUs – New tender process for the procurement of only HDDs – SW stack: TBD, with the feedback of other NRENs:

• Storage system: gluster/ceph?
• Monitoring system:

– Our Zabbix? – Something common with other NRENs (collectd anyone?) – outsourced? (Dataloop/New relic?)

• Logs: ELK
• Deployment receipes: salt/puppet/nothing?

14 TF-Storage: Belnet Update

Build your own

dd.mm.yyyy

Belnet Costs:

– CAPEX: TBD (depending on 1U/4U systems, disk size) – OPEX: TBD (we need more time and feedback!) – It's important for us to clarify all of that, because the...

Impact Day to Day:

– Important!

• Maintain your own HW could cost you less
• Your SW team need (and will also be) experts

16 TF-Storage: Belnet Update

Conclusions

dd.mm.yyyy

Legacy Low cost vendor BYO Vendor ecosystem Depends Depends Operational work Investment in new infrastructure and knowledge Customization possibilities Cost per TB Customer Satisfaction Unknown Unknown Unknown Expected customer satisfaction

17 TF-Storage: Belnet Update

Brainstorm topics

dd.mm.yyyy

Quick questions:

– For your HW maintenance, have you an idea of how much does it cost you outside business hours? Is it still a good idea to outsource it? – Do we really need support on some SW if we are building a network of knowledge together? – What's the possible SLA on this network of knowledge? – Is it our mission to build/maintain this network of knowledge? – Is it our mission to deliver this sort of low cost infrastructure?

18 TF-Storage: Belnet Update

Summary & Next steps

dd.mm.yyyy

Belnet is advancing, at its own pace Low cost storage in general IS interesting

(and could be even better depending on your decisions: SLA/outsourcing)

Our technology choices could be defined by yours

Organize a round table, anyone?

The cost model needs further improvements

If you have time to look at it, contact me...

How do we share our expertise/code ?

–

brainstorm topics?

Questions?