B e l n e t U p d a t e o n Storage systems Jean-Philippe Evrard & Mario Vandaele 22/09/2014
Belnet Objectives Please our customers – However, it’s possible they don’t know what they technically want Reduce our costs – Enters the “low cost solutions”... 21/09/2014 TF-Storage: Belnet Update 2
Low cost storage: What’s in it for… The customers: – Lower prices – New services possibilities – Better SLAs The management: – Cost reductions – No black boxes (better internal knowledge) – Additional resiliency (new virtualization storage backends…) dd.mm.yyyy TF-Storage: Belnet Update 3
3 storage families Legacy storage systems – Proprietary SW & HW – Netapp, EMC, Dell… “Low cost storage” vendors – « commodity » hardware and proprietary/free SW – Storiant, Nexenta BYO – « commodity » hardware and whatever you’ll pick for the SW dd.mm.yyyy TF-Storage: Belnet Update 4
For each storage type… Technology available/used by Belnet Costs for Belnet (see also our Excel file) – Not including any of our « overhead » (PM & other internal costs) – Not including DC costs (energy, rackspace) – Not including connectivity costs Impact on our Ops team (Day to Day impact) dd.mm.yyyy TF-Storage: Belnet Update 5
Legacy storage systems Technology: – Black boxes – iSCSI – Sometimes NFS/CIFS/other systems « drivers » Belnet: (Dell Equalogic +) NetApp SANs dd.mm.yyyy TF-Storage: Belnet Update 6
Legacy storage systems Belnet Costs: – CAPEX: 25k€ per 50TB – OPEX: 3k€ per year per 50TB Impact Day to Day: – Almost 0. If problems, we have our supplier dd.mm.yyyy TF-Storage: Belnet Update 7
Low cost storage vendors Technology: – “commodity” hardware – Variable software stack – Data available often as block or file Belnet example: Storiant dd.mm.yyyy TF-Storage: Belnet Update 8
Low cost storage vendors Storiant Technology: – “commodity” hardware – Based on DAS boxes on storage servers (2 per rack) with SSDs – Max 4 boxes of 60 drives per storage server – ZFS for the filesystem – Proprietary drive management and web frontend – Intended for “cold storage” (glacier) – OOTB monitoring (based on Nagios) – Data available as block or file dd.mm.yyyy TF-Storage: Belnet Update 9
Low cost storage vendors Belnet Costs: – CAPEX: ~300k€ min investment (1.4PB) – OPEX: ~35k€ for the min investment – CAPEX: ~60±15k€ for later disks (per 480TB) – OPEX: 8.5±3k€ for later disks (per 480TB) Impact Day to Day: – Depends on the integrator. Here we have: HW support (outsourced, but you could change the model). ● SW support (you can just “use the solution”) ● dd.mm.yyyy TF-Storage: Belnet Update 10
Build your own Phase1: Belnet PoC Technology: – 1U Pizza boxes (2.5k€) with 12 drives – Hot swappable disks (per 3 disk, not disk per disk :/ ) – PSU are not redundant – Multiple 10Gbps Ethernet – No SSDs – GlusterFS for first PoC – Ceph for second PoC – Custom scripts for HDD power management and XFS mgmt – Integration of our own software Zabbix auto-discovery for monitoring ● Saltstack for de/provisionning ● dd.mm.yyyy TF-Storage: Belnet Update 11
Build your own Phase2: Belnet possible future: – Larger multi-host boxes (4U) with redundant PSUs – New tender process for the procurement of only HDDs – SW stack: TBD, with the feedback of other NRENs: Storage system: gluster/ceph? ● Monitoring system: ● – Our Zabbix? – Something common with other NRENs (collectd anyone?) – outsourced? (Dataloop/New relic?) Logs: ELK ● Deployment receipes: salt/puppet/nothing? ● dd.mm.yyyy TF-Storage: Belnet Update 12
Build your own Belnet Costs: – CAPEX: TBD (depending on 1U/4U systems, disk size) – OPEX: TBD (we need more time and feedback!) – It's important for us to clarify all of that, because the... Impact Day to Day: – Important! Maintain your own HW could cost you less ● Your SW team need (and will also be) experts ● dd.mm.yyyy TF-Storage: Belnet Update 14
Conclusions Legacy Low cost vendor BYO Vendor ecosystem Depends Depends Operational work Investment in new infrastructure and knowledge Customization possibilities Cost per TB Customer Satisfaction Unknown Unknown Unknown Expected customer satisfaction dd.mm.yyyy TF-Storage: Belnet Update 16
Brainstorm topics Quick questions: – For your HW maintenance, have you an idea of how much does it cost you outside business hours? Is it still a good idea to outsource it? – Do we really need support on some SW if we are building a network of knowledge together? – What's the possible SLA on this network of knowledge? – Is it our mission to build/maintain this network of knowledge? – Is it our mission to deliver this sort of low cost infrastructure? dd.mm.yyyy TF-Storage: Belnet Update 17
Summary & Next steps Belnet is advancing, at its own pace Low cost storage in general IS interesting (and could be even better depending on your decisions: SLA/outsourcing) Our technology choices could be defined by yours Organize a round table, anyone? The cost model needs further improvements If you have time to look at it, contact me... How do we share our expertise/code ? – brainstorm topics? dd.mm.yyyy TF-Storage: Belnet Update 18
Questions?
Recommend
More recommend
