A Dropbox-like Personal Cloud for OpenStack Swift Pedro Garca Lpez - PowerPoint PPT Presentation

A Dropbox-like Personal Cloud for OpenStack Swift Pedro García López OpenStack Summit Adrián Moreno Martínez May 2014 - Atlanta Cristian Cotes González

CloudSpaces project ● Open Service Platform for the Next Generation of Open Personal Clouds: http://cloudspaces.eu ● Objectives: o Interoperability and privacy ● Partners: o URV , Eurecom, EPFL, eyeOS (Web Desktop), TISSAT (cloud provider), NEC.

Personal Cloud We understand it as… ● Storage ● Synchronization ● Sharing

Our motivations ● Not a cloud open-source solution available  StackSync and OpenStack ● Users lack control of their data  Privacy, Client-side encryption, secure sharing… ● Vendor lock-in  Interoperability

Big picture

First stage ● Metadata separated from data → StackSync Server → OpenStack Swift Metadata DB StackSync Server OpenStack Swift Metadata Data flow flow Desktop client

Second stage Metadata DB ● MOM (RabbitMQ) → push notifications StackSync Server StackSync Server StackSync Server ● Elastic sync protocol RabbitMQ OpenStack Swift Metadata Data flow (Push) flow Desktop client

Current stage Metadata DB ● API as a Swift’s proxy module Metadata flow StackSync Server StackSync Server StackSync Server ● Website ● Android app OpenStack Swift RabbitMQ StackSync API Metadata Data flow (push) flow Desktop client Mobile client

Lifecycle and architecture

Lifecycle StackSync Server StackSync Server StackSync Server RabbitMQ OpenStack Swift Client 3 Client 1 Client 2

Desktop client Metadata DB Metadata flow StackSync Server OpenStack Swift RabbitMQ StackSync API Metadata Data flow (push) flow Desktop client Mobile client

Tasks ● File system watcher ● Chunking, compression, and encryption

Features ● File versioning ● File sharing ● OS integration ● Overlay icons ● Notifications

StackSync server Metadata DB Metadata flow StackSync Server OpenStack Swift RabbitMQ StackSync API Metadata Data flow (push) flow Desktop client Mobile client

Operations { "is_folder":false, "mimetype":"application/pdf", ● getAccount "status":"NEW", "parent_file_id":73990274, "file_id":-534824681, ● getChanges "path":"/documents/", "filename":"StackSync_Guide.pdf", "checksum":-2678858962222278590, ● commit "user":"John Doe", "client_modified":"2013-03-08 10:36:41.997", "size":775412, ● … "version":1, "server_modified":"2013-03-08 10:36:41.997" }

Syncing conflicts ● Two users modify the same file at the same time ● Whichever version is processed first wins ● The loser creates a conflicted copy of the file with his changes User 1 StackSync Server User 2

Workspace ● Represents the relation between files and users ● Every user has its own workspace ● Every shared folder has its workspace ● A workspace maps into a Swift container Workspace 1 Container 1 OpenStack Swift User 1 Shared container Shared workspace Workspace 2 Container 2 User 2

Encryption settings Client-side Server-side Plain encryption encryption

Swift mappings StackSync OpenStack Swift installation tenant user user workspace container admin tenant admin file N chunks

Sharing process 1. Send share proposal 2. Notify the addressee 3. Proposal gets accepted 4. Create container and set up ACL OpenStack Swift X-Container-Read and X-Container-Write StackSync Server User 1 User 2

RabbitMQ Metadata DB Metadata flow StackSync Server OpenStack Swift RabbitMQ StackSync API Metadata Data flow (push) flow Desktop client Mobile client

RabbitMQ ● Communication between clients and server ● Push notifications ● Load-balancing ● Multicast messages per workspace

StackSync Server StackSync Server StackSync Server

StackSync API Metadata DB Metadata flow StackSync Server OpenStack Swift RabbitMQ StackSync API Metadata Data flow (push) flow Desktop client Mobile client

StackSync API ● Located in the proxy’s pipeline ● Used by website & mobile clients ● Communicates with StackSync server to get metadata and authorization ● Activates when the X-StackSync-API header is set

StackSync API Auth ● Module located before the API ● OAuth 1.0a implementation ● Sets the user information in the wsgi environment for the API

GET /api/file/4688390 Header: X-StackSync-API: v1

Performance

Traffic overhead

Push vs pull

Server elasticity Number of instances Number of requests Time (minutes)

Using StackSync

On premises Deployment scenarios On cloud provider Private Hybrid Public OpenStack Swift OpenStack Swift OpenStack Swift StackSync Server StackSync Server StackSync Server

Current deployments ● Universitat Rovira i Virgili ● Tissat (Data center) ● Rediris (Spanish university network)

Community ● Available for: ● Documentation available at stacksync.org ● Easy to deploy for swifters

Conclusions ● StackSync is a ready-to-use Personal Cloud for OpenStack Swift ● What’s next ● Advanced privacy features ● Interoperability ● iOS and OS X clients

Thanks for your attention! stacksync.org github.com/stacksync