for Multitenant Filesystems Giorgos Kappes, Andromachi - - PowerPoint PPT Presentation

▶
for multitenant filesystems
SMART_READER_LITE
LIVE PREVIEW

for Multitenant Filesystems Giorgos Kappes, Andromachi - - PowerPoint PPT Presentation

Jul 04, 2023 477 likes •770 views

Virtualization Aware Access Control for Multitenant Filesystems Giorgos Kappes, Andromachi Hatzieleftheriou, Stergios V. Anastasiadis gkappes, ahatziel, stergios (at) cs.uoi.gr Department of Computer Science and Engineering University of

slide-1
SLIDE 1

Department of Computer Science and Engineering University of Ioannina, Greece

Virtualization Aware Access Control for Multitenant Filesystems

gkappes, ahatziel, stergios (at) cs.uoi.gr

15th TERENA TF-Storage Task Force Meeting

Giorgos Kappes, Andromachi Hatzieleftheriou, Stergios V. Anastasiadis

slide-2
SLIDE 2

Storage Consolidation

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

2

Benefits

  • Efficient usage and management of storage resources
  • Sharing support
  • High capacity utilization
  • Reduced cost

... Local Storage Host1 Local Storage

VM VM

HostN Shared Storage ...

VM VM

HostN

VM VM

Host1

VM VM

slide-3
SLIDE 3

Storage Interfaces

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

3

Direct virtual disk access through block interface

+ Isolation, snapshoting, versioning, migration + Heterogeneous clients — Semantic loss hardens consistency, sharing, manageability — Reduced performance due to layering and FS nesting

Direct filesystem sharing through file interface

+ Isolation, snapshoting, versioning, migration + Semantic awareness: sharing, consistency, manageability + Elimination of layering and nesting: increased performance — Complicates support for heterogeneous clients

slide-4
SLIDE 4

Storage Multitenancy

Goal

  • Storage infrastructure shared among different tenants

Requirements

  • Scalability: Support enormous number of end users
  • Isolation: Isolate the user identities and access control of

different tenants

  • Sharing: Flexible data sharing within or between tenants
  • Compatibility: Compatibility with existing applications
  • Manageability: Flexible resource management

Research focus

  • Efficient and secure multitenancy in VM filesystems
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

4

slide-5
SLIDE 5

Ceph Storage

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

5

A distributed and unified storage platform

  • Everything is stored in the form of objects
  • Scalable, no single point of failure, software-based, flexible

Ceph object store (RADOS)

  • A reliable, distributed, autonomous object store

Storage interfaces:

  • Direct access through librados
  • Object-level (RADOSGW): HTTP REST gateway
  • Block-level (RBD): A reliable and distributed block device
  • File-level (CEPHFS): A POSIX-compliant distributed FS
slide-6
SLIDE 6

Ceph FS

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

6

A distributed object-based filesystem

  • Parallelization of file I/O
  • Elimination of the potential metadata bottleneck

Separate management of file metadata and data

  • Metadata managed by Metadata Servers (MDS)
  • Data managed by Data Servers (OSD)

Client OSD OSD MDS MON Auth Auth Auth Ceph Storage Cluster

slide-7
SLIDE 7

ID: 76

Metadata Server

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

7

Goal

  • Create filesystem hierarchy on top of objects
  • Manage metadata: hierarchy, permissions, timestamps, etc.

Metadata is stored in RADOS

  • Inodes are embedded inside folders
  • Folders are stored as single objects or as multiple fragments

/etc crontab group cron.d … ID: 75

crontab 101 cron.d 102 group 103

…

anacron mdadm

anacron 175 mdadm 106

dentry inode Object

slide-8
SLIDE 8

Metadata Server

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

8

Metadata caching and failure recovery

  • Recently updated metadata is cached
  • The MDS journals metadata updates for failure recovery

Other features

  • High availability: Multiple standby MDSs
  • Scalability: Multiple active MDSs, subtree partitioning
slide-9
SLIDE 9

Stores file data and metadata in object-form

  • Objects: files with identifier, binary data, object metadata

Multiple active OSDs

  • Scalability
  • Fault tolerance

Data placement

  • CRUSH maps objects to PGs and PGs to OSDs
  • PGs are logically grouped into pools
  • Clients run the CRUSH algorithm themselves

Data Server

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

9

OBJ OBJ OBJ OBJ OBJ OSD1 OSD2 OSD3 Placement Group 1 Placement Group 2

slide-10
SLIDE 10

Maintains the state of the cluster

  • The state is maintained into the cluster map
  • Clients check in periodically to obtain a fresh map copy

Cluster map

  • Composition of: MON map, MDS map, OSD map, CRUSH map
  • The maps track the state of the servers, PGs, storage, etc.
  • The cluster map gets updated each time the state changes

Monitor quorum

  • Consensus about the cluster map is established with Paxos
  • An odd number of monitors is required

Monitor

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

10

slide-11
SLIDE 11

Provides Access to the filesystem

  • Linux kernel client
  • FUSE client

Clients calculate the final position of an object

  • First step: calculate object id (OID)

OID: INODE number, object fragment number

  • Second step: choose a Placement Group inside a pool

PG: hash of OID, total number of PGs, pool number

  • Third step: choose the OSDs to store the object

List of OSDs: Run CRUSH to map the PG to OSDs

Client

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

11

slide-12
SLIDE 12

Access control decisions happen at the MDS

  • The MDS authorizes a request and provides capabilities to clients
  • The clients present the capabilities to OSDs
  • The OSDs validate the capabilities

Access control

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

12

Client MDS OSD OSD (1) Authorization Request (2) Capability

slide-13
SLIDE 13

Motivation

Problem of multitenancy

  • Shared FS namespace
  • Crosstalk between tenants
  • Complicated security

Native multitenancy at the filesystem level

  • Clean way to isolate multiple tenants
  • Shared hardware, operating system, fileservers
  • Configurable isolation, sharing, performance, manageability
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

13

FS NATIVE USERS Shared File System TENANT1 UID: 1000 GID: 1000 UID: 1100 TENANTN UID: 1000 GID: 1000 UID: 1050 GID: 1000 UID: 2000 UID: 1000

slide-14
SLIDE 14

Prior approaches

Centralized

  • The principals’ identities of all tenants centrally maintained

— Poor scalability, isolation and manageability

Peer-to-peer

  • The principals of each tenant managed locally
  • Tenants communicate to publicize their principals’ identities

— Overhead to periodically synchronize the tenants

Mapping

  • Local principal IDs mapped to global unique IDs

— Mapping overhead, sharing complications, security violations

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

14

slide-15
SLIDE 15

The Dike Approach

Hierarchical identification and authentication

  • The tenants manage their principals
  • The provider manages the tenants

Native multitenant authorization

  • Separate ACLs per tenant and provider
  • Namespace isolation through filesystem views

Efficient permission management and storage

  • Shared common permissions
  • Inheritance of permissions
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

15

slide-16
SLIDE 16

Identification

Principals

  • Tenant principals: Use/manage tenant resources
  • Native FS principals: Manage the FS

Tenant Authentication Server (TAS)

  • Certifies local clients and principals

Filesystem Authentication Server (FAS)

  • Certifies filesystem services, tenants, native principals
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

16

TENANT1 Users1 Clients Tenant Authentication Server FILESYSTEM SERVERS MDS OSD OSD Filesystem Authentication Service Filesystem Authentication Server TENANTN UsersN Clients Tenant Authentication Server

slide-17
SLIDE 17

Authentication

Metadata ticket

  • Securely specifies tenant principal
  • Provides access to MDS

Data ticket

  • Securely specifies tenant principal

and permissions

  • Provides access to OSDs
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

17

(1) Principal authenticated by TAS (2) Principal requests FS access (3) Client contacts TAS (4) Client receives Metadata ticket (5) Client contacts MDS (6) MDS issues Data ticket (7) Client contacts OSD (8) Client accesses data

Steps

(2) Connect (3) Request

Clients Tenant Authentication Server Tenant principals Filesystem Authentication Server

(8) Data Authenticate

Native principals

TENANTS PROVIDER OSD OSD

MDS

(7) Data ticket

slide-18
SLIDE 18

Authorization

Access control isolation

  • Separate ACLs per tenant, provider
  • Metadata accessible through views

Filesystem view

  • Used by native principals to

manage tenants

Tenant view

  • Used by tenants to access or manage tenant resources

File sharing

  • Private to a principal
  • Shared across principals of one or more tenants
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

18

Tenant1 Policy Tenant2 Policy ... ... TenantN Policy Authorization Request Authorization Decision Client TENANT1 Metadata Ticket Tenant1 Policy Tenant2 Policy ... ... TenantN Policy MDS Data Ticket Per file access policy

slide-19
SLIDE 19

Common Permissions

Goal

  • Reduce filesystem load by reducing ACLs

Per tenant permission inheritance

  • Permissions can be inherited to child

files/folders

Per tenant common permissions

  • Child files can share parent’s ACL

ACLs

  • Tree folder ACLs: Folder permissions
  • Tree file ACLs: Shared child files permissions
  • Private file ACLs: Child file permissions explicitly set by user
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

19 Folder Folder Tenanti Tenanti

Tree Folder ACLs Tree File ACLs

Tenanti Tenanti

Tree Folder ACLs Tree File ACLs Private File ACL

Tenantk

slide-20
SLIDE 20

Security Analysis

Captured credential

  • Fresh tamperproof credentials cannot be forged

Compromised tenant principal account

  • Compromised tenant view is isolated
  • Attack limited to principal’s private or shared files
  • Cross-tenant policy violation is prohibited

Attack by revoked tenant

  • Restricted through deleted tenant view
  • Tenant cannot access other views

Compromised provider administrator account

  • Handle via good practices (e.g., restricted remote access)
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

20

slide-21
SLIDE 21

Prototype

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

21

Identification

  • Each tenant has its own public key pair
  • The tenant ID is derived by applying a cryptographic hash

function on the public key

  • Clients know their tenant’s ID

Session

  • During FS mount clients initiate a new session with the MDS
  • The create session request message contains the tenant ID
  • The MDS tags the new session with the tenant ID
  • The session can be used only by the principals of the

identified tenant

slide-22
SLIDE 22

Prototype

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

22

Permission storage

  • Tenant view: Extended Attributes (EAs) indexed by Tenant ID

– Folders: maintain per tenant tree file/folder permissions – Files: maintain per tenant private file permissions if specified

  • Filesystem view: Regular fields
  • EAs with tenant permissions not directly accessed by clients

INODES UID GID MODE ... XATTRS ... Filesystem view Permissions Client Client Client Client Client MDS Client Client MON Client Client OSD OBJECT POOLS Auth EXTENDED ATTRIBUTES MAP ... Permissions Permissions Permissions TID1 TID2 ... TIDN Tenant view Permissions

slide-23
SLIDE 23

Prototype

  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

23

Capabilities

  • Include principal and tenant identifiers
  • Sent to clients with tenant file access

Management of tenant access policies

  • grant_tenant_access, revoke_tenant_access

– New client and MDS calls to grant/revoke tenant access to/from a file or a directory

  • change_tenant_policy

– Tool for administrate tenant access policies

slide-24
SLIDE 24

Experimentation Environment

Configuration: AWS EC2 Instances

  • m1.xlarge: x3, 4 VCPU, 15 GB RAM, Linux 3.9.3
  • t1.micro: x32, 1 VCPU, 615 MB RAM, Linux 3.9.3

Filesystem configuration

  • Ceph/Dike: m1.xlarge, 1xOSD+MON, 1xOSD+MDS, 1xOSD
  • Gluster/Heka: m1.xlarge, 3 fileservers
  • Replication factor 3

Microbenchmark

  • mdtest
  • 48000 created files and folders
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

24

slide-25
SLIDE 25

Results

Dike Client scalability

  • 1  32 clients: Similar to Ceph

Dike Tenant scalability

  • 1k  5k tenants: 2% extra overhead
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

25

slide-26
SLIDE 26

Results

Dike limited overhead

  • 1k tenants overhead: up to 14%
  • 5k tenants overhead: up to 16%

ID mapping multitenancy too costly

  • 1k tenants overhead: up to 49%
  • 5k tenants overhead: up to 84%
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

26

Dike native multitenancy

Limited overhead Scalable to thousands tenants

slide-27
SLIDE 27

Conclusions

Native filesystem multitenancy with sharing support

  • Hierarchical identification scheme
  • Namespace isolation: Per tenant and provider ACLs
  • Per tenant common permissions and inheritance

Performance and security analysis

  • Limited multitenancy overhead up to 16%
  • Dike scalable to several thousand tenants
  • Tenant principals not able to violate cross-tenant policy

Future work

  • I/O intensive application experimentation
  • Weaker trust assumptions
  • G. Kappes, Department of Computer Science & Engineering, University of Ioannina, Greece

27