Integrating Keystone with Large-scale Centralized Authentication - - PowerPoint PPT Presentation

Integrating Keystone with Large-scale Centralized Authentication

Chris Janiszewski

OpenStack Solutions Architect chris.openstack@redhat.com blog: chrisj.cloud

Ken Holden

OpenStack Solutions Architect kholden@redhat.com blog: holdenthecloud.com

• Level set
• Benefits
• Issues and Bottlenecks
• Best Practices
• Tuning and Results

Agenda

• Authentication and Authorization
• Keystone
• Tokens
• LDAP
• Domains
• Kerberos and SSO

Level Set

• Keystone (OpenStack Identity Service)
• Methods of Authentication
• Authorization tokens
• Transport Layer Security (TLS) with X.509

Authentication and Authorization

.. provides identity, token, catalog, and policy services for use specifically by services in the OpenStack family Features:

• External authentication
• Federation
• Multi-factor authentication

Considerations:

• Invalid login attempts
• No enforcement policies on password strength, expiration

Keystone

• generated for authorization and access
• default value for expiry is one hour
• ften passed within the structure of a larger

context

• Token types:

○ UUID ○ PKI and PKIZ ○ FERNET

Tokens

• The Lightweight Directory Access Protocol

(LDAP) - is an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network

• LDAP simplifies integration of Identity

authentication

• Identity service MUST NOT be allowed to write

to LDAP services

LDAP

Domains

Keystone V2 Keystone V3

*Pictures found on Matt’s Dorn Blog - http://madorn.com/keystone-v3-api.html#.WwM2hXWUvgk

*

• Keystone can now be executed in a web

server like Apache HTTPD

• The Kerberos ticket-granting ticket can

be used to securely provide tickets for a given service

• Web server handle authentication is not

exclusive

• Web server can provide support for X.509
• r Kerberos authentication
• Keystone provides support for password

authentication (with SQL or LDAP)

Kerberos

Carnival Ticket Booth vs. Kerberos (server)

Identity Service: Authentication Mechanisms

TGT

• Grants a ticket called a

Ticket Granting Ticket (TGT) which says which services you have access to. This ticket also has an expiration called a Time-To-Live (TTL).

• You present the TGT to the

service you want access to and the service will use that to determine if you can access the service.

• Consolidated repository
• Simplified management
• Ideal for multiple OpenStack Deployments with

shared resources

• Performance of LDAP vs SQL
• SSL/TLS
• Back-end database flexibility
• Well-defined client API
• Decoupling of the roles

LDAP Benefits

• Few people understand LDAP
• No writes allowed
• Is OpenStack a front end for your users?
• Very strict schema
• Performance considerations

LDAP Considerations

• Token Performance
• LDAP Size
• Database
• Horizon Dashboard
• LDAP response

Issues and Bottlenecks

• Enable Debug and Verbosity - Impact can be significant
• Use slow hard drives for Controller’s OS and Databases
• Use non-secured APIs, Dashboard, and LDAP queries

Generally a good idea NOT to...

• The faster you can request tokens, the faster OpenStack responds
• Fernet tokens offer superior performance over UUID

Tokens

Who remembers the good old days when OpenStack started to crawl ? # du -sh /var/lib/mysql/keystone 149G /var/lib/mysql/keystone MariaDB [keystone]> select count(*) from token; +----------+ | count(*) | +----------+ | 20717136 | Number of UUID tokens +----------+

Just say No to UUID

• Active Directory DNS: Use Domain Name

url = ldaps://example.com Or obtain a list of Domain Controllers from DNS via SRV Records dig -t SRV _ldap.tcp.example.com

• LDAP: list multiple LDAP servers

url = ldaps://ldap1,ldaps://ldap2

Where do I query?

• Do all 100,000+ users REALLY need access to OpenStack?
• Keystone doesn’t always cleanup after itself

MariaDB [keystone]> select count(*) from id_mapping; +----------+ | count(*) | +----------+ | 129052 | Number of LDAP Accounts without filtering +----------+ MariaDB [keystone]> select count(*) from id_mapping; +----------+ | count(*) | +----------+ | 129052 | Number doesn’t decrease when adding filtering later +----------+

Limit number of LDAP records

• Filtering users and groups

user_filter = (&(|(memberOf=CN=OpenStack-Admins)(memberOf=CN=OpenStack-Users))) group_filter = (&(objectClass=Group)(&(|(cn=OpenStack)(cn=OpenStack-Admins)(cn=OpenSt ack-Users))))

• Using AND / OR

(&(|(memberOf=foo)(memberOf=bar))) (&(memberOf=for)(memberOf=bar))

• Active Directory Nested groups

(memberOf:1.2.840.113556.1.4.1941:=cn=OpenStack)

LDAP - Filtering

• Member of group1 OR group2

ldapsearch -LLL -H ldap://example.com -E pr=2/noprompt -b 'dc=example,dc=com' -D 'example\USER' -w PASSWORD '(&(|(memberOf=CN=OpenStack-Admins)(memberOf=CN=OpenStack-Users)))'

• Member of group1 AND group2

ldapsearch -LLL -H ldap://example.com -E pr=2/noprompt -b 'dc=example,dc=com' -D 'example\USER -w PASSWORD '(&(memberOf=CN=OpenStack-Admins)(memberOf=CN=OpenStack-Users))'

• Members of Active Directory Nested Group

ldapsearch -LLL -H ldap://example.com -E pr=2/noprompt -b 'dc=example,dc=com' -D 'example\USER -w PASSWORD '(&(objectClass=organizationalPerson)(sAMAccountName=*)(memberOf: 1.2.8 40.113556.1.4.1941:=cn=OpenStack))'

ldapsearch

Default value is 0 which disables paging

$ ldapsearch -LLL -H ldap://ldapserver.domain.com \

• b 'dc=domain,dc=com' -D 'DOMAIN\USERNAME' \
• w PASSWORD |grep sAMAccountName |wc -l

Size limit exceeded (4) 825 $ ldapsearch -LLL -H ldap://ldapserver.domain.com \

• E pr=1/noprompt -b 'dc=domain,dc=com' \
• D 'DOMAIN\USERNAME' \
• w PASSWORD |grep sAMAccountName |wc -l

10052

LDAP Tuning - Pagination

• View how keystone queries LDAP by enabling debug

crudini --set keystone.conf ldap debug_level 3 crudini --set keystone.conf DEFAULT insecure_debug true crudini --set keystone.conf DEFAULT debug true # restart keystone

• Watch the magic...

tail -f /var/log/keystone/keystone.log | grep LDAP\ search

filterstr=(&(sAMAccountName=kholden)(&(|(memberOf=CN=OpenStack-Admins,OU=People, DC=lab,DC=lan)(memberOf=CN=OpenStack-Users,OU=People,DC=example,DC=com)))(object Class=person)) attrs=['sAMAccountName', 'userPassword', 'userAccountControl', 'mail', 'description'] attrsonly=0

• Disable debug when done

LDAP Debug

• Keystone Pooling

○ Decrease load on Domain Controller especially with TLS ○ Separate pool for User Authentication ○ use_auth_pool requires use_pool enabled

crudini --set keystone.DOMAIN.conf ldap use_pool True crudini --set keystone.DOMAIN.conf ldap pool_size 200 crudini --set keystone.DOMAIN.conf ldap pool_retry_max 20 crudini --set keystone.DOMAIN.conf ldap pool_retry_delay 0.1 crudini --set keystone.DOMAIN.conf ldap pool_connection_timeout -1 crudini --set keystone.DOMAIN.conf ldap pool_connection_lifetime 600 crudini --set keystone.DOMAIN.conf ldap use_auth_pool True crudini --set keystone.DOMAIN.conf ldap auth_pool_size 1000 crudini --set keystone.DOMAIN.conf ldap auth_pool_connection_lifetime 60 # restart keystone

Keystone Pooling

• Enable Keystone Caching with memcached backend

systemctl enable memcached && systemctl start memcached crudini --set keystone.conf cache enabled true crudini --set keystone.conf cache backend dogpile.cache.memcached crudini --set keystone.conf cache backend_argument url: LOCAL_IP:11211 crudini --set keystone.conf catalog caching true crudini --set keystone.conf domain_config caching true crudini --set keystone.conf federation caching true crudini --set keystone.conf revoke caching true crudini --set keystone.conf role caching true crudini --set keystone.conf token caching true crudini --set keystone.conf token cache_on_issue true crudini --set keystone.conf identity caching true crudini --set keystone.conf identity cache_time 600 # restart keystone

Keystone Caching

Results

• Challenges with Implementation

○ Proxies in front of Domain Controllers ○ Firewalled Domain Controllers ○ Omitted Distinguished Names to save licensing costs ○ Service Account Password Change ○ Certificate / CA change ○ Changes require keystone restart ○ Fernet Token rotation ○ The Defiant Windows Admin

Meanwhile, in the real world...

• Multiple OpenStack Environments with One LDAP

○ Unique IDs for roles, Domains, and Projects are unique to each site ○ Determine Project, Domain and Role IDs from one site and update other sites with same IDs (BACKUP DB FIRST)

select * from project where name='PROJECT_NAME'; select * from project where name='DOMAIN_NAME'; select * from project where name='ROLE_NAME'; update project set id='SITE1_PROJECT_ID' where name='PROJECT_NAME'; update project set id='SITE1_DOMAIN_ID' where name='DOMAIN_NAME'; update project set id='SITE1_ROLE_ID' where name='ROLE_NAME';

Multi-site