Issuing temporary credentials for MySQL using Hashicorp Vault - - PowerPoint PPT Presentation
Issuing temporary credentials for MySQL using Hashicorp Vault Walter Heck - CTO at OlinData Percona Live Europe 2017 Sounds familiar? Hey, Jane Doe from that department youve never heard of wants to do some analysis and she needs
“Hey, Jane Doe from that department you’ve never heard of wants to do some analysis and she ‘needs’ direct access to our production database. Can you set that up in the next 30 minutes please, I know you have nothing better to do anyway. Cheers, not-your-manager.”
GitHub, AWS IAM, etc}
○ 3 az’s, 1 region
○ Secrets backend: consul ○ Auth backend -> github
○ (any flavor/version > 5.0)
Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, and more. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.
using 256-bit AES in GCM mode with a randomly generated nonce.
The key features of Vault are:
Terraform (Infra as Code) Manual (One time operations)
○ Officially supported by Hashicorp root@ip-10-1-103-8:~# consul members Node Address Status Type Build Protocol DC Segment ip-10-1-103-104 10.1.103.104:8301 alive server 0.9.3 2 dc1 <all> ip-10-1-104-233 10.1.104.233:8301 alive server 0.9.3 2 dc1 <all> ip-10-1-105-147 10.1.105.147:8301 alive server 0.9.3 2 dc1 <all> ip-10-1-103-8 10.1.103.8:8301 alive client 0.9.3 2 dc1 <default>
○ By default 5, need 3 minimum to unseal (Shamir's Secret Sharing) ○ Don’t lose them, you lose everything ○ Use gpg init for easier distribution (https://www.vaultproject.io/docs/con cepts/pgp-gpg-keybase.html)
root@ip-10-1-103-8:~# vault init Unseal Key 1: p5Luba1DNJcFSvThese2rj/fJ4iJQMA8bUBG5fuvIsS Unseal Key 2: 3+M+ajrPVCS96fKeysxUOEfM4JxsT40sosMVHfq1bqA Unseal Key 3: aW1qaxI2H7u57YAreG26Fuchao0XEaWq/f79dljE3iLA Unseal Key 4: tNSeeA6WWkAMK5Notjs/gEqf+8KbqQ32ypcfh3oecsfu Unseal Key 5: nkbtNRGOUxiXPiRealtNBTai9bzVaMmkkbCVRzbaoFn8 Initial Root Token: d57d945b-yoaa-f476-5660-3f6645692555 Vault initialized with 5 keys and a key threshold of 3. Please securely distribute the above keys. When the vault is re-sealed, restarted, or stopped, you must provide at least 3 of these keys to unseal it again. Vault does not store the master key. Without at least 3 keys, your vault will remain permanently sealed.
vault seal will seal the vault
keys by default
root@ip-10-1-103-8:~# vault unseal Key (will be hidden): Sealed: true Key Shares: 5 Key Threshold: 3 Unseal Progress: 1 Unseal Nonce: d1747cd1-a850-bf79-9175-7ac1aaffdddd root@ip-10-1-103-8:~# vault unseal Key (will be hidden): Sealed: true Key Shares: 5 Key Threshold: 3 Unseal Progress: 2 Unseal Nonce: d1747cd1-a850-bf79-9175-7ac1aaffdddd root@ip-10-1-103-8:~# vault unseal Key (will be hidden): Sealed: false Key Shares: 5 Key Threshold: 3 Unseal Progress: 0 Unseal Nonce: root@ip-10-1-103-8:~# vault status Sealed: false Key Shares: 5 Key Threshold: 3 Unseal Progress: 0 Unseal Nonce: Version: 0.8.2 Cluster Name: vault-cluster-d2bd39fc Cluster ID: d6957662-bc13-826e-5a10-1effde41a718 High-Availability Enabled: true Mode: standby Leader Cluster Address: https://10.1.104.220:8201
2 1 3 ✓
root@ip-10-1-103-8:~# vault seal Error sealing: Error making API request. URL: PUT https://127.0.0.1:8200/v1/sys/seal Code: 500. Errors: * 1 error occurred: * missing client token root@ip-10-1-103-8:~# vault auth Token (will be hidden): Successfully authenticated! You are now logged in. token: d57d945b-b0aa-f476-5660-3f6645692555 token_duration: 0 token_policies: [root] root@ip-10-1-103-8:~# vault seal Vault is now sealed.
grant or forbid access to certain paths and operations
cat <<EOF | vault policy-write core-policy /dev/stdin path "sys/*" { policy = "deny" } path "database/creds/readonly" { policy = "read" capabilities = ["list", "sudo"] } path "database/creds/demodb_admin" { policy = "read" capabilities = ["list", "sudo"] } path "database/roles/*" { policy = "read" capabilities = ["read", "list"] } EOF
○ AWS, Gcloud, LDAP, Radius, Okta and more available
tokens
○ Beware! Losing a personal token is a security risk
in https://github.com/settings/tokens.
○ Generate a new Token that has the scope read:org.
root@ip-10-1-103-8:~# vault auth-enable github Successfully enabled 'github' at 'github'! root@ip-10-1-103-8:~# vault auth -methods Path Type Accessor Default TTL Max TTL Replication Behavior Description github/ github auth_github_db842730 system system replicated token/ token auth_token_84532020 system system replicated token based credentials root@ip-10-1-103-8:~# vault write auth/github/config
Success! Data written to: auth/github/config root@ip-10-1-103-8:~# vault write auth/github/map/teams/core value=core-policy Success! Data written to: auth/github/map/teams/core root@ip-10-1-103-8:~# vault auth -method=github token=10a8acd3f4ec0b2399146abb0ba6b70211bb6990 Successfully authenticated! You are now logged in. The token below is already saved in the session. You do not need to "vault auth" again with the token. token: 58b52458-4e25-6642-f1f1-a24eda37913d token_duration: 2764799 token_policies: [core-policy default]
the tree
for a number of database engines
○ Postgres, mongo, oracle, MS SQL server
argument for the role is flexible and can contain whatever SQL statement you want
max_open_connections and
root@ip-10-1-103-194:~# vault mount database Successfully mounted 'database' at 'database'! root@ip-10-1-103-8:~# vault write database/config/mysql \ > plugin_name=mysql-database-plugin \ > connection_url="user:mypwd@tcp(perconalive.olindata.local:3306)/" \ > allowed_roles="readonly" The following warnings were returned from the Vault server: * Read access to this endpoint should be controlled via ACLs as it will return the connection details as is, including passwords, if any. root@ip-10-1-103-8:~# vault write database/roles/readonly \ > db_name=mysql \ > creation_statements="CREATE USER '{{name}}'@'%' \ > IDENTIFIED BY '{{password}}';GRANT SELECT ON *.* TO '{{name}}'@'%';" \ > default_ttl="1h" \ > max_ttl="24h" Success! Data written to: database/roles/readonly
1. User auths against Vault 2. User asks vault for creds 3. Vault creates records in consul and issues a grant statement to MySQL 4. Vault returns username+password back to user … user does their thing 5. After X amount of time, vault removes grant from MySQL
really useful in prod
○ Use HTTP API instead
root@ip-10-1-103-8:~# vault auth -method=github token=a8acd3f4ec0b2399146abb0ba6b70211bb699010 Successfully authenticated! You are now logged in. The token below is already saved in the session. You do not need to "vault auth" again with the token. token: 76ee8b56-61e5-cbcd-2710-7f7d08668568 token_duration: 2764799 token_policies: [core-policy default] root@ip-10-1-103-8:~# vault read database/creds/readonly Key Value
database/creds/readonly/1b889400-092d-e634-6444-d1217d c93690 lease_duration 1h0m0s lease_renewable true password A1a-77r41spp13x57vy5 username v-github-wal-readonly-23q9t9vxx2
○ AWS, LDAP, Kerberos, SSH, etc.
Vault on AWS https://gist.github.com/chris-moreton/f523650c1863f0181e22e2020d0f2268 Consul Cluster ASG on AWS https://github.com/dwmkerr/terraform-consul-cluster Vault with MySQL https://www.percona.com/blog/2016/11/14/using-vault-mysql/