Use Plays with HashiCorp Vault

Integrating Hashicorp Vault with Ansible allows you to manage sensitive data and secrets securely. Vault provides a centralized way to store and access secrets, such as API keys, passwords, and certificates, which can be dynamically retrieved during playbook execution. This enhances security by avoiding hard-coded secrets in your playbooks and inventory files.

Authentification

To access Vault, you first need to authenticate to Vault. To do so, you need to have an input token or a userpass connection.

I will only talk about a userpass connection there but it is easy to modify it to an input token connection.

This documentation will focus only on retrieving the password for the hosts and creating an inventory from it. First, let's define some variables to store our username and password :

---
- name: Get Userpass
  gather_facts: false
  connection: local
  hosts: localhost
  vars_prompt:
    - name: "username"
      prompt: "Enter your username"
      private: no
    - name: "password"
      prompt: "Enter your password"
      private: yes
  tasks:
    - name: Print Username
      debug:
        msg: "{{ username }}"
      delegate_to: localhost
    - name: Print Password
      debug:
        msg: "{{ password }}"
      delegate_to: localhost

connection: local : don't connect via ssh to any hosts, stay local.
hosts: localhost : don't execute the playbook on any hosts, only on the host localhost.
gather_facts: fasle : don't connect via ssh, even to the localhost.

Now we have the username and password of the user who wants to connect to Vault stored in a variable.

Get the User Token

To continue the authentication process, we need to store the Vault address in a variable. Then we make a POST request to obtain the token for this user.

---
- name: Connection Vault
  gather_facts: false
  connection: local
  hosts: localhost
  vars:
    vault_addr: 'https://vault.skignes.fr:8200'
  vars_prompt:
    - name: "vault_username"
      prompt: "Enter your Vault username"
      private: no
    - name: "vault_password"
      prompt: "Enter your Vault password"
      private: yes
  tasks:
    - name: Authentificate with Vault
      uri:
        url: "{{ vault_addr }}/v1/auth/userpass/login/{{ vault_username}}"
        method: POST
        body: "{{ {'password': vault_password} | to_json }}"
        headers:
          Content-Type: "applications/json"
        status_code: 200
        return_content: yes
	  register: vault_auth_response
	  changed_when: false
	  no_log: true

    - name: Set Vault token
	  set_fact:
	    vault_token: "{{ vault_auth_response.json.auth.client_token }}"
	  no_log: true

Authentification with Vault : make a POST request to the Vault to get the response of the login.
Set Vault token : get the token of the user to access the secrets inside the Vault.
no_log: true : don't log any information about these tasks, even in debug mode.

Add Hosts inside Inventory

We have a connection between our Ansible Playbook and our Vault. We now need to get the secrets from Vault.

Get a SSH password

To get the secrets from Vault we need to make a new request to the Vault server.

---
- name: Get SSH Password
  gather_facts: false
  connection: local
  hosts: localhost
  vars:
    vault_addr: 'https://vault.skignes.fr:8200'
    # API path for the directory where the secrets are
    password_path: '/kv/data/my_directory'
  vars_prompt:
    - name: "vault_username"
      prompt: "Enter your Vault username"
      private: no
    - name: "vault_password"
      prompt: "Enter your Vault password"
      private: yes
  tasks:
    - name: Authenticate with Vault
      uri:
        url: "{{ vault_addr }}/v1/auth/userpass/login/{{ vault_username }}"
        method: POST
        body: "{{ {'password': vault_password} | to_json }}"
        headers:
          Content-Type: "application/json"
        status_code: 200
        return_content: yes
      register: vault_auth_response
      changed_when: false
      no_log: true

    - name: Set Vault token
      set_fact:
        vault_token: "{{ vault_auth_response.json.auth.client_token }}"
      no_log: true

    - name: Fetch Password
      set_fact: 
        ssh_password: "{{ lookup('community.hashi_vault.hashi_vault', 'secret={{ password_path }} token={{ vault_token }} url={{ vault_addr }}')['key'] }}"
      no_log: true

password_path : It is a new variable, and it represents the API path to the directory of secrets.
['key'] : Inside the Fetch Password task, there is ['key'] at the end. It is the name of the secret you want to retrieve (for example: skignes).

Add it inside the Inventory

So when we got the password and we know the username for the machine we can add it to the inventory.

---
- name: Create Inventory
  gather_facts: false
  connection: local
  hosts: localhost
  vars:
    vault_addr: 'http://vault.skignes.fr:8200'
    # API path for the directory where the secrets are
    password_path: '/kv/data/my_directory'
  vars_prompt:
    - name: "vault_username"
      prompt: "Enter your Vault username"
      private: no
    - name: "vault_password"
      prompt: "Enter your Vault password"
      private: yes
  tasks:
    - name: Authenticate with Vault
      uri:
        url: "{{ vault_addr }}/v1/auth/userpass/login/{{ vault_username }}"
        method: POST
        body: "{{ {'password': vault_password} | to_json }}"
        headers:
          Content-Type: "application/json"
        status_code: 200
        return_content: yes
      register: vault_auth_response
      changed_when: false
      no_log: true

    - name: Set Vault token
      set_fact:
        vault_token: "{{ vault_auth_response.json.auth.client_token }}"
      no_log: true

    - name: Fetch Password
      set_fact:
        ssh_password: "{{ lookup('community.hashi_vault.hashi_vault', 'secret={{ password_path }} token={{ vault_token }} url={{ vault_addr }}')['key'] }}"
      no_log: true

    - name: Save SSH connection
      add_host: 
        name: 127.0.0.1
        ansible_ssh_user: "skignes"
        ansible_ssh_pass: "{{ ssh_password }}"
        ansible_ssh_port: 12345
      changed_when: false
      no_log: true

So if you read the Save SSH connection, you should have seen that it uses the same variables as when you create a plain text inventory.

name : The IP address of the connection you want to add (it can be a FQDN).
ansible_ssh_user: It is the username to connect to the host.
ansible_ssh_pass : It is the password to connect to the host.
ansible_ssh_port : It is the port to connect to the host.

When this new host is created, we can use it inside other plays as a normal host, just as if we had defined it in a plain text inventory.

The new host is not available inside the Play where we are defining the new hosts. Only on a new play.

Final Playbook

So now that we have everything to create an Inventory file, we can write a nice playbook to execute the ls command on every host.

---
- name: List everything
  gather_facts: false
  connection: local
  hosts: localhost
  vars:
    vault_addr: 'http://vault.skignes.fr:8200'
    # API path for the directory where the secrets are
    password_path: '/kv/data/my_directory'
  vars_prompt:
    - name: "vault_username"
      prompt: "Enter your Vault username"
      private: no
    - name: "vault_password"
      prompt: "Enter your Vault password"
      private: yes
  tasks:
    - name: Authenticate with Vault
      uri:
        url: "{{ vault_addr }}/v1/auth/userpass/login/{{ vault_username }}"
        method: POST
        body: "{{ {'password': vault_password} | to_json }}"
        headers:
          Content-Type: "application/json"
        status_code: 200
        return_content: yes
      register: vault_auth_response
      changed_when: false
      no_log: true

    - name: Set Vault token
      set_fact:
        vault_token: "{{ vault_auth_response.json.auth.client_token }}"
      no_log: true

    - name: Fetch Password
      set_fact:
        ssh_password: "{{ lookup('community.hashi_vault.hashi_vault', 'secret={{ password_path }} token={{ vault_token }} url={{ vault_addr }}')['key'] }}"
      no_log: true

    - name: Save SSH connection
      add_host:
        name: 127.0.0.1
        ansible_ssh_user: "skignes"
        ansible_ssh_pass: "{{ ssh_password }}"
        ansible_ssh_port: 12345
      changed_when: false
      no_log: true

- name: Execute a command
  hosts: all,!localhost
  tasks: 
    - name: Execute the 'ls' command
      command: chdir=/home/ ls
      register: ls_output
      changed_when: false

    - name: Print the output of the 'ls' command
      debug: 
        var: ls_output.stdout_lines

You need to exclude localhost because it is the host that will execute the playbook if no inventory file is provided.

Now you have a dynamic inventory with Hashicorp Vault !

Vault with Ansible