HashiCorp Vault: The Key to Secrets Management 🔐

I’ve embarked on my latest deep-dive into the HashiCorp ecosystem, and let me tell you there’s a lot to unpack! My focus right now? Vault. It’s one of those tools that, once you understand its capabilities, you can’t help but wonder how you ever managed without it.

Vault is all about secrets. Not the kind you whisper to a friend, but the kind that keep your infrastructure and applications secure—things like tokens, passwords, certificates, API keys, you name it. It provides a unified way to manage and protect these secrets, and it does so with a level of security that’s built for dynamic, distributed environments.

So, what makes Vault so special?

Secrets Management

Vault is like your digital Fort Knox for secrets. Whether it’s credentials for databases or API keys, Vault lets you store them securely. You have the option to keep secrets static or generate them dynamically on the fly.

• Dynamic Secrets: Instead of using static secrets that can go stale, Vault generates secrets on-demand with a lease time. Think of it like a guest pass that automatically expires keeping your environment secure without the hassle of manual rotation.
• Data Encryption: Vault doesn’t just store secrets; it provides encryption as a service. This means you can encrypt data both in transit and at rest without directly dealing with the encryption keys. Super useful if you want to add an extra layer of security to your apps.
• Access Control: Vault integrates with a bunch of identity providers to centralise authentication. Policies control who or what can access which secrets, making it easier to manage permissions at scale.
• Audit Logging: Every time someone or something accesses Vault, it’s logged. This is crucial for tracking, analyzing, and maintaining compliance over how your secrets are used and by whom.
• Secret Rotation and Revocation: Stale secrets are a security nightmare, and Vault’s got that covered. It automates secret rotation and revocation, ensuring that even if something slips through the cracks, it won’t stay vulnerable for long.

HashiCorp Vault is a powerhouse when it comes to securing modern infrastructure and applications. Whether you’re dealing with micro-services, dynamic cloud setups, or just a ton of sensitive data, Vault helps bring sanity to the madness of secrets management.

Over the next few posts I’ll be covering the capabilities of Vault and ultimately training myself up to tackle the Vault Operations Professional exam that is a mix of knowledge and practical scenarios that require hands-on experience.

Join along and learn something new, let’s dig in.

Enabling and Configuring Secret Engines

Let’s dive into one of the core pieces of HashiCorp Vault—Secret Engines. If you’ve just started your journey with Vault, this is where you’ll learn how to actually manage secrets like database credentials, tokens, certificates, and more.

Vault doesn’t store everything in one generic blob. Instead, it has secret engines, which are purpose-built plugins that handle different types of secrets. Whether you’re working with cloud provider keys, dynamic database credentials, or X.509 certificates, there’s a secret engine designed for that.

In this post, we’ll break down what secret engines are, walk through enabling and configuring a couple of the most common ones, and set up some exercises to get hands-on experience.

What Are Secret Engines?

Think of secret engines in Vault as components for securely managing, generating, or encrypting various types of secrets. Vault doesn’t store secrets the same way across the board. Instead, it has different plugins—each built to handle a specific type of secret. You can enable multiple engines within a single Vault instance or HA cluster, allowing you to interact with multiple services or platforms securely.

Common Secret Engines

Here are a few of the secret engines you’ll use most often:

• Key/Value (KV) Secrets Engine: Think of this as a highly secure key-value store for arbitrary secrets like API keys, passwords, or configuration files. It’s simple and highly flexible, but doesn’t offer some of the advanced functionality of other secret engines (like dynamic secrets).

Link to Offical Documentation

• Database Secrets Engine: A powerhouse for dynamically generating database credentials on the fly. Instead of using long-lived, static credentials that are hard to rotate, this engine allows Vault to create short-lived, unique credentials with minimal effort. Supported databases include MySQL, PostgreSQL, and MongoDB, to name a few.

Link to Offical Documentation

• AWS Secrets Engine: This engine helps you manage AWS iam credentials. You can dynamically generate AWS access keys and secret keys based on iam policies, rotating them automatically.

Link to Offical Documentation

• PKI Secrets Engine: This one is all about certificates. With Vault’s PKI engine, you can generate dynamic X.509 certificates, acting as a certificate authority to issue and renew certificates.

Link to Offical Documentation

Enabling and Configuring Secret Engines

Let’s get practical. Enabling a secrets engine in Vault is pretty straightforward. Vault allows you to enable engines at specific paths, which means you can enable multiple instances of the same engine with different configurations if you need to.

💡

One of the things I appreciate about HashiCorp Vault is its straightforward and intuitive command-line syntax. Whether you’re reading secrets, managing policies, or configuring authentication methods, the structure remains clean and consistent. 👌

How to Enable a Secret Engine

Here’s the syntax to enable a secret engine in Vault:

$ vault secrets enable -path=<path> <engine_type>

The following example enables the Key/Value secrets engine (version 2):

$ vault secrets enable -path=secret kv-v2

This enables the KV engine at the secret/ path. You can now start using it to store secrets.

Configuration

Once a secret engine is enabled, you can configure it by writing specific parameters to the engine’s path. The configuration varies based on the type of engine you are enabling. For instance, the database secret engine requires you to configure a plugin and connection settings to your database.

Let’s dig into a couple of examples to see this in action.

Storing a Secret

Now that the KV engine is enabled, let’s store a secret in Vault. We’ll store a simple username and password combination at secret/myapp/config.

$ vault kv put secret/myapp/config username="admin" password="passw0rd"

Retrieve the Secret

To retrieve the secret we just stored, use the vault kv get command.

$ vault kv get secret/myapp/config

Delete or Destroy Versions

If you want to delete a secret (or even specific versions), the KV engine supports that too. V2 gives us much more flexibility on this than V1.

$ vault kv delete secret/myapp/config

As you can see interacting with Vault secrets engines is simple yet powerful and this can also all be done programmatically through the API for your applications.

Configure Database Secrets Engine for MySQL

Dynamic secrets are where Vault really shines. In the following example we’ll enable the Database Secrets Engine to dynamically create short-lived credentials for a MySQL database.

Enable the Database Secrets Engine

First, let’s enable the database engine. This command will enable the engine at its default path of database/

$ vault secrets enable database

Configure the MySQL Plugin:

Now, we need to configure the database plugin by providing it with the necessary connection details to communicate with the MySQL instance. Replace the placeholders with actual values like your MySQL host and password.

$ vault write database/config/my-mysql-database \  
plugin_name=mysql-database-plugin \\
connection_url="{{username}}:{{password}}@tcp(127.0.0.1:3306)/" \\
allowed_roles="my-role" \\
username="root" \\
password="your_mysql_password"

💡

With this configuration we have defined how Vault will connect to the database and with which credentials. These can be rotated immediately so that only Vault knows them. This tutorial here runs through it in detail.

Create a Role

Next, we’ll create a role that Vault will use to generate dynamic credentials for our MySQL instance. The role defines what permissions those dynamically generated users will have.

$ vault write database/roles/my-role \\
db_name=my-mysql-database \\
creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}'; GRANT SELECT ON . TO '{{name}}'@'%';" \\
default_ttl="1h" \\
max_ttl="24h"

Here, we’ve told Vault to generate MySQL users with SELECT privileges and credentials that are valid for one hour, but can be extended up to 24 hours.

Generate Credentials

Finally, let’s generate dynamic MySQL credentials by reading from the database/creds/my-role path.

$ vault read database/creds/my-role

Vault will generate a new MySQL user with a random password and the credentials will automatically expire after the TTL set within the role.

💡

Note as it is Vault that is generating the credential on the fly within the database, the Vault node or cluster will need access to the database network port to perform this.

Best Practices

As with most security tools, proper planning for implementation will help you in the future. When enabling and configuring secret engines, here are a few best practices to keep in mind:

• Use Consistent Naming Conventions: Having a consistent path structure makes it easier to manage secrets and policies. For example, use /appname/environment/secrets for environment-specific secrets.
• Restrict Access Using Policies: Always ensure that access to secrets is tightly controlled. Use Vault’s policy system to define granular access controls and follow the principle of least privilege.
• Regularly Rotate Secrets: Especially when using static secrets, make sure to rotate them regularly. Vault can automate this process for dynamic secrets.

Wrapping Up

In this post, we covered the essentials of enabling and configuring Vault’s secret engines. Whether you’re storing static key-value pairs or generating dynamic database credentials, Vault provides a secure and flexible way to manage secrets across your infrastructure.

By practicing the commands above, you’ll gain the hands-on experience needed to confidently manage secret engines in a production environment. You can head over to the HashiCorp site and download the Vault binary for your operating system or spin up a vault container.

Remember: security is not a one-time task. It’s a continuous process of configuring, managing, and monitoring. Vault’s secret engines are just one piece of the puzzle, but they’re a powerful tool in your security toolkit.

Stay tuned for more as we dive into production hardening, auto unsealing, and other advanced Vault topics in upcoming posts!

I hope this helps someone.

Cheers