In How to manage secrets with dotnet user secrets I walked through the process of how to use the built in secret manager in Dotnet to safely store and use secrets for your dotnet based projects. The solution detailed there could be a great solution if you're single developer or you're working on a really small team, and you're managing really small scale deployments. However, that is not typically how developers tend to work in Enterprise environments and we often need far more scalable solutions to solve this particular issue.
When developing larger applications and environments you may need to have different secrets for different environments and need to a be able share these secrets with many developers who may be geographically disperesed.
Fortunately most cloud providers and platforms provide and mechanism to share sensitive information, primarily to faciliate sharing across multiple different environments and even regions. However, making use of these services for development can also be beneficial.
In this post we are going to take a walk-through making use of Azure Key Vault in an ASP.net REST API applications.
What is Azure Key Vault
Azure Key Vault is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys. Key Vault service supports two types of containers: vaults and managed Hardware Security Module(HSM) pools.
Azure Key Vault
helps solve the following problems:
- Secrets Management - Azure Key Vault can be used to Securely store and tightly control access to tokens, passwords, certificates, API keys, and other secrets
- Key Management - Azure Key Vault can be used as a Key Management solution. Azure Key Vault makes it easy to create and control the encryption keys used to encrypt your data.
- Certificate Management - Azure Key Vault lets you easily provision, manage, and deploy public and private Transport Layer Security/Secure Sockets Layer (TLS/SSL) certificates for use with Azure and your internal connected resources.
Check out Azure Key Vault basic concepts to gain a broader understanding and common terminology used with Key Vault.
Why use Azure key Vault
Using a Secret Manager like Azure Key Vault is very different compared to use the Dotnet Secret manager in that the data doesn't simply stay in a file on your server or local computer. This information is stored in hardware device and the device offers you many features like auditing, tamper-proofing, encryption, etc. What Microsoft provides in the form of Azure Key Vault is an interface using which you can access the HSM device in a secure way.
What about the cost?
- Zero setup fee
- Secrets: $0.03/10,000 requests
- Keys: $1 per key per month
- Certificates: $3 per renewal request
Best Practices
Always try use separate Key Vaults for your projects and even environments in your projects. Don't try use one Key Vault for everything. The recommended approach is to use a vault per application per environment and per region.
The benefit of this approach is that it helps not to share secrets across environments and regions. However, there is also a major security benefit in that it will also minimise the threat of any breaches.
Continuous Architecture in Practice discusses Security as an Architectural Concern and the 3 main principles of secrets management:
- Choosing Good Secrets
- keeping them secret
- changing them reliably
It is also within this context, the primary reasons why you and your organisation shouldn't choose just one secret manager for all your secrets. The key take away is that you should ideally have a KeyVault for each service or application.
If you run into a particular case where you find yourself in situation where it is necessary to share secrets across many different application, then it may be an opportunity to store those particular secrets in a shared Vault enabling the opportunity to manage those particular secrets effectively.
Best practices for using Azure Key Vault
Continuous Architecture in Practice
Software Architecture In the age of Agility and Devops
in-depth guidance for addressing today's key quality attributes and cross-cutting concerns such as security, performance, scalability, resilience, data, and emerging technologies. Each key technique is demonstrated through a start-to-finish case study reflecting the authors’ deep experience with complex software environments.
How to create an Azure key Vault
There are a number of ways you can create an Azure Key vault i.e. directly using the Azure Portal Dashboard, or using Terraform or Pulumi etc. However, for the purpose of this article I am going to assume you have an Azure Account and Subscription and have installed the Azure CLI .
My preferred method of Installing the Azure CLI is by making use of Homebrew
brew install azure-cli
Once your Azure CLI is installed ensure you have authenticated and assigned your default subscription.
public class UsernameFactoryTests
{
private UsernameFactory _factory;
public UsernameFactoryTests()
{
_factory = new UsernameFactory();
}
[Fact]
public void ShouldGetFirstNameFirst()
{
//arrange
var user = "Gary Woodfine";
//act
var username = _factory.GetUserName(user);
//assert
Assert.Equal("Gary", username.FirstName);
Assert.Equal("Woodfine", username.LastName);
}
[Fact]
public void ShouldGetLastNameFirst()
{
//arrange
var user = "Woodfine, Gary";
//act
var username = _factory.GetUserName(user);
//assert
Assert.Equal("Gary", username.FirstName);
Assert.Equal("Woodfine", username.LastName);
}
}
Before creating an Azure Key Vault we'll need to create our Resource Group
A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to add resources to resource groups based on what makes the most sense for your organization.
Manage Azure Resource Groups by using Azure CLI
In my case I want to create a Development Resource Group for all the resources that are going to be used by my project, in my particular case I am using the ukwest region, but you should set it to whatever region is best for your particular use case.
az group create --name VaultTutorialRG --location ukwest
Manage Azure Resource Groups by using Azure CLI
Typically we want to create a Resource Group for our project and the different environments in our project, so as above I have created Resource Group for my Development project.
Now that we have created our Resource Group we can start creating all the resources we will need for our project. In the case of this tutorial we're going to focus on creating the Azure Key Vault. We can create our Azure Key Vault using the Azure CLI.
We'll use a similar naming convention for the name of our Azure resource we're creating, typically I use the name of the project with the capitalised Initials of the resource and the post-fix of the environment. VaultTutorialKV-dev.
az keyvault create --name VaultTutorialKV-dev --resource-group VaultTutorialRG --location ukwest
Create a key vault using the Azure CLI
We'll wait a few seconds and then our new key vault will be created and we should get confirmation.
With our Key Vault freshly created we can now go ahead and add our first secret to it. The first step is to actually create the Key. My my purposes I am going to create a key and name it SecretKey
az keyvault key create --vault-name VaultTutorialKV-dev --name SecretKey
This will create my key file but at the moment it does not actually create a secret value. I will go ahead and set this value now.
az keyvault secret set --vault-name VaultTutorialKV-dev --name SecretKey --value IGotTheKeyIGotTheSecret
Manage Key Vault using the Azure CLI
At this stage we have created our Azure Key Vault and added our secret we want to use.
All Code Samples for this Tutorial are available
How to use Azure Key Vault in dotnet Web API projects
We're going to create a new REST API project making use of the API Template Pack . I already have the API Template Pack installed so will create a new API Solution project and name it Diogel.
dotnet new apisolution --name VaultTutorial --root Threenine
How to use the API Solution Template
This will generate a new API Solution project template ready for us to start implementing a REST API using the Vertical Slice Architecture and REPR pattern
In order to make use of the Azure Key Vault in our project we need to add some additional nuget references to our Api project
dotnet add src/Api/Api.csproj package Azure.Identity dotnet add src/Api/Api.csproj package Microsoft.Extensions.Azure dotnet add src/Api/Api.csproj package Azure.Security.KeyVault.Secrets dotnet add src/Api/Api.csproj package Azure.Extensions.AspNetCore.Configuration.Secrets dotnet add src/Api/Api.csproj package Microsoft.Extensions.Configuration.AzureKeyVault
Our Next step we want to create a new class in our Common Project that will be a class that we will use to create a Strongly Typed settings value to store our Key Vault Name. This is not a essential but I like to do this ensure that we have a strongly typed setting we can reuse in our code.
dotnet new class --name KeyVaultSettings --output ./src/Common/
Once the class is generated we can add our new property to store the Key Vault name, which we'll name Vault
public class KeyVaultSettings
{
public string? Vault { get; set; }
}
We can also add some configuration values to our appsettings.json to provide a name of the Vault we want to use for our secrets
{
"KeyVault": {
"vault": "VaultTutorialKV-dev"
},
"AllowedHosts": "*"
}
We also want to add an additional Application Constants file which we'll use to add Constants we will want to use throughout our application to minimize the use of magic strings
dotnet new class --name ApplicationConstants --output ./src/Common/
How to get a single secret using Azure Secret Client
We can start configuring our application now, so we need to add the following lines to our Program.cs to configure the Dependency Injection of our Azure Clients. We need to first retrieve the value from our appsettings.json , then we will use the AddAzureClients extension method to add it to our application dependency injection container. We will then use addSecretClient to make the Azure Key Vault client to our application.
var keyVault = builder.Configuration.GetSection(ApplicationConstants.KeyVault).Get<KeyVaultSettings>();
builder.Services.AddAzureClients(clientBuilder =>
{
// Add a KeyVault client
clientBuilder.AddSecretClient(new Uri($"https://{keyVault?.Vault}.vault.azure.net/"));
clientBuilder.UseCredential(new DefaultAzureCredential());
});
- DefaultAzureCredential is used for authentication.
DefaultAzureCredentialchooses the best authentication mechanism based on your environment, allowing you to move your app seamlessly from development to production with no code changes.
The DefaultAzureCredential is appropriate for most scenarios where the application is intended to ultimately be run in Azure. This is because the DefaultAzureCredential combines credentials commonly used to authenticate when deployed, with credentials used to authenticate in a development environment.
Check out the Azure Identity client library for .NET - version 1.8.2 for more details on Azure Active Directory (Azure AD) token authentication support across the Azure SDK. It provides a set of TokenCredential implementations which can be used to construct Azure SDK clients which support Azure AD token authentication.
For now that is all we have to do. The Azure Key vault client is now ready to be used where we need to use it.
The next step we can do is make use of the API Template Pack to add Query endpoint to illustrate how we could use it our application. Lets add the end point making using of the terminal.
dotnet new query --name Get --output ./src/Api/Activities/WhatYouGot --resource WhatYouGot
This will generate the files for our endpoint as follows
We can edit the Get.Response.cs file to add a property for our return.
public class Response
{
public string WhatYouGot { get; set; }
}
With this in place we can now edit our Handler file as follows to get the value from Azure Key Vault. We will inject the Azure Secret Client into our handler.
public class Handler : IRequestHandler<Query, SingleResponse<Response>>
{
private readonly SecretClient _secretClient;
public Handler(SecretClient secretClient)
{
_secretClient = secretClient;
}
public async Task<SingleResponse<Response>> Handle(Query request, CancellationToken cancellationToken)
{
var secret = await _secretClient.GetSecretAsync("SecretKey", cancellationToken: cancellationToken);
return new SingleResponse<Response>(new Response{ WhatYouGot = secret.Value.Value });
}
}
If we run our application to execute our endpoint using the swagger we'll see it execute and our secret value will be displayed.
How to get configuration data using Azure Key Vault
While to above approach is pretty cool and provides a mechanism for getting secret data into your while running, it's not typically how I normally use Key Vault. Typically I use it to store all sensitive configuration data for the application at start up. So items like Database Connection strings, API Keys etc. We typically want to get all this Data when the application is starting up.
Fortunately this is really easy to do using the Azure extensions and it literally requires just a couple of lines of code. If we add the code below to our Program.cs
var keyVault = builder.Configuration.GetSection(ApplicationConstants.KeyVault).Get<KeyVaultSettings>();
builder.Configuration.AddAzureKeyVault(new Uri($"https://{keyVault?.Vault}.vault.azure.net/"),
new DefaultAzureCredential(),
new KeyVaultSecretManager());
var connectionString = builder.Configuration.GetConnectionString(ConnectionsStringName);
builder.Services.AddDbContext<VaultTutorialKVContext>(x => x.UseNpgsql(connectionString)).AddUnitOfWork<VaultTutorialKVContext>();
We can use the Azure CLI to upload our Secret to Key Vault as follows:
az keyvault secret set --vault-name VaultTutorialKV-dev --name "ConnectionStrings--DefaultConnection" --value " User ID=VaultTutorialKV;Password=Password12@;Host=localhost;Port=5432;Database=VaultTutorialKV;Pooling=true;Integrated Security=true;"
We can then update our appsettings.Development.json to remove our connection string stored there.
"ConnectionStrings": {
"DefaultConnection": "<Stored in Key Vault>"
},
Conclusion
This post introduced the Azure Key Vault and how you can utilise it in your applications to securely store and use sensitive data in your application. It also discusses some of the best practices to follow when making use of secret managers in your cloud native applications.
- What is this Directory.Packages.props file all about? - January 25, 2024
- How to add Tailwind CSS to Blazor website - November 20, 2023
- How to deploy a Blazor site to Netlify - November 17, 2023