This article describes the current and preferred method for managing how secrets are configured for custom applications. 


The primary tools used by this process are:

  • Azure DevOps
    • Release pipelines
    • Pipeline Libraries
  • Azure Key Vault
  • Terraform


This does mean that a new deployment is necessary whenever a secret dependency is updated since secrets are provided to the application when the application is deployed.  A different strategy should be used ff it's necessary to update a secret at run-time, without a deployment.


Release Pipeline


The primary functionality of this process is done by the "Variable Substitution" action during the release pipeline for a project.  This action can be a standalone step in a release pipeline, as shown in the first screenshot below.  It may also be executed as part of a more sophisticated step, such as the management of an Azure App Service (second screenshot) or deployment to an IIS web application (third screenshot).


The documentation for these actions is as follows:


Replace tokens with variable values in XML or JSON configuration files
File transformation and variable substitution task: Update tokens in your XML based configuration files and then replaces those tokens with variable values.
Currently only XML, JSON file formats are supported for variable substitution.


Also: 

Provide new line separated list of JSON files to substitute the variable values. Files names are to be provided relative to the root folder.
To substitute JSON variables that are nested or hierarchical, specify them using JSONPath expressions.

For example, to replace the value of ‘ConnectionString’ in the sample below, you need to define a variable as ‘Data.DefaultConnection.ConnectionString’ in the build or release pipeline (or release pipeline's environment).
{
  "Data": {
    "DefaultConnection": {
      "ConnectionString": "Server=(localdb)\SQLEXPRESS;Database=MyDB;Trusted_Connection=True"
    }
  }
}
Variable Substitution is run after configuration transforms.

Note: pipeline variables are excluded in substitution.



Variable Substitution for the appsettings.json file


Variable Substitution for the appsettings.json file


Not Used in this pipeline example

Release Variables


Variables are specified for the release pipeline, which are used by the release pipeline steps to substitute placeholders from the settings file in source code with the real values used in the deployed application.  


Dot separated variable names are matched the schema from the configuration file matched in the release pipeline step. For example "AppSettings.OAPassword" would match


{
  "AppSettings": {
    "OAPassword": "SuperSecretPassword"
  }
}


The value of the matching "OAPassword" variable from the variable groups will be substituted for "SuperSecretPassword" value. The "$" and parenthesis wrap variable names from variable groups available within the context of the prescribed "Scope".  For example "$(Foo)" reference the "Foo" variable.  References to these pipeline variables, or variable group variables, can be made with the $() wrapping within steps of the release pipeline.


The string used to name the pipeline variable needs to match the string used to define the variable in the configuration file from source code.  It does not need to match string used for the variable in the variable group.  Likewise, the string used for the variable name in the variable group, does not need to match the string for the secret record in Key Vault.  The matching only needs to happen when you're using the $() reference mechanism. 


For example, from the schema given above, the pipeline variable needs to be "AppSettings.OAPassword" and the value in the pipeline variable might be "$(OpenAirPassword)".  The string "OpenAirPassword" needs to match the variable group records name.  The variable group record with the name "OpenAirPassword" might then reference a Key Vault record called "OpenAirAPIUserPassword".  So, the "OpenAirAPIUserPassword" record from Key Vault, is picked up the "OpenAirPassword" variable group record, which is in turn referenced by the pipeline variable with "$(OpenAirPassword)" and substituted for the "SuperSecretPassword" placeholder value in the configuration file during the deployment.  





Variables from a variable group are usually references to Azure Key Vault secrets as described below. 



Secrets don't have to come from variables groups tied to release libraries and key vault records.  They can be put directly in the variable table, but this should ONLY be done sparingly when this will be the ONLY location this secret is used.  Checking the column for the lock will protect the value of the variable as a secret in both the UI and in the log outputs from pipeline steps.  


The "Scope" is generally used to separate a production from beta environment and release phases.  The scope can be set accordingly to match.  



Release Library


The release library is used to create collections of variables that can be used by pipelines within the project.  They can contain manually prescribed variables, but we mostly use it to gather secrets from Azure Key Vault to make available to release pipelines.  


A service connection for the workspace must be defined using a service principal that has permission to read the secrets from Key Vault. You can then pick the secrets you'd like to make available to the variable group after specifying with service connection and the key vault that service connection has permissions for.



Azure Subscription - Service Connection


Referenced Azure Key Vault Record



Project Service Connections


Azure Key Vault


Azure Key Vault secrets should be created with Terraform when the related project is using Terraform to prescribe and configure the related resources.  For example, in circumstances such as the CGGS apps configurations that need secrets from the GP1 tenant, the secret is defined in the Terraform config for the CGGS Azure resources with an empty string in the Terraform configuration, and the Terraform object for the secret is defined to ignore changes to the Key Vault secret value. The Key Vault and secret record are Azure CGGS resources, but the value will come from a GP1 object and be manually placed in the secret by the IT team.  


We don't want to open the GP1 tenant to CGGS service principals.  Therefore, the GP1 application and secret, username and password, or other objects with a secret involved, are manually created in the GP1 tenant if there is no other Terraform config with an existing related purpose.  The value of the secret is then manually placed in the CGGS Key Vault secret record.


The service principal used with the service connection in Azure DevOps needs to have read access to the Key Vault secret(s) in order to pull them into the Azure DevOps release configurations.