Pipeline Refinement
Hard to believe, but this is already the sixth post in a series dealing with the basics of using HashiCorp Terraform to deploy resources in Azure using Azure DevOps. If you want to follow along with me in refining your pipeline codes and understanding the how and why, it would be a good idea to review and work through the previous episodes in the series:
- Prerequisites
- Repositories and Pipelines
- Build Pipeline and Resource Deployment
- Pipeline Security and Governance
- State File Storage Security
I will be writing with the assumption that you have all code at the same level as me after working through those posts, with a validation and build pipelines created and a develop and main branch in your Azure DevOps Repo.
In this episode I’m going to be looking at refining our pipeline code by adding formatting checks and putting some of our more sensitive variables into and Azure DevOps library. If you’ve been with me on the whole journey you now should have a relatively good understanding of the structure of pipelines, and be able to use them to deploy resources into Azure using basic Terraform code.
I’m writing this series as a memory aid for myself with the hope that others can use it that might be on the same learning path as me. It’s not a detailed guide into Terraform, DevOps or Azure, it probably doesn’t meet all the best practices that a DevOps engineer would use, but it’s what I’ve found works for me when learning how something works. I’m writing in the middle of 2024, so by the time you read this things might have moved on a little, but I suspect the core concepts will remain the same. As always I’d like to credit my friend and colleague James Meegan for his original documentation which has been used as a foundation for the series.
Terraform Format Checking
Hopefully by now you’re in the habit of running “terraform fmt –recursive” in your VS Code terminal before committing and syncing your code to your Azure DevOps repository. This not only makes sure the commands run correctly, but also makes it easier for anyone coming after you trying to read and understand the layout of your code. If you forget that step though, there’s nothing in the DevOps helpers that can check the format or stop the code from running if there’s something wrong with it. Because the DevOps agent is running Terraform commands however, we can put a line of script into our pipelines that runs the “terraform fmt” command. If we put that script before our existing steps, within the steps code block, it will check the formatting before anything runs, and the pipeline run will fail if it sees any issues. So, in our validation pipeline, we’ll add the following:
# Check Terraform Formatting
- script: |
terraform fmt -check -recursive --diff
displayName: Check Terraform Formatting
Note here that just as with our scripts in the last post, I’ve used the pipe (|) symbol after the “script” command to allow me to put the actual code on a separate line and make things easier to read. If you save this file, commit and sync, you can actually test that it works by adding something incorrectly formatted to one of your .tf files, saving and syncing that and watching the pipeline job:
Fix and sync your code, and watch it all work successfully:
We shouldn’t need to do this for our apply pipeline, because anything that is pulled into our main branch will already have passed the check in develop when that validation pipeline runs. If you want to though, remember that it’s your code, and you can do anything you want with it!
Use Pipeline Variables
In our last post, we started using variables to replace information that would be regularly used. It helps us prevent typos and potentially allows us to save a little time and effort. These variables and their values were defined within the code itself. We’ve also got in our pipeline code things like the storage account names, which might be considered sensitive information, and they would definitely differ for a different project or client, meaning the code isn’t immediately re-usable.
We fix this by moving our variable definitions to a variable library within Azure DevOps. We can then point our pipeline at the library and call the values of the variables from there. Not only does this meet best practice but it can allow us to encrypt sensitive values if we wish. So how do we choose what data to put in our variables library? Firstly, look for values that are repeated, like the service connection, the resource group, the storage account name etc. Then consider anything that seems like a hard-coded value that might change depending on the client or project. For your own code you should choose any values you want. For this example I’m going to start with the hard coded values for the backend state file storage account:
Now that we’ve determined which of our values will be replaced by variables, we need to come up with sensible names for those variables that can be easily understood at a later time by yourself or others. These names should be entered in your code, replacing the current values, in the format:
‘$(variable_name)’
We’ll make a start as usual with our validation pipeline. In the code snippet below, you can see that I’ve done this and also commented out the original values – this is just to help me remember the original values and the comments will be removed when the data is successfully stored in our variable library:
# Terraform Initialisation
- task: TerraformTaskV4@4
displayName: Run Terraform Init
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '$(backend_sc)' # 'terraform-series-sc'
backendAzureRmResourceGroupName: '$(backend_rg)' # 'uks-tfstatefiles-rg-01'
backendAzureRmStorageAccountName: '$(backend_saname)' # 'ukstfstatefilessa01'
backendAzureRmContainerName: '$(backend_container)' # 'tfstatefiles'
backendAzureRmKey: 'terraformstatefiles/terraform_series.tfstate'
We now need to create the variable library. In Azure DevOps, under Pipelines, select “Library”. Once there, select “+ Variable group”:
Give your variable group a name and description, then click “+ Add” when we can add our key value pairs of variable name and value, repeating the “+Add” for each one:
Remember to note the name of the variable library because we will need to refer to it from our pipelines. We now have a list of all our required variables in that library:
When done adding variables, remember to click “Save” at the top! You can come back and add more variables at any time. By default, Azure DevOps libraries are secured so that no YAML pipelines are able to access them. We need to make sure our library is accessible from our pipelines, so we need to remove those restrictions. After clicking Save, select “Pipeline permissions”:
In the resulting box, you can either press the three dots () “More Actions” button and open it to any of your pipelines, or you can select the “+” button and add whichever pipelines you wish:
I’ve done the latter and added both my apply and validation pipelines individually. So we’ve now got our variables library, it contains our variable key / value pairs, and our pipelines have permissions to use it. All that remains is to tell the pipeline where to look for its variables. We do that in the pipeline file itself by adding a “variables” block towards the top of the code, which includes a (properly indented!) “group” argument giving the name of the variables library. Switch to your validate pipeline in VS Code and add the appropriate lines after your “trigger” block:
# Look in the Azure DevOps variables library for variable key / value pairs
variables:
- group: 'test-variable-group'
Now seems a good time to save your file and push it up to your repository, and make sure the pipeline runs OK. If there are any issues, check your variable spellings, their values and the name of the variable library. When all is working as it should, you can remove the comments you set earlier as reminders, and then also replace any other examples of those references being used anywhere in your validation pipeline. After doing that, again commit and sync your code to make sure all runs OK.
We’ve now got a much more refined validation pipeline, but our apply pipeline remains as it was. You have the information now though to update that. You don’t need a new variables library, you can just add the same variables block as you did to the validation pipeline, pointing at the same library. You can replace all the same entries in your apply pipeline that use those key / value pairs. Look through both pipeline files. Are there any more repeating or potentially sensitive values that can be moved to your variables library? If so, move them, change things, play – you’re not going to break anything. If your pipeline doesn’t run, undo your last change and try again. Maybe play with VS Code’s search function; look for the value you want to replace, search for it and replace each time it’s found. It will all increase your awareness and understanding of the underlying concepts.
When you’re happy with your code, save your files, do a formatting check, then commit and sync. If your validation pipeline runs successfully, perform a pull into your main branch and make sure your apply pipeline runs OK. Again, if there are problems, check for spelling errors or differences in the name of your referenced library.
And that’s it for this one. In the next post we’ll be looking at Terraform modules and directories, but right now our pipelines are in a reasonably good place and we should have a sensible understanding of the YAML structure, and how to keep things a little more secure.
Until next time…
– The Zoo Keeper
4 comments