NGINX Tutorial: How to Use GitHub Actions to Automate Microservices Canary Deployments

Original: https://www.nginx.com/blog/nginx-tutorial-github-actions-automate-microservices-canary-deployments/
Chris Harrison
Christopher Harrison of GitHub
Enterprise Advocate
March 21, 2023

This post is one of four tutorials that help you put into practice concepts from Microservices March 2023: Start Delivering Microservices:

Automating deployments is critical to the success of most projects. However, it’s not enough to just deploy your code. You also need to ensure downtime is limited (or eliminated), and you need the ability to roll back quickly in the event of a failure. Combining canary deployment and blue‑green deployment is a common approach to ensuring new code is viable. This strategy includes two steps:

If you’re unfamiliar with the different use cases for distributing traffic between different versions of an app or website (traffic splitting), read How to Improve Resilience in Kubernetes with Advanced Traffic Management on our blog to gain a conceptual understanding of blue‑green deployments, canary releases, A/B testing, rate limiting, circuit breaking, and more. While the blog is specific to Kubernetes, the concepts are broadly applicable to microservices apps.

Tutorial Overview

In this tutorial, we show how to automate the first step of a canary blue‑green deployment using GitHub Actions. In the four challenges of the tutorial you use Microsoft Azure Container Apps to deploy a new version of your application, then use Azure Traffic Manager to shift traffic from the old environment to the new environment:

Note: While this tutorial uses Azure Container Apps, the concepts and techniques can be applied to any cloud‑based host.

Prerequisites and Setup

Prerequisites

If you want to do this tutorial in your own environment, you need:

Set Up

Create and configure the necessary base resources. Fork and clone the repository for the tutorial, log in to the Azure CLI, and install the extension for Azure Container Apps.

  1. In your home directory, create the microservices-march directory. (You can also use a different directory name and adapt the instructions accordingly.)

    Note: Throughout the tutorial the prompt on the Linux command line is omitted, to make it easier to copy and paste the commands into your terminal.

    mkdir ~/microservices-march
cd ~/microservices-march

  2. Fork and clone the Microservices March platform repository to your personal GitHub account, using either the GitHub CLI or GUI.

    • If using the GitHub GUI:

      1. Click Fork on the upper right corner of the window and select your personal GitHub account on the Owner menu.

        screenshot of GitHub GUI showing fork of the repository for this tutorial

      2. Clone the repository locally, substituting your account name for <your_GitHub_account>:

        git clone https://github.com/<your_GitHub_account>/platform.git
cd platform

    • If using the GitHub CLI, run:

      gh repo fork microservices-march/platform -–clone

  3. Login to the Azure CLI. Follow the prompts to log in using a browser:

    az login
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "cfd11e0f-1435-450s-bdr1-ffab73b4er8e",
    "id": "60efapl2-38ad-41w8-g49a-0ecc6723l97c",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Azure subscription 1",
    "state": "Enabled",
    "tenantId": "cfda3e0f-14g5-4e05-bfb1-ffab73b4fsde",
    "user": {
      "name": "user@example.com",
      "type": "user"
    }
  }
]

  4. Install the containerapp extension:

    az extension add --name containerapp -upgrade
The installed extension 'containerapp' is in preview.

Challenge 1: Create and Deploy an NGINX Container App

In this initial challenge, you create an NGINX Azure Container App as the initial version of the application used as the baseline for the canary blue‑green deployment. Azure Container Apps is a Microsoft Azure service you use to easily execute application code packaged in a container in a production‑ready container environment.

  1. Create an Azure resource group for the container app:

    az group create --name my-container-app-rg --location westus
{
  "id": "/subscriptions/0efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg",
  "location: "westus",
  "managedBy": null,
  "name": "my-container-app-rg",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

  2. Deploy the container to Azure Container Apps (this step may take a while):

    az containerapp up \
    --resource-group my-container-app-rg \
    --name my-container-app \
    --source ./ingress \
    --ingress external \
    --target-port 80 \
    --location westus
... 
- image:
    registry: cac085021b77acr.azurecr.io
    repository: my-container-app
    tag: "20230315212413756768"
    digest: sha256:90a9fc67c409e244195ec0ea190ce3b84195ae725392e8451...
  runtime-dependency:
    registry: registry.hub.docker.com
    repository: library/nginx
    tag: "1.23"
    digest: sha256:aa0afebbb3cfa473099a62c4b32e9b3fb73ed23f2a75a65ce...
  git: {} 
Run ID: cf1 was successful after 27s
Creating Containerapp my-container-app in resource group my-container-app-rg
Adding registry password as a secret with name "ca2ffbce7810acrazurecrio-cac085021b77acr" 
Container app created. Access your app at https://my-container-app.delightfulmoss-eb6d59d5.westus.azurecontainerapps.io/
...

  3. In the output in Step 2, find the name and the URL of the container app you’ve created in the Azure Container Registry (ACR). They’re highlighted in orange in the sample output. You will substitute the values from your output (which will be different from the sample output in Step 2) for the indicated variables in commands throughout the tutorial:

    • Name of container app – In the image.registry key, the character string before .azurecr.io. In the sample output in Step 2, it is cac085021b77acr.

      Substitute this character string for <ACR_name> in subsequent commands.

    • URL of the container app – The URL on the line that begins Container app created. In the sample output in Step 2, it is https://my-container-app.delightfulmoss-eb6d59d5.westus.azurecontainerapps.io/.

      Substitute this URL for <ACR_URL> in subsequent commands.

  4. Enable revisions for the container app as required by for a blue‑green deployment:

    az containerapp revision set-mode \
    --name my-container-app \
    --resource-group my-container-app-rg \
    --mode multiple
"Multiple"

  5. (Optional) Test that the deployment is working by querying the /health endpoint in the container:

    curl <ACR_URL>/health
OK

Challenge 2: Set Up Permissions for Automating Azure Container App Deployments

In this challenge, you obtain the JSON token that enables you to automate Azure container app deployments.

You start by obtaining the ID for the Azure Container Registry (ACR), and then the principal ID for your Azure managed identity. You then assign the built‑in Azure role for ACR to the managed identity, and configure the container app to use the managed identity. Finally you obtain the JSON credentials for the managed identity, which will be used by GitHub Actions to authenticate to Azure.

While this set of steps may seem tedious, you only need to perform them once when creating a new application and you can fully script the process. The tutorial has you perform the steps manually to become familiar with them.

Note: This process for creating credentials for deployment is specific to Azure.

  1. Look up the principal ID of your managed identity. It appears in the PrincipalID column of the output (which is divided across two lines for legibility). You’ll substitute this value for <managed_identity_principal_ID> in Step 3:

    az containerapp identity assign \
    --name my-container-app \
    --resource-group my-container-app-rg \
    --system-assigned \
    --output table
PrincipalId                          ...                           
------------------------------------ ...  
    39f8434b-12d6-4735-81d8-ba0apo14579f ...
 
    ... TenantId
    ... ------------------------------------
        ... cfda3e0f-14g5-4e05-bfb1-ffab73b4fsde

  2. Look up the container app’s resource ID in ACR, replacing <ACR_name> with the name you recorded in Step 3 of Challenge 1. You’ll substitute this value for <ACR_resource_ID> in the next step:

    az acr show --name <ACR_name> --query id --output tsv
/subscriptions/60efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg/providers/Microsoft.ContainerRegistry/registries/cac085021b77acr

  3. Assign the built‑in Azure role for ACR to the container app’s managed identity, replacing <managed_identity_principal_ID> with the managed identity obtained in Step 1, and <ACR_resource_ID> with the resource ID obtained in Step 2:

    az role assignment create \
    --assignee <managed_identity_principal_ID> \
    --role AcrPull \
    --scope <ACR_resource_ID>
{
  "condition": null,
  "conditionVersion": null,
  "createdBy": null,
  "createdOn": "2023-03-15T20:28:40.831224+00:00",
  "delegatedManagedIdentityResourceId": null,
  "description": null,
  "id": "/subscriptions/0efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg/providers/Microsoft.ContainerRegistry/registries/cac085021b77acr/providers/Microsoft.Authorization/roleAssignments/f0773943-8769-44c6-a820-ed16007ff249",
  "name": "f0773943-8769-44c6-a820-ed16007ff249",
  "principalId": "39f8ee4b-6fd6-47b5-89d8-ba0a4314579f",
  "principalType": "ServicePrincipal",
  "resourceGroup": "my-container-app-rg",
  "roleDefinitionId": "/subscriptions/60e32142-384b-43r8-9329-0ecc67dca94c/providers/Microsoft.Authorization/roleDefinitions/7fd21dda-4fd3-4610-a1ca-43po272d538d",
  "scope": "/subscriptions/ 0efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg/providers/Microsoft.ContainerRegistry/registries/cac085021b77acr",
  "type": "Microsoft.Authorization/roleAssignments",
  "updatedBy": "d4e122d6-5e64-4bg1-9cld-2aceeb0oi24d",
  "updatedOn": "2023-03-15T20:28:41.127243+00:00"
}

  4. Configure the container app to use the managed identity when pulling images from ACR, replacing <ACR_name> with the container app name you recorded in Step 3 in Challenge 1 (and also used in Step 2 above):

    az containerapp registry set \
    --name my-container-app \
    --resource-group my-container-app-rg \
    --server <ACR_name>.azurecr.io \
    --identity system
[
  {
    "identity": "system",
    "passwordSecretRef": "",
    "server": "cac085021b77acr.azurecr.io",
    "username": ""
  }
]

  5. Look up your Azure subscription ID. 

    az account show --query id --output tsv
0efafl2-38ad-41w8-g49a-0ecc6723l97c

  6. Create a JSON token which contains the credentials to be used by the GitHub Action, replacing <subscription_ID> with your Azure subscription ID. Save the output to paste in as the value of the secret named AZURE_CREDENTIALS in Add Secrets to Your GitHub Repository. You can safely ignore the warning about --sdk-auth being deprecated; it’s a known issue:

    az ad sp create-for-rbac \
    --name my-container-app-rbac \
    --role contributor \
    --scopes /subscriptions/<subscription_ID>/resourceGroups/my-container-app-rg \
    --sdk-auth \
    --output json
Option '--sdk-auth' has been deprecated and will be removed in a future release.
...
{
  "clientId": "0732444d-23e6-47fb-8c2c-74bddfc7d2er",
  "clientSecret": "qg88Q~KJaND4JTWRPOLWgCY1ZmZwN5MK3N.wwcOe",
  "subscriptionId": "0efafl2-38ad-41w8-g49a-0ecc6723l97c",
  "tenantId": "cfda3e0f-14g5-4e05-bfb1-ffab73b4fsde",
  "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
  "resourceManagerEndpointUrl": "https://management.azure.com/",
  "activeDirectoryGraphResourceId": "https://graph.windows.net/",
  "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
  "galleryEndpointUrl": "https://gallery.azure.com/",
  "managementEndpointUrl": "https://management.core.windows.net/"
}

Challenge 3: Create a Canary Blue-Green Deployment GitHub Action

In this challenge, you add secrets to your GitHub repo (used to manage sensitive data in your GitHub Action workflows), create an Action workflow file, and execute the Action workflow.

For a detailed introduction to secrets management, see the second tutorial for Microservices March 23, How to Securely Manage Secrets in Containers on our blog.

Add Secrets to Your GitHub Repository

To deploy a new version of the application, you need to create a series of secrets in the GitHub repository you forked in Set Up. The secrets are the JSON credentials for the managed identity created in Challenge 2, and some sensitive deployment‑specific parameters necessary to deploy new versions of the NGINX image to Azure. In the next section you’ll use these secrets in a GitHub Action to automate the canary blue‑green deployment.

Create a GitHub Action Workflow File

With the managed identity and secrets in place, you can create a workflow file for a GitHub Action that automates the canary blue‑green deployment.

Note: Workflow files are defined in YAML format, where whitespace is significant. Be sure to preserve the indentation shown in the steps below.

  1. Create a file for the Action workflow.

    • If using the GitHub GUI:

      1. Navigate to your GitHub repository.
      2. Select Actions > New workflow > Skip this and set up a workflow yourself.

    • If using the GitHub CLI, create the .github/workflows directory and create a new file called main.yml:

      mkdir .github/workflows
touch .github/workflows/main.yml

  2. Using your preferred text editor, add the text of the workflow to main.yml. The easiest method is to copy in the text that appears in Full Workflow File. Alternatively, you can build the file manually by adding the set of snippets annotated in this step.

    Note: Workflow files are defined in YAML format, where whitespace is significant. If you copy in the snippets, be sure to preserve the indentation (and to be extra sure, compare your file to Full Workflow File.

    • Define the workflow’s name:

      name: Deploy to Azure

    • Configure the workflow to run when a push or pull request is made to the main branch:

      on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

    • In the jobs section, define the build-deploy job, which checks out the code, logs into Azure, and deploys the application to Azure Container App:

      jobs:
  build-deploy:
    runs-on: ubuntu-22.04
    steps:
      - name: Check out the codebase
        uses: actions/checkout@v3

      - name: Log in to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Build and deploy Container App
        run: |
          # Add the containerapp extension manually
          az extension add --name containerapp --upgrade
          # Use Azure CLI to deploy update
          az containerapp up -n ${{ secrets.CONTAINER_APP_NAME }}\
            -g ${{ secrets.RESOURCE_GROUP }} \
            --source ${{ github.workspace }}/ingress \
            --registry-server ${{ secrets.ACR_NAME }}.azurecr.io

    • Define the test-deployment job, which obtains the staging URL of the newly deployed revision and uses a GitHub Action to ping the API endpoint /health to ensure the new revision is responding. If the health check succeeds, the Azure Traffic Manager on the container app is updated to point all traffic at the newly deployed container.

      Note: Be sure to indent the test-deployment key at the same level as the build-deploy key you defined in the previous bullet:

        test-deployment:
    needs: build-deploy
    runs-on: ubuntu-22.04
    steps:
      - name: Log in to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Get new container name
        run: |
          # Add the containerapp extension manually
          az extension add --name containerapp --upgrade

          # Get the last deployed revision name
          REVISION_NAME=`az containerapp revision list -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --query "[].name" -o tsv | tail -1`
          # Get the last deployed revision's fqdn
          REVISION_FQDN=`az containerapp revision show -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision "$REVISION_NAME" --query properties.fqdn -o tsv`
          # Store values in env vars
          echo "REVISION_NAME=$REVISION_NAME" >> $GITHUB_ENV
          echo "REVISION_FQDN=$REVISION_FQDN" >> $GITHUB_ENV

      - name: Test deployment
        id: test-deployment
        uses: jtalk/url-health-check-action@v3 # Marketplace action to touch the endpoint
        with:
          url: "https://${{ env.REVISION_FQDN }}/health" # Staging endpoint

      - name: Deploy succeeded
        run: |
          echo "Deployment succeeded! Enabling new revision"
          az containerapp ingress traffic set -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision-weight "${{ env.REVISION_NAME }}=100"

Full Workflow File

This is the complete text for the Action workflow file.

name: Deploy to Azure
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  build-deploy:
    runs-on: ubuntu-22.04
    steps:
      - name: Check out the codebase
        uses: actions/checkout@v3

      - name: Log in to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
     
      - name: Build and deploy Container 
        run: |
          # Add the containerapp extension manually
          az extension add --name containerapp -upgrade
       
          # Use Azure CLI to deploy update
          az containerapp up -n ${{ secrets.CONTAINER_APP_NAME }} \
            -g ${{ secrets.RESOURCE_GROUP }} \
            --source ${{ github.workspace }}/ingress \
            --registry-server ${{ secrets.ACR_NAME }}.azurecr.io
  test-deployment:
    needs: build-deploy
    runs-on: ubuntu-22.04
    steps:
      - name: Log in to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Get new container name
        run: |
          # Install the containerapp extension for the Azure CLI
          az extension add --name containerapp --upgrade
          # Get the last deployed revision name
          REVISION_NAME=`az containerapp revision list -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --query "[].name" -o tsv | tail -1`
          # Get the last deployed revision's fqdn
          REVISION_FQDN=`az containerapp revision show -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision "$REVISION_NAME" --query properties.fqdn -o tsv`
          # Store values in env vars
          echo "REVISION_NAME=$REVISION_NAME" >> $GITHUB_ENV
          echo "REVISION_FQDN=$REVISION_FQDN" >> $GITHUB_ENV

      - name: Test deployment
        id: test-deployment
        uses: jtalk/url-health-check-action@v3 # Marketplace action to touch the endpoint
        with:
          url: "https://${{ env.REVISION_FQDN }}/health" # Staging endpoint

      - name: Deploy succeeded
        run: |
          echo "Deployment succeeded! Enabling new revision"
          az containerapp ingress traffic set -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision-weight "${{ env.REVISION_NAME }}=100"

Execute the Action Workflow

Challenge 4: Test the GitHub Actions Workflow

In this challenge, you test the workflow. You first simulate a successful update to your Ingress load balancer and confirm the application has been updated. You then simulate an unsuccessful update (which leads to an internal server error) and confirm that the published application remains unchanged.

Make a Successful Update

Create a successful update and watch the workflow succeed.

Make an Unsuccessful Update

Now create an unsuccessful update and watch the workflow fail. This basically involves repeating the steps in Make a Successful Update but with a different value for the return directive.

Resource Cleanup

You probably want to remove the Azure resources you deployed in the tutorial to avoid any potential charges down the line:

az group delete -n my-container-app-rg -y

You can also delete the fork you created if you wish.

Next Steps

Congratulations! You’ve learned how to use GitHub Actions to perform a canary blue‑green deployment of a microservices app. Check out these articles in the GitHub docs to continue exploring and growing your knowledge of DevOps:

If you’re ready to try out the second step of a canary deployment (testing in production) then check out the tutorial from Microservices March 2022, Improve Uptime and Resilience with a Canary Deployment on our blog. It uses NGINX Service Mesh to gradually transition to a new app version. Even if your deployments aren’t yet complex enough to need a service mesh, or you’re not using Kubernetes, the principles still apply to simpler deployments only using an Ingress controller or load balancer.

To continue your microservices education, check out Microservices March 2023. In Unit 3, Accelerate Microservices Deployments with Automation, you’ll learn more about automating deployments.

Banner reading 'Microservices March 2023: Sign Up for Free, Register Today'

Retrieved by Nick Shadrin from nginx.com website.