Secret management is a crucial factor in CI/CD provider selection.

SaaS at your own risk

in recent years, DevOps practices have increasingly become popular among organizations looking to move fast, and improve their SDLC process. With Continuous integration and Continuous deployment (CI/CD) being the most popular of these practices, CI/CD tool selection has become increasingly difficult. Security has also become a growing concern in the software supply chain.

When putting together a CI/CD stack, secret management should be at the core of this decision. Not compromising on other features like proper event triggers, available building blocks, the possibility to declare pipeline as code, pipeline reusability, proper UI for visualization, etc.

The recent CircleCi and Travis-Ci hacks further drive home the importance of CI/CD secret management. Selecting a 3rd party SaaS CI/CD service provider and managing secrets on their platform, pose a very high threat level to the security of your infrastructure. Although, there are interesting SaaS solutions like Buildkite that support dedicated secrets management systems like AWS Secrets Manager, GCP Secret Manager, or Hashicorp Vault.

GitHub actions vs Gitlab Ci

This leaves us with SCM specific CI/CD tools, like GitHub actions and Gitlab Ci. GitHub's marketplace for actions provides a host of external integrations, making integrating with external solutions easier. Using officially verified actions created by AWS, GCP, and Hashicorps gives GitHub an edge over Gitlab in our use case.

A secure pipeline

As earlier established, we don't want to Jam secrets into the GitHub UI. Secrets remain in the AWS secrets manager or AWS Systems Manager Parameter Store, AWS CodeBuild is used as the main build agent, and GitHub actions the main pipeline orchestrator.

Using AWS CodeBuild with GitHub Actions and Secret Manager offers several benefits, including:

• Rich event trigger from GitHub actions

• GitHub needs only codebuild:StartBuild, codebuild:BatchGetBuilds, logs:GetLogEvents permissions.

• Large ephemeral AWS Codebuild managed containers.

• Codebuild project in a private subnet with access to resources within the VPC e.g RDS instance.

• Secrets management in AWS secrets manager or AWS Systems Manager Parameter Store.

• Possibility to audit secret access in AWS CloudTrail.

Implementation

To implement this solution create:

• An AWS Codebuild project with the appropriate configuration, and IAM role.

• Prepare a buildspec.yml file. This file contains the steps and stages to be run in the AWS Codebuild container.

A sample buildspec.yml file:

version: 0.2
env:
  #Read secrets from parameter store
  parameter-store:
    key: "value"
    key: "value"
  #Read secrets from secrets manager
  secrets-manager:
    key: secret-id:json-key:version-stage:version-id
phases:
  build:
    commands:
      - echo "build stage"
  deploy:
    commands:
      - echo "deploy stage"

• Use the official AWS actions in the GitHub action workflow yaml file. The most important ones being: aws-credentials-action-for-github-actions and aws-codebuild-run-build-action-for-github-actions.

A sample GitHub actions yaml file:

- name: Configure AWS Credentials  
  uses: aws-actions/configure-aws-credentials@v1
  with:
    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    aws-region: us-east-2
- name: Run CodeBuild
  uses: aws-actions/aws-codebuild-run-build@v1
  with:
    #AWS Codebuild project properties
    project-name: CodeBuildProjectName
    buildspec-override: path/to/buildspec.yaml
    compute-type-override: compute-type
    environment-type-override: environment-type
    image-override: ecr-image-uri
    env-vars-for-codebuild: |
      custom,
      requester,
      event-name
  env:
    custom: my environment variable
    requester: ${{ github.actor }}
    event-name: ${{ github.event_name }}

Sample codes to create the necessary resources can be found on my GitHub.

This architecture combines the benefits of cloud providers while minimizing security risks.