Deploying

Helm

To get started with deploying a Django project we need to first create a helm chart in the project:

helm create <project>

This will create a helm chart that has everything you need to deploy a project. Further, to build this entry and deploy it we need to a

CI/CD

Add the following into your CI/CD to deploy to AWS. This will build and image.

deploytag \
    --cloud aws \
    --cloud-aws-secret-id <cluster/secret> \
    --app-aws-secret-ids <app/secret> \
    build --dotenv-file=<secret_file_in_container>  \
    --container-registry 1234.dkr.ecr.us-west-2.amazonaws.com \
    --project-id prefix \
    --image app_image

What is happening here?

  • You are specifying the Cloud
  • You are pulling the credentials used to push to the correct cloud with the cloud-aws-secret-id
  • You are getting the application level environment vairables with app-aws-secret-ids
  • The build is building a Docker image and pushing it to the repo.

    • container-registry is the registry to push the image to. Here you specify your account in ECR.
    • project-id is a prefix that you can affix to the iamge.
    • image is the name of the image.

When you want to deploy your image you can use the following:

deploytag \
    --cloud aws \
    --cloud-aws-secret-id <cluster/secrets> \
    --app-aws-secret-ids <app/secret> \
    deploy \
    --chart-name <helm/chart> \
    --helm-set image.repository=1234.dkr.ecr.us-west-2.amazonaws.com/<project-id>/<image> \
    --helm-set image.pullPolicy=Always
  • You are pulling the credentials used to push to the correct cloud with the cloud-aws-secret-id
  • You are getting the application level environment vairables with app-aws-secret-ids
  • The deploy is going to deploy the application to the cluster.

    • It uses the current git branch name and uses that as the tag of the Docker image.
    • It takes the secrets from app-aws-secret-ids and adds it to the Pods in the cluster as a volume that mounts a secret.
  • chart-name is the location of the Helm chart
  • helm-set passes all commands to the helm upgrade --install

Ths deploy also sets additional variables include DEPLOYTAG_BRANCH into the environment variable.

The code is located here: https://github.com/opszero/deploytag/tree/go-rewrite

Branch Deploys

You can configure the deploys to only happen on git branches with certain prefix such as feature/, epic/, bug/.

CircleCI

Here is how to put it all together in CircleCI

references:
  deploy_image: &deploy_image
    docker:
      - image: opszero/deploytag:go-rewrite

# use CircleCI 2.1
version: 2.1
jobs: # a collection of steps
  test: # runs not using Workflows must have a `build` job as entry point
    parallelism: 1 # run three instances of this job in parallel
    docker: # run the steps with Docker
      - image: circleci/python:3 # ...with this image as the primary container; this is where all `steps` will run
        environment: # environment variables for primary container
          BUNDLE_JOBS: 3
          BUNDLE_RETRY: 3
      - image: circleci/postgres:11.1-alpine-postgis-ram
        environment: # environment variables for database
          POSTGRES_USER: postgres
          POSTGRES_DB: bigco_test
          POSTGRES_PASSWORD: "postgres"
      - image: circleci/redis:5.0
    steps: # a collection of executable commands
      - checkout # special step to check out source code to working directory
      - run:
          name: Pip Install
          command: |
            pipenv install --dev
      - run:
          name: Install psql client
          command: |
            sudo apt-get update && sudo apt-get install postgresql-client
      - run:
          name: Run pytest
          command: |
            cp .env.test .env
            pipenv run pytest

      # Save test results for timing analysis
      - store_test_results:
          path: test_results

  build:
    <<: *deploy_image
    steps:
      - setup_remote_docker
      - checkout
      - run:
          name: Build
          command: |
            deploytag --cloud aws \
                      --cloud-aws-secret-id staging/cluster \
                      dns \
                      --cloudflare-zone-id 9236a8ed9a7141da712b2863de6326bb \
                      --record '{.Branch}-guest-server-frontend-aws' \
                      --record '{.Branch}-guest-server-server-aws'

            deploytag \
              --cloud aws \
              --cloud-aws-secret-id staging/cluster \
              --app-aws-secret-ids=staging/bigco \
              build --dotenv-file=env_secrets  \
              --container-registry 1234.dkr.ecr.us-west-2.amazonaws.com \
              --project-id bigco-inc \
              --image bigco

  deploy_feature:
    <<: *deploy_image
    steps:
      - checkout
      - run:
          name: Deploy
          command: |
            deploytag \
              --cloud aws \
              --cloud-aws-secret-id=staging/cluster \
              --app-aws-secret-ids=staging/bigco \
              deploy \
              --chart-name charts/bigco \
              --helm-set image.repository=1234.dkr.ecr.us-west-2.amazonaws.com/bigco-inc/counterpart \
              --helm-set image.pullPolicy=Always

  deploy_prod:
    <<: *deploy_image
    steps:
      - checkout
      - run:
          name: Deploy
          command: |
            deploytag \
              --cloud aws \
              --cloud-aws-secret-id=prod/cluster \
              --app-aws-secret-ids=prod/bigco  \
              deploy \
              --chart-name charts/bigco \
              --helm-set image.repository=1234.dkr.ecr.us-west-2.amazonaws.com/bigco-inc/counterpart \
              --helm-set image.pullPolicy=Always

# Workflows
workflows:
  version: 2
  test_and_deploy:
    jobs:
      - test:
          context: org-global
      - build:
          context: org-global
          requires:
            - test
          filters:
            branches:
              only:
                - master
                - qa
                - /^(bug|epic|feature_deploy)\/.*/
      - deploy_feature:
          context: org-global
          requires:
            - build
          filters:
            branches:
              only:
                - qa
                - /^(bug|epic|feature_deploy)\/.*/
      - deploy_prod:
          context: org-global
          requires:
            - build
          filters:
            branches:
              only:
                - master

Troubleshooting

UWSGI Logs

kubectl get pods --all-namespaces
kubectl exec -it -n <namespace> <pod> -c <container> -- bash
tail -f logs/log.log