Serverless FastAPI CI/CD with CircleCi

Automate testing and deployments by creating a Serverless FastAPI CI/CD (Continuous Integration and Continuous Deployment) pipeline using CircleCi and Github.

TL;DR

Automate testing and deployments by creating a Serverless FastAPI CI/CD (Continuous Integration and Continuous Deployment) pipeline using CircleCi and Github.

The Serverless FastAPI will run on an AWS Lambda and AWS API Gateway will handle routing all requests to the Lambda.

I originally came across Mangum through this awesome blog post here that shows an example of Continuously Deploying a FastAPI to AWS Lambda with SAM CLI and Travis.

Setting up Serverless FastAPI with AWS Lambda and API Gateway

In this walkthrough we are going to assume you have already setup the necessary AWS Resources.

If you want to see a way to do this you can check out this post. It walks through setting up the Serverless FastAPI Lambda and API Gateway in AWS.

Requirements

AWS Account
Github Account
Familiarity with using git on the command line

Resource Files

Grab the demo application if you need it. You can find the starter files for the Serverless FastApi Example here.

If you want the finished CICD code you can find that here.

Once you have cloned the repository go ahead and setup the app.

Checkout the starter files branch

git checkout serverless-fastapi-cicd-starter

Setup Virtual Environment

virtualenv -p python3.7 env
source ./env/bin/activate

Install Dependences

pip install -r requirements.txt

Run Application

uvicorn app.main:app --reload

FastAPI CI/CD with CircleCi

Create Github Repository

If you do not have a Github account now you need to go create one. If you are starting from scratch you will need to go ahead and create a new repository.

If you forked the starter files you can skip this step.

You'll also want to go ahead and add a .gitignore file in the root of your project while you're at it.

Here is a Python .gitignore boiler plate if you need one. (Starter repo should already have this)

Once you are ready, go ahead and checkout a new branch I am going to call this one cicd-demo-dev

git checkout -b cicd-demo-dev

Packaging Lambda for CircleCi

Instead of packaging our Lambda on our local machine and uploading to AWS directly we are going to have CircleCi do this for us.

Every time we push a change to the cicd-demo-dev branch in Github, CircleCi will checkout the code from that branch and create a new build.

Because of this it won't have the same local dependencies we have in our virtual environment so we need a way to inform CircleCi which dependencies to install in order to build our lambda.

This is where a requirements.txt file comes into play. In your terminal cd into the root of your FastAPI project and run the following command

pip freeze -r requirements.txt

This will create a new file named requirements.txt and append a list of all the dependencies currently installed in the application.

If you forked the example repo this should already be in the project.

Setup CircleCi Project

Once you have a repository setup with your FastAPI Lambda code and you have created your requirements.txt file it's time to setup CircleCi.

When you go to create a new CircleCi account it will prompt you to link it to your Github account. Go ahead and do this then it should forward you to the "Projects" dashboard.

You should see the name of your repository with a button next to it that says "Setup Project."

If for some reason you didn't land on this page click the "Add Projects" button in the left-hand sidebar.

Once you have a repository setup with your FastAPI Lambda code and you have created your requirements.txt file it's time to setup CircleCi.

When you go to create a new CircleCi account it will prompt you to link it to your Github account. Go ahead and do this then it should forward you to the "Projects" dashboard.

You should see the name of your repository with a button next to it that says "Setup Project."

If for some reason you didn't land on this page click the "Add Projects" button in the left-hand sidebar.

If your repository isn't showing up then you may need to grant CircleCi access to your Github Repositories manually. In which case you can find this in your Github Account under Settings and Third-party access.

Click "Setup Project" then click "Use Existing Config" then when prompted click "Start Building"

This is going to start a build and fail right away since there isn't a config file in your project yet. This is ok for now more on this in a moment.

CircleCi Config

CircleCi uses a file called config.yml inside of a folder named .circleci in the root of your project. This naming is important and has to be followed in order for CircleCi to find it.

In the root of your project create a new folder called .circleci with a file inside of it called config.yml This naming is important and has to be followed in order for CircleCi to find it. Now your file tree should look something like this.

.
├── .circleci
│   └── config.yml
├── app
│   ├── __init__.py
│   ├── api
│   │   ├── __init__.py
│   │   └── api_v1
│   │       ├── __init__.py
│   │       ├── endpoints
│   │       │   ├── __init__.py
│   │       │   └── users.py
│   │       └── api.py
│   └── main.py
├── .gitignore
└── requirements.txt

Now what to add to your config.yml file. If you want the finished version of this file you can grab it here.

Workflows, Jobs, and Steps Oh My!

These config files primarily center around Workflows, Jobs, and Steps

Workflow - Describes a series of Jobs to run

Job - Specifies a series of Steps to run.

Step - An individual command for CircleCi to run.

These will look something like this starting out.

version: 2.1

orbs:  
  python: circleci/python@1.0.0

jobs:  
  build-and-test:    
    executor:
      name: python/default
      tag: '3.7'
    steps:
      - checkout
      - run:
          name: Setup Virtual env
          command: |            
            virtualenv -p python3.7 env            
            echo "source ./env/bin/activate" >> $BASH_ENV
      - run:
          name: Install Dependencies
          command: pip install -r requirements.txt
    
workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          filters:            
            branches:
              only:
                - cicd-demo-dev

Let's break down whats going on here.

CircleCi command that checks out the repository code.
Setting up the same virtual environment that we used when we were developing the app locally. The $BASH_ENV is an alias for the .bashrc file used inside the container so on each step our virtual environment will be activated by default.
Installing all of the needed dependencies based off of the requirements.txt file.
Inside the Workflows block we are telling CircleCi to run the build-and-test job but adding the filter to only to run this particular job when changes are made to the cicd-demo-dev branch only.

Orbs and Executors

You'll also notice the Orbs and Executors blocks. CircleCi has the ability to run each individual job in its own container with its own configurations. So we also need to specify what kind of an environment we want these jobs to run in and what dependencies we will need. There are few ways of doing this we are going to be using Orbs and Executors.

Orb - Reusable snippets of code that help automate repeated processes, speed up project setup, and make it easy to integrate with third-party tools.

Executor - This is where you declare which of the Orbs you predefined to use for each Job

Lets go ahead and check this into git and see what happens in CircleCi.

git add .
git commit -m "added circleci config file"
git push --set-upstream origin cicd-demo-dev

Success! Let's not get ahead of ourselves just yet we aren't really doing anything useful yet. Now let's see if we can get CircleCi to automate some tests.

Automate Tests

First we need to add a testing framework to our application. So back in the codebase lets install Pytest.

pip install pytest

For the sake of this walkthrough we are going to make a really simple test.

Create a tests directory in the root of the project
Add a test_main.py file and an __init__.py file inside that directory and add the following snippet.

from fastapi.testclient import TestClient

from app.main import app

client = TestClient(app)

def test_read_main():
    response = client.get("/")
    assert response.status_code == 200
    assert response.json() == {"message": "Hello World!"}

Run the test

pytest

Great! Our tests are working, let's add it to our steps in the config.yml

version: 2.1

orbs:  
  python: circleci/python@0.3.2

jobs:  
  build-and-test:
    executor:
      name: python/default
      tag: '3.7'
    steps:
      - checkout
      - run:
          name: Setup Virtual env
          command: |            
            virtualenv -p python3.7 env            
            echo "source ./env/bin/activate" >> $BASH_ENV
      - run:
          name: Install Dependencies
          command: pip install -r requirements.txt
      - run:
          name: Test
          command: pytest
    
workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          filters:            
            branches:
              only:
                - cicd-demo-dev

Next we need to make sure Pytest is included in our requirements.txt file.

pip freeze > requirements.txt

Now let's check those changes into git and kick off a new build.

Awesome! We have automated our build and testing every-time we commit a change to Github! Now we'll know right away if any breaking changes have been made before deploying. Pretty cool stuff!

Deploy Lambda to AWS

Now that we have our first job working we need to add another job that will deploy the lambda to the desired environment depending on which branch the change has been made to.

Package Lambda

But, before we can do that we need to package the lambda first. So let's add the following steps to our build-and-test job.

version: 2.1

orbs:  
  python: circleci/python@0.3.2

jobs:  
  build-and-test:
    executor:
      name: python/default
      tag: '3.7'
    steps:
      - checkout
      - run:
          name: Setup Virtual env
          command: |            
            virtualenv -p python3.7 env            
            echo "source ./env/bin/activate" >> $BASH_ENV
      - run:
          name: Install Dependencies
          command: pip install -r requirements.txt
      - run:
          name: Test
          command: pytest
      - run:
          name: Create Zipfile archive of Dependencies
          command: |
            cd env/lib/python3.7/site-packages
            zip -r9 ../../../../function.zip .
      - run:
          name: Add App to Zipfile
          command: zip -g ./function.zip -r app
    
workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          filters:            
            branches:
              only:
                - cicd-demo-dev

Let's walk through these three steps.

Change directories into the site-packages folder in our virtual environment and zip up the dependencies.
Add the app folder with our API in it to the zip file.

Go ahead and check those changes into Git and make sure your build is still succeeding before moving on.

Persist Files Across Multiple Jobs

In order for our Deploy job to have access to the function.zip file we can persist data to a workspace in CircleCi. Then attach it to any job that needs access that data downstream.

Workspaces are a workflow-aware storage mechanism. A workspace stores data unique to the job, which may be needed in downstream jobs. Each workflow has a temporary workspace associated with it. The workspace can be used to pass along unique data built during a job to other jobs in the same workflow.https://circleci.com/docs/2.0/concepts/#caches-workspaces-and-artifacts

version: 2.1

orbs:  
  python: circleci/python@0.3.2

jobs:  
  build-and-test:
    executor:
      name: python/default
      tag: '3.7'
    steps:
      - checkout
      - run:
          name: Setup Virtual env
          command: |            
            virtualenv -p python3.7 env            
            echo "source ./env/bin/activate" >> $BASH_ENV
      - run:
          name: Install Dependencies
          command: pip install -r requirements.txt
      - run:
          name: Test
          command: pytest
      - run:
          name: Create Zipfile archive of Dependencies
          command: |
            cd env/lib/python3.7/site-packages
            zip -r9 ../../../../function.zip .
      - run:
          name: Add App to Zipfile
          command: zip -g ./function.zip -r app
      - persist_to_workspace:
          root: .
          paths:
            - function.zip
    
workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          filters:            
            branches:
              only:
                - cicd-demo-dev

So now when we create our second job which will handle uploading the zip file to S3 and updating the Lambda we can access the already packaged function.zip file.

Configure Deploy Job

Before we add this we also need to add another orb in order to have the AWS CLI already preinstalled in the container we are going to be running Job #2 in.

orbs:  
	python: circleci/python@0.3.2
	aws-cli: circleci/aws-cli@1.2.1

This orb allows us to easily configure the AWS CLI inside CircleCi and provision the resources we need by running the following commands (More on this later). Check out the docs for circleci/aws-cli@1.2.1 for more info on this orb.

Deploy Job

deploy-dev: 
    executor: aws-cli/default  
    steps:    
      - attach_workspace:
          at: ./
      - aws-cli/setup:
          aws-region: AWS_DEFAULT_REGION
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
      - run:
          name: Upload to S3        
          command: aws s3 cp function.zip s3://<YOUR_S3_BUCKET_NAME>/function.zip
      - run:        
          name: Deploy new Lambda        
          command: aws lambda update-function-code --function-name <YOU_FUNCTION_NAME> --s3-bucket <YOUR_S3_BUCKET_NAME> --s3-key function.zip

So what is going on here?

Attaching the workspace so we have access to the function.zip file
Configure AWS Credentials in order for CircleCi to have access to your AWS resources (we'll come back to this).
upload the new version of the lambda code to S3
Update the lambda function code in with the new changes in the repository.

Update Workflow

Don't forget now we also have to add the deploy-dev job to the workflow in order for CircleCi to run it.

workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          filters:            
            branches:
              only:
                - cicd-demo-dev
      - deploy-dev:
          requires:
            - build-and-test    
          filters:        
            branches:
              only:
                - cicd-demo-dev

Notice the requires key. This ensures deploy-dev will only run on a successful completion of build-and-test. This is so if the tests or the build fails then we won't deploy broken code to our environments.

Configure AWS

Create AWS IAM User

CircleCi will need access to your AWS account in order to upload to S3 and update the Lambda. So let's create a User with the least amount of permissions it needs to accomplish this.

Log into the AWS console and navigate to the IAM dashboard and click Users and then click Add User.

Set User Details

Give your new user a name and check the Programmatic access box only. Since this user will only be used by CircleCi it does not need the ability to log into the console.

Set Permissions

We only want to give it access to our S3 bucket and to our Lambda so we are going to attach a policy directly.

So click Attach existing policies directly
Then click Create policy
Click the JSON tab and enter this snippet.

{
	"Version": "2012-10-17",
	"Statement": [{
			"Action": [
				"s3:DeleteObject",
				"s3:GetObject",
				"s3:PutObject"
			],
	    "Effect": "Allow",
	    "Resource": "<YOUR_BUCKET_ARN_HERE>/*"
	  },
		{            
			"Sid": "PermissionsToCreateAndUpdateFunction",            
			"Effect": "Allow",            
			"Action": [                
				"lambda:CreateFunction",                
				"lambda:GetFunction",                
				"lambda:UpdateFunctionCode"            
			],            
			"Resource": [                
				"<YOUR_LAMBDA_ARN_HERE>"            
			]        
		}
	]
}

Each object in the Statement of the policy has an Action, Effect, and a Resource. In the first Statement object we are granting this user the permission to perform the Actions of Delete, Get, and Put an object to the S3 bucket that matches the AWS resource with that ARN only.

To find the ARN of your bucket check the box to the left of your bucket and a window with more details will pop up. Click the Copy Bucket ARN button.

Don't forget to add the / at the end of your bucket ARN this allows you to perform these actions on all objects within that S3 bucket.*

We will do the same thing with the second Statement object for the lambda. To find the ARN of your Lambda function open up your lambda and look up in the top right corner of of the dashboard.

Once you've updated the ARNs click Review policy, give it a name and click Create policy.

Head back to the Add user > Set permissions page and click the refresh icon on the top right of the policies table.

You should be able to search for your newly created policy and check the box. Then click Next:Tags

AWS Access Keys

Now AWS will show you the User's Access key ID and Secret access key. You need to copy these and or download them somewhere secure and that you'll have access to.

If you lose them no worries you can deactivate those keys and generate new ones fairly easily by going into the user, under Access Keys in the Security tab.

Add AWS Keys to CircleCi

CircleCi has a handy feature called Contexts that allows you to assign Environment Variables on an Organization wide level within a defined context.

This is useful when we want to define different Environment Variables for a each environment, DEV, STAGE, PROD, etc.

Create Context

We do this by clicking on the Organization Settings Icon/Text on the left-hand sidebar in the CircleCi dashboard.

Click Contexts then Create Context

Give this one a name followed by -dev since this will be for the development environment then click Create Context.

ex. serverless-fastapi-demo-dev

Add Environment Variables

Once inside your context click Add Environment Variables.

We want to add the following environment variables in all caps just like this.

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION

go ahead and create those using the credentials from the User you created earlier.

The default region will be a string with the AWS region your resources are in it will look something like this us-east-1.

You can find this out by looking in the top right corner of the AWS console whenever you have your resource's dashboard open.

Deploy!

Finally we need to update the config.yml to reference the new context.

version: 2.1

orbs:  
  python: circleci/python@0.3.2
  aws-cli: circleci/aws-cli@1.2.1

jobs:  
  build-and-test:
    executor:
      name: python/default
      tag: '3.7'
    steps:
      - run:
          name: Setup Virtual env
          command: |            
            virtualenv -p python3.7 env            
            echo "source ./env/bin/activate" >> $BASH_ENV
      - run:
          name: Install Dependencies
          command: pip install -r requirements.txt
      - run:
          name: Test
          command: pytest
      - run:
          name: Create Zipfile archive of Dependencies
          command: |
            cd env/lib/python3.7/site-packages
            zip -r9 path/to/root/function.zip .
      - run:
          name: Add App to Zipfile
          command: zip -g ./function.zip -r app
      - persist_to_workspace:
          root: .
          paths:
            - function.zip
  
  deploy-dev: 
    executor: aws-cli/default  
    steps:    
      - attach_workspace:
          at: ./
      - aws-cli/setup:
          aws-region: AWS_DEFAULT_REGION
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
      - run:
          name: Upload to S3        
          command: aws s3 cp function.zip s3://<YOUR_S3_BUCKET_NAME>/function.zip
      - run:        
          name: Deploy new Lambda        
          command: aws lambda update-function-code --function-name <YOUR_LAMBDA_NAME> --s3-bucket <your_s3_bucket_name> --s3-key function.zip

    
workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          context: <YOUR_CONTEXT_NAME>
          filters:            
            branches:
              only:
                - cicd-demo-dev
      - deploy-dev:
          context: <YOUR_CONTEXT_NAME>
          requires:
            - build-and-test    
          filters:        
            branches:
              only:
                - cicd-demo-dev

Now commit all of our code changes and push to git. This should automatically kick off another workflow in CircleCi and build, test, and deploy our Serverless FastAPI Lambda.

Add Environment Variables

Finally, you will want some sort of plan on how you are going to handle Environment Variables and or sensitive credentials you don't want to check into Git throughout your CICD pipeline.

Let's say for example you have a MONGO_DB_URL key for your database that is mongodb://prod_user:prod_password:27017 in production but for your development environment it is mongodb://dev_user:dev_password:27017 how do we differentiate between environments?

There are a slew of different ways you can handle this. We will be using a pretty straight forward method of a .env file combined with CircleCi's Contexts.

This practice is common enough that it has a name, these environment variables are commonly placed in a file .env, and the file is called a "dotenv".https://fastapi.tiangolo.com/advanced/settings/

Install Dotenv

pip install python-dotenv

Don't forget to add this to the requirements.txt file.

pip freeze > requirements.txt

Config.py

Next let's add a new folder in the app directory called core and then a file where we will handle managing the ENVs called config.py along with an __init__.py file inside the core folder.

# config.py
from pydantic import BaseSettings

class Settings(BaseSettings):
    prefix: str = "/api/v1"
    secret_key: str = None

    class Config:
      env_file = ".env"

settings = Settings()

FastAPI comes with Pydantic baked in for data validation. Among other features this allows us to to declare default values like we have here for prefix.

Within our Settings class we create a sub class and set the env_file to look for a .env file in the root of our application.

Create .env file

So let's also add this .env file in the root of our project and include our key value pairs.

This is the file that will hold potentially sensitive credentials so we don't want to check this into Github. Making sure this is added in the .gitignore file will make sure we don't do this by mistake.

# .env
SECRET_KEY="secret_value"

Notice two things here.

In the .env file we are using the ALL_CAPS convention for our key. This will automatically be converted to the lower case we declared in our config.py file.
We did not add PREFIX. This is so we can see the default value we declared in our config.py file in action.

Now your file tree should look something like this.

.
├── .circleci
│   └── config.yml
├── app
│   ├── api
│   │   ├── __init__.py
│   │   └── api_v1
│   │       ├── __init__.py
│   │       ├── api.py
│   │       └── endpoints
│   │           ├── __init__.py
│   │           └── users.py
│   ├── core
│   │   ├── __init__.py
│   │   └── config.py
│   └── tests
│       ├── __init__.py
│       └── test_main.py
├── .gitignore
├── requirements.txt
└── .env

Now that that is set up we have access to these within our application so let's update the main.py file to use them.

from fastapi import FastAPI
from mangum import Mangum
from app.core import config

from app.api.api_v1.api import router as api_router

app = FastAPI()

@app.get("/")
async def root():
    return {"message": f"This is our secret key value: {config.settings.secret_key}" }

app.include_router(api_router, prefix=config.settings.prefix)
handler = Mangum(app)

We added three lines here.

Import the config object
Update the root "/" route to return our SECRET_VALUE so we can check to see if it works.
Update the prefix argument to pass in the default value instead of hard coding it in.

Add ENVs to CircleCi

There are a number of ways we can implement how to insert the ENVs in at build time. We are going to add them to our Context just like we did the AWS credentials.

This way you can define a separate Context for each Workflow to trigger based off of the corresponding branches in Github.

ex. serverless-fastapi-demo-dev would be triggered when a change is made to the development branch, and serverless-fastapi-demo-prod when a change is made to the master branch.

In the CircleCi dashboard navigate back to the Organization Settings and to the Context we setup earlier.

Click on Add Environment Variable
Add in our SECRET_KEY and its corresponding value

Update Config.yml

Since we don't check the .env file into Git, CircleCi won't have it when it checks out the code to run the builds. CircleCi will add the values we declared in the Project Settings environment variables into the env object which we can output into a new .env file.

So our final config.yml file should look like the following.

version: 2.1

orbs:  
  python: circleci/python@0.3.2
  aws-cli: circleci/aws-cli@1.2.1

jobs:  
  build-and-test:
    executor:
      name: python/default
      tag: '3.7'
    steps:
      - run:
          name: Setup Virtual env
          command: |            
            virtualenv -p python3.7 env            
            echo "source ./env/bin/activate" >> $BASH_ENV
      - run:
          name: Install Dependencies
          command: pip install -r requirements.txt
			- run:
          name: Create ENV file
          command: env > .env
      - run:
          name: Test
          command: pytest
      - run:
          name: Create Zipfile archive of Dependencies
          command: |
            cd env/lib/python3.7/site-packages
            zip -r9 path/to/root/function.zip .
      - run:
          name: Add App to Zipfile
          command: zip -g ./function.zip .env -r app
      - persist_to_workspace:
          root: .
          paths:
            - function.zip
  
  deploy-dev: 
    executor: aws-cli/default  
    steps:    
      - attach_workspace:
          at: ./
      - aws-cli/setup:
          aws-region: AWS_DEFAULT_REGION
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
      - run:
          name: Upload to S3        
          command: aws s3 cp function.zip s3://<YOUR_S3_BUCKET_NAME>/function.zip
      - run:        
          name: Deploy new Lambda        
          command: aws lambda update-function-code --function-name <YOUR_LAMBDA_NAME> --s3-bucket <your_s3_bucket_name> --s3-key function.zip

    
workflows:  
  build-test-and-deploy:    
    jobs:      
      - build-and-test:
          context: <YOUR_CONTEXT_NAME>
          filters:            
            branches:
              only:
                - cicd-demo-dev
      - deploy-dev:
          context: <YOUR_CONTEXT_NAME>
          requires:
            - build-and-test    
          filters:        
            branches:
              only:
                - cicd-demo-dev

Note the .env file is created before running the tests because more than likely there will be some ENVs that the tests will rely on.

We also need to add the .env to our zip file during the packaging of the lambda.

Deploy New ENVs

Check these changes into Git and...wait what happened? Our build failed. We changed the return value for the route we wrote a test for and now our test is failing. This is the beauty of CICD in action.

We were notified of the failed test and did not deploy the changes.

Let's update our test and recommit these changes.

# test_main.py
from fastapi.testclient import TestClient

from app.main import app

client = TestClient(app)

def test_read_main():
    response = client.get("/")
    assert response.status_code == 200
    assert response.json() == {"message": "This is our secret key value: Secret Value"}

Check the API Gateway endpoint and awesome! Our Environment Variables have been deployed successfully to our Lambda!

Summary

Phew! This was quite a lot but we made. Automated our testing and deployments using CircleCi and Github!