AWS CodeBuild is a fully managed build service in the cloud. CodeBuild compiles your source code, runs unit tests, and produces artifacts that are ready to deploy.
CodeBuild is often utilized to build Docker images used for deploying applications to AWS container services like ECS and EKS.
There will be situations where you will need to run the CodeBuild project locally:
- You have a complex application with complicated build instructions. It is much faster and easier to run it locally and fix encountered problems than do the same in the cloud.
- The lack of budget for new AWS resources or it takes a long time to provision AWS CodeBuild because often it involves some bureaucracy in the company.
- If you just starting to learn how to work with CodeBuild or just do some quick tests etc.
In this tutorial, I will create a Docker image with a simple SpringBoot REST application. Complete source code you can find here. To make it work both Docker and Docker Compose have to be installed on PC.
1. Build The Image
The first step is to build the aws/codebuild/standard:5.0.
-
Clone official AWS CodeBuild curated Docker image GitHub repository by executing the command
$git clone https://github.com/aws/aws-codebuild-docker-images.git
-
Go to the directory /ubuntu/standard/5.0 and execute the command to build the image
$docker build -t aws/codebuild/standard:5.0 .
The Dockerfile has over 100 instructions, so the build process can take up to 2 hours, so be patient.
Successful image build
The build can fail so you will have to start it over again.
The error message example
2. Download the Build Script
Download build script by running the command:
$ wget https://raw.githubusercontent.com/aws/aws-codebuild-docker-images/master/local_builds/codebuild_build.sh
For convenience, I keep this file inside the root folder of the project. To make this script executable run the command:
$ chmod +x codebuild_build.sh
3. How It Works
The goal is to compile the code of the application, package it into a jar file, create the Docker image with this jar file, and upload the image to a Registry. I will show you two examples, one use as a Registry AWS ECR and another a local instance of Docker Registry.
All build settings and configurations are located in the buildspec.yaml file. This file has to be located inside a project directory.
To start a build process run command:
./codebuild_build.sh -i aws/codebuild/standard:5.0 -a ./artifact/ -c
Arguments for this command:
Required:
-i - specifys an image
-a - specifys an artifact output directoryOptional:
-c - when you need to use the AWS configuration and credentials from your local host
-b - overrides buildspec.yaml file
More you can find here.
CodeBuild local setup diagram
When codebuild_build.sh is executed the following events occur (to understand better follow the diagram):
- A docker container is created based on amazon/aws-codebuild-local:latest image.
- Inside the container, a docker-compose.yaml file is created with configuration for a local build including some arguments passed to the build script execution command. This container has access to the host`s Docker and Docker Compose. From inside the container the command is executed to run docker-compose.yaml file. Based on this file 2 containers are created.
- One container (agent-resources_agent_1) is created from the same amazon/aws-codebuild-local:latest image and is marked as an inner container has a volume mapped to a project source folder.
- The second container (agent-resources_build_1) is based on aws/codebuild/standard:5.0 image, and it has volumes mapped to the first container. These volumes give him access to a source folder. Two more volumes are mapped: one is to /.aws directory where the credentials file is located. This allows the local CodeBuild have access to AWS Services if necessary. The second volume is to save any artifacts files created during the build process. In this example, it saves imagedefinitions.json specified under artifacts: files section of a buildspec.yaml file.
All build processes take place in agent-resources_build_1. Instructions from buildspec.yaml are executed, source code is copied from a project folder to a directory inside the container, necessary dependencies are pulled from the Maven repository, the jar file is packaged, then a Docker image is built with a jar file and uploaded to an AWS ECR or Docker local Registry.
4. Using AWS ECR to Push Created Image
For this, to work an AWS ECR repository has to be created. How to create a repository? You can find out in this tutorial.
PC has to have a config and a credentials file configured. Step by step configuration you can find here. The credentials need to have the necessary permissions to perform operations with the repository.
The buildspec.yaml file look like below.
version: 0.2
env:
variables:
AWS_REGION: us-east-1
AWS_ACCOUNT_ID: 708995052028
AWS_SERVICE_NAME: .dkr.ecr.
AWS_DOMAIN_NAME: .amazonaws.com
IMAGE_NAME: aws-codebuild-locally-build-docker-image
CONTAINER_NAME: aws-codebuild-locally-build-docker-container
phases:
install:
runtime-versions:
java: corretto8
commands:
pre_build:
commands:
- REPOSITORY_URL="$AWS_ACCOUNT_ID$AWS_SERVICE_NAME$AWS_REGION$AWS_DOMAIN_NAME"
- echo $REPOSITORY_URL
- echo Setting up docker
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2 &
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
- echo Logging in to Amazon ECR...
- aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $REPOSITORY_URL
build:
commands:
- echo Build started on `date`
- 'mvn clean package'
post_build:
commands:
- docker image build -t $IMAGE_NAME .
- docker image tag $IMAGE_NAME:latest $REPOSITORY_URL/$IMAGE_NAME:latest
- echo image name = $IMAGE_NAME:latest
- echo image url = $REPOSITORY_URL/$IMAGE_NAME:latest
- docker push $REPOSITORY_URL/$IMAGE_NAME:latest
- echo Writing image definitions file...
# name is a container name from the task definition
- printf '[{"name":"'"$CONTAINER_NAME"'","imageUri":"'"$REPOSITORY_URL"'/'"$IMAGE_NAME"':latest"}]' > imagedefinitions.json
- cat imagedefinitions.json
- echo Build completed on `date`
artifacts:
files:
- imagedefinitions.json
buildspec.yaml
You just have to substitute environment variables with your values.
After the build is complete, the image will be in AWS ECR and the imagedefinitions.json file in /artifact directory.
5. Using Local Docker Registry to Push Created Image
Let's imagine a situation when one does not have access to AWS ECR. It is not a problem as we can create a local Docker Registry.
In a project go to src/main/resources/docker-compose and execute the command:
$docker-compose up
This command will spin Docker a container with Docker Registry.
To run the build run command bellow, as an argument pass the name of buildspec-local-repository.yaml file. Here we don't need to pass -c as we won't need to interact with AWS Services.
./codebuild_build.sh -i aws/codebuild/standard:5.0 -a ./artifact/ -b ./buildspec-local-repository.yaml
The buildspec-local-repository.yaml file look like below. This file points CodeBuild to a localhost:5000 where the local Docker Registry is running.
version: 0.2
env:
variables:
AWS_REGION:
AWS_ACCOUNT_ID:
AWS_SERVICE_NAME:
AWS_DOMAIN_NAME: localhost:5000
IMAGE_NAME: aws-codebuild-locally-build-docker-image
CONTAINER_NAME: aws-codebuild-locally-build-docker-container
phases:
install:
runtime-versions:
java: corretto8
commands:
pre_build:
commands:
- REPOSITORY_URL="$AWS_ACCOUNT_ID$AWS_SERVICE_NAME$AWS_REGION$AWS_DOMAIN_NAME"
- echo $REPOSITORY_URL
- echo Setting up docker
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2 &
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
- echo Logging in to Amazon ECR...
# Commented out because this CodeBuild project won't need to communicate with AWS Services
# - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $REPOSITORY_URL
build:
commands:
- echo Build started on `date`
- pwd
- ls
- 'mvn clean package'
- ls
post_build:
commands:
- docker image build -t $IMAGE_NAME .
- docker image tag $IMAGE_NAME:latest $REPOSITORY_URL/$IMAGE_NAME:latest
- echo image name = $IMAGE_NAME:latest
- echo image url = $REPOSITORY_URL/$IMAGE_NAME:latest
- docker push $REPOSITORY_URL/$IMAGE_NAME:latest
- echo Writing image definitions file...
# name is a container name from the task definition
- printf '[{"name":"'"$CONTAINER_NAME"'","imageUri":"'"$REPOSITORY_URL"'/'"$IMAGE_NAME"':latest"}]' > imagedefinitions.json
- cat imagedefinitions.json
- echo Build completed on `date`
artifacts:
files:
- imagedefinitions.json
buildspec-local-repository.yaml
After the build is complete, the image will be in the local Docker Registry and the imagedefinitions.json file in /artifact directory.
To check the image in a local Docker Registry execute the command:
curl -X GET http://localhost:5000/v2/_catalog
Quick demo
Summary
In this tutorial, I've explained how to set up a local CodeBuild project in simple steps. The two above setups can completely make you build tests independent of AWS cloud service. They are useful for beginners and for experienced developers.
Thank you for reading!
If you have any questions or suggestions, please feel free to write to me on my LinkedIn account.