circuit

How to Use the CloudFormation Sub Function

Since I started using AWS CloudFormation 4 years ago, I have seen many people using the Fn::Join command to merge information such as static text and variables in their CloudFormation templates. While this works, it can get messy and difficult to understand when things get complicated. Fn::Sub helps simplify our template definitions.




First: Fn::Join

The Join function allows us to connect text and variables. The syntax can be represented in several ways using either JSON or YAML formats.

In JSON, we write our Join function like this:

{
  "Fn::Join" :
    [ "delimiter", [
      comma-delimited list of values
    ]
  ]
}

For YAML, we can write the Join function as

!Join [ 'delimiter', [ "string', !Ref variable, 'string ] ]

or

Fn::Join
  - delimiter
  - -
    - string
    - !Ref variable
    - string

While the Fn::Join command does the job, the syntax can be hard to comprehend. It just doesn't feel intuitive, especially the second format.

When using YAML, you can call the Join function using either !Join or Fn::Join.

For example, if we want to create an S3 bucket with a predefined name consisting of some strings separated by a hyphen, we can define it in our CloudFormation template as:

Resources:
  Bucket:
    Type: AWS::S3:Bucket
    Properties:
      BucketName: !Join [ "-", [ !Ref NamePrefix, "env", !Ref env ] ]

If NamePrefix has a value of databucket and env has a value of test, the resulting string is databucket-env-test.

Second: Fn::Sub

The Fn::Sub command essentially does the same thing, albeit in a more intuitive manner. The difference is that while Fn::Join constructs a string using the provided values separated by the delimiter, Fn::Sub replaces the variables in the provided string. When defining the Fn::Sub command in your CloudFormation template, you can choose to provide a variable map, or use the ${} substitution syntax.

The ${} substitution syntax is very easy to use:

BucketName: !Sub "${AppIdentifier}-${Service}-${Resource}-${Name}"

where each of the variables maps to a CloudFormation parameter. This shorthand is easy to understand. This example substitutes four parameters, but can easily include both defined and variable text.

BucketName: !Sub "${AppIdentifier}-deployment-bucket-${Name}"

Naming buckets, or any resource for that matter should be avoided as many resources cannot be updated when a custom name is used. Use a “Name” tag instead. Like domain names, resource names are for people.

Using a Variable Map

The alternative is using a variable map, where you define the string format, and a map of values to insert. This method allows using other intrinsic functions. The format in JSON is

{
  "Fn::Sub" :
    [ String,
      {
        Var1Name: Var1Value,
        Var2Name: Var2Value
      }
    ]
}

and in YAML is

Fn::Sub:
  - "string"
  - Var1Name: Var1Value
    Var2Name: Var2Value

The “string” is defined using the same ${} syntax, such as

Fn::Sub:
  - "${p1}-${p2}"
  - p1: !Ref part1
    p2: !Ref part2

We will see this in action later in the article.

Using !Sub with Systems Manager Parameter Store

In the various work projects I have been involved with, I proposed a path based naming structure for our Parameter Store keys. It means everyone involved can remember what the key is composed of and reference the key in both CloudFormation templates, and application code. For example:

/app-identifier/service/resource/parameter

This means parameter names would look like /n4jt/s3/bucket/name. Once the object is created, environment prefixes are used within the bucket to separate data for the various non-production environments. Production resources are created using a different account, and bucket policies restrict the resources the accounts can access.

Building this parameter store key using !Join in the CloudFormation template looks like:

Resources:
  Bucket:
    Type: AWS::S3:Bucket
    Properties:
      BucketName: !Join [ "/", [ "", !Ref AppIdentifier, !Ref Service, !Ref Resource, !Ref BucketName ] ]

This works, but is a lot more difficult to read than

Resources:
  Bucket:
    Type: AWS::S3:Bucket
    Properties:
      BucketName: !Sub "/${AppIdentifier}/${Service}/${Resource}/${BucketName}"

The !Sub approach is a lot easier to comprehend, and when debugging issues in the resource creation, it is easier to perform variable substitution to determine if there is an error.

How are they different?

The end result with Fn::Join and Fn::Sub appear to be the same: construct a value from text and variable data. Both are processed when the CloudFormation stack is launched, allowing for template re-use with the parameter values defining the string value.

In both cases we can use other CloudFormation intrinsic functions to alter the string. This however, is only available if using the !Join or !Sub function with a mapping. Let's look at an example with all three.

AWSTemplateFormatVersion: 2010-09-09

Description: Demonstrating Sub and Join
Metadata:
  Author: Chris Hare

Parameters:
  part1:
    Type: String
    Description: Part 1 of the bucket name
  part2:
    Type: String
    Description: Part 2 of the bucket name
  part3:
    Type: String
    Description: Part 3 of the bucket name

Conditions:
  PartName: !Equals [ !Ref part3, "end"]

Resources:

  Bucket1:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Join [ "-", [ "join", !Ref part1, !Ref part2, !Ref part3 ] ]

  Bucket2:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub "sub1-${part1}-${part2}-${part3}"

  Bucket3:
    Type: AWS::S3::Bucket
    Properties:
      BucketName:
        Fn::Sub:
          - "sub2-${p1}-${p2}-${p3}"
          - p1: !Ref part1
            p2: !Ref part2
            p3: !If [ PartName, !Ref part3, "part3-skipped" ]

Outputs:
  Name1:
    Value: !Ref Bucket1
  Name2:
    Value: !Ref Bucket2
  Name3:
    Value: !Ref Bucket3

This CloudFormation template creates three S3 buckets. Bucket1 uses the !Join function, Bucket2 uses !Sub without a variable mapping, and Bucket3 uses !Sub with a variable mapping.

How does the Fn::If fit into the picture? The PartName condition returns True if the part3 parameter is equal to end. When we incldue the !If in the mapping, we insert the value of the part3 parameter if the condition evaluates to true, or the provided string if the condition evaluates as false.

Now, if we look at the outputs of the stack using the CLI, we would see:

$ aws cloudformation describe-stacks --stack-name sample-sub
{
  "Stacks": [
    {
      "StackId": "arn:aws:cloudformation:us-east-1:1234567890:stack/sample-sub/15aa4e20-ddd4-11eb-b920-0acd55d80e95",
      "StackName": "sample-sub",
      "Description": "Demonstrating Sub and Join",
      "Parameters": [
        {
          "ParameterKey": "part1",
          "ParameterValue": "prefix"
        },
        {
          "ParameterKey": "part2",
          "ParameterValue": "middle"
        },
        {
          "ParameterKey": "part3",
          "ParameterValue": "end"
        }
      ],
      "CreationTime": "2021-07-05T21:00:53.220Z",
      "RollbackConfiguration": {
      "RollbackTriggers": []
      },
    "StackStatus": "CREATE_COMPLETE",
    "DisableRollback": false,
    "NotificationARNs": [],
    "Outputs": [
      {
        "OutputKey": "Name3",
        "OutputValue": "sub2-prefix-middle-end"
      },
      {
        "OutputKey": "Name1",
        "OutputValue": "join-prefix-middle-end"
      },
      {
        "OutputKey": "Name2",
        "OutputValue": "sub1-prefix-middle-end"
      }
    ],
    "Tags": [],
    "EnableTerminationProtection": false,
    "DriftInformation": {
      "StackDriftStatus": "NOT_CHECKED"
      }
    }
  ]
}

All three of the buckets are created regardless of which path is taken. When using !Join and !Sub with a variable mapping, we can use other intrinsic functions. The sample template uses a condition to check if the value of the part3 parameter is end, and if so, inserts an alternative value.

Fn::Sub, Fn::Join and the Serverless Framework

If you are using the Serverless Framework to deploy your resources, then using Fn::Sub is more challenging, as the Serverless Framework uses the same ${} syntax as Fn::Sub. Using the Serverless Framework serverless-cloudformation-sub-variables plugin helps solve this problem.

Conclusion

It is hard to say which one of these CloudFormation Intrinsic Functions is better than the other. They both do similar things, specifically replacing variables in strings. My opnion is Fn::Sub is easier to write and easier to comprehend later. I have written a lot of CloudFormation templates using both, but these days when I need to perform variable substitution to create a string value, I generally use Fn::Sub.

References

CloudFormation Fn::Join Syntax

CloudFormation Fn:Sub Syntax

CloudFormation protip: use !Sub instead of !Join

Understanding AWS CloudFormation !Sub Syntax

Improve Your CloudFormation Game with these 9 Tips

CloudFormation Intrinsic Functions




Continue Learning