First: Fn::Join
The Join function allows us to connect text and variables. The syntax can be represented in several ways using either JSON or YAML formats.
In JSON, we write our Join function like this:
{
"Fn::Join" :
[ "delimiter", [
comma-delimited list of values
]
]
}
For YAML, we can write the Join function as
!Join [ 'delimiter', [ "string', !Ref variable, 'string ] ]
or
Fn::Join
- delimiter
- -
- string
- !Ref variable
- string
While the Fn::Join command does the job, the syntax can be hard to comprehend. It just doesn't feel intuitive, especially the second format.
When using YAML, you can call the Join function using either !Join or Fn::Join.
For example, if we want to create an S3 bucket with a predefined name consisting of some strings separated by a hyphen, we can define it in our CloudFormation template as:
Resources:
Bucket:
Type: AWS::S3:Bucket
Properties:
BucketName: !Join [ "-", [ !Ref NamePrefix, "env", !Ref env ] ]
If NamePrefix has a value of databucket and env has a value of test, the resulting string is databucket-env-test.
Second: Fn::Sub
The Fn::Sub command essentially does the same thing, albeit in a more intuitive manner. The difference is that while Fn::Join constructs a string using the provided values separated by the delimiter, Fn::Sub replaces the variables in the provided string. When defining the Fn::Sub command in your CloudFormation template, you can choose to provide a variable map, or use the ${} substitution syntax.
The ${} substitution syntax is very easy to use:
BucketName: !Sub "${AppIdentifier}-${Service}-${Resource}-${Name}"
where each of the variables maps to a CloudFormation parameter. This shorthand is easy to understand. This example substitutes four parameters, but can easily include both defined and variable text.
BucketName: !Sub "${AppIdentifier}-deployment-bucket-${Name}"
Naming buckets, or any resource for that matter should be avoided as many resources cannot be updated when a custom name is used. Use a “Name” tag instead. Like domain names, resource names are for people.
Using a Variable Map
The alternative is using a variable map, where you define the string format, and a map of values to insert. This method allows using other intrinsic functions. The format in JSON is
{
"Fn::Sub" :
[ String,
{
Var1Name: Var1Value,
Var2Name: Var2Value
}
]
}
and in YAML is
Fn::Sub:
- "string"
- Var1Name: Var1Value
Var2Name: Var2Value
The “string” is defined using the same ${} syntax, such as
Fn::Sub:
- "${p1}-${p2}"
- p1: !Ref part1
p2: !Ref part2
We will see this in action later in the article.
Using !Sub with Systems Manager Parameter Store
In the various work projects I have been involved with, I proposed a path based naming structure for our Parameter Store keys. It means everyone involved can remember what the key is composed of and reference the key in both CloudFormation templates, and application code. For example:
/app-identifier/service/resource/parameter
This means parameter names would look like /n4jt/s3/bucket/name. Once the object is created, environment prefixes are used within the bucket to separate data for the various non-production environments. Production resources are created using a different account, and bucket policies restrict the resources the accounts can access.
Building this parameter store key using !Join in the CloudFormation template looks like:
Resources:
Bucket:
Type: AWS::S3:Bucket
Properties:
BucketName: !Join [ "/", [ "", !Ref AppIdentifier, !Ref Service, !Ref Resource, !Ref BucketName ] ]
This works, but is a lot more difficult to read than
Resources:
Bucket:
Type: AWS::S3:Bucket
Properties:
BucketName: !Sub "/${AppIdentifier}/${Service}/${Resource}/${BucketName}"
The !Sub approach is a lot easier to comprehend, and when debugging issues in the resource creation, it is easier to perform variable substitution to determine if there is an error.
How are they different?
The end result with Fn::Join and Fn::Sub appear to be the same: construct a value from text and variable data. Both are processed when the CloudFormation stack is launched, allowing for template re-use with the parameter values defining the string value.
In both cases we can use other CloudFormation intrinsic functions to alter the string. This however, is only available if using the !Join or !Sub function with a mapping. Let's look at an example with all three.
AWSTemplateFormatVersion: 2010-09-09
Description: Demonstrating Sub and Join
Metadata:
Author: Chris Hare
Parameters:
part1:
Type: String
Description: Part 1 of the bucket name
part2:
Type: String
Description: Part 2 of the bucket name
part3:
Type: String
Description: Part 3 of the bucket name
Conditions:
PartName: !Equals [ !Ref part3, "end"]
Resources:
Bucket1:
Type: AWS::S3::Bucket
Properties:
BucketName: !Join [ "-", [ "join", !Ref part1, !Ref part2, !Ref part3 ] ]
Bucket2:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub "sub1-${part1}-${part2}-${part3}"
Bucket3:
Type: AWS::S3::Bucket
Properties:
BucketName:
Fn::Sub:
- "sub2-${p1}-${p2}-${p3}"
- p1: !Ref part1
p2: !Ref part2
p3: !If [ PartName, !Ref part3, "part3-skipped" ]
Outputs:
Name1:
Value: !Ref Bucket1
Name2:
Value: !Ref Bucket2
Name3:
Value: !Ref Bucket3
This CloudFormation template creates three S3 buckets. Bucket1 uses the !Join function, Bucket2 uses !Sub without a variable mapping, and Bucket3 uses !Sub with a variable mapping.
How does the Fn::If fit into the picture? The PartName condition returns True if the part3 parameter is equal to end. When we incldue the !If in the mapping, we insert the value of the part3 parameter if the condition evaluates to true, or the provided string if the condition evaluates as false.
Now, if we look at the outputs of the stack using the CLI, we would see:
$ aws cloudformation describe-stacks --stack-name sample-sub
{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:us-east-1:1234567890:stack/sample-sub/15aa4e20-ddd4-11eb-b920-0acd55d80e95",
"StackName": "sample-sub",
"Description": "Demonstrating Sub and Join",
"Parameters": [
{
"ParameterKey": "part1",
"ParameterValue": "prefix"
},
{
"ParameterKey": "part2",
"ParameterValue": "middle"
},
{
"ParameterKey": "part3",
"ParameterValue": "end"
}
],
"CreationTime": "2021-07-05T21:00:53.220Z",
"RollbackConfiguration": {
"RollbackTriggers": []
},
"StackStatus": "CREATE_COMPLETE",
"DisableRollback": false,
"NotificationARNs": [],
"Outputs": [
{
"OutputKey": "Name3",
"OutputValue": "sub2-prefix-middle-end"
},
{
"OutputKey": "Name1",
"OutputValue": "join-prefix-middle-end"
},
{
"OutputKey": "Name2",
"OutputValue": "sub1-prefix-middle-end"
}
],
"Tags": [],
"EnableTerminationProtection": false,
"DriftInformation": {
"StackDriftStatus": "NOT_CHECKED"
}
}
]
}
All three of the buckets are created regardless of which path is taken. When using !Join and !Sub with a variable mapping, we can use other intrinsic functions. The sample template uses a condition to check if the value of the part3 parameter is end, and if so, inserts an alternative value.
Fn::Sub, Fn::Join and the Serverless Framework
If you are using the Serverless Framework to deploy your resources, then using Fn::Sub is more challenging, as the Serverless Framework uses the same ${} syntax as Fn::Sub. Using the Serverless Framework serverless-cloudformation-sub-variables plugin helps solve this problem.
Conclusion
It is hard to say which one of these CloudFormation Intrinsic Functions is better than the other. They both do similar things, specifically replacing variables in strings. My opnion is Fn::Sub is easier to write and easier to comprehend later. I have written a lot of CloudFormation templates using both, but these days when I need to perform variable substitution to create a string value, I generally use Fn::Sub.
References
CloudFormation Fn::Join Syntax
CloudFormation protip: use !Sub instead of !Join
Understanding AWS CloudFormation !Sub Syntax