Search for a command to run...
Protect your important assets with those simple tricks
👨💻 Sofware Engineer · 💬 Consultant · 🚀 Indie Maker · ⚡️ Serverless
🔧 serverless-appsync-plugin · serverless-appsync-simulator · middy-appsync
🐈 and 🍵
No comments yet. Be the first to comment.
If you're on Spain's regulated electricity tariff (PVPC), timing is everything. Prices change every hour based on supply, demand, and energy source — meaning your dishwasher costs three times more to
Invoking Private API Gateway endpoints from EventBridge
How to Invoke Private API Gateway From Step Functions
On-Demand Self-Destructed Sandbox AWS Accounts
Using JSONata capabilities to unmarshall DynamoDB data natively in Step Functions
Benoit Boure
16 posts
I am a software engineer with a focus on serverless technologies. I blog about Serverless Follow me on Twitter - Need help?
On this page
CloudFormation is an AWS service that allows you to maintain Infrastructure as Code (IaC). Whether you are using it natively (with JSON or YML) or through a third-party service such as the Serverless Framework, AWS CDK or SAM, it is a great way to make your infrastructure reproducible across various stages. It also makes the deployment process easily automatable through CI/CD pipelines. In other words, it makes managing your infrastructure less prone to human errors.
Although automating things sounds like a good idea, one of the downsides of CloudFormation is that it is hard to understand what is going on under the hood and what exactly is going to happen to your stack during the process, turning every single deployment into a potential 7 minutes of terror story. Imagine that an entire resource gets deleted and all its data with it. Make just one mistake and you will only find out when it's too late. What if it is a production database? It happens more than you think. 😱😱😱
To avoid this kind of disasters, I will show you 5 ways to protect your resources from deletion with CloudFormation.
The first technique is to understand which actions will effectively be executed during the update before they happen. CloudFormation offers a tool that lets you pre-visualize all the modifications that would be applied by a change in your template.
To use it, follow these simple steps:
It might take a few seconds for the changeset to be generated. Once it is done, the console will show you a detailed summary of what actions would be executed if you decided to proceed with the update.
Example:
As you can see, you can easily spot what resources will be Modified, or Removed and if they require replacement. Once you are confident enough that this is what you intend to do, you can hit the Execute button with a certain peace of mind 🧘
This method is useful when you want to visually confirm a change that you are unsure about. However, it is not always convenient. Let's explore other solutions.
With the DeletionPolicy attribute, you can control what CloudFormation should do with a resource in the event of it being removed from the template, or if the stack is deleted altogether. The default value is Delete which is probably not what you want in some cases. By changing the value to Retain, you are telling CloudFormation to keep the resource instead.
Example:
Resources:
MyTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: mytable
One thing to notice here is that this method will not make your deployment fail. CloudFormation will execute all your changes. The difference is that any instruction to delete a resource with a Retain policy will be ignored and the resource will be "detached" from the stack instead. This also means that if you try to add the resource back to the stack, any subsequent deployment might fail because CloudFormation will try to re-create the resource that already exists (e.g: the DynamoDB table already exists with that name). If that happens, you can check this guide for Importing Existing Resources into a CloudFormation Stack.
⚠️ Attention!
This capability doesn't apply to resources whose physical instance is replaced during stack update operations. For example, if you edit a resource's properties such that CloudFormation replaces that resource during a stack update.
This extract from the official documentation is very important. What it means is that if you change a property of a resource that requires replacement (e.g.: changing a DynamoDB table's name), the deletion policy will not apply, and it would still be deleted and re-created. Before you change a property, you should pay attention to the Update requires section of the CloudFormation documentation for that resource's attribute.
With certain types of resources, like EC2 volumes or RDS instances, you can also use Snapshot. In that case, the asset would still be deleted but a backup would be executed first.
You can read more about this strategy by reading the official documentation.
A more advanced way of protecting your resources is through Stack Policies. With Stack Policies, you can constraint what actions are allowed to be executed or not according to specific rules that you define. When you add a policy, all resources are protected by default. You need to explicitly Allow the changes on the resources that you want to update. You can think of it as an IAM policy, but the difference here is that it only applies during stack updates.
Example:
The following policy allows any change on all resources, except for the resource whose id is MyDynamoDBTable. By explicitly denying Update:Delete and Update:Replace, the resource is protected against deletion and replacement. On the other hand, modifications are still allowed (e.g.: Add a Global Secondary Index).
{
"Statement": [
{
"Effect": "Allow",
"Action": ["Update:*"],
"Principal": "*",
"Resource": "*"
},
{
"Effect": "Deny",
"Action": ["Update:Delete", "Update:Replace"],
"Principal": "*",
"Resource": ["LogicalResourceId/MyDynamoDBTable"]
}
]
}
To learn how to write custom stack policies, refer to the documentation
If all you worry about is someone (or a process) tearing down a whole stack by mistake, what you need is Stack termination protection. When enabled, CloudFormation will reject any attempt of deleting the stack.
To enable termination protection:
Last but not least, if you are too paranoid about deleting precious resources and all the data they contain, the best thing you can do is isolate them into their own stack. Place each one of them in a dedicated template and touch them only if and when you need to. By doing so, you will not risk destroying them while deploying other stacks that change more often.
Each solution has its own pros and cons. They also behave differently in different situations. To help you better understand the differences, I created a simple cheat sheet.
| Will my resource be protected if | It is removed from the stack | It requires replacement | The stack is deleted |
| Changeset review | Manual (1) | Manual (1) | No |
| DeletionPolicy | Yes | No | Yes |
| Stack Policy | Yes (2) | Yes (2) | No |
| Stack Termination Protection | No | No | Yes |
| Resource Isolation | No (3) | No (3) | No (3) |
(1) You will need to manually review and approve the changes.
(2) Provided you configure the policy properly
(3) On its own, resource isolation will not protect any resource. You'll need to combine it with other solutions
As you can see, there is no one-fits-all solution (none of the rows has all Yeses). You will need to use more than one if you want full protection.
I just showed you 5 ways to avoid accidental deletion of CloudFormation resources:
Use the one that best fits your needs and your particular use-cases. If you need complete protection, you can combine them together and benefit from several safety nets at the same time.
Hopefully, these measures will help you and your team sleep better at night 😴.
If you would like to read more content like this, follow me on Twitter and subscribe to my brand new newsletter on Hashnode.