Using aws-nuke as a Cleanup and Cost Saving Tool

Photo by Visual Stories || Micheile on Unsplash

At C.E.S.A.R, we use lots of AWS accounts to develop different services for many of our clients. Besides that, for each service or project, we use to have different accounts per environment (e.g., development, QA, integration, staging, etc.). This brings many issues regarding billing costs, since a lot of resources can be created and forgotten after issues are resolved, generating a bunch of garbage and headaches 💸.

Also, even if the process of deleting useless resources are well defined and strictly followed by both developers and QA teams, it requires some scripting and maintenance over time, since the software keeps changing, as well as the AWS infrastructure. On top of that, when the devs responsible for these scripts change (e.g., DevOps team), it requires some time for the new team to learn the cleanup process and code (shell scripts, python, node, etc.).

To keep it simple and speedup the cleanup process, we adopted a nice tool named aws-nuke. With this tool, the cleanup process is being simplified, since it requires one single command to wipe out the full AWS resources from an account… so, here comes the caution notice, from the aws-nuke developers themselves:

Photo by Hiroshi Kimura on Unsplash

Be aware that aws-nuke is a very destructive tool, hence you have to be very careful while using it. Otherwise you might delete production data. We strongly advise you to not run this application on any AWS account, where you cannot afford to lose all resources.

Nevertheless, we do not use that one single command (to rule them all 🌝) to wipe all resources out, but, instead, we use some filtering techniques by using a configuration file with preset filters, as we will be discussing in the following sections, with a simple example. For the full documentation, please visit this website.

Installing

The tool can be executed either by installing it or just running a container image. The MacOS, Linux and Windows binaries can be found on the releases page.

To execute from a container, try the following command, by using docker:

docker run --rm -it quay.io/rebuy/aws-nuke:v2.11.0 version

Running aws-nuke to show the current version, with docker

If using the binary, just execute:

aws-nuke version

Filtering

To simplify the setup, we’ll be using the locally installed binary instead of the containerized way, since with the container approach, it would be necessary to mount volumes (AWS credentials file and the config file). Also, we are going the use the default profile from ~/.aws/credentials, but you can use other profile by specifying the —-profile <your_profile>option.

As already mentioned, you can run the a single command to list all the resources to be deleted, without, by default, deleting it, for security reasons. Before executing the command, a minimal configuration file should be created, with the following content, named as config.yml:

regions:
  - us-east-2
  - global
account-blocklist:
  - "999999999999" # production
accounts:
  "000000000000": {}

The tool forces you to specify production accounts that must not be touched, again for security reasons, specified within the account-blocklist key. In the accounts key, specify your AWS root account ID, instead of using the 000000000000 example. Also, we should specify the regions that the tool would search for resources and manage to destroy them. Now, execute the following command, that will only list the resources that would be deleted, without actually deleting them.

aws-nuke -c config.yml

CAUTION, AGAIN 🚨: to actually wipe everything out, instead of just listing, you can add the flag —-no-dry-runto the previous command. WE ARE NOT DOING THIS it in this article without doing some filtering, since we do not wish to remove some of our precious resources, but, instead, we want specific the resources to be deleted, based on some patterns.

Suppose we would like to delete a cloudformation stack and some S3 buckets, only. To get the aws-nuke supported resources, to check if both resources are listed, just run:

aws-nuke resource-types

A complete list is returned, and you should find the resources we are looking for, that is, S3Bucket and CloudFormationStack. Now, let's change our config.yml file:

regions:
  - us-east-2
  - global
account-blocklist:
  - "999999999999" # production
resource-types:
  targets:
    - CloudFormationStack
    - S3Bucket
accounts:
  "000000000000":
    presets:
      - "cf-filter"
      - "s3-filter"
presets:
  cf-filter:
    filters:
      CloudFormationStack:
        - property: 'Name'
          type: glob
          value: "stack-example-*"
          invert: true
  s3-filter:
    filters:
      S3Bucket:
        - property: 'Name'
          type: glob
          value: "bucket-example-*"
          invert: true

Remember to change the 000000000000 with your AWS root account ID. In this example, we specified to delete cloudformation and S3 resources only, by using preset filters. With the usage of the glob type, we are instructing aws-nuke to delete cloudformation stacks that start with the string "stack-example-" and end with anything else, as defined by the wildcard character (*). The same occurs with the S3 resources, but with a different string pattern. For any other types and filtering options, please visit this website.

By default, the aws-nuke tool deletes all the AWS resources, except the filtered ones. But, in our example, we want the inverse, that is, to delete the ones defined by the filters (e.g., to delete stacks and buckets created by a particular developer or issue, that is no longer necessary). Then, to invert that logic, we should set the invert key to the value true (invert: true) for both resources.

Destroying

With the config.yml file created in the previous section, let's run the cleaning process. In order to list what will be performed, just run the command below. You should be prompted to type your AWS account alias, for security concerns.

aws-nuke -c config.yml

As you can see in the figure, only 3 resources would be removed, that is, the ones that matches the glob pattern. As already explained, this command does not delete the resources, but list them only. To delete them, you should add the —-no-dry-run option🚨. When asked, type your account alias after the confirmation of what will be deleted.

aws-nuke -c config.yml --no-dry-run

Nice, the resources have been deleted with a simple configuration file and a quite useful tool, without requiring extra development efforts. If you have to create an automation for it, by using pipelines, for example, you can use an extra option, the —-force. But, be careful, because no confirmation will be asked before deleting the resources.