5 Steps: Using Amazon ECS Exec to pass through Fargate/ECS into containers

Published: 2021-03-16

Lastmod: 2023-01-17

(Illustration: pass through! Image source: by Tomas Tuma on Unsplash)

Today I was chasing the information of AWS Pi Week to celebrate AWS 15th Anniversary for S3 Birthday, but I am also very happy to see that Amazon ECS announced a new feature that everyone asks for a long long time: Amazon ECS Exec.

This article is created based on these reference:

Massimo Re Ferre’s article on the official blog: “NEW – Using Amazon ECS Exec to access your containers on AWS Fargate and Amazon EC2"，
And Amazon ECS official document: “Using Amazon ECS Exec for debugging”

Then let’s try it through the AWS CLI interface, let’s try to see Amazon ECS Exec traverse directly to a container on Amazon ECS (EC2).

Highlights

Massimo’s article and Amazon ECS official documents have detailed steps, but before our trying, I would recommend to look at the beginning part of the official docs mentioned about considerations (may also change over time). Here is my summary:

Architecture
- Amazon ECS Exec uses AWS Systems Manager (SSM) Session Manager to establish a connection with the container, and uses AWS IAM policies to control permissions to execute commands.
- Amazon ECS agent or AWS Fargate agent will plant a SSM agent in a designated container, which is a combination of inside and outside.
Considerations
- ECS Exec is only supported on Linux containers.
- ECS Exec is not currently supported using the AWS Management Console.
- ECS Exec is not currently supported in the Asia Pacific (Osaka) Region.（Feeling curious, what is missing from the bottom in Osaka region?)
- You can’t enable ECS Exec for existing tasks. It can only be enabled for new tasks.
- When a user runs commands against a container using ECS Exec, these commands are run as the root user. The SSM agent and its child processes run as root even when you specify a user ID for the container.
- ECS Exec will use some CPU and memory. You’ll want to accomodate for that when specifying the CPU and memory resource allocations in your task definition.
Preparation
- AWS CLI + Session Manager plugin
  - ECS Exec is not yet supported by AWS CLI v2 for now (the week of announcement). Massimo’s article says it will be updated in the coming weeks.
  - I thought AWS CLI v2 is preferred ?!
- Amazon EC2 and AWS Fargate version requirements:
  - Amazon ECS optimized AMI that was released after January 20th, 2021, with an agent version of 1.50.2 or greater.
  - AWS Fargate platform version 1.4.0 or higher.

After we are mentally prepared, let’s start to take a look :)

The following operating scenario assumes that there is an existing project running on Amazon ECS (EC2 launch type), adjusted to an environment that can execute ECS Exec traversal, and successfully executes ECS Exec into the container.

0.1 Preparation: Upgrade Amazon ECS optimized AMI

For the existing ECS cluster (EC2 launch type), remember to update the Amazon ECS optimized AMI step to meet the operating conditions of ECS Exec.

You can use the method you are familiar with to upgrade the AMI. I built a Makefile to include all the AWS CLI commands that happen to use from time to time.（As I grow older, I can’t remember the commands…

1
2
3
4
5
6
7
❯ make get-ami-id
Getting latest Amazon ECS optimized AMI ID (Amazon Linux 2) of assigned region (region-codename)...
AMI_ID = ami-0abcdefghijklmnop

❯ make create-asg-lc

❯ make update-asg

0.2 Preparation: Install Session Manager plugin

Install Session Manager plugin

1
2
3
❯ session-manager-plugin

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.

1. Grant permissions: ECS task IAM role

The ECS Exec feature requires a task IAM role to grant containers the permissions needed for communication between the managed SSM agent (execute-command agent) and the SSM service.

There is a Task role in the ECS task definition that can specify a set of Amazon ECS task IAM role, this IAM role needs to include the following permissions, or it can be defined as an IAM policy and referenced to multiple IAM roles. (I am used to one ECS service (task definition) corresponding to one task role, and that makes the distinction clean.)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
   "Version": "2012-10-17",
   "Statement": [
       {
       "Effect": "Allow",
       "Action": [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:OpenDataChannel"
       ],
      "Resource": "*"
      }
   ]
}

AWS CLI operation example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
❯ aws iam create-policy --policy-name ecsExecPolicy --policy-document file://ecs-exec-policy.json
{
    "Policy": {
        "PolicyName": "ecsExecPolicy",
        "PolicyId": "ANxxxxxxxxxxxxxxxxxxx",
        "Arn": "arn:aws:iam::123456789012:policy/ecsExecPolicy",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "CreateDate": "2021-03-17T09:03:45+00:00",
        "UpdateDate": "2021-03-17T09:03:45+00:00"
    }
}

❯ aws iam attach-role-policy --policy-arn arn:aws:iam:123456789012:policy/ecsExecPolicy --role-name YourEcsTaskRoleNameHere

2. Modify task definition

Modify task definition parameters so that initProcessEnabled = true, will run the init process in the container, it will clear the SSM agent and its child processes that may become zombies.

Docs says it is optional, so it’s up to you. You can modify it or not, depending on if you want or don’t want to raise some zombies (Huh?! XDD

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
    "containerDefinitions": [
        {
            "linuxParameters": {
                "initProcessEnabled": true
            }
        }
    ],
    "family": "ecs-exec-task"
}

3. Enable Amazon ECS Exec

According to the conditions, you have to run a new ECS task again to have the SSM agent in it. There are four actions that can accomplish this:

Because the context of this article is that there are already existing tasks running on Amazon ECS, so first try the AWS CLI command [update-service](https://docs.aws.amazon.com/cli/latest/reference/ecs /update-service.html) with the parameter --enable-execute-command. (This article was written (March 18, 2021) at the time AWS CLI v2 did not support this parameter.)

If it is a newly created ECS service, please refer to create-service.
If not using ECS service, but want to run the task directly, please refer to start-task or run- task.

1
2
3
4
5
6
7
8
9
❯ aws --version
aws-cli/1.19.32 Python/3.8.5 Linux/4.19.121-linuxkit botocore/1.20.32

❯ aws ecs update-service \
  --cluster your-cluster-name \
  --service your-service-name \
  --task-definition your-td-family-name:N \
  --force-new-deployment \
  --enable-execute-command

After enabling ECS Exec for a task, you can use the describe-tasks command to check that lastStatus of the ExecuteCommandAgent should be RUNNING, and the enableExecuteCommand property should be set to true. This task is set up and ready to go :)

1
2
3
❯ aws ecs describe-tasks \
    --cluster your-cluster-name \
    --tasks your-task-id

Here is the key inspections of the output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
    "tasks": [
        {
            ...
            "containers": [
                {
                    ...
                    "managedAgents": [
                        {
                            "lastStartedAt": "2021-03-18T112:13:14.151600+08:00",
                            "name": "ExecuteCommandAgent",
                            "lastStatus": "RUNNING"
                        }
                    ]
                }
            ],
            ...
            "enableExecuteCommand": true,
            ...
        }
    ]
}

4. Get pass through！ Go! Amazon ECS Exec!

Use the command execute-command to start traversing! Go!

The --command parameter can be replaced with any command you want to execute in your container.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
❯ aws ecs execute-command --cluster your-cluster-name \
    --task your-task-id \
    --container your-container-name \
    --interactive \
    --command "/bin/bash"

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-0936520b5bac0473e
root@46baf93bef74:/var/www/html#

Successfully traversed!

Bottom line

Amazon ECS Exec is quite functional. It is the same as most AWS services. After IAM grants the corresponding permissions, then update the service to the corresponding version (or higher), and it will work.

Although this article lists a total of 5 steps from step #0 to step #4, after the environment setting is completed, you only need to find the ECS task id you want to pass through and go directly with step #4.

Because Amazon ECS Exec has CloudFormation support when the Amazon ECS Exec function is released, just wait for AWS CDK L2 support. But it can already be implemented in CDK :) You can refer to [Pahud’s example which uses CDK to implement the environment settings required by Amazon ECS Exec] (https://github.com/pahud/ecs-exec-cdk-demo), more convenient than AWS CLI. You can choose the tools you are good at to use, either way is great.

If your ECS Exec journey is successful, welcome to share with your local tech community, such as AWS User Group Taiwan (AWSUG TW) :)