A Practical Example of Container Portability: DynaTrace ActiveGate in AWS ECS

Pre-Flight General Knowledge Requirements

• Container
• Container Orchestration
• Runtime Abstraction
• Configuration as Code
• Basic Networking

The Problem

• DynaTrace ActiveGate is not officially support on AWS ECS
• Avoid using an EC2 instance if at all possible
• Need to be able to automatically update ActiveGate on a schedule
• 99.999% uptime goal

The Workload

• ECS cluster with persistent services
• Each services has a OneAgent sidecar
  • To be covered in a future article
• ActiveGate is the only service that will egress to the a remote network

Scope

While containers may be portable; orchestration feature coverage varies widely. For brevity this topic is out of scope of this article.

The Solution

Using the Kubernetes configuration YAML as a starting point we create AWS ECS task_definition configuration for the containers. Mapping as much of the Kubernetes YAML key/value to the appropriate ECS JSON key/value. CPU, MEM, mounted disks, ports, etc.

If you think of containers are a boundary abstraction, like a web API, then the orchestration system is does not matter. There are features no solution will support that others do not, sure, but in my experience the unique features are rarely critical to making a process run.

The Configurations

As a practical example here is the complete DynaTrace ActiveGate Kubernetes YAML:

apiVersion: v1
kind: Service
metadata:
name: dynatrace-activegate
namespace: dynatrace
spec:
type: ClusterIP
selector:
  activegate: kubernetes-monitoring-and-routing
ports:
- protocol: TCP
  port: 443
  targetPort: ag-https
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: dynatrace-activegate
namespace: dynatrace
labels:
  activegate: kubernetes-monitoring-and-routing
spec:
serviceName: ""
selector:
  matchLabels:
    activegate: kubernetes-monitoring-and-routing
template:
  metadata:
runtime/default
    labels:
      activegate: kubernetes-monitoring-and-routing
  spec:
    serviceAccountName: dynatrace-activegate
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/arch
              operator: In
              values:
              - amd64
            - key: kubernetes.io/os
              operator: In
              values:
              - linux
    containers:
    - name: activegate
      image: <YOUR_ENVIRONMENT_URL>/linux/activegate
      imagePullPolicy: Always
      ports:
        - name: ag-https
          containerPort: 9999
      env:
      - name: DT_ID_SEED_NAMESPACE
        value: dynatrace
      - name: DT_ID_SEED_K8S_CLUSTER_ID
        value: <YOUR_KUBE-SYSTEM_NAMESPACE_UUID>
      - name: DT_CAPABILITIES
        value: kubernetes_monitoring,MSGrouter,restInterface
      - name: DT_DNS_ENTRY_POINT
        value: https://$(DYNATRACE_ACTIVEGATE_SERVICE_HOST):$(DYNATRACE_ACTIVEGATE_SERVICE_PORT)/communication
      volumeMounts:
      - name: dynatrace-tokens
        mountPath: /var/lib/dynatrace/secrets/tokens
      - name: truststore-volume
        mountPath: /opt/dynatrace/gateway/jre/lib/security/cacerts
        readOnly: true
        subPath: k8s-local.jks
      - name: ag-lib-gateway-config
        mountPath: /var/lib/dynatrace/gateway/config
      - name: ag-lib-gateway-temp
        mountPath: /var/lib/dynatrace/gateway/temp
      - name: ag-lib-gateway-data
        mountPath: /var/lib/dynatrace/gateway/data
      - name: ag-log-gateway
        mountPath: /var/log/dynatrace/gateway
      - name: ag-tmp-gateway
        mountPath: /var/tmp/dynatrace/gateway
      livenessProbe:
        failureThreshold: 2
        httpGet:
          path: /rest/state
          port: ag-https
          scheme: HTTPS
        initialDelaySeconds: 30
        periodSeconds: 30
        successThreshold: 1
        timeoutSeconds: 1
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /rest/health
          port: ag-https
          scheme: HTTPS
        initialDelaySeconds: 30
        periodSeconds: 15
        successThreshold: 1
        timeoutSeconds: 1
      resources:
        requests:
          cpu: 250m
          memory: 512Mi
        limits:
          cpu: 250m
          memory: 512Mi
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop:
          - all
        privileged: false
        readOnlyRootFilesystem: true
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
    initContainers:
    - name: certificate-loader
      image: YOUR_ENVIRONMENT_URL>/linux/activegate
      workingDir: /var/lib/dynatrace/gateway
      command: ['/bin/bash']
      args: ['-c', '/opt/dynatrace/gateway/k8scrt2jks.sh']
      volumeMounts:
      - mountPath: /var/lib/dynatrace/gateway/ssl
        name: truststore-volume
    imagePullSecrets:
    - name: dynatrace-docker-registry
    volumes:
    - name: dynatrace-tokens
      secret:
        secretName: dynatrace-tokens
    - name: truststore-volume
      emptyDir: {}
    - name: ag-lib-gateway-config
      emptyDir: {}
    - name: ag-lib-gateway-temp
      emptyDir: {}
    - name: ag-lib-gateway-data
      emptyDir: {}
    - name: ag-log-gateway
      emptyDir: {}
    - name: ag-tmp-gateway
      emptyDir: {}
updateStrategy:
  type: RollingUpdate

Focusing on volumes and containers yaml keys, everything else can be ignored for now as being Kubernetes specific. Now we have the following to work with:

containers:
  - name: dynatrace-activegate
    image: dynatrace/activegate
    args:
      - '--set-network-zone=<your.network.zone>'
    volumeMounts:
      - name: host-root
        mountPath: /mnt/root
env:
  - name: DT_ID_SEED_NAMESPACE
    value: dynatrace
  - name: DT_ID_SEED_K8S_CLUSTER_ID
    value: <YOUR_KUBE-SYSTEM_NAMESPACE_UUID>
  - name: DT_CAPABILITIES
    value: kubernetes_monitoring,MSGrouter,restInterface
  - name: DT_DNS_ENTRY_POINT
    value: https://$(DYNATRACE_ACTIVEGATE_SERVICE_HOST):$(DYNATRACE_ACTIVEGATE_SERVICE_PORT)/communication
volumes:
  - name: host-root
    hostPath:
      path: /

Seems reasonably easy to convert this to ECS container_definition JSON syntax and add a couple ECS specific configurations like executionRoleArn and networkMode.

{
   "containerDefinitions": [ 
      { 
         "image": "dynatrace/activegate",
         "name": "dynatrace-activegate",
         "portMappings": []
      }
   ],
   "executionRoleArn": "arn:aws:iam::012345678910:role/ecsTaskExecutionRole",
   "family": "fargate-task-definition",
   "networkMode": "awsvpc",
   "runtimePlatform": {
        "operatingSystemFamily": "LINUX"
    },
   "requiresCompatibilities": [ 
       "FARGATE" 
    ]
}

With a little more effort add more ECS configuration. logConfiguration, essential, portMappings…

{
   "containerDefinitions": [ 
      { 
        command = [
          "--authtoken=REDACTED",
          "--tenant=REDACTED",
          "--token=REDACTED",
          "--networkzone=REDACTED"
        ]
         "essential": true,
         "image": "dynatrace/activegate",
         "logConfiguration": { 
            "logDriver": "awslogs",
            "options": { 
               "awslogs-group" : "/ecs/fargate-task-definition",
               "awslogs-region": "us-east-1",
               "awslogs-stream-prefix": "ecs"
            }
         },
         "name": "dynatrace-activegate",
         "portMappings": []
      }
   ],
   "cpu": "256",
   "executionRoleArn": "arn:aws:iam::012345678910:role/ecsTaskExecutionRole",
   "family": "fargate-task-definition",
   "memory": "512",
   "networkMode": "awsvpc",
   "runtimePlatform": {
        "operatingSystemFamily": "LINUX"
    },
   "requiresCompatibilities": [ 
       "FARGATE" 
    ]
}

And there it is. The majority of the configuration to get ActiveGate running in AWS ECS; you will need a few ENV VAR entries for remote DNS and maybe credentials. Even though it is not officially supported, it will, and does, run as expected.

Larger Impact

The wider impact of this is that once we understand the core concepts of containerization being a boundary and general container orchestration simply being a hosting solution it enables service portability. Docker-Swarm to Kubernetes? no problem. AWS ECS to Docker Swarm? no problem. OpenShift to Podman? Again, no problem. The promise of container portability delivers.

Additional Resources

• DynaTrace ActiveGate
  • https://www.dynatrace.com/support/help/setup-and-configuration/dynatrace-activegate
  • https://www.dynatrace.com/support/help/setup-and-configuration/setup-on-container-platforms/kubernetes/get-started-with-kubernetes-monitoring/set-up-k8s-monitoring-daemonset#install
• AWS ECS
  • https://docs.aws.amazon.com/AmazonECS/latest/developerguide/example_task_definitions.html#example_task_definition-webserver