Skip to main content

Installing JTL Reporter on AWS ECS (EC2 or Fargate)

Prerequisites#

Preparation#

Configure PostgreSQL#

You will need the following information:

  • PostgreSQL Hostname (from here on: <psqlhost>)
  • PostgreSQL Username (from here on: <psqluser>)
  • PostgreSQL Password (from here on: <psqlpassword>)

Log into the PostgreSQL Database via psql and execute the following commands:

CREATE DATABASE jtl_report;
\c jtl_report
CREATE EXTENSION "uuid-ossp";
CREATE SCHEMA IF NOT EXISTS jtl;
CREATE TABLE jtl.projects(
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
project_name character varying(50) NOT NULL UNIQUE
);
CREATE TABLE jtl.items (
id uuid DEFAULT uuid_generate_v4() PRIMARY KEY,
test_name character varying(40) NOT NULL,
project_id uuid NOT NULL REFERENCES jtl.projects(id),
jtl_data jsonb NOT NULL,
note character varying(150),
environment character varying(20),
upload_time timestamp without time zone DEFAULT now(),
start_time timestamp without time zone,
duration integer
);
CREATE TABLE jtl.item_stat (
id uuid DEFAULT uuid_generate_v4() PRIMARY KEY,
item_id uuid NOT NULL REFERENCES jtl.items(id),
stats jsonb NOT NULL
);

Configure AWS Elastic File System (EFS) for MongoDB Persistent Storage#

Amazon DocumentDB does not support several features required by JTL Reporter to serve as a drop-in replacement for MongoDB (see Using JTL-Reporter with AWS DocumentDB), so we are going to use AWS Elastic File System for the MongoDB persistent storage.

Create an EFS File System and Mount Target(s) accessible from the ECS Cluster (see the EFS documentation for a walkthrough). Note the File System ID, which looks like fs-0a1b2c3d (from now on: <efsid>).

ECS Task#

Required Variables#

You require the following information:

  • <efsid>
  • <psqlhost>
  • <psqlpassword>
  • <psqluser>
  • Any string to serve as the JWT Token (from here on: <jwt_token>)
  • Any string to serve as the JWT Token Login (from here on: <jwt_token_login>)

Additional Notes#

The hostnames for the docker containers in the task are hard coded. This is no problem when bringing up the service via docker-compose, but in ECS Fargate you cannot assign hostnames to the containers in a Task. To work around this, we will append the hostnames for the backend, frontend, listener, and mongodb to /etc/hosts on the containers (which contains the IP address of the Task, so it will resolve the local hostnames to the correct IP address).

Do do that, we modify the command of the containers in the ContainerDefinitions to first echo the hostnames into /etc/hosts, then start the actual service.

For MongoDB, we also first echo all commands to prepare the database into /docker-entrypoint-initdb.d/mongo-init.js before starting the service. We also attach the EFS volume we created earlier to the /data/db/ directory to serve as persistent storage.

IMPORTANT: Only one MongoDB can access the database files on the EFS volume concurrently, so you need to ensure that only one JTL Reporter service is running at a time and spin down the old service before replacing it with a newer version.

ECS Task Definition#

Below is a minimum viable ECS Task Definition CloudFormation resource. You will have to adjust it to your environment and configuration.

Resources:
JtlTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Cpu: 256
Memory: 512
ContainerDefinitions:
- Name: jtl_reporter_fe
Command:
- /bin/sh
- -c
- echo " be fe listener mongodb" >> /etc/hosts && nginx -g "daemon off;"
DependsOn:
- Condition: START
ContainerName: jtl_reporter_be
- Condition: START
ContainerName: jtl_reporter_mongodb
Essential: true
Image: novyl/jtl-reporter-fe:v3.4.1
PortMappings:
- ContainerPort: 80
HostPort: 80
Protocol: tcp
- Name: jtl_reporter_mongodb
Command:
- /bin/sh
- -c
- >
echo 'db.createCollection("data-chunks"); newCol = db.getCollection("data-chunks"); newCol.createIndex({ dataId: -1 }, { name: "data-id-index" });' > /docker-entrypoint-initdb.d/mongo-init.js &&
/usr/local/bin/docker-entrypoint.sh mongod
Environment:
- Name: MONGO_INITDB_DATABASE
Value: jtl-data
Essential: true
Image: mongo:4.2.5-bionic
MountPoints:
- ContainerPath: /data/db/
SourceVolume: mongodb-efs
- Name: jtl_reporter_be
Command:
- /bin/sh
- -c
- echo " be fe listener mongodb" >> /etc/hosts && npm run start
Environment:
- Name: DB_HOST
Value: <psqlhost>
- Name: DB_PASS
Value: <psqlpassword>
- Name: DB_USER
Value: <psqluser>
- Name: JWT_TOKEN
Value: <jwt_token>
- Name: JWT_TOKEN_LOGIN
Value: <jwt_token_login>
- Name: MONGO_CONNECTION_STRING
Value: mongodb://mongodb:27017
Essential: true
Image: novyl/jtl-reporter-be:v3.4.2
- Name: jtl_reporter_migration
Command:
- /bin/sh
- -c
- echo " be fe listener mongodb" >> /etc/hosts && npm run migrate up
Environment:
- Name: DATABASE_URL
Value: postgres://<psqluser>:<psqlpassword>@<psqlhost>/jtl_report
Essential: false
Image: novyl/jtl-reporter-be:v3.4.2
- Name: jtl_reporter_listener
Command:
- /bin/sh
- -c
- echo " be fe listener mongodb" >> /etc/hosts && npm run start
Environment:
- Name: JWT_TOKEN
Value: <jwt_token>
- Name: MONGO_CONNECTION_STRING
Value: mongodb://mongodb:27017
Essential: true
Image: novyl/jtl-reporter-listener-service:v1.0.1
NetworkMode: awsvpc # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html#cfn-ecs-taskdefinition-networkmode
Volumes:
- EfsVolumeConfiguration:
FilesystemId: <efsid>
Name: mongodb-efs