This is a simple Scala application written using the Scalatra framework that retrieves a list of instances off AWS using the EC2 API.

Source Code

The source code is in a private Bitbucket repository. Please contact Siddhu Warrier if you wish to access it.

git clone git@github.com:siddhuwarrier/aws-api.git

API Documentation

You can view the API documentation by loading up https://staging.awsapi.siddhuw.info/api-explorer/swagger.json in a Swagger client.

Endpoint	Params	Response
POST /auth	BODY {"username": username, "password": password}	200 with JSON containing JWT token, 401 UNAUTHORIZED if username and/or password are invalid)
GET /versions		200 with JSON containing git commit hash of latest commit
GET /health		200 with JSON containing git commit hash of latest commit if running
GET /api/aws/regions	AUTHORIZATION header set with bearer token returned in (Bearer JWT token)	200 with JSON of list of regions, 401 UNAUTHORIZED, 429 TOO MANY REQUESTS if you exceed the rate limit (managed using a token bucket algorithm
GET /api/aws/ec2/instances	AUTHORIZATION header set with bearer token returned in (Bearer JWT token)	200 with JSON of list of instances, 401 UNAUTHORIZED, 429 TOO MANY REQUESTS if you exceed the rate limit (managed using a token bucket algorithm, 500 INTERNAL SERVER ERROR (if AWS credentials invalid), 503 SERVICE UNAVAILABLE (if AWS inaccessible)

AWS deployment

This microservice is deployed in an ECS container (Fargate) deployed in a VPC behind an ALB to AWS eu-west-2 London and can be accessed at https://staging.awsapi.siddhuw.info. (use the endpoints above with the right username and password to use it). The entire deployment process is automated:

• The AWS infrastructure is built using Terraform. See the git submodule in the terraform subdirectory (git submodule update --init --recursive).
• Every pull-request is built using AWS CodeBuild.
• On merge to master, an AWS CodePipeline is used to check out source, build a Docker image deployed to AWS ECR, and update the ECS containers.

This is how user requests are routed and handled.

User -> AWS ALB -> ECS Container (Fargate) <-> RDS Aurora (Postgres)

Security

• NO AWS CREDENTIALS ARE DEPLOYED TO AWS. Instead, I use IAM roles associated with the ECS task in conjunction with the AWS default credentials chain to access the credentials.
• Neither the ECS containers nor the RDS instance are directly accessible from the public internet. A bastion host ( t2.micro) is deployed in eu-west-2 with SSH access (secured using a RSA-4096 keypair), which can be used to access the RDS database using psql (this is what I used to add users that can authenticate against the app).
• All requests to the /api/aws endpoints are authenticated, requiring a JWT bearer token that can be generated by hitting the auth endpoint with a valid username and password. The username and the salted password hash are stored in the Postgres DB (I'd initially started out with Twitter OAuth, but for reasons I've now forgotten 6 years hence, I had to remove that). Please ask me if you'd like credentials to access it.

Build and deploy instructions

Pre-requisites

• Java 11
• Maven 3.6
• Docker Engine (if you wish to run the service locally)
• AWS credentials (if you wish to run the service locally)

To run just tests, type mvn test. This will also run the integration tests (which use an in-memory H2 database).

To generate a code coverage report, run

mvn scoverage:report -Pcoverage

The scoverage report is produced in target/site/scoverage/index.html. Due to a bug in the scoverage Maven plugin, none of the hyperlinks from the main page work. However, you can open each individual HTML file for each class manually.

Running locally

To run the app locally, first set your AWS access key and secret access key in a file in the root of your repo called .env (in an AWS deployment, we would use IAM roles):

AWS_ACCESS_KEY_ID=<enter-your-access-key>
AWS_SECRET_ACCESS_KEY=<enter-your-secret-access-key>

run:

docker-compose build
docker-compose up -d

You can view the logs by typing:

docker logs -f <directory-name>_microservice_1

Note: You may receive some warnings as Tomcat starts up.

The API should be accessible on http://localhost:8080. However, please note that you cannot access the API without first authenticating.

The PostGres Docker container is initialised with a single user admin with the password burak-crush-pineapple (note: the password is stored hashed and salted in the DB).

You can get a JWT token to make requests using this username and password as follows:

curl --silent http://localhost:8080/auth -X POST -d "{\"username\":\"admin\", \"password\":\"burak-crush-pineapple\"}" -H "Content-Type:application/json"

Note on Logging

All logs are written in the logstash format. This would potentially allow for it to be fed into ElasticSearch and viewed on Kibana.

Future Work/Improvements

• Migration scripts for DB schema changes.
• API documentation using Swagger.
• Move hmac_key used to sign JWT tokens out of source control into configuration management.
• The OWSAP Enterprise Security API requires all logging go through log4j. Either switch to log4j or replace ESAPI.