TechnoDev DevOps CI/CD Platform
A CI/CD platform for four containerised Lambda microservices, built inside a private VPC because managed CI/CD services are unavailable in the lab. GitHub Actions run on self-hosted EC2 runners in private subnets, images go to ECR, REST and WebSocket API Gateways expose the functions, and CloudWatch, VPC Flow Logs, Secrets Manager, and Cognito cover observability and security.
- Role
- DevOps engineer (module exercise). Network, runners, data services, container registry, Lambda deployment, API Gateway, pipelines, observability. Application code, SQL schema, frontend, and workflow definitions were provided by the module.
- Context
- LKS Nasional 2026 Cloud Computing, DevOps Automation module, on AWS Academy Learner Lab. TechnoDev is the fictional company in the brief.
- Period
- 2026-07 (Public repository created 2026-07-27.)
- Evidence
- ArchitectureDocumentationTechnical breakdownSource code
01
The Case
TechnoDev, the fictional e-commerce company in this competition module, is moving to microservices and needs automated delivery. The AWS Academy Learner Lab does not offer CodePipeline or CodeBuild, so the pipeline has to be built from GitHub Actions with runners that live inside a private AWS network and still reach GitHub, ECR, Lambda, and RDS.
The module's ten tasks cover the network, the runners, the data layer, the container registry and Lambda functions, both API Gateways, the Amplify frontend, the CI and CD workflows, secrets and monitoring, and an end-to-end verification.
02
Evidence
End-to-end architecture figure from the module
architecture
GitHub Actions workflows: test-runner.yaml, ci.yaml, deploy.yaml
code
LKS Nasional 2026 DevOps Automation module (18 pages)
document
03
Architecture
Text version of the diagram
Source and CI/CD
- Developer external
-
GitHub repository
GitHub
devops-learner-lab with branch protection requiring one review.
-
GitHub Actions
GitHub Actions
ci.yaml on pull requests, deploy.yaml on merge to main. Workflow definitions provided by the module.
VPC devops-vpc (us-west-2)
-
Self-hosted runners (2x EC2)
Amazon EC2
Ubuntu 24.04 t3.medium in private subnets 210.0.10.0/24 and 210.0.11.0/24; Docker, AWS CLI, Git; reached through Session Manager.
-
NAT Gateway
NAT Gateway
Egress for private subnets; Internet Gateway for public subnets.
Compute
-
ECR (4 repositories)
Amazon ECR
Mutable tags, scan on push.
-
Lambda (4 container functions)
AWS Lambda
user-api, order-api, notif-worker, websocket-api; Python 3.11 base image; VPC-attached. Function code provided by the module.
API layer
-
REST API devops-api
Amazon API Gateway
/users, /orders, /products with {proxy+}; Lambda proxy integration; stage prod.
-
WebSocket API devops-ws-api
Amazon API Gateway
$connect, $disconnect, $default, sendMessage; ping, get_orders, order_status_update, send_notification actions.
Data and messaging
-
RDS PostgreSQL 15
Amazon RDS
db.t3.small, Multi-AZ, encrypted, 7-day backups, private subnet group.
-
SQS queues + DLQ
Amazon SQS
orders and notifications queues redrive to devops-dlq after 3 receives.
-
SNS topics
Amazon SNS
devops-notifications and devops-alerts with e-mail subscriptions.
Frontend and auth
-
Amplify frontend
AWS Amplify
Manual deployment of the provided SPA.
-
Cognito user pool
Amazon Cognito
Hosted UI, SPA client, JWT authorizer on the REST API.
Security and observability
-
Secrets Manager
AWS Secrets Manager
devops/rds-credentials fetched at runtime by all four functions.
-
CloudWatch dashboard
Amazon CloudWatch
Lambda invocations, errors, p50/p99 duration; RDS CPU and connections; SQS queue depth.
-
VPC Flow Logs
VPC Flow Logs
ALL traffic to CloudWatch Logs.
Connections
- Developer → GitHub repository · pull request
- GitHub repository → GitHub Actions · triggers
- GitHub Actions → Self-hosted runners (2x EC2) · jobs
- Self-hosted runners (2x EC2) → NAT Gateway · egress
- Self-hosted runners (2x EC2) → ECR (4 repositories) · docker push
- Self-hosted runners (2x EC2) → Lambda (4 container functions) · update-function-code
- Self-hosted runners (2x EC2) → Amplify frontend · create-deployment
- ECR (4 repositories) → Lambda (4 container functions) · image
- REST API devops-api → Lambda (4 container functions) · AWS_PROXY
- WebSocket API devops-ws-api → Lambda (4 container functions) · routes
- Lambda (4 container functions) → RDS PostgreSQL 15 · port 5432 (PostgreSQL)
- Lambda (4 container functions) → SQS queues + DLQ · send / consume
- SQS queues + DLQ → Lambda (4 container functions) · event source mapping
- Lambda (4 container functions) → SNS topics · publish
- Lambda (4 container functions) → Secrets Manager · get_secret_value
- Amplify frontend → REST API devops-api · REST
- Amplify frontend → WebSocket API devops-ws-api · WebSocket
- Cognito user pool → REST API devops-api · authorizer
- Lambda (4 container functions) → CloudWatch dashboard · metrics
- RDS PostgreSQL 15 → CloudWatch dashboard · metrics
- SQS queues + DLQ → CloudWatch dashboard · metrics
- NAT Gateway → VPC Flow Logs · flow records
04
Technical Breakdown
infrastructure
VPC devops-vpc (210.0.0.0/16) with two public and two private subnets across us-west-2a
and us-west-2b, an Internet Gateway, one NAT Gateway with an Elastic IP, and separate public
and private route tables. Two EC2 runners (Ubuntu 24.04 LTS, t3.medium, LabRole instance
profile) with Docker Engine, AWS CLI v2, Git, and the GitHub Actions runner labelled
self-hosted, linux, x64.
Four ECR repositories with image scanning on push hold the container images built from the
provided Dockerfiles on the Lambda Python 3.11 base image. Four Lambda functions run from those
images inside the private subnets with the devops-sg-lambda security group.
data
RDS PostgreSQL 15 (devops-db, db.t3.small, 20 GB gp3) initialised with the provided
schema.sql from a runner over the private network. SQS standard queues devops-orders-queue
(30 s visibility) and devops-notifications-queue (60 s visibility) with devops-dlq
(14-day retention) as the redrive target; an event source mapping feeds the notification worker
in batches of ten. SNS topics devops-notifications and devops-alerts with confirmed e-mail
subscriptions.
security
Four security groups by least privilege, VPC-attached Lambda functions, Secrets Manager for the database credentials, and a Cognito user pool (e-mail as username, password policy, hosted UI) whose JWTs are validated by an API Gateway authorizer on the REST API.
observability
CloudWatch dashboard devops-dashboard with Lambda invocations, errors, and p50/p99 duration,
RDS CPU utilisation and connections, and SQS queue depth. VPC Flow Logs capture ALL traffic to
the log group /aws/vpc/devops-vpc/flowlogs, so both ACCEPT and REJECT records are available
for troubleshooting.
deployment
ci.yaml runs on pull requests against main: flake8 on each handler, a Docker build per
service, an import check of every handler inside its container, and presence checks for the
frontend and SQL schema. deploy.yaml runs on push to main: it logs in to ECR, builds and
pushes the four images tagged latest and with the commit SHA, updates each function's image,
zips and deploys the frontend to Amplify through a manual deployment, and smoke-tests the REST
API. Both target runs-on: self-hosted. The workflow definitions were supplied by the module;
the work was provisioning everything they depend on and getting them green on the runners.
05
Key Findings
Engineering decisions
- Two self-hosted GitHub Actions runners on EC2 (Ubuntu 24.04, t3.medium) in private subnets across two Availability Zones, administered only through Session Manager, with outbound access through a NAT Gateway.
- Four least-privilege security groups: runners self-referencing, Lambda outbound-only, RDS on port 5432 only from the Lambda and runner groups, and a self-referencing default group.
- Four containerised Lambda functions from ECR images (user-api, order-api, notif-worker, websocket-api) attached to the private subnets; a REST API with greedy {proxy+} resources and a WebSocket API with $connect, $disconnect, $default, and sendMessage routes.
- Dead-letter queue created before the application queues; both queues redrive to it after three failed receives; the notification worker consumes SQS with batch size 10.
- Database credentials moved from Lambda environment variables to Secrets Manager and fetched at runtime.
- CI on pull requests (flake8, four Docker builds, handler import checks, frontend and schema checks); CD on merge to main (ECR push, Lambda image update, Amplify deployment, API smoke test).
- Private runners are the security boundary. Both runners sit in private subnets without public IPs. Administration goes through Session Manager, package downloads through the NAT Gateway, and the runner service is registered as a system service so it survives reboots.
- Security groups, not passwords, gate the database. RDS accepts port 5432 only from the Lambda and runner security groups. It is Multi-AZ, encrypted, backed up for seven days, and never publicly accessible.
- Create the dead-letter queue first. Both application queues carry a redrive policy to it from the start, so a message that fails three times is isolated rather than lost.
- Greedy proxy routes keep the API small. Three resources with
{proxy+}children forward every nested path to the right function through Lambda proxy integration. The read-only product catalogue is served by the order function, which already holds the database connection. - Secrets move out of environment variables. The database password is stored once in Secrets Manager and read at runtime by all four functions.
06
Challenges and Solutions
Challenges
- AWS CodePipeline and CodeBuild are unavailable in the Learner Lab.
- Runners without public IPs still need to reach GitHub, ECR, and package repositories.
- Failed messages must not be lost and the database must never be reachable from the internet.
Solutions
- GitHub Actions with self-hosted runners registered from inside the VPC.
- Private subnets with a NAT Gateway for egress and Session Manager for administration.
- SQS dead-letter queue with 14-day retention and redrive policies; RDS Multi-AZ, encrypted, private, in a subnet group spanning both private subnets.
07
Result
The three workflows (runner verification, CI, CD) are in the public repository and target the self-hosted runners. The module's acceptance test is a CRUD sequence through the REST API (create user, create order, update status, verify the SNS notification, cascade delete) and a WebSocket sequence (ping, get_orders, status broadcast to a second client, disconnect cleanup). Execution logs from the lab session were not kept, so the outcome is documented by the module requirements rather than by retained evidence.
Outcomes on record
- Runner verification, CI, and CD workflows are committed to the public repository and target the self-hosted runners.
- The module defines an end-to-end CRUD test and a WebSocket broadcast test as the acceptance criteria; execution logs were not kept. Not documented
08
Stack
AWS services
Infrastructure and IaC
Containers and orchestration
Databases and storage