DevSecOps Fundamentals Project

Table of Contents

Introduction

Every engineer has that one project — the one that starts as a simple idea and evolves into a full-blown learning laboratory. For me, that project is this Publisher Web App: a system designed to publish announcements across multiple social media channels simultaneously. The primary use case? The AWS Certification Announcer, a community tool where members submit their AWS certification achievements and the platform automatically publishes them to Facebook, Instagram, WhatsApp, LinkedIn, Email, etc.

What began as a practical need for the AWS User Group quickly became an exercise in applying every DevSecOps principle I wanted to put into practice, but couldn’t until now. This article walks through the fundamentals of the project — the architecture, the security posture, the deployment strategy, and the lessons learned along the way.

The entire project is open source and available at github.com/Walsen/devsecops-poc .

The Vision: More Than Just Posting

The goal was never to build “yet another social media scheduler.” The vision was to create a platform that could serve as a reference implementation for modern cloud-native development — one that demonstrates how to build software that is secure by design, portable across deployment models, and maintainable over time.

The platform needed to:

Hexagonal Architecture: The Foundation

If there is one architectural decision that made everything else possible, it is the adoption of hexagonal architecture (also known as Ports & Adapters).

This pattern separates the system into three concentric zones:

The application core — domain entities, use cases, and port interfaces — has absolutely zero knowledge of the outside world. It doesn’t know if it’s running on ECS or Lambda. It doesn’t know if the database is PostgreSQL or DynamoDB. It only knows about abstractions.

The Domain Model

The domain is built with pure Python dataclasses:

Ports and Adapters in Practice

The MessageRepository port is the same interface regardless of deployment mode. Only the adapter changes:

# Port (shared across both modes)
class MessageRepository(ABC):
    @abstractmethod
    async def get_by_id(self, id: UUID) -> Message | None: ...

    @abstractmethod
    async def save(self, message: Message) -> None: ...

# Container adapter
class PostgresMessageRepository(MessageRepository):
    def __init__(self, session: AsyncSession):
        self._session = session

# Serverless adapter
class DynamoMessageRepository(MessageRepository):
    def __init__(self, table_name: str):
        self._table = boto3.resource("dynamodb").Table(table_name)

This is the key enabler for the dual-mode deployment strategy. Business logic is completely decoupled from infrastructure.

Three Services, One Stream

The platform is composed of three microservices connected by Amazon Kinesis Data Streams:

Each service has its own main.py that serves as the composition root — the only place where concrete implementations are imported and wired together. No other module imports from infrastructure/ directly.

Dual-Mode Deployment: Containers or Serverless

This is where hexagonal architecture pays off in a very tangible way. The platform supports two fully independent deployment modes, switchable at deploy time via a single CI/CD parameter:

Both modes share the same domain and application layers. Only the infrastructure adapters change. The deploy workflow selects the mode via infra_type input (containers or serverless), routing to the corresponding CDK project. Stack names are fully independent, so both can coexist in the same AWS account during migration.

Why This Matters

Different stages of a project have different needs:

You can start cheap with serverless and migrate to containers when traffic justifies the fixed cost — or vice versa — without rewriting a single line of business logic.

Running Both Simultaneously

Since stack names are independent, you can run both modes in parallel during a migration window. This allows smoke testing against the new mode before cutting over, gradual traffic shifting using weighted DNS routing (Route 53), and instant rollback by switching DNS back.

Zero Trust Security: Trust Nothing, Verify Everything

Security is not a feature you bolt on at the end. It is a design principle that permeates every layer of the system. The platform implements a Zero Trust architecture based on three pillars:

Network Security

All traffic flows through multiple security layers:

The VPC is segmented into public, private, and isolated subnets. Security groups enforce micro-segmentation — the ALB can only talk to the API, the API can only talk to the Worker and the database, and the database accepts connections only from authorized services.

Authentication Flow

Every request is validated against Cognito JWTs with strict algorithm restriction (RS256 only), audience and issuer validation, and JWKS caching with TTL refresh. OAuth credentials for social providers are stored in AWS Secrets Manager.

API Security Middleware

The API service implements a comprehensive security middleware stack:

Data Protection

Encryption is applied at every layer:

Secure Supply Chain

The build pipeline is hardened against supply chain attacks with multiple security gates:

Defense in Depth: Multiple Scanners

No single scanner catches everything. The project uses six complementary tools:

Each tool covers blind spots the others miss. Semgrep catches SQL injection and XSS patterns. Bandit finds Python-specific issues like unsafe YAML loading. pip-audit and Trivy handle known CVEs. Gitleaks prevents credential leaks. Checkov ensures the infrastructure itself is secure — no public S3 buckets, no missing encryption, no overly permissive IAM.

Container Security

Production containers are built with security as a first-class concern:

CI/CD with GitHub Actions OIDC

The project uses GitHub Actions OIDC to assume AWS IAM roles without storing long-lived credentials. A CDK bootstrap stack (GitHubOIDCStack) creates the OIDC provider and two IAM roles:

No AWS access keys are stored in GitHub Secrets. Every deployment uses short-lived credentials (1 hour) obtained via OIDC federation.

The Golden Thread: Distributed Security Tracing

One of the most interesting aspects of the project is the Golden Thread — a distributed tracing exercise that demonstrates end-to-end request correlation across all infrastructure layers using a single correlation ID (X-Request-ID).

The correlation ID flows through:

/* correlation_id=golden-thread-test-1739... */ INSERT INTO messages (id, content_text, ...) VALUES (...)

fields @timestamp, service, event, correlation_id, method, path, status_code
| filter correlation_id = "YOUR_TRACE_ID"
| sort @timestamp asc

AI-Powered Content Adaptation

The Worker service supports two publishing strategies, selectable via configuration:

The AgentPublisher uses the Strands Agents SDK with Amazon Bedrock (Claude) to intelligently adapt content for each platform:

The agent reasons about which tools to call, executes them, observes results, and continues until the task is complete. Content filtering middleware protects against prompt injection and PII leakage in AI-generated output.

AI-Powered Security Testing Agent

Perhaps the most novel addition to the project is the Security Testing Agent — an AI-powered interactive penetration testing assistant built with the Strands Agents SDK and Amazon Bedrock. Instead of manually running security tests and interpreting results, you have a conversation with an agent that can run tests, diagnose failures, query AWS resources, and even auto-fix test code.

The interaction looks like this:

🔒 Penetration Testing Agent Ready!

You: Run all fast tests
Agent: I'll run the fast test suite in parallel, skipping slow scans...
       ✅ All tests passed in 8 seconds!

You: The TLS test is failing, can you debug it?
Agent: Let me run the TLS test and check the logs...
       [runs test, queries CloudFormation for endpoints, checks CloudWatch]
       The certificate for api.ugcbba.click expired. Here's what I found...

Strict Guardrails

The agent enforces strict security boundaries:

The Test Suite

The agent wraps a comprehensive pytest-based security test suite running inside a Kali Linux container:

The agent is smart about which tests to run — it prioritizes recently failed tests, skips slow scans unless explicitly requested, and caches successful results for 5 minutes to avoid redundant runs. Parallel execution via pytest-xdist makes the full fast suite complete in about 8 seconds.

Automated Penetration Testing Framework

Beyond the AI agent, the project includes a standalone automated penetration testing framework using Dockerized Kali Linux. This is the foundation that the agent builds on, but it can also be used independently in CI/CD pipelines.

# Build the Kali container
docker build -f testing/Dockerfile.kali -t pentest:latest testing/

# Quick smoke test
docker run --rm -e TARGET_URL=http://your-alb.amazonaws.com pentest smoke

# Full test suite
docker run --rm -e TARGET_URL=http://your-alb.amazonaws.com pentest all

# Generate report
docker run --rm -v $(pwd):/tests -e TARGET_URL=$TARGET_URL pentest report

The framework integrates with GitHub Actions for scheduled weekly scans and on-demand testing, with results uploaded as workflow artifacts.

Threat Detection and Incident Response

The platform implements automated threat detection and response:

GuardDuty monitors for threats. When a finding with severity >= 4 is detected, an EventBridge rule triggers a Lambda function that automatically blocks the offending IP in the WAF IP set and sends a detailed alert to Slack. Security Hub aggregates findings from GuardDuty, CloudTrail, and AWS Config rules for a unified compliance view.

Security Observability

CloudWatch dashboards and alarms provide real-time visibility:

The Developer Experience

A good DevSecOps project is not just about security and architecture — it is also about making the developer’s life easier. The project uses Devbox for isolated development environments and Just as a task runner.

The justfile provides a comprehensive set of commands:

# Development
just dev          # Start local services and tail logs
just up           # Start PostgreSQL + LocalStack
just test         # Run all tests
just lint-local   # Lint all services locally

# Security
just security-scan    # SAST + SCA + Secrets scan
just trivy-scan       # Container vulnerability scan
just sbom-scan        # Generate and scan SBOMs
just iac-scan         # Infrastructure scan with Checkov
just pentest-full     # Run the full penetration test suite

# AWS Resource Management (Cost Saving)
just aws-up       # Scale up ECS services + start RDS
just aws-down     # Scale down to zero (save money)
just aws-status   # Check resource status

The aws-up and aws-down commands are particularly useful — they allow scaling the entire infrastructure to zero when not in use, saving significant costs during development.

Project Structure

The codebase is organized for clarity and separation of concerns:

.
├── api/                    # API service (FastAPI + hexagonal)
│   ├── src/
│   │   ├── domain/         # Business entities, value objects
│   │   ├── application/    # Use cases, ports, DTOs
│   │   ├── infrastructure/ # Adapters (DB, Kinesis, Secrets)
│   │   └── presentation/   # HTTP routes, middleware
│   └── tests/
│
├── worker/                 # Worker service (Kinesis consumer)
│   ├── src/
│   │   ├── domain/         # Ports for channels, publishers
│   │   ├── application/    # Delivery service
│   │   ├── infrastructure/ # Publisher adapters (Direct, AI Agent)
│   │   └── channels/       # Channel gateways (FB, IG, LI, Email, SMS)
│   └── tests/
│
├── scheduler/              # Scheduler service (cron + ECS)
├── api-lambda/             # API Lambda handler (serverless)
├── worker-lambda/          # Worker Lambda handler (serverless)
├── scheduler-lambda/       # Scheduler Lambda handler (serverless)
├── web/                    # Frontend (React + Vite + TypeScript)
│
├── testing/                # Penetration testing framework
│   ├── pentest_agent.py    # AI security testing agent (Strands SDK)
│   ├── test_pentest.py     # Security test cases (pytest)
│   ├── Dockerfile.kali     # Kali Linux container with tools
│   └── justfile            # Pentest task runner
│
├── infra/                  # CDK infrastructure (containers)
│   └── stacks/             # Network, Security, Auth, Data, Compute, Edge, Monitoring
│
├── infra-fs/               # CDK infrastructure (serverless)
│   └── stacks/             # Data, Auth, API, Worker, Scheduler, Security, Frontend
│
├── docs/                   # Comprehensive documentation
├── .github/workflows/      # CI/CD (deploy + security scan)
├── devbox.json             # Development environment
├── docker-compose.yml      # Local services
└── justfile                # Task runner

Lessons Learned

Building this project reinforced several convictions:

Hexagonal architecture is not academic overhead. It is the single decision that enabled dual-mode deployment, clean testing, and a codebase that remains navigable as it grows. The upfront investment in defining ports and adapters pays for itself many times over.

Security scanning must be layered. No single tool catches everything. The combination of SAST, SCA, secrets scanning, IaC scanning, and DAST provides genuine defense in depth. The key is making these scans fast and automatic — if they slow down the developer, they will be bypassed.

Correlation IDs are non-negotiable. The ability to trace a single request from CloudFront through the API, into Kinesis, through the Worker, and into the database is invaluable for debugging, incident response, and security forensics. Implement them from day one.

Cost management is a feature. The aws-up / aws-down pattern for scaling resources to zero when not in use is simple but effective. For a project like this, it is the difference between a $200/month bill and a $5/month bill during development.

AI agents need guardrails. The Strands Agents integration is powerful, but without content filtering, prompt injection detection, and output validation, it is a liability. The agent can only execute pre-approved operations through well-defined tool interfaces — never raw API calls. The same principle applies to the Security Testing Agent: it operates within a strict allowlist of tests and read-only AWS access, proving that AI-powered automation and security boundaries can coexist.

Conclusion

This project is a living laboratory. It is not finished, and it probably never will be — because the landscape of threats, tools, and best practices is always evolving. But it serves its purpose: a concrete, open-source reference for how to build modern cloud-native applications with security woven into every layer.

If you are starting a new project and wondering where to begin with DevSecOps, my advice is simple: start with the architecture. Get hexagonal architecture right, define your ports and adapters, and the rest — security, testing, deployment flexibility — becomes dramatically easier.

The code is open source. Fork it, break it, improve it. That is what it is there for.

Links

Random Recommended Posts

Devbox: Portable and Isolated Development Environments

The dependency manager I like the most for development environments.

2025-01-16

The Future of Work

What does it mean to be a traditional software engineer in the AI era.

2025-11-09