Multi-Tier DevOps Notes Application

I built a production-ready, 3-tier containerized web application built to demonstrate modern cloud architecture, secure networking, persistent storage caching, and an automated continuous integration/continuous deployment (CI/CD) pipeline targeting an isolated Azure Virtual Machine.

---

System Architecture

The application is structured as a tightly decoupled microservices ecosystem running entirely within a custom internal Docker bridge network (notesnet). The only exposed doorways to the public internet are ports 8080 and 8443.

                  [ Public Internet ]
                           │
                    Port 8080 / 8443 (TLS)
                           ▼
                 ┌───────────────────┐
                 │    Nginx Proxy    │
                 └─────────┬─────────┘
                           │ (Internal Routing via notesnet)
            ┌──────────────┴──────────────┐
            ▼                             ▼
┌───────────────────────┐     ┌───────────────────────┐
│   frontend (React)    │     │     api (FastAPI)     │
│   Served via Nginx    │     │   Uvicorn Production  │
└───────────────────────┘     └───────────┬───────────┘
                                          │
                            ┌─────────────┴─────────────┐
                            ▼                           ▼
                ┌───────────────────────┐   ┌───────────────────────┐
                │   postgres (v16)      │   │     redis (v7)        │
                │ Persistent pg_data    │   │ Cache (allkeys-lru)   │
                └───────────────────────┘   └───────────────────────┘

Component Breakdown

• Frontend Tier: A React single-page application built with Vite. It features a responsive UI for note management and communicates with the backend via a single unified origin path (/api), completely avoiding Cross-Origin Resource Sharing (CORS) vulnerabilities.
• Application Tier: A high-performance FastAPI (Python 3.12) backend using asynchronous connection pooling (asyncpg) to maximize database throughput and native lifespan handlers to warm up resource connections on startup.
• Database Tier: PostgreSQL 16 serving as the primary relational system with automatic schema initiation, indexing on time-series fields, and state persistence handled via a managed Docker named volume (pg_data).
• Caching Tier: Redis 7 managing an ultra-fast cache layer for reading payloads (notes:all). Configured with a 60s TTL and eviction policies (allkeys-lru) to guarantee memory stability on host workloads.
• Reverse Proxy Tier: A standalone Nginx container acting as the secure gateway, handling TLS termination, structural logging, gzip compression compression, security header injects (X-Frame-Options), and context path routing (stripping /api/ down to upstream targets).

---

The DevOps Journey: Core Engineering & Challenges

Transitioning this application from local code into a resilient, automated production environment presented several critical DevOps challenges, which were solved using standard industry best practices.

1. Ultra-Lean Multi-Stage Builds (Docker Optimization)

• The Challenge: Standard Node.js images with node_modules and full source tools create heavy, insecure containers exceeding 1.2 GB.
• The Solution: Implemented a multi-stage Dockerfile for the React frontend. Stage 1 utilizes a Node base to compile static files via Vite, and Stage 2 discards the entire development environment, copying only the minified production assets into a tiny Alpine Nginx container. The final image size dropped from 1.2 GB to ~25 MB.
• Layer Cache Optimization: In both Python and Node Dockerfiles, dependency manifests (package.json, requirements.txt) are copied before the main source code. Docker caches these installation layers, cutting subsequent compilation times from minutes to under 3 seconds.

2. Network Isolation & Security

• The Challenge: Exposing databases, cache systems, or raw API servers directly to public network interfaces invites security vulnerabilities.
• The Solution: Explicitly removed ports: blocks from the application and database containers in production. They run on an isolated private bridge network (notesnet). Nginx is the only service mapped to the host's ports, forcing all external interactions to go through a single, auditable reverse proxy layer.

3. Pipeline Efficiency & Zero-Downtime Operations

• The Challenge: Running an automated deployment can result in service blips, configuration errors, or redundant work (e.g., re-running slow Docker builds for simple syntax fixes).
• The Solution:
• Fail Fast: Separated testing from image creation. Dependencies are cached and run natively on the GitHub Actions runner host first (pytest and ruff). If a bug exists, the pipeline fails instantly without spinning up heavy cloud container tasks.
• Sequential Rolling Updates: Instead of running a destructive docker compose down && docker compose up, the pipeline deploys sequentially using --no-deps flags. The backend API container spins up first and is given time to clear its health status before the frontend asset server is updated.
• Graceful Nginx reloads: Instead of restarting the Nginx engine (which drops user sockets), the pipeline sends a live signal handler (nginx -s reload). Old workers complete their serving duties gracefully while new worker processes seamlessly take over traffic using the fresh image deployments.

---

Automated CI/CD Pipeline

The entire system is governed by a secure GitHub Actions workflow (.github/workflows/deploy.yml) structured into three distinct, linear phases:

Phase 1: Test & Quality Gate

• Triggers on both direct pushes to main and incoming pull_request commits.
• Provisions an ephemeral ubuntu-latest VM.
• Installs Python and Node runtimes with automatic package layer caching.
• Executes full backend unit test configurations via pytest and runs style/syntax static validation with ruff.

Phase 2: Secure Build & Push

• Executes only after Phase 1 returns a completely clean success status.
• Uses explicit security blocks (permissions: packages: write) to harness GitHub's built-in short-lived workflow identity (secrets.GITHUB_TOKEN). This eliminates the need to manage manually rotated Personal Access Tokens (PATs).
• Instruments advanced Docker Buildx utilities linked to GitHub’s native Actions cache platform (type=gha), creating highly efficient, distributed image layer reuse.
• Extracts an explicit 7-character Git commit hash (git rev-parse --short HEAD) to tag immutable production images alongside a rolling :latest pointer, ensuring full build auditability.

Phase 3: Automated Azure Deployment

• Establishes an automated security bridge to the remote Azure Virtual Machine using target SSH keys (appleboy/ssh-action and scp-action).
• Transfers runtime files (docker-compose.yml, local configs, and runtime variables) directly into the host deployment directory.
• Logs into the container registry from the cloud VM shell using short-lived tokens to pull the newly built image hashes.
• Performs an incremental roll out of updated microservice containers while verifying live runtime indicators.
• Runs a final integration smoke-test using local health endpoints (/api/health). If the container fails to start properly, the pipeline halts, streams the last 50 error log lines directly back to the GitHub panel interface, and marks the workflow as a failure to protect uptime.