<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Power of Eloquence</title>
  
  <subtitle>Mastering the Art of Technical Craftsmanship</subtitle>
  <link href="http://awongcm.io/atom.xml" rel="self"/>
  
  <link href="http://awongcm.io/"/>
  <updated>2026-03-30T12:13:51.448Z</updated>
  <id>http://awongcm.io/</id>
  
  <author>
    <name>Andy Wong</name>
    
  </author>
  
  <generator uri="https://hexo.io/">Hexo</generator>
  
  <entry>
    <title>Maximizing GitHub Copilot CLI: A Senior Engineer&#39;s Guide to AI-Powered Development</title>
    <link href="http://awongcm.io/blog/2026/02/15/maximizing-github-copilot-cli-a-senior-engineer-s-guide-to-ai-powered-development/"/>
    <id>http://awongcm.io/blog/2026/02/15/maximizing-github-copilot-cli-a-senior-engineer-s-guide-to-ai-powered-development/</id>
    <published>2026-02-15T04:25:00.000Z</published>
    <updated>2026-03-30T12:13:51.448Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_maximising_copilot_cli_senior_engineer_guide.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>As software engineers, we’re constantly seeking ways to streamline our workflows and boost productivity. GitHub Copilot CLI has emerged as a powerful tool that brings AI assistance directly into your terminal, transforming how we interact with code across different stacks. This comprehensive guide will walk you through setting up an effective AI-powered development workflow with GitHub Copilot CLI, whether you’re building front-end applications, back-end services, data pipelines, or cloud infrastructure.</p><blockquote><p><strong>Note</strong>: GitHub Copilot CLI is currently in <strong>public preview</strong> and features are subject to change. Always refer to the <a href="https://docs.github.com/en/copilot/how-tos/copilot-cli">official GitHub documentation</a> for the most up-to-date information.</p></blockquote><h2 id="What-Makes-Copilot-CLI-Different"><a href="#What-Makes-Copilot-CLI-Different" class="headerlink" title="What Makes Copilot CLI Different?"></a>What Makes Copilot CLI Different?</h2><p>Unlike traditional code assistants, Copilot CLI is <strong>terminal-native</strong> and <strong>agentic</strong>—it doesn’t just answer questions, it can act as your coding partner. You can delegate tasks, and Copilot will autonomously execute them while you maintain oversight through explicit approval mechanisms.</p><span id="more"></span><p><strong>Latest Update (February 2026):</strong> GitHub Copilot CLI now supports advanced AI models including Claude Opus 4.6, released on February 5, 2026. Claude Opus 4.6 features enhanced agentic capabilities, agent teams (multiple agents working in parallel), a 1M token context window (in beta), and record-breaking performance on coding benchmarks including 65.4% on Terminal-Bench 2.0.</p><h2 id="Part-1-Setting-Up-Your-AI-Development-Environment"><a href="#Part-1-Setting-Up-Your-AI-Development-Environment" class="headerlink" title="Part 1: Setting Up Your AI Development Environment"></a>Part 1: Setting Up Your AI Development Environment</h2><h3 id="1-1-Installation-and-Prerequisites"><a href="#1-1-Installation-and-Prerequisites" class="headerlink" title="1.1 Installation and Prerequisites"></a>1.1 Installation and Prerequisites</h3><p><strong>Requirements:</strong></p><ul><li>Node.js 22 or later</li><li>A GitHub account with Copilot access (Pro, Pro+, Business, or Enterprise)</li><li>Your organization must enable CLI preview features (for Business&#x2F;Enterprise users)</li></ul><p><strong>Installation:</strong></p><pre><code class="hljs bash">npm install -g @github/copilotcopilot</code></pre><p>On first launch, you’ll be prompted to authenticate:</p><pre><code class="hljs bash">/login</code></pre><p>Follow the on-screen instructions to authenticate with your GitHub account.</p><h3 id="1-2-Custom-Instructions-The-Foundation-of-Productivity"><a href="#1-2-Custom-Instructions-The-Foundation-of-Productivity" class="headerlink" title="1.2 Custom Instructions: The Foundation of Productivity"></a>1.2 Custom Instructions: The Foundation of Productivity</h3><p>Custom instructions teach Copilot about your team’s conventions, build processes, and coding standards. According to the official documentation, all custom instruction files now <strong>combine</strong> instead of using priority-based fallbacks.</p><p><strong>Instruction File Locations:</strong></p><table><thead><tr><th>Location</th><th>Scope</th><th>Use Case</th></tr></thead><tbody><tr><td><code>~/.copilot/copilot-instructions.md</code></td><td>Global (all sessions)</td><td>Personal preferences, common aliases</td></tr><tr><td><code>.github/copilot-instructions.md</code></td><td>Repository-wide</td><td>Team standards, build commands</td></tr></tbody></table><blockquote><p><strong>Important</strong>: The exact hierarchy and all supported file locations may vary. Check the <a href="https://docs.github.com/en/copilot/how-tos/copilot-cli">official documentation</a> for the current list of supported custom instruction files.</p></blockquote><h4 id="Example-Full-Stack-Application-Instructions"><a href="#Example-Full-Stack-Application-Instructions" class="headerlink" title="Example: Full-Stack Application Instructions"></a>Example: Full-Stack Application Instructions</h4><p>Here’s a production-ready <code>.github/copilot-instructions.md</code> for a TypeScript full-stack project:</p><pre><code class="hljs markdown"><span class="hljs-section">## Build &amp; Development Commands</span><span class="hljs-bullet">-</span> <span class="hljs-code">`npm run dev`</span> - Start development server (frontend + backend)<span class="hljs-bullet">-</span> <span class="hljs-code">`npm run build`</span> - Production build<span class="hljs-bullet">-</span> <span class="hljs-code">`npm run test`</span> - Run all tests with coverage<span class="hljs-bullet">-</span> <span class="hljs-code">`npm run test:watch`</span> - Run tests in watch mode<span class="hljs-bullet">-</span> <span class="hljs-code">`npm run lint:fix`</span> - Auto-fix linting issues<span class="hljs-bullet">-</span> <span class="hljs-code">`npm run type-check`</span> - TypeScript type checking<span class="hljs-section">## Code Style &amp; Architecture</span><span class="hljs-bullet">-</span> Use TypeScript strict mode across all files<span class="hljs-bullet">-</span> Frontend: Prefer functional React components with hooks<span class="hljs-bullet">-</span> Backend: Use dependency injection pattern for services<span class="hljs-bullet">-</span> Always add JSDoc comments for public APIs<span class="hljs-bullet">-</span> Follow Repository Pattern for data access<span class="hljs-bullet">-</span> Use async/await over promises chains<span class="hljs-section">## Testing Requirements</span><span class="hljs-bullet">-</span> Unit test coverage must exceed 80%<span class="hljs-bullet">-</span> Integration tests for all API endpoints<span class="hljs-bullet">-</span> Use Jest for unit tests, Playwright for E2E<span class="hljs-bullet">-</span> Mock external services in tests<span class="hljs-section">## Git Workflow</span><span class="hljs-bullet">-</span> Branch naming: <span class="hljs-code">`feature/`</span>, <span class="hljs-code">`bugfix/`</span>, <span class="hljs-code">`hotfix/`</span><span class="hljs-bullet">-</span> Commit format: Conventional Commits (feat, fix, docs, etc.)<span class="hljs-bullet">-</span> Run <span class="hljs-code">`npm run lint:fix &amp;&amp; npm test`</span> before commits<span class="hljs-bullet">-</span> Create feature branches from <span class="hljs-code">`develop`</span><span class="hljs-section">## Stack-Specific Guidelines</span><span class="hljs-section">### Frontend (React + TypeScript)</span><span class="hljs-bullet">-</span> Use CSS Modules or styled-components<span class="hljs-bullet">-</span> Implement proper error boundaries<span class="hljs-bullet">-</span> Lazy load routes with React.lazy()<span class="hljs-bullet">-</span> Follow accessibility standards (WCAG 2.1)<span class="hljs-section">### Backend (Node.js + Express)</span><span class="hljs-bullet">-</span> All routes must have proper error handling<span class="hljs-bullet">-</span> Use middleware for cross-cutting concerns<span class="hljs-bullet">-</span> Validate all inputs with Zod schemas<span class="hljs-bullet">-</span> Implement rate limiting on public endpoints<span class="hljs-section">### Database (PostgreSQL)</span><span class="hljs-bullet">-</span> Use migrations for all schema changes<span class="hljs-bullet">-</span> Never use raw SQL in application code<span class="hljs-bullet">-</span> Implement proper indexing strategies<span class="hljs-bullet">-</span> Use transactions for multi-step operations</code></pre><h4 id="Stack-Specific-Instruction-Templates"><a href="#Stack-Specific-Instruction-Templates" class="headerlink" title="Stack-Specific Instruction Templates"></a>Stack-Specific Instruction Templates</h4><p><strong>Data Stack (Python + Spark + Airflow):</strong></p><pre><code class="hljs markdown"><span class="hljs-section">## Data Engineering Standards</span><span class="hljs-bullet">-</span> Python 3.11+ with type hints mandatory<span class="hljs-bullet">-</span> All DAGs must include data quality checks<span class="hljs-bullet">-</span> Use PySpark for distributed processing<span class="hljs-bullet">-</span> Schema evolution handled via Delta Lake<span class="hljs-section">## Workflow</span><span class="hljs-bullet">-</span> <span class="hljs-code">`poetry run pytest`</span> - Run data pipeline tests<span class="hljs-bullet">-</span> <span class="hljs-code">`poetry run black . &amp;&amp; poetry run ruff check --fix`</span> - Format &amp; lint<span class="hljs-bullet">-</span> All transformations must be idempotent<span class="hljs-bullet">-</span> Implement proper data lineage tracking</code></pre><p><strong>Cloud Infrastructure (Terraform + AWS):</strong></p><pre><code class="hljs markdown"><span class="hljs-section">## Infrastructure as Code Standards</span><span class="hljs-bullet">-</span> Use Terraform 1.6+ with proper state locking<span class="hljs-bullet">-</span> All resources must have proper tags<span class="hljs-bullet">-</span> Implement least privilege IAM policies<span class="hljs-bullet">-</span> Use AWS CDK for complex constructs<span class="hljs-section">## Commands</span><span class="hljs-bullet">-</span> <span class="hljs-code">`terraform fmt &amp;&amp; terraform validate`</span> - Format &amp; validate<span class="hljs-bullet">-</span> <span class="hljs-code">`terraform plan -out=plan.tfplan`</span> - Generate plan<span class="hljs-bullet">-</span> Always review plans before apply<span class="hljs-bullet">-</span> Use workspaces for environment separation</code></pre><h3 id="1-3-Tool-Permissions-Security-Meets-Productivity"><a href="#1-3-Tool-Permissions-Security-Meets-Productivity" class="headerlink" title="1.3 Tool Permissions: Security Meets Productivity"></a>1.3 Tool Permissions: Security Meets Productivity</h3><p>Copilot CLI requires explicit permission for potentially destructive operations. During a session, Copilot will request permission when it wants to use a tool.</p><p><strong>Permission Options:</strong></p><p>When Copilot wants to use a tool (like <code>chmod</code>, <code>git</code>, or <code>npm</code>), you’ll be prompted with:</p><ol><li><strong>Allow</strong> - Allow this one time</li><li><strong>Yes, and approve TOOL for the rest of the session</strong> - Auto-approve this tool for the current session</li><li><strong>No, and tell Copilot what to do differently (Esc)</strong> - Reject and provide alternative guidance</li></ol><p><strong>Command-line Permission Flags:</strong></p><pre><code class="hljs bash"><span class="hljs-comment"># Allow all tools (use with caution)</span>copilot --allow-all-tools<span class="hljs-comment"># Allow all paths</span>copilot --allow-all-paths</code></pre><blockquote><p><strong>Security Note</strong>: Using <code>--allow-all-tools</code> gives Copilot the same access as you have to files and commands. Only use in trusted environments.</p></blockquote><h3 id="1-4-Model-Selection-Choosing-the-Right-AI-for-the-Job"><a href="#1-4-Model-Selection-Choosing-the-Right-AI-for-the-Job" class="headerlink" title="1.4 Model Selection: Choosing the Right AI for the Job"></a>1.4 Model Selection: Choosing the Right AI for the Job</h3><p>GitHub Copilot CLI’s default model is <strong>Claude Sonnet 4.5</strong>. You can switch models using the <code>/model</code> command within an interactive session.</p><p><strong>Available Models</strong> (as of February 2026):</p><p>Use <code>/model</code> within a CLI session to see the complete list of available models. According to GitHub’s official repository, available models include:</p><ul><li>Claude Sonnet 4.5 (default)</li><li>Claude Sonnet 4</li><li>Claude Opus 4.6 (latest, added in CLI version 0.0.406 on Feb 7, 2026)</li><li>GPT-5</li></ul><blockquote><p><strong>Note</strong>: The exact list of available models may vary based on your subscription and region. Use the <code>/model</code> command to see what’s available in your environment.</p></blockquote><p><strong>What’s New in Claude Opus 4.6:</strong></p><p>Claude Opus 4.6, released by Anthropic on February 5, 2026, represents a significant advancement:</p><ul><li><strong>Agent Teams</strong>: Multiple AI agents can work in parallel on different components of complex tasks</li><li><strong>1M Token Context Window</strong> (in beta): Work with massive codebases—roughly 750,000 words or about 3,000 pages of code</li><li><strong>Enhanced Agentic Capabilities</strong>: 65.4% on Terminal-Bench 2.0, highest score for agentic coding systems</li><li><strong>Improved Code Review</strong>: Can detect and correct its own mistakes during implementation</li><li><strong>128K Output Tokens</strong>: Doubled from the previous 64K limit</li><li><strong>Adaptive Thinking</strong>: Dynamically adjusts reasoning depth based on task complexity</li></ul><p><strong>Switching Models:</strong></p><pre><code class="hljs bash"><span class="hljs-comment"># Within CLI session</span>/model<span class="hljs-comment"># Follow the prompts to select your desired model</span></code></pre><p><strong>Strategic Considerations:</strong></p><ul><li>Use the <strong>default model (Sonnet 4.5)</strong> for most day-to-day coding tasks</li><li>Switch to <strong>Opus 4.6</strong> for complex architectural decisions, difficult debugging, or agentic tasks</li><li>Different models may consume different amounts of your monthly premium request quota</li></ul><h2 id="Part-2-Mastering-Plan-Driven-Development"><a href="#Part-2-Mastering-Plan-Driven-Development" class="headerlink" title="Part 2: Mastering Plan-Driven Development"></a>Part 2: Mastering Plan-Driven Development</h2><h3 id="2-1-Why-Planning-Matters"><a href="#2-1-Why-Planning-Matters" class="headerlink" title="2.1 Why Planning Matters"></a>2.1 Why Planning Matters</h3><p>Plan mode transforms vague requests into concrete, reviewable implementation roadmaps. It helps catch misunderstandings before any code is written.</p><p><strong>Activation:</strong></p><ul><li>Press <code>Shift + Tab</code> to cycle between modes (interactive → plan → autopilot)</li><li>Or use <code>/plan</code> command in interactive mode</li></ul><p><strong>The Planning Workflow:</strong></p><ol><li><strong>You provide the goal</strong></li><li><strong>Copilot asks clarifying questions</strong> (architecture decisions, edge cases, constraints)</li><li><strong>Copilot generates a structured plan</strong> with checkboxes and detailed steps</li><li><strong>You review and modify</strong> the plan</li><li><strong>Copilot implements</strong> the approved plan</li></ol><h3 id="2-2-Real-World-Planning-Examples"><a href="#2-2-Real-World-Planning-Examples" class="headerlink" title="2.2 Real-World Planning Examples"></a>2.2 Real-World Planning Examples</h3><h4 id="Example-1-Adding-OAuth2-Authentication"><a href="#Example-1-Adding-OAuth2-Authentication" class="headerlink" title="Example 1: Adding OAuth2 Authentication"></a>Example 1: Adding OAuth2 Authentication</h4><p><strong>Prompt:</strong></p><pre><code class="hljs plaintext">/plan Add OAuth2 authentication with Google and GitHub providers</code></pre><p><strong>Copilot’s Clarifying Questions:</strong></p><ul><li>Should users be able to link multiple OAuth providers to one account?</li><li>Do you want to support traditional email&#x2F;password alongside OAuth?</li><li>What’s your preference for session management - JWT or server-side sessions?</li><li>Any specific libraries you’re already using for auth?</li></ul><p><strong>Generated Plan:</strong></p><pre><code class="hljs markdown"><span class="hljs-section"># Implementation Plan: OAuth2 Authentication</span><span class="hljs-section">## Overview</span>Implement social authentication using OAuth2 with Google and GitHub providers,supporting multiple provider linking and JWT-based session management.<span class="hljs-section">## Dependencies</span><span class="hljs-bullet">-</span> [ ] Install passport (^0.7.0)<span class="hljs-bullet">-</span> [ ] Install passport-google-oauth20 (^2.0.0)<span class="hljs-bullet">-</span> [ ] Install passport-github2 (^0.1.12)<span class="hljs-bullet">-</span> [ ] Install jsonwebtoken (^9.0.0)<span class="hljs-section">## Infrastructure Changes</span><span class="hljs-bullet">-</span> [ ] Add OAuth credentials to environment variables<span class="hljs-bullet">  -</span> GOOGLE<span class="hljs-emphasis">_CLIENT_</span>ID, GOOGLE<span class="hljs-emphasis">_CLIENT_</span>SECRET<span class="hljs-bullet">  -</span> GITHUB<span class="hljs-emphasis">_CLIENT_</span>ID, GITHUB<span class="hljs-emphasis">_CLIENT_</span>SECRET<span class="hljs-bullet">  -</span> JWT<span class="hljs-emphasis">_SECRET</span><span class="hljs-emphasis"></span><span class="hljs-emphasis">## Database Migrations</span><span class="hljs-emphasis">- [ ] Create `oauth_</span>providers` table<span class="hljs-bullet">  -</span> user<span class="hljs-emphasis">_id (FK to users)</span><span class="hljs-emphasis">  - provider (enum: google, github)</span><span class="hljs-emphasis">  - provider_</span>user<span class="hljs-emphasis">_id</span><span class="hljs-emphasis">  - access_</span>token (encrypted)<span class="hljs-bullet">  -</span> refresh<span class="hljs-emphasis">_token (encrypted)</span><span class="hljs-emphasis">  - unique constraint on (user_</span>id, provider)<span class="hljs-section">## Backend Implementation</span><span class="hljs-bullet">-</span> [ ] Create <span class="hljs-code">`/api/auth`</span> routes module<span class="hljs-bullet">  -</span> POST /auth/google (initiate Google OAuth)<span class="hljs-bullet">  -</span> GET /auth/google/callback<span class="hljs-bullet">  -</span> POST /auth/github (initiate GitHub OAuth)<span class="hljs-bullet">  -</span> GET /auth/github/callback<span class="hljs-bullet">  -</span> POST /auth/link/:provider (link additional provider)  <span class="hljs-bullet">-</span> [ ] Implement Passport strategies<span class="hljs-bullet">  -</span> GoogleStrategy with profile data mapping<span class="hljs-bullet">  -</span> GitHubStrategy with profile data mapping<span class="hljs-bullet">  -</span> Error handling for provider failures  <span class="hljs-bullet">-</span> [ ] Create authentication service<span class="hljs-bullet">  -</span> findOrCreateUserFromOAuth()<span class="hljs-bullet">  -</span> linkProviderToUser()<span class="hljs-bullet">  -</span> generateJWTToken()<span class="hljs-bullet">  -</span> Token refresh logic<span class="hljs-section">## Frontend Implementation</span><span class="hljs-bullet">-</span> [ ] Create OAuth login buttons component<span class="hljs-bullet">-</span> [ ] Add provider linking UI in settings<span class="hljs-bullet">-</span> [ ] Implement OAuth redirect handling<span class="hljs-bullet">-</span> [ ] Add loading states during auth flow<span class="hljs-bullet">-</span> [ ] Error messaging for auth failures<span class="hljs-section">## Security Considerations</span><span class="hljs-bullet">-</span> [ ] Implement CSRF protection for OAuth state parameter<span class="hljs-bullet">-</span> [ ] Encrypt stored OAuth tokens<span class="hljs-bullet">-</span> [ ] Add rate limiting to auth endpoints<span class="hljs-bullet">-</span> [ ] Implement token rotation policy<span class="hljs-section">## Testing</span><span class="hljs-bullet">-</span> [ ] Unit tests for strategies and auth service<span class="hljs-bullet">-</span> [ ] Integration tests for OAuth flows<span class="hljs-bullet">-</span> [ ] E2E tests for complete login process<span class="hljs-bullet">-</span> [ ] Test account linking scenarios<span class="hljs-section">## Documentation</span><span class="hljs-bullet">-</span> [ ] Update API documentation<span class="hljs-bullet">-</span> [ ] Add OAuth setup guide for developers<span class="hljs-bullet">-</span> [ ] Document environment variable requirements</code></pre><p><strong>Implementation:</strong></p><pre><code class="hljs plaintext">Proceed with the plan. Start with database migrations.</code></pre><h4 id="Example-2-Refactoring-for-Microservices"><a href="#Example-2-Refactoring-for-Microservices" class="headerlink" title="Example 2: Refactoring for Microservices"></a>Example 2: Refactoring for Microservices</h4><p><strong>Prompt:</strong></p><pre><code class="hljs plaintext">/plan Refactor the monolithic user service into separate microservices for authentication, profile management, and notifications</code></pre><p>This complex architectural change benefits from planning because Copilot will:</p><ul><li>Identify all coupling points</li><li>Propose API contracts between services</li><li>Plan database separation strategy</li><li>Address data consistency concerns</li><li>Suggest deployment sequencing</li></ul><h3 id="2-3-When-to-Use-Plan-Mode"><a href="#2-3-When-to-Use-Plan-Mode" class="headerlink" title="2.3 When to Use Plan Mode"></a>2.3 When to Use Plan Mode</h3><table><thead><tr><th>Scenario</th><th>Use Plan Mode?</th><th>Reasoning</th></tr></thead><tbody><tr><td>Adding a major feature</td><td>✅ Yes</td><td>Complex, multi-file, needs architecture decisions</td></tr><tr><td>Refactoring core business logic</td><td>✅ Yes</td><td>High risk, requires careful sequencing</td></tr><tr><td>Fixing a typo</td><td>❌ No</td><td>Simple, single-file change</td></tr><tr><td>Quick bug fix</td><td>❌ No</td><td>Usually isolated, fast iteration better</td></tr><tr><td>Database schema migration</td><td>✅ Yes</td><td>Needs careful planning of rollback strategy</td></tr><tr><td>Adding a new API endpoint</td><td>⚠️ Depends</td><td>Simple CRUD? No. Complex with validation? Yes</td></tr></tbody></table><h3 id="2-4-Recommended-Workflow-Explore-→-Plan-→-Review-→-Implement-→-Verify"><a href="#2-4-Recommended-Workflow-Explore-→-Plan-→-Review-→-Implement-→-Verify" class="headerlink" title="2.4 Recommended Workflow: Explore → Plan → Review → Implement → Verify"></a>2.4 Recommended Workflow: Explore → Plan → Review → Implement → Verify</h3><p>This five-phase workflow maximizes success rates:</p><p><strong>Phase 1: Explore</strong></p><pre><code class="hljs plaintext">Read the authentication system but don&#x27;t write code yet. Explain the current architecture and identify potential bottlenecks.</code></pre><p><strong>Phase 2: Plan</strong></p><pre><code class="hljs plaintext">/plan Implement password reset flow with email verification and rate limiting</code></pre><p><strong>Phase 3: Review &amp; Iterate</strong><br>Review the generated plan. Ask for modifications:</p><pre><code class="hljs plaintext">Add a step to implement magic link authentication as an alternative to password reset</code></pre><p><strong>Phase 4: Implement</strong></p><pre><code class="hljs plaintext">Proceed with the updated plan. Implement the password reset first.</code></pre><p><strong>Phase 5: Verify</strong></p><pre><code class="hljs plaintext">Run all authentication tests and fix any failures. Then run the linter.</code></pre><h2 id="Part-3-Advanced-Workflows-for-Different-Stacks"><a href="#Part-3-Advanced-Workflows-for-Different-Stacks" class="headerlink" title="Part 3: Advanced Workflows for Different Stacks"></a>Part 3: Advanced Workflows for Different Stacks</h2><h3 id="3-1-Front-End-Development-React-x2F-Vue-x2F-Angular"><a href="#3-1-Front-End-Development-React-x2F-Vue-x2F-Angular" class="headerlink" title="3.1 Front-End Development (React&#x2F;Vue&#x2F;Angular)"></a>3.1 Front-End Development (React&#x2F;Vue&#x2F;Angular)</h3><p><strong>Component Generation with Design Systems:</strong></p><pre><code class="hljs plaintext">Create a reusable DataTable component following our design systemRequirements:- Support sorting, filtering, and pagination- Responsive design with mobile-first approach- Accessibility compliant (WCAG 2.1 AA)- Include loading and error states- TypeScript with proper prop types</code></pre><p><strong>Visual Design Implementation:</strong></p><blockquote><p><strong>Note</strong>: Check current documentation for image upload capabilities, as features are evolving.</p></blockquote><p><strong>State Management Integration:</strong></p><pre><code class="hljs plaintext">Refactor the shopping cart to use Zustand for state managementMigrate from Redux while maintaining the same API for componentsAdd persistence to localStorageInclude tests for all state transitions</code></pre><h3 id="3-2-Back-End-Development-Node-js-x2F-Python-x2F-Go"><a href="#3-2-Back-End-Development-Node-js-x2F-Python-x2F-Go" class="headerlink" title="3.2 Back-End Development (Node.js&#x2F;Python&#x2F;Go)"></a>3.2 Back-End Development (Node.js&#x2F;Python&#x2F;Go)</h3><p><strong>API Development:</strong></p><pre><code class="hljs plaintext">/plan Create a RESTful API for product management with the following endpoints:- GET /api/products (list with pagination, filtering, sorting)- GET /api/products/:id (single product)- POST /api/products (create, admin only)- PUT /api/products/:id (update, admin only)- DELETE /api/products/:id (soft delete, admin only)Requirements:- OpenAPI/Swagger documentation- Request validation with Zod- Proper error handling with standard error codes- Rate limiting (100 req/min for authenticated users)- Include integration tests</code></pre><p><strong>Database Optimization:</strong></p><pre><code class="hljs plaintext">Analyze query patterns in the products table and propose indexing strategiesConsider:- Current query patterns in @/services/product-service.ts- Write vs read ratio (70% reads, 30% writes)- Don&#x27;t over-index, as it impacts write performance</code></pre><p><strong>Microservices Communication:</strong></p><pre><code class="hljs plaintext">Implement an event-driven communication pattern between the order service and inventory service- Use RabbitMQ for message broker- Implement retry logic with exponential backoff- Add dead letter queue for failed messages- Ensure idempotency for inventory updates- Include monitoring and alerting</code></pre><h3 id="3-3-Data-Engineering-Python-x2F-Spark-x2F-Airflow"><a href="#3-3-Data-Engineering-Python-x2F-Spark-x2F-Airflow" class="headerlink" title="3.3 Data Engineering (Python&#x2F;Spark&#x2F;Airflow)"></a>3.3 Data Engineering (Python&#x2F;Spark&#x2F;Airflow)</h3><p><strong>ETL Pipeline Development:</strong></p><pre><code class="hljs plaintext">/plan Create an Airflow DAG for customer analytics ETL pipelineSource: PostgreSQL customer databaseTransformations:- Clean and normalize customer data- Calculate customer lifetime value- Segment customers by behavior patterns- Aggregate purchase historyDestination: Snowflake data warehouseRequirements:- Incremental loading (process only new/changed records)- Data quality checks at each stage- Proper error handling and alerting- Idempotent operations- Include unit tests for transformation logic</code></pre><p><strong>Data Quality Implementation:</strong></p><pre><code class="hljs plaintext">Add comprehensive data quality checks to the customer pipelineChecks needed:- Schema validation (expected columns and types)- Null value checks for required fields- Range checks for numeric fields (age, purchase amounts)- Format validation (email, phone numbers)- Cross-field validation (end_date &gt; start_date)- Statistical anomaly detectionOn failure: quarantine bad records, alert data team, continue processing good records</code></pre><h3 id="3-4-Cloud-Infrastructure-Terraform-x2F-CloudFormation"><a href="#3-4-Cloud-Infrastructure-Terraform-x2F-CloudFormation" class="headerlink" title="3.4 Cloud Infrastructure (Terraform&#x2F;CloudFormation)"></a>3.4 Cloud Infrastructure (Terraform&#x2F;CloudFormation)</h3><p><strong>Infrastructure as Code:</strong></p><pre><code class="hljs plaintext">/plan Create Terraform configuration for a production-grade EKS cluster on AWSRequirements:- Multi-AZ deployment for high availability- Auto-scaling node groups (min 3, max 10 nodes)- Separate node groups for system and application workloads- VPC with public and private subnets- Proper security groups and NACLs- IAM roles following least privilege- Enable cluster autoscaler- Integration with AWS Load Balancer Controller- Monitoring with CloudWatch Container Insights- Secrets management with AWS Secrets Manager</code></pre><p><strong>Security Hardening:</strong></p><pre><code class="hljs plaintext">Review our Terraform configurations for security issues:- Overly permissive IAM policies- Unencrypted resources- Public exposure of sensitive services- Missing backup configurations- Inadequate logging and monitoring- Missing resource tagsGenerate a report with findings and proposed fixes</code></pre><h3 id="3-5-DevOps-amp-CI-x2F-CD"><a href="#3-5-DevOps-amp-CI-x2F-CD" class="headerlink" title="3.5 DevOps &amp; CI&#x2F;CD"></a>3.5 DevOps &amp; CI&#x2F;CD</h3><p><strong>GitHub Actions Workflow:</strong></p><pre><code class="hljs plaintext">Create a comprehensive GitHub Actions workflow for our Node.js applicationPipeline stages:1. Lint and format checking2. Unit tests with coverage (fail if &lt; 80%)3. Integration tests against test database4. Security scanning (npm audit, Snyk)5. Build Docker image6. Push to ECR7. Deploy to staging (auto)8. E2E tests on staging9. Deploy to production (manual approval)Include:- Caching for dependencies and build artifacts- Parallel job execution where possible- Slack notifications for failures- Deployment status updates to PR</code></pre><p><strong>Kubernetes Deployment:</strong></p><pre><code class="hljs plaintext">Create production-ready Kubernetes manifests for our microservicesServices: api-gateway, auth-service, product-service, notification-serviceRequirements:- Use Deployments with rolling update strategy- Configure resource requests and limits- Health checks (liveness and readiness probes)- Horizontal Pod Autoscaling based on CPU/memory- ConfigMaps for configuration- Secrets for sensitive data- Network policies for inter-service communication- Ingress with TLS termination- Include monitoring with Prometheus metrics</code></pre><h2 id="Part-4-Session-Management-and-Context"><a href="#Part-4-Session-Management-and-Context" class="headerlink" title="Part 4: Session Management and Context"></a>Part 4: Session Management and Context</h2><h3 id="4-1-Understanding-Session-Context"><a href="#4-1-Understanding-Session-Context" class="headerlink" title="4.1 Understanding Session Context"></a>4.1 Understanding Session Context</h3><p>Copilot CLI features <strong>automatic context compaction</strong> to enable long-running sessions. When your conversation approaches 95% of the token limit, Copilot automatically compresses your history in the background.</p><p><strong>How it works:</strong></p><ol><li>Older conversation history is summarized</li><li>Critical information and decisions are preserved</li><li>Recent context remains in full detail</li><li>The process happens automatically without interrupting your workflow</li></ol><h3 id="4-2-Session-Management-Commands"><a href="#4-2-Session-Management-Commands" class="headerlink" title="4.2 Session Management Commands"></a>4.2 Session Management Commands</h3><p><strong>View current session info:</strong></p><pre><code class="hljs bash">/session</code></pre><p><strong>View context usage:</strong></p><pre><code class="hljs bash">/context</code></pre><p>This shows a breakdown of how your context window is being used.</p><p><strong>Manual compaction (rarely needed):</strong></p><pre><code class="hljs bash">/compact</code></pre><p>Press Escape to cancel if you change your mind.</p><h3 id="4-3-Best-Practices-for-Sessions"><a href="#4-3-Best-Practices-for-Sessions" class="headerlink" title="4.3 Best Practices for Sessions"></a>4.3 Best Practices for Sessions</h3><p><strong>Start fresh between unrelated tasks:</strong></p><pre><code class="hljs bash"><span class="hljs-comment"># After completing authentication feature</span>/new<span class="hljs-comment"># Start fresh for payment integration</span>&gt; Implement Stripe payment integration</code></pre><p><strong>Clear context when switching focus:</strong></p><p>Think of <code>/new</code> like ending a meeting and starting a new one - context is reset, but you can always reference previous work:</p><pre><code class="hljs bash"><span class="hljs-comment"># In a new session</span>&gt; Review the authentication implementation patterns we used previously and apply similar approaches <span class="hljs-keyword">for</span> payment authorization</code></pre><h2 id="Part-5-Delegation-Capabilities"><a href="#Part-5-Delegation-Capabilities" class="headerlink" title="Part 5: Delegation Capabilities"></a>Part 5: Delegation Capabilities</h2><h3 id="5-1-The-delegate-Command"><a href="#5-1-The-delegate-Command" class="headerlink" title="5.1 The /delegate Command"></a>5.1 The <code>/delegate</code> Command</h3><blockquote><p><strong>Important</strong>: Verify the current capabilities of the <code>/delegate</code> command in the <a href="https://docs.github.com/en/copilot/how-tos/copilot-cli">official documentation</a>, as this feature may work differently than described below or may be available through different mechanisms.</p></blockquote><p>Delegation allows you to offload work to Copilot coding agent. According to GitHub’s documentation, you can delegate tasks in scenarios including:</p><ul><li>Codebase maintenance (security fixes, dependency upgrades, refactoring)</li><li>Documentation updates</li><li>Feature development</li><li>Improving test coverage</li><li>Prototyping new projects</li></ul><p><strong>Example usage:</strong></p><pre><code class="hljs bash">/delegate Add comprehensive API documentation using OpenAPI 3.0 spec</code></pre><h3 id="5-2-When-to-Delegate-vs-Work-Locally"><a href="#5-2-When-to-Delegate-vs-Work-Locally" class="headerlink" title="5.2 When to Delegate vs. Work Locally"></a>5.2 When to Delegate vs. Work Locally</h3><table><thead><tr><th>Delegate to Agent</th><th>Work Locally</th></tr></thead><tbody><tr><td>Documentation updates</td><td>Core features on the critical path</td></tr><tr><td>Test coverage improvements</td><td>Active debugging sessions</td></tr><tr><td>Dependency updates</td><td>Interactive exploration of codebase</td></tr><tr><td>Non-blocking refactoring</td><td>Performance optimization requiring testing</td></tr></tbody></table><h2 id="Part-6-Multi-Repository-Workflows"><a href="#Part-6-Multi-Repository-Workflows" class="headerlink" title="Part 6: Multi-Repository Workflows"></a>Part 6: Multi-Repository Workflows</h2><h3 id="6-1-Working-Across-Multiple-Codebases"><a href="#6-1-Working-Across-Multiple-Codebases" class="headerlink" title="6.1 Working Across Multiple Codebases"></a>6.1 Working Across Multiple Codebases</h3><p>Copilot CLI can work with multiple repositories simultaneously.</p><p><strong>Option 1: Start from Parent Directory</strong></p><pre><code class="hljs bash"><span class="hljs-built_in">cd</span> ~/projects/my-microservicescopilot</code></pre><p>Copilot now has access to all child repositories:</p><pre><code class="hljs plaintext">~/projects/my-microservices/├── api-gateway/├── auth-service/├── product-service/├── notification-service/└── shared-libraries/</code></pre><p><strong>Option 2: Add Directories Dynamically</strong></p><blockquote><p><strong>Note</strong>: The exact command syntax for adding directories may vary. Refer to the official documentation for current commands.</p></blockquote><h3 id="6-2-Real-World-Multi-Repo-Scenarios"><a href="#6-2-Real-World-Multi-Repo-Scenarios" class="headerlink" title="6.2 Real-World Multi-Repo Scenarios"></a>6.2 Real-World Multi-Repo Scenarios</h3><p><strong>Scenario 1: Coordinated API Changes</strong></p><pre><code class="hljs plaintext">I need to update our user authentication API. Changes span multiple repositories:- api-gateway (update routing)- auth-service (implement new logic)- mobile-app (update client SDK)- docs (update API documentation)Start by showing me the current authentication flow across all repos.Then create a plan for migrating to OAuth 2.1 with PKCE.</code></pre><p><strong>Scenario 2: Shared Library Refactoring</strong></p><pre><code class="hljs plaintext">The logger module in our shared libraries is being used inconsistently.Search all microservices for logger usage patterns.Propose a standardized interface.Update the logger implementation.Then update all microservices to use the new interface.</code></pre><h2 id="Part-7-Productivity-Patterns-for-Common-Tasks"><a href="#Part-7-Productivity-Patterns-for-Common-Tasks" class="headerlink" title="Part 7: Productivity Patterns for Common Tasks"></a>Part 7: Productivity Patterns for Common Tasks</h2><h3 id="7-1-Codebase-Onboarding"><a href="#7-1-Codebase-Onboarding" class="headerlink" title="7.1 Codebase Onboarding"></a>7.1 Codebase Onboarding</h3><p>When joining a new project:</p><pre><code class="hljs plaintext">I&#x27;m new to this codebase. Help me understand:1. Overall architecture and design patterns2. How authentication/authorization works3. Database schema and ORM patterns4. Build and deployment process5. Testing strategy and conventions</code></pre><h3 id="7-2-Test-Driven-Development"><a href="#7-2-Test-Driven-Development" class="headerlink" title="7.2 Test-Driven Development"></a>7.2 Test-Driven Development</h3><pre><code class="hljs plaintext">I want to implement user registration with email verification.First, write comprehensive failing tests covering:- Happy path (successful registration)- Duplicate email handling- Invalid email format- Weak password rejection- Email verification flow- Expired verification tokensUse Jest and follow our testing patterns</code></pre><p>After reviewing and approving the tests:</p><pre><code class="hljs plaintext">Now implement the minimum code to make all tests pass.</code></pre><h3 id="7-3-Code-Review-Assistance"><a href="#7-3-Code-Review-Assistance" class="headerlink" title="7.3 Code Review Assistance"></a>7.3 Code Review Assistance</h3><pre><code class="hljs plaintext">Review the changes in my current branch.Focus on:- Potential bugs and edge cases- Security vulnerabilities- Performance concerns- Code style consistency- Missing tests- Documentation gapsProvide a detailed report with specific suggestions.</code></pre><h3 id="7-4-Bug-Investigation"><a href="#7-4-Bug-Investigation" class="headerlink" title="7.4 Bug Investigation"></a>7.4 Bug Investigation</h3><pre><code class="hljs plaintext">The /api/products endpoint returns 500 errors intermittently (about 5% of requests).Investigation steps:1. Search application logs for related errors2. Check database query performance3. Look for race conditions in concurrent requests4. Review error handling in the products controller5. Check for memory leaks or resource exhaustionAnalyze the evidence and identify the root cause.</code></pre><h3 id="7-5-Performance-Optimization"><a href="#7-5-Performance-Optimization" class="headerlink" title="7.5 Performance Optimization"></a>7.5 Performance Optimization</h3><pre><code class="hljs plaintext">Analyze the performance of our product search featureCurrent metrics:- Average response time: 850ms- P95: 1.2s- P99: 2.5sTarget: Average &lt; 200ms, P95 &lt; 400msIdentify bottlenecks and propose optimizations:- Database query optimization- Caching strategies (Redis)- Index improvements- Pagination efficiency- API response payload size</code></pre><h3 id="7-6-Security-Auditing"><a href="#7-6-Security-Auditing" class="headerlink" title="7.6 Security Auditing"></a>7.6 Security Auditing</h3><pre><code class="hljs plaintext">Conduct a security audit of our authentication system:Check for:- SQL injection vulnerabilities- XSS attack vectors- CSRF protection- Weak password policies- Insecure session management- Missing rate limiting- Exposed sensitive data in logs- Insufficient input validationGenerate a security report with severity ratings and remediation steps.</code></pre><h2 id="Part-8-Team-Collaboration-and-Standards"><a href="#Part-8-Team-Collaboration-and-Standards" class="headerlink" title="Part 8: Team Collaboration and Standards"></a>Part 8: Team Collaboration and Standards</h2><h3 id="8-1-Establishing-Team-Conventions"><a href="#8-1-Establishing-Team-Conventions" class="headerlink" title="8.1 Establishing Team Conventions"></a>8.1 Establishing Team Conventions</h3><p>Create a team-wide <code>.github/copilot-instructions.md</code>:</p><pre><code class="hljs markdown"><span class="hljs-section"># Team Development Standards</span><span class="hljs-section">## When to Use Copilot CLI Features</span><span class="hljs-section">### Plan Mode (`/plan` or Shift+Tab)</span><span class="hljs-strong">**ALWAYS use for:**</span><span class="hljs-bullet">-</span> New features touching &gt; 3 files<span class="hljs-bullet">-</span> Database schema changes<span class="hljs-bullet">-</span> API contract modifications<span class="hljs-bullet">-</span> Refactoring affecting multiple modules<span class="hljs-bullet">-</span> Security-sensitive changes<span class="hljs-strong">**SKIP for:**</span><span class="hljs-bullet">-</span> Bug fixes in single file<span class="hljs-bullet">-</span> Documentation updates<span class="hljs-bullet">-</span> Code formatting<span class="hljs-bullet">-</span> Dependency updates<span class="hljs-section">## Code Review Requirements</span>All code (human or AI-generated) must:<span class="hljs-bullet">-</span> Pass CI/CD pipeline (tests, linting, security scans)<span class="hljs-bullet">-</span> Be reviewed by at least one team member<span class="hljs-bullet">-</span> Include tests with &gt;80% coverage<span class="hljs-bullet">-</span> Update relevant documentation<span class="hljs-bullet">-</span> Follow conventional commit format<span class="hljs-section">## Model Selection Guidance</span><span class="hljs-bullet">-</span> <span class="hljs-strong">**Default to Sonnet 4.5**</span> for most work<span class="hljs-bullet">-</span> <span class="hljs-strong">**Use Opus 4.6**</span> for architectural decisions, complex bugs, agentic tasks<span class="hljs-bullet">-</span> Consider premium request quota when choosing models</code></pre><h3 id="8-2-Shared-Repository-Instructions"><a href="#8-2-Shared-Repository-Instructions" class="headerlink" title="8.2 Shared Repository Instructions"></a>8.2 Shared Repository Instructions</h3><p>For a full-stack e-commerce application:</p><pre><code class="hljs markdown"><span class="hljs-section"># E-Commerce Platform Development Guide</span><span class="hljs-section">## Architecture Overview</span><span class="hljs-bullet">-</span> Frontend: Next.js 14 (App Router) + TypeScript<span class="hljs-bullet">-</span> Backend: NestJS + PostgreSQL + Redis<span class="hljs-bullet">-</span> Infrastructure: AWS ECS + RDS + ElastiCache<span class="hljs-bullet">-</span> CI/CD: GitHub Actions → ECR → ECS<span class="hljs-section">## Key Patterns</span><span class="hljs-section">### Frontend</span><span class="hljs-bullet">-</span> Server Components by default, Client Components only when needed<span class="hljs-bullet">-</span> Data fetching in Server Components, not API routes<span class="hljs-bullet">-</span> Use React Server Actions for mutations<span class="hljs-bullet">-</span> Global state: Zustand (client-side only)<span class="hljs-bullet">-</span> Forms: React Hook Form + Zod validation<span class="hljs-section">### Backend</span><span class="hljs-bullet">-</span> CQRS pattern for complex domains (orders, inventory)<span class="hljs-bullet">-</span> Repository pattern for data access<span class="hljs-bullet">-</span> Event-driven architecture for cross-service communication<span class="hljs-bullet">-</span> Queue processing with Bull for async tasks<span class="hljs-section">### Database</span><span class="hljs-bullet">-</span> Prisma ORM for type-safe queries<span class="hljs-bullet">-</span> Migrations in <span class="hljs-code">`prisma/migrations/`</span><span class="hljs-bullet">-</span> Never use raw SQL in application code<span class="hljs-bullet">-</span> Always use transactions for multi-table operations<span class="hljs-section">## Testing Strategy</span><span class="hljs-bullet">-</span> Unit: Jest (&gt;80% coverage required)<span class="hljs-bullet">-</span> Integration: Supertest for API tests<span class="hljs-bullet">-</span> E2E: Playwright for critical user journeys<span class="hljs-bullet">-</span> Load: k6 for performance testing<span class="hljs-section">## Deployment</span><span class="hljs-bullet">-</span> Main branch → Auto-deploy to staging<span class="hljs-bullet">-</span> Release branches → Manual deploy to production<span class="hljs-bullet">-</span> Rollback procedure documented in RUNBOOK.md</code></pre><h2 id="Part-9-Troubleshooting-and-Getting-Help"><a href="#Part-9-Troubleshooting-and-Getting-Help" class="headerlink" title="Part 9: Troubleshooting and Getting Help"></a>Part 9: Troubleshooting and Getting Help</h2><h3 id="9-1-Common-Issues-and-Solutions"><a href="#9-1-Common-Issues-and-Solutions" class="headerlink" title="9.1 Common Issues and Solutions"></a>9.1 Common Issues and Solutions</h3><p><strong>Issue: Copilot doesn’t see recent file changes</strong></p><p>Try the session refresh approach or restart the CLI session.</p><p><strong>Issue: Responses seem inconsistent with project conventions</strong></p><pre><code class="hljs bash"><span class="hljs-comment"># Review your custom instructions</span><span class="hljs-built_in">cat</span> .github/copilot-instructions.md<span class="hljs-comment"># Ensure they&#x27;re being loaded properly</span></code></pre><p><strong>Issue: Permission denied errors</strong></p><p>Make sure you’ve granted appropriate tool permissions when prompted, or use permission flags when starting Copilot CLI.</p><h3 id="9-2-Built-in-Help-System"><a href="#9-2-Built-in-Help-System" class="headerlink" title="9.2 Built-in Help System"></a>9.2 Built-in Help System</h3><p><strong>View general help:</strong></p><pre><code class="hljs bash">copilot -h</code></pre><p><strong>Within CLI session:</strong></p><pre><code class="hljs bash">/help</code></pre><p>This shows all available slash commands.</p><h3 id="9-3-Getting-Support"><a href="#9-3-Getting-Support" class="headerlink" title="9.3 Getting Support"></a>9.3 Getting Support</h3><p><strong>Official resources:</strong></p><ol><li><strong>Documentation</strong>: <a href="https://docs.github.com/en/copilot/how-tos/copilot-cli">GitHub Copilot CLI Docs</a></li><li><strong>GitHub Community</strong>: Community Discussions</li><li><strong>Feedback</strong>: Use <code>/feedback</code> within CLI for bug reports or feature requests</li><li><strong>Support</strong>: GitHub Support for enterprise customers</li></ol><h2 id="Part-10-Best-Practices-and-Recommendations"><a href="#Part-10-Best-Practices-and-Recommendations" class="headerlink" title="Part 10: Best Practices and Recommendations"></a>Part 10: Best Practices and Recommendations</h2><h3 id="10-1-Stay-Updated"><a href="#10-1-Stay-Updated" class="headerlink" title="10.1 Stay Updated"></a>10.1 Stay Updated</h3><p>GitHub Copilot CLI is in active development and evolving rapidly.</p><p><strong>Regular checks:</strong></p><pre><code class="hljs bash"><span class="hljs-comment"># Check for CLI updates</span>npm update -g @github/copilot<span class="hljs-comment"># Check current version</span>copilot --version</code></pre><p><strong>Stay informed:</strong></p><ul><li>Watch the <a href="https://github.com/github/copilot-cli">GitHub Copilot CLI repository</a> for releases</li><li>Follow GitHub Blog for major announcements</li><li>Review release notes regularly</li></ul><h3 id="10-2-Evolving-Your-Custom-Instructions"><a href="#10-2-Evolving-Your-Custom-Instructions" class="headerlink" title="10.2 Evolving Your Custom Instructions"></a>10.2 Evolving Your Custom Instructions</h3><p>Your custom instructions should evolve with your team:</p><p><strong>Quarterly review process:</strong></p><ol><li>What patterns is Copilot frequently getting wrong? → Add explicit guidance</li><li>What questions do we repeatedly ask? → Add to instructions</li><li>Have our tools or practices changed? → Update commands and tech stack</li><li>Are there new team members’ pain points? → Improve onboarding instructions</li><li>What works well? → Document and reinforce</li></ol><p><strong>Version control your instructions:</strong></p><pre><code class="hljs bash"><span class="hljs-comment"># Treat .github/copilot-instructions.md like production code</span>git <span class="hljs-built_in">log</span> .github/copilot-instructions.md<span class="hljs-comment"># Review changes in PRs</span>git diff main HEAD -- .github/copilot-instructions.md</code></pre><h3 id="10-3-Responsible-AI-Use"><a href="#10-3-Responsible-AI-Use" class="headerlink" title="10.3 Responsible AI Use"></a>10.3 Responsible AI Use</h3><p><strong>Remember:</strong></p><ol><li><strong>Always review AI-generated code</strong> - Copilot is a tool, not a replacement for human judgment</li><li><strong>Security first</strong> - Review generated code for security vulnerabilities</li><li><strong>Test thoroughly</strong> - All generated code should have appropriate test coverage</li><li><strong>Understand what you commit</strong> - Don’t commit code you don’t understand</li><li><strong>IP considerations</strong> - Follow your organization’s policies on AI-generated code</li></ol><h3 id="10-4-Building-an-AI-Assisted-Culture"><a href="#10-4-Building-an-AI-Assisted-Culture" class="headerlink" title="10.4 Building an AI-Assisted Culture"></a>10.4 Building an AI-Assisted Culture</h3><p><strong>Team practices:</strong></p><ol><li><strong>Share wins and learnings</strong> - Regular demos of effective Copilot usage</li><li><strong>Contribute to shared instructions</strong> - Treat them like production code</li><li><strong>Measure impact</strong> - Track metrics before and after adoption</li><li><strong>Continuous learning</strong> - Allocate time for experimentation</li><li><strong>Celebrate improvements</strong> - Recognize productivity gains</li></ol><h2 id="Conclusion-Embracing-AI-Assisted-Development"><a href="#Conclusion-Embracing-AI-Assisted-Development" class="headerlink" title="Conclusion: Embracing AI-Assisted Development"></a>Conclusion: Embracing AI-Assisted Development</h2><p>GitHub Copilot CLI represents a significant evolution in developer tooling. With the recent release of advanced AI models like Claude Opus 4.6 (February 5, 2026), the capabilities continue to expand, offering features like agent teams and extended context windows.</p><p><strong>Key takeaways:</strong></p><ol><li><strong>Invest in custom instructions</strong> - They’re the foundation of consistent results</li><li><strong>Use plan mode for complex tasks</strong> - Structured planning improves outcomes</li><li><strong>Choose appropriate models</strong> - Balance capability with quota management</li><li><strong>Establish team standards</strong> - Shared conventions maximize productivity</li><li><strong>Always review and validate</strong> - AI is a powerful assistant, but you remain responsible</li><li><strong>Stay current</strong> - The tool is evolving; keep up with new features</li><li><strong>Measure and iterate</strong> - Track what works and continuously improve</li></ol><p>The developers who thrive will be those who effectively integrate these tools into their workflow while maintaining strong engineering practices.</p><p><strong>Getting Started:</strong></p><ol><li>Install GitHub Copilot CLI: <code>npm install -g @github/copilot</code></li><li>Authenticate: <code>copilot</code> then <code>/login</code></li><li>Create your first <code>.github/copilot-instructions.md</code></li><li>Try a simple <code>/plan</code> workflow</li><li>Share your experience with your team</li><li>Iterate and improve</li></ol><hr><p><strong>Disclaimer</strong>: This guide is based on the state of GitHub Copilot CLI as of February 2026. Features, commands, and capabilities are subject to change as the tool is in public preview. Always consult the <a href="https://docs.github.com/en/copilot/how-tos/copilot-cli">official GitHub documentation</a> for the most current and accurate information. Finally, this guide is not only for senior engineers but can be adapted for developers at all levels to maximize their productivity with AI-assisted development. </p><p>Till next time, Happy coding! 🚀</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_maximising_copilot_cli_senior_engineer_guide.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;As software engineers, we’re constantly seeking ways to streamline our workflows and boost productivity. GitHub Copilot CLI has emerged as a powerful tool that brings AI assistance directly into your terminal, transforming how we interact with code across different stacks. This comprehensive guide will walk you through setting up an effective AI-powered development workflow with GitHub Copilot CLI, whether you’re building front-end applications, back-end services, data pipelines, or cloud infrastructure.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: GitHub Copilot CLI is currently in &lt;strong&gt;public preview&lt;/strong&gt; and features are subject to change. Always refer to the &lt;a href=&quot;https://docs.github.com/en/copilot/how-tos/copilot-cli&quot;&gt;official GitHub documentation&lt;/a&gt; for the most up-to-date information.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;What-Makes-Copilot-CLI-Different&quot;&gt;&lt;a href=&quot;#What-Makes-Copilot-CLI-Different&quot; class=&quot;headerlink&quot; title=&quot;What Makes Copilot CLI Different?&quot;&gt;&lt;/a&gt;What Makes Copilot CLI Different?&lt;/h2&gt;&lt;p&gt;Unlike traditional code assistants, Copilot CLI is &lt;strong&gt;terminal-native&lt;/strong&gt; and &lt;strong&gt;agentic&lt;/strong&gt;—it doesn’t just answer questions, it can act as your coding partner. You can delegate tasks, and Copilot will autonomously execute them while you maintain oversight through explicit approval mechanisms.&lt;/p&gt;</summary>
    
    
    
    
    <category term="ai, github-copilot, cli, developer-productivity, software-development, best-practices, coding-assistant, ai-agents, software-engineering, full-stack-development, data-engineering, cloud-infrastructure, agentic-ai, plan-driven-development, multi-repo-workflows" scheme="http://awongcm.io/tags/ai-github-copilot-cli-developer-productivity-software-development-best-practices-coding-assistant-ai-agents-software-engineering-full-stack-development-data-engineering-cloud-infrastructure-agentic-ai-plan-driven-development-multi-repo-workflows/"/>
    
  </entry>
  
  <entry>
    <title>GitHub Copilot AI Models: A Developer&#39;s Guide to Choosing the Right Model</title>
    <link href="http://awongcm.io/blog/2026/01/24/github-copilot-ai-models-a-developer-s-guide-to-choosing-the-right-model/"/>
    <id>http://awongcm.io/blog/2026/01/24/github-copilot-ai-models-a-developer-s-guide-to-choosing-the-right-model/</id>
    <published>2026-01-24T04:19:44.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_github_copilot_models_developers_guide.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>GitHub Copilot now supports multiple AI models from leading providers including OpenAI, Anthropic, Google, and xAI, giving developers unprecedented flexibility to choose the right tool for their specific coding scenarios. Understanding the strengths and trade-offs of each model can significantly improve your productivity and code quality. This guide breaks down when to use each model and provides practical tips for getting the most out of GitHub Copilot in 2026.</p><span id="more"></span><h2 id="Available-Models-Overview"><a href="#Available-Models-Overview" class="headerlink" title="Available Models Overview"></a>Available Models Overview</h2><p>As of January 2026, GitHub Copilot offers access to an extensive range of AI models:</p><p><strong>OpenAI Models:</strong></p><ol><li>GPT-4o - Latest GPT-4 optimized model</li><li>GPT-4 Turbo - Fast GPT-4 variant</li><li>GPT-5 (1x) - Latest GPT-5 model</li><li>GPT-5-Codex (Preview, 1x) - GPT-5 optimized for coding</li><li>GPT-5.1 (1x) - Enhanced GPT-5 version</li><li>GPT-5.1-Codex (1x) - GPT-5.1 optimized for coding</li><li>GPT-5.1-Codex-Max (1x) - Maximum performance coding model</li><li>GPT-5.1-Codex-Mini (Preview, 0.33x) - Lightweight coding model</li><li>GPT-5.2 (1x) - Latest GPT-5.2 model</li><li>GPT-5.2-Codex (1x) - GPT-5.2 optimized for coding</li><li>o1-preview - Advanced reasoning model</li><li>o1-mini - Faster reasoning model</li></ol><p><strong>Anthropic Models:</strong></p><ol><li>Claude Sonnet 4 (1x) - Claude 4 balanced model</li><li>Claude Sonnet 4.5 (1x) - Latest Claude Sonnet version</li><li>Claude Opus 4.5 (3x) - Most capable Claude 4 model</li><li>Claude Haiku 4.5 (0.33x) - Fast, efficient Claude 4 model</li></ol><p><strong>Google Models:</strong></p><ol><li>Gemini 2.5 Pro (1x) - Advanced Gemini model</li><li>Gemini 3 Pro (Preview, 1x) - Latest Gemini Pro version</li><li>Gemini 3 Flash (Preview, 0.33x) - Fast Gemini variant</li></ol><p><strong>xAI Models:</strong></p><ol><li>Grok Code Fast 1 (0x) - xAI’s coding-focused model</li></ol><p><strong>Other Models:</strong></p><ol><li>Raptor mini (Preview, 0x) - Lightweight preview model</li></ol><h2 id="Model-Comparison-amp-Best-Use-Cases"><a href="#Model-Comparison-amp-Best-Use-Cases" class="headerlink" title="Model Comparison &amp; Best Use Cases"></a>Model Comparison &amp; Best Use Cases</h2><h3 id="GPT-4o-The-Optimized-Flagship"><a href="#GPT-4o-The-Optimized-Flagship" class="headerlink" title="GPT-4o: The Optimized Flagship"></a>GPT-4o: The Optimized Flagship</h3><p><strong>Best for:</strong></p><ul><li>Balanced performance and speed</li><li>General-purpose development tasks</li><li>Multi-modal interactions</li><li>Real-time code assistance</li><li>Production-grade applications</li></ul><p><strong>When to use:</strong></p><ul><li>Need reliable, fast responses</li><li>Working on standard to moderately complex tasks</li><li>Require consistent quality across various coding scenarios</li><li>Building customer-facing features</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier varies by implementation</li><li>May not match GPT-5 series for cutting-edge capabilities</li><li>Optimized for speed over maximum reasoning depth</li></ul><p><strong>Pro tip:</strong> GPT-4o strikes an excellent balance between speed and capability, making it ideal for daily production work.</p><h3 id="GPT-4-Turbo-The-Speed-Specialist"><a href="#GPT-4-Turbo-The-Speed-Specialist" class="headerlink" title="GPT-4 Turbo: The Speed Specialist"></a>GPT-4 Turbo: The Speed Specialist</h3><p><strong>Best for:</strong></p><ul><li>Rapid code completion</li><li>Quick iterations during development</li><li>Real-time suggestions</li><li>High-throughput scenarios</li><li>Time-sensitive projects</li></ul><p><strong>When to use:</strong></p><ul><li>Need fastest possible responses</li><li>Working under tight deadlines</li><li>Require immediate feedback during coding</li><li>Building prototypes quickly</li></ul><p><strong>Trade-offs:</strong></p><ul><li>May sacrifice some reasoning depth for speed</li><li>Better for well-defined problems than complex reasoning</li><li>Not ideal for architectural decisions</li></ul><p><strong>Pro tip:</strong> Use GPT-4 Turbo when velocity matters more than maximum sophistication.</p><h3 id="GPT-5-1x-The-Next-Generation-Model"><a href="#GPT-5-1x-The-Next-Generation-Model" class="headerlink" title="GPT-5 (1x): The Next-Generation Model"></a>GPT-5 (1x): The Next-Generation Model</h3><p><strong>Best for:</strong></p><ul><li>Advanced reasoning tasks</li><li>Complex code generation</li><li>Multi-step problem solving</li><li>Sophisticated algorithm implementation</li><li>Novel problem approaches</li></ul><p><strong>When to use:</strong></p><ul><li>Tackling new or unusual challenges</li><li>Need cutting-edge AI capabilities</li><li>Working on innovative solutions</li><li>Require deep contextual understanding</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>May be slower than GPT-4 variants</li><li>Newer model with evolving capabilities</li></ul><p><strong>Pro tip:</strong> GPT-5 represents the next generation of AI capabilities—use it when you need the latest advancements.</p><h3 id="GPT-5-Codex-Preview-1x-The-Code-First-Innovator"><a href="#GPT-5-Codex-Preview-1x-The-Code-First-Innovator" class="headerlink" title="GPT-5-Codex (Preview, 1x): The Code-First Innovator"></a>GPT-5-Codex (Preview, 1x): The Code-First Innovator</h3><p><strong>Best for:</strong></p><ul><li>Cutting-edge code generation</li><li>Complex programming patterns</li><li>Advanced code optimization</li><li>Novel algorithm development</li><li>Experimental coding approaches</li></ul><p><strong>When to use:</strong></p><ul><li>Need latest code-specific capabilities</li><li>Working on innovative coding solutions</li><li>Exploring new programming paradigms</li><li>Require advanced code understanding</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Preview status (may have limitations)</li><li>Evolving capabilities</li></ul><p><strong>Pro tip:</strong> GPT-5-Codex is ideal for developers who want to leverage the latest code-specific AI advancements.</p><h3 id="GPT-5-1-1x-The-Enhanced-Model"><a href="#GPT-5-1-1x-The-Enhanced-Model" class="headerlink" title="GPT-5.1 (1x): The Enhanced Model"></a>GPT-5.1 (1x): The Enhanced Model</h3><p><strong>Best for:</strong></p><ul><li>Improved reasoning over GPT-5</li><li>Complex code generation</li><li>Multi-file refactoring</li><li>Advanced problem-solving</li><li>Production-ready code</li></ul><p><strong>When to use:</strong></p><ul><li>Need enhanced capabilities over GPT-5</li><li>Working on sophisticated features</li><li>Require reliable, high-quality output</li><li>Building complex systems</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Balanced speed and quality</li><li>Standard computational cost</li></ul><p><strong>Pro tip:</strong> GPT-5.1 offers improvements over GPT-5 while maintaining the same cost structure.</p><h3 id="GPT-5-1-Codex-1x-The-Advanced-Code-Specialist"><a href="#GPT-5-1-Codex-1x-The-Advanced-Code-Specialist" class="headerlink" title="GPT-5.1-Codex (1x): The Advanced Code Specialist"></a>GPT-5.1-Codex (1x): The Advanced Code Specialist</h3><p><strong>Best for:</strong></p><ul><li>Advanced code-specific tasks</li><li>Complex algorithm implementation</li><li>Code optimization and refactoring</li><li>Technical debt reduction</li><li>Performance-critical code</li></ul><p><strong>When to use:</strong></p><ul><li>Need specialized code understanding</li><li>Working on complex algorithms</li><li>Implementing performance optimizations</li><li>Refactoring legacy code</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Specialized for code tasks</li><li>May be overkill for simple questions</li></ul><p><strong>Pro tip:</strong> Use GPT-5.1-Codex when you need advanced code-specific capabilities with enhanced reasoning.</p><h3 id="GPT-5-1-Codex-Max-1x-The-Maximum-Performance-Model"><a href="#GPT-5-1-Codex-Max-1x-The-Maximum-Performance-Model" class="headerlink" title="GPT-5.1-Codex-Max (1x): The Maximum Performance Model"></a>GPT-5.1-Codex-Max (1x): The Maximum Performance Model</h3><p><strong>Best for:</strong></p><ul><li>Maximum code generation quality</li><li>Extremely complex algorithms</li><li>Critical performance optimization</li><li>Advanced architectural patterns</li><li>Mission-critical code</li></ul><p><strong>When to use:</strong></p><ul><li>Need absolute best code quality</li><li>Working on performance-critical systems</li><li>Implementing complex design patterns</li><li>Solving the toughest coding challenges</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>May be slower than lighter models</li><li>Best reserved for complex tasks</li></ul><p><strong>Pro tip:</strong> GPT-5.1-Codex-Max is your go-to for the most demanding code generation tasks.</p><h3 id="GPT-5-1-Codex-Mini-Preview-0-33x-The-Efficient-Code-Assistant"><a href="#GPT-5-1-Codex-Mini-Preview-0-33x-The-Efficient-Code-Assistant" class="headerlink" title="GPT-5.1-Codex-Mini (Preview, 0.33x): The Efficient Code Assistant"></a>GPT-5.1-Codex-Mini (Preview, 0.33x): The Efficient Code Assistant</h3><p><strong>Best for:</strong></p><ul><li>Lightweight code generation</li><li>Quick code completions</li><li>Standard coding patterns</li><li>High-volume coding tasks</li><li>Cost-effective development</li></ul><p><strong>When to use:</strong></p><ul><li>Need efficient code assistance</li><li>Working on standard implementations</li><li>Require fast responses</li><li>Budget-conscious development</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 0.33x (very efficient!)</li><li>Preview status</li><li>Less sophisticated than full Codex models</li></ul><p><strong>Pro tip:</strong> With its low multiplier, GPT-5.1-Codex-Mini is perfect for high-volume coding without premium cost concerns.</p><h3 id="GPT-5-2-1x-The-Latest-Flagship"><a href="#GPT-5-2-1x-The-Latest-Flagship" class="headerlink" title="GPT-5.2 (1x): The Latest Flagship"></a>GPT-5.2 (1x): The Latest Flagship</h3><p><strong>Best for:</strong></p><ul><li>Complex code generation requiring deep context understanding</li><li>Multi-file refactoring tasks</li><li>Architectural decisions and design patterns</li><li>Advanced reasoning and problem-solving</li><li>Critical production code</li></ul><p><strong>When to use:</strong></p><ul><li>Working on mission-critical systems</li><li>Need highest accuracy for complex algorithms</li><li>Dealing with legacy codebases requiring careful analysis</li><li>Solving challenging technical problems</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Slower than lightweight models</li><li>Higher computational cost</li></ul><p><strong>Pro tip:</strong> GPT-5.2 represents the cutting edge of OpenAI’s capabilities. Use it when quality and accuracy are paramount.</p><h3 id="GPT-5-2-Codex-1x-The-Ultimate-Code-Specialist"><a href="#GPT-5-2-Codex-1x-The-Ultimate-Code-Specialist" class="headerlink" title="GPT-5.2-Codex (1x): The Ultimate Code Specialist"></a>GPT-5.2-Codex (1x): The Ultimate Code Specialist</h3><p><strong>Best for:</strong></p><ul><li>Code-specific tasks and optimizations</li><li>Understanding complex code patterns</li><li>Advanced code generation</li><li>Technical debt reduction</li><li>Performance optimization</li></ul><p><strong>When to use:</strong></p><ul><li>Need specialized code understanding</li><li>Working on performance-critical code</li><li>Implementing complex algorithms</li><li>Code optimization tasks</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Specialized for code, may be overkill for general questions</li><li>Best-in-class for code tasks</li></ul><p><strong>Pro tip:</strong> GPT-5.2-Codex is the most advanced code-specific model available—use it for your toughest coding challenges.</p><h3 id="o1-preview-The-Advanced-Reasoning-Specialist"><a href="#o1-preview-The-Advanced-Reasoning-Specialist" class="headerlink" title="o1-preview: The Advanced Reasoning Specialist"></a>o1-preview: The Advanced Reasoning Specialist</h3><p><strong>Best for:</strong></p><ul><li>Complex logical reasoning</li><li>Mathematical problem solving</li><li>Algorithm design and analysis</li><li>Multi-step reasoning tasks</li><li>Formal verification approaches</li><li>Deep analytical thinking</li></ul><p><strong>When to use:</strong></p><ul><li>Need deep analytical thinking</li><li>Working on algorithmically complex problems</li><li>Require formal reasoning capabilities</li><li>Solving mathematical or logical challenges</li><li>Complex system design requiring step-by-step analysis</li></ul><p><strong>Trade-offs:</strong></p><ul><li>May be slower due to reasoning depth</li><li>Optimized for reasoning over speed</li><li>Best for specific analytical tasks</li><li>Premium request multiplier varies</li></ul><p><strong>Pro tip:</strong> o1-preview excels at problems requiring step-by-step logical reasoning and mathematical analysis. Use it when you need to think through complex problems systematically.</p><h3 id="o1-mini-The-Efficient-Reasoner"><a href="#o1-mini-The-Efficient-Reasoner" class="headerlink" title="o1-mini: The Efficient Reasoner"></a>o1-mini: The Efficient Reasoner</h3><p><strong>Best for:</strong></p><ul><li>Fast reasoning tasks</li><li>Moderate complexity problems</li><li>Quick analytical solutions</li><li>Balanced reasoning and speed</li><li>Cost-effective reasoning</li></ul><p><strong>When to use:</strong></p><ul><li>Need reasoning capabilities with faster responses</li><li>Working on moderately complex logical tasks</li><li>Require efficient resource usage</li><li>Balancing quality and speed</li><li>Standard algorithm design</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Less depth than o1-preview</li><li>Better for focused problems than broad analysis</li><li>Optimized for efficiency</li><li>Faster but less comprehensive reasoning</li></ul><p><strong>Pro tip:</strong> o1-mini provides reasoning capabilities at a more accessible speed and cost point—perfect for everyday reasoning tasks.</p><h3 id="Claude-Sonnet-4-1x-The-Balanced-Claude"><a href="#Claude-Sonnet-4-1x-The-Balanced-Claude" class="headerlink" title="Claude Sonnet 4 (1x): The Balanced Claude"></a>Claude Sonnet 4 (1x): The Balanced Claude</h3><p><strong>Best for:</strong></p><ul><li>Balanced code generation</li><li>Standard development tasks</li><li>Quality-focused implementations</li><li>Thoughtful code suggestions</li><li>General-purpose development</li></ul><p><strong>When to use:</strong></p><ul><li>Need reliable Claude performance</li><li>Working on standard features</li><li>Require good balance of speed and quality</li><li>Building production applications</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Solid all-around performance</li><li>May not match Sonnet 4.5 for cutting-edge quality</li></ul><p><strong>Pro tip:</strong> Claude Sonnet 4 provides excellent balanced performance for most development tasks.</p><h3 id="Claude-Sonnet-4-5-1x-The-Quality-Champion"><a href="#Claude-Sonnet-4-5-1x-The-Quality-Champion" class="headerlink" title="Claude Sonnet 4.5 (1x): The Quality Champion"></a>Claude Sonnet 4.5 (1x): The Quality Champion</h3><p><strong>Best for:</strong></p><ul><li>Code reviews and quality improvements</li><li>Security-conscious code generation</li><li>Detailed explanations and documentation</li><li>Refactoring with emphasis on best practices</li><li>Maintainable, production-ready code</li></ul><p><strong>When to use:</strong></p><ul><li>Need thorough, thoughtful code suggestions</li><li>Working on security-sensitive features</li><li>Want detailed reasoning behind suggestions</li><li>Require strong adherence to coding standards</li><li>Complex logic requiring careful reasoning</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Can be more verbose in explanations</li><li>Slightly slower than lightweight models</li></ul><p><strong>Pro tip:</strong> Claude Sonnet 4.5 excels at writing clean, maintainable code with strong attention to best practices and security.</p><h3 id="Claude-Opus-4-5-3x-The-Premium-Powerhouse"><a href="#Claude-Opus-4-5-3x-The-Premium-Powerhouse" class="headerlink" title="Claude Opus 4.5 (3x): The Premium Powerhouse"></a>Claude Opus 4.5 (3x): The Premium Powerhouse</h3><p><strong>Best for:</strong></p><ul><li>Extremely complex reasoning tasks</li><li>Mission-critical code requiring highest quality</li><li>Comprehensive code analysis</li><li>Advanced architectural decisions</li><li>Complex multi-step problem solving</li></ul><p><strong>When to use:</strong></p><ul><li>Working on the most challenging problems</li><li>Need the absolute best quality output</li><li>Complex system design</li><li>Critical security implementations</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 3x (use strategically!)</li><li>Slower response times</li><li>Best reserved for truly complex tasks</li></ul><p><strong>Pro tip:</strong> Claude Opus 4.5 is your “big gun”—use it sparingly for the toughest challenges where quality is non-negotiable.</p><h3 id="Claude-Haiku-4-5-0-33x-The-Speed-Champion"><a href="#Claude-Haiku-4-5-0-33x-The-Speed-Champion" class="headerlink" title="Claude Haiku 4.5 (0.33x): The Speed Champion"></a>Claude Haiku 4.5 (0.33x): The Speed Champion</h3><p><strong>Best for:</strong></p><ul><li>Real-time code completion</li><li>Quick syntax fixes</li><li>Boilerplate code generation</li><li>Rapid prototyping</li><li>Repetitive coding tasks</li><li>Standard CRUD operations</li></ul><p><strong>When to use:</strong></p><ul><li>Need instant feedback while typing</li><li>Working on repetitive tasks</li><li>Generating standard code patterns</li><li>Time-sensitive development sprints</li><li>Writing scaffolding code</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 0.33x (very efficient!)</li><li>Less sophisticated reasoning for complex problems</li><li>Not suitable for architectural decisions</li></ul><p><strong>Pro tip:</strong> With its low multiplier and fast responses, Claude Haiku 4.5 is perfect for high-velocity coding sessions.</p><h3 id="Gemini-2-5-Pro-1x-The-Advanced-Google-Model"><a href="#Gemini-2-5-Pro-1x-The-Advanced-Google-Model" class="headerlink" title="Gemini 2.5 Pro (1x): The Advanced Google Model"></a>Gemini 2.5 Pro (1x): The Advanced Google Model</h3><p><strong>Best for:</strong></p><ul><li>Complex reasoning with Google’s approach</li><li>Multimodal code generation</li><li>Advanced problem solving</li><li>Alternative to GPT&#x2F;Claude models</li></ul><p><strong>When to use:</strong></p><ul><li>Need Google’s advanced capabilities</li><li>Working with multimodal tasks</li><li>Want alternative AI perspective</li><li>Exploring different approaches</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Mature, established model</li><li>Strong multimodal capabilities</li></ul><p><strong>Pro tip:</strong> Gemini 2.5 Pro offers a powerful alternative perspective to OpenAI and Anthropic models.</p><h3 id="Gemini-3-Pro-Preview-1x-The-Google-Flagship"><a href="#Gemini-3-Pro-Preview-1x-The-Google-Flagship" class="headerlink" title="Gemini 3 Pro (Preview, 1x): The Google Flagship"></a>Gemini 3 Pro (Preview, 1x): The Google Flagship</h3><p><strong>Best for:</strong></p><ul><li>Complex reasoning with Google’s latest approach</li><li>Multimodal code generation</li><li>Advanced problem solving</li><li>Alternative to GPT&#x2F;Claude models</li></ul><p><strong>When to use:</strong></p><ul><li>Want to try Google’s latest flagship</li><li>Need different perspective on problems</li><li>Working with images and code together</li><li>Exploring alternative AI approaches</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 1x</li><li>Public preview status</li><li>Availability may vary</li></ul><p><strong>Pro tip:</strong> Gemini 3 Pro represents Google’s latest AI advancements—experiment with it for fresh perspectives.</p><h3 id="Gemini-3-Flash-Preview-0-33x-The-Google-Speedster"><a href="#Gemini-3-Flash-Preview-0-33x-The-Google-Speedster" class="headerlink" title="Gemini 3 Flash (Preview, 0.33x): The Google Speedster"></a>Gemini 3 Flash (Preview, 0.33x): The Google Speedster</h3><p><strong>Best for:</strong></p><ul><li>Fast iterations</li><li>Multimodal tasks (code + images)</li><li>Quick prototyping</li><li>Standard coding patterns</li></ul><p><strong>When to use:</strong></p><ul><li>Need Google’s latest technology</li><li>Working with visual elements</li><li>Rapid development cycles</li><li>Experimenting with new approaches</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 0.33x</li><li>Public preview (may have limitations)</li><li>Less mature than established models</li></ul><p><strong>Pro tip:</strong> Gemini 3 Flash combines speed with multimodal capabilities at an efficient price point.</p><h3 id="Grok-Code-Fast-1-0x-The-Unlimited-Alternative"><a href="#Grok-Code-Fast-1-0x-The-Unlimited-Alternative" class="headerlink" title="Grok Code Fast 1 (0x): The Unlimited Alternative"></a>Grok Code Fast 1 (0x): The Unlimited Alternative</h3><p><strong>Best for:</strong></p><ul><li>High-volume code generation</li><li>Alternative coding perspectives</li><li>Fast iterations</li><li>Unlimited usage scenarios</li><li>Experimental approaches</li></ul><p><strong>When to use:</strong></p><ul><li>Want to try xAI’s approach</li><li>Need unlimited usage without premium costs</li><li>Looking for different coding styles</li><li>Exploring alternative AI perspectives</li><li>High-frequency coding tasks</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 0x (unlimited!)</li><li>Newer model with evolving capabilities</li><li>Different approach than established models</li></ul><p><strong>Pro tip:</strong> With 0x multiplier, Grok Code Fast 1 is perfect for high-volume experimentation and unlimited daily coding without consuming premium requests.</p><h3 id="Raptor-mini-Preview-0x-The-Unlimited-Specialist"><a href="#Raptor-mini-Preview-0x-The-Unlimited-Specialist" class="headerlink" title="Raptor mini (Preview, 0x): The Unlimited Specialist"></a>Raptor mini (Preview, 0x): The Unlimited Specialist</h3><p><strong>Best for:</strong></p><ul><li>GitHub-specific workflows</li><li>High-frequency coding tasks</li><li>Standard development patterns</li><li>Unlimited daily usage</li><li>Cost-conscious development</li></ul><p><strong>When to use:</strong></p><ul><li>Need unlimited model access</li><li>Working on typical GitHub workflows</li><li>Require efficient, focused responses</li><li>Budget is a primary concern</li><li>Standard development tasks</li></ul><p><strong>Trade-offs:</strong></p><ul><li>Premium request multiplier: 0x (unlimited!)</li><li>Preview status with potential limitations</li><li>Optimized for specific use cases</li><li>Fine-tuned for GitHub workflows</li></ul><p><strong>Pro tip:</strong> Raptor mini is ideal for developers who need unlimited access for standard coding tasks without any premium request concerns.</p><h2 id="Practical-Tips-amp-Tricks"><a href="#Practical-Tips-amp-Tricks" class="headerlink" title="Practical Tips &amp; Tricks"></a>Practical Tips &amp; Tricks</h2><h3 id="1-Match-Model-to-Task-Complexity"><a href="#1-Match-Model-to-Task-Complexity" class="headerlink" title="1. Match Model to Task Complexity"></a>1. Match Model to Task Complexity</h3><p><strong>Simple tasks</strong> (syntax, boilerplate, formatting):<br>→ Use <strong>Claude Haiku 4.5</strong>, <strong>Grok Code Fast 1</strong>, <strong>Raptor mini</strong>, or <strong>GPT-4 Turbo</strong></p><p><strong>Medium complexity</strong> (feature implementation, standard algorithms):<br>→ Use <strong>GPT-4o</strong>, <strong>Claude Sonnet 4</strong>, <strong>Gemini 3 Flash</strong>, or <strong>o1-mini</strong></p><p><strong>High complexity</strong> (architecture, complex algorithms, critical systems):<br>→ Use <strong>GPT-5.2</strong>, <strong>Claude Sonnet 4.5</strong>, <strong>GPT-5</strong>, or <strong>o1-preview</strong></p><p><strong>Maximum quality</strong> (mission-critical, highest stakes):<br>→ Use <strong>Claude Opus 4.5</strong>, <strong>GPT-5.2</strong>, or <strong>o1-preview</strong></p><p><strong>Code-specific optimization</strong>:<br>→ Use <strong>GPT-5.2-Codex</strong>, <strong>GPT-5.1-Codex-Max</strong>, or <strong>GPT-5-Codex</strong></p><p><strong>Advanced reasoning</strong>:<br>→ Use <strong>o1-preview</strong> or <strong>o1-mini</strong></p><h3 id="2-Optimize-for-Premium-Request-Usage"><a href="#2-Optimize-for-Premium-Request-Usage" class="headerlink" title="2. Optimize for Premium Request Usage"></a>2. Optimize for Premium Request Usage</h3><p><strong>Models with 0x multiplier</strong> (unlimited usage):</p><ul><li>Grok Code Fast 1</li><li>Raptor mini</li></ul><p><strong>Most efficient models</strong> (low multipliers):</p><ul><li>Claude Haiku 4.5 (0.33x)</li><li>Gemini 3 Flash (0.33x)</li><li>GPT-5.1-Codex-Mini (0.33x)</li></ul><p><strong>Standard models</strong> (1x multiplier):</p><ul><li>Claude Sonnet 4&#x2F;4.5</li><li>GPT-5, GPT-5.1, GPT-5.2</li><li>GPT-4o</li><li>GPT-5-Codex, GPT-5.1-Codex, GPT-5.1-Codex-Max, GPT-5.2-Codex</li><li>Gemini 2.5 Pro, Gemini 3 Pro</li></ul><p><strong>Premium models</strong> (use strategically):</p><ul><li>Claude Opus 4.5 (3x)</li></ul><p><strong>Pro tip:</strong> Start with 0x multiplier models (Grok, Raptor mini) for unlimited daily work, use 0.33x models for efficient high-volume tasks, and reserve 3x models for critical challenges.</p><h3 id="3-Speed-vs-Quality-Trade-off-Strategy"><a href="#3-Speed-vs-Quality-Trade-off-Strategy" class="headerlink" title="3. Speed vs. Quality Trade-off Strategy"></a>3. Speed vs. Quality Trade-off Strategy</h3><p><strong>Prototyping phase:</strong></p><ul><li>Primary: Claude Haiku 4.5, Grok Code Fast 1, or GPT-4 Turbo</li><li>Alternative: Raptor mini or Gemini 3 Flash</li></ul><p><strong>Implementation phase:</strong></p><ul><li>Primary: GPT-4o or Claude Sonnet 4</li><li>Alternative: GPT-5.1-Codex or GPT-5</li></ul><p><strong>Refinement phase:</strong></p><ul><li>Primary: Claude Sonnet 4.5 or GPT-5.2</li><li>Alternative: GPT-5.2-Codex or o1-mini</li></ul><p><strong>Review phase:</strong></p><ul><li>Primary: Claude Sonnet 4.5 or GPT-4o</li><li>Alternative: GPT-5.2 or o1-preview</li></ul><p><strong>Reasoning phase:</strong></p><ul><li>Primary: o1-preview</li><li>Alternative: o1-mini or GPT-5.2</li></ul><h3 id="4-Language-and-Framework-Recommendations"><a href="#4-Language-and-Framework-Recommendations" class="headerlink" title="4. Language and Framework Recommendations"></a>4. Language and Framework Recommendations</h3><p><strong>Python&#x2F;JavaScript&#x2F;TypeScript:</strong></p><ul><li>Any model works well</li><li>Use GPT-4o or Grok Code Fast 1 for daily work</li><li>Use Claude Sonnet 4.5 for quality-critical code</li></ul><p><strong>Rust&#x2F;Go&#x2F;Modern languages:</strong></p><ul><li>GPT-5.2 or Claude Sonnet 4.5 for advanced features</li><li>Claude Haiku 4.5 for standard patterns</li><li>o1-preview for complex algorithms</li></ul><p><strong>Legacy or niche languages:</strong></p><ul><li>GPT-5.2 or Claude Opus 4.5 for best understanding</li><li>Claude Sonnet 4.5 for careful, conservative suggestions</li></ul><p><strong>Code-heavy tasks:</strong></p><ul><li>GPT-5.2-Codex for optimization</li><li>GPT-5.1-Codex-Max for complex code generation</li><li>GPT-5-Codex for cutting-edge approaches</li></ul><p><strong>Mathematical&#x2F;algorithmic tasks:</strong></p><ul><li>o1-preview for complex reasoning</li><li>o1-mini for efficient problem-solving</li></ul><h3 id="5-Model-Updates-amp-New-Additions"><a href="#5-Model-Updates-amp-New-Additions" class="headerlink" title="5. Model Updates &amp; New Additions"></a>5. Model Updates &amp; New Additions</h3><p><strong>New Models to Explore:</strong></p><ul><li><strong>GPT-4o</strong> - Optimized GPT-4 for balanced performance</li><li><strong>GPT-4 Turbo</strong> - Speed-focused GPT-4 variant</li><li><strong>GPT-5 series</strong> - Next-generation capabilities</li><li><strong>o1-preview &amp; o1-mini</strong> - Advanced reasoning models</li><li><strong>Grok Code Fast 1</strong> - xAI’s unlimited usage alternative</li></ul><p><strong>Key Changes:</strong></p><ul><li>GPT-4o and GPT-4 Turbo provide optimized GPT-4 experiences</li><li>GPT-5 series offers cutting-edge capabilities</li><li>o1 models bring specialized reasoning abilities</li><li>Grok provides unlimited usage alternative</li><li>Expanded Codex variants for code-specific tasks</li></ul><p><strong>Pro tip:</strong> Experiment with new models in non-critical work to understand their strengths before using them in production.</p><h3 id="6-Cost-Optimization-Workflow"><a href="#6-Cost-Optimization-Workflow" class="headerlink" title="6. Cost Optimization Workflow"></a>6. Cost Optimization Workflow</h3><p><strong>Daily development routine:</strong></p><ol><li><strong>Active typing</strong>: Claude Haiku 4.5 (0.33x) or Grok Code Fast 1 (0x)</li><li><strong>Feature implementation</strong>: GPT-4o or Grok Code Fast 1 (0x)</li><li><strong>Complex logic</strong>: Claude Sonnet 4.5 (1x) or o1-mini</li><li><strong>Code review</strong>: Claude Sonnet 4.5 (1x) or GPT-4o</li><li><strong>Critical issues</strong>: Claude Opus 4.5 (3x) or o1-preview - use sparingly</li></ol><p><strong>Budget-conscious approach:</strong></p><ul><li>Use 0x models (Grok Code Fast 1, Raptor mini) as default</li><li>Use low-multiplier models (Haiku, Gemini Flash, Codex-Mini) for high-volume tasks</li><li>Reserve 1x+ models for important work</li><li>Use 3x models only for critical challenges</li></ul><p><strong>Pro tip:</strong> With Grok Code Fast 1 and Raptor mini at 0x, you can do unlimited coding without any premium request concerns.</p><h3 id="7-Multimodal-Capabilities"><a href="#7-Multimodal-Capabilities" class="headerlink" title="7. Multimodal Capabilities"></a>7. Multimodal Capabilities</h3><p><strong>Models supporting images:</strong></p><ul><li>Gemini 3 Flash</li><li>Gemini 3 Pro</li><li>Gemini 2.5 Pro</li><li>GPT-4o (check documentation)</li><li>Select other models (check documentation)</li></ul><p><strong>Use cases:</strong></p><ul><li>Screenshot of UI mockups → Generate HTML&#x2F;CSS</li><li>Error message screenshots → Debug issues</li><li>Diagram images → Explain architecture</li><li>Code screenshots → Analyze and improve</li><li>Design mockups → Implement interfaces</li></ul><p><strong>Pro tip:</strong> Leverage multimodal capabilities when working with visual elements—it can significantly speed up UI development and debugging.</p><h3 id="8-Team-Collaboration-Guidelines"><a href="#8-Team-Collaboration-Guidelines" class="headerlink" title="8. Team Collaboration Guidelines"></a>8. Team Collaboration Guidelines</h3><p><strong>Establish team standards:</strong></p><pre><code class="hljs plaintext">Decision Tree for Model Selection:Is it a security-critical feature?  → Yes: Claude Sonnet 4.5 or Claude Opus 4.5  → No: Continue...Is it a complex reasoning/algorithmic task?  → Yes: o1-preview or o1-mini  → No: Continue...Is it a complex architectural decision?  → Yes: GPT-5.2, Claude Opus 4.5, or o1-preview  → No: Continue...Is it code-specific optimization?  → Yes: GPT-5.2-Codex, GPT-5.1-Codex-Max, or GPT-5-Codex  → No: Continue...Is it boilerplate or repetitive?  → Yes: Claude Haiku 4.5, Grok Code Fast 1, or Raptor mini  → No: Continue...Is budget/usage a concern?  → Yes: Grok Code Fast 1 or Raptor mini (0x)  → No: GPT-4o or Claude Sonnet 4Default: Grok Code Fast 1 or GPT-4o</code></pre><h3 id="9-IDE-Specific-Considerations"><a href="#9-IDE-Specific-Considerations" class="headerlink" title="9. IDE-Specific Considerations"></a>9. IDE-Specific Considerations</h3><p><strong>Visual Studio Code:</strong></p><ul><li>All models available</li><li>Use “Auto” mode for intelligent selection</li><li>Can add custom models via AI Toolkit</li><li>Great support for all model families</li></ul><p><strong>JetBrains IDEs:</strong></p><ul><li>Full model support</li><li>Requires Copilot plugin 1.5.61+ for latest Codex models</li><li>Excellent integration with reasoning models</li></ul><p><strong>Visual Studio:</strong></p><ul><li>Comprehensive model support</li><li>Great for .NET development with any model</li><li>Strong support for GPT and Claude models</li></ul><p><strong>Xcode:</strong></p><ul><li>All models supported</li><li>Requires plugin 0.45.0+ for latest features</li><li>Good for Swift&#x2F;iOS development</li></ul><p><strong>Eclipse:</strong></p><ul><li>Full support available</li><li>Requires plugin 0.13.0+ for latest models</li><li>Works well with all model families</li></ul><h3 id="10-Prompt-Engineering-by-Model-Family"><a href="#10-Prompt-Engineering-by-Model-Family" class="headerlink" title="10. Prompt Engineering by Model Family"></a>10. Prompt Engineering by Model Family</h3><p><strong>For Claude models:</strong></p><ul><li>Be explicit about requirements and constraints</li><li>Ask for explanations when needed (especially Sonnet&#x2F;Opus)</li><li>Request “best practices” or “secure implementation”</li><li>Works well with structured, detailed prompts</li><li>Emphasize code quality and maintainability</li></ul><p><strong>For GPT models:</strong></p><ul><li>More conversational prompts work well</li><li>Good at inferring intent from context</li><li>Effective with iterative refinement</li><li>Handles ambiguous requests better</li><li>GPT-4o excels at balanced tasks</li></ul><p><strong>For o1 models:</strong></p><ul><li>Frame problems as reasoning tasks</li><li>Break down complex problems into steps</li><li>Ask for analytical thinking</li><li>Request mathematical or logical explanations</li><li>Best for step-by-step problem solving</li></ul><p><strong>For Gemini models:</strong></p><ul><li>Leverage multimodal capabilities</li><li>Provide visual context when available</li><li>Good for creative problem-solving</li><li>Works well with images and code together</li></ul><p><strong>For Grok:</strong></p><ul><li>Direct, straightforward prompts</li><li>Focus on efficiency</li><li>Good for quick iterations</li><li>Works well with high-volume requests</li></ul><h2 id="Real-World-Scenarios"><a href="#Real-World-Scenarios" class="headerlink" title="Real-World Scenarios"></a>Real-World Scenarios</h2><h3 id="Scenario-1-Building-a-New-REST-API"><a href="#Scenario-1-Building-a-New-REST-API" class="headerlink" title="Scenario 1: Building a New REST API"></a>Scenario 1: Building a New REST API</h3><ol><li><strong>Architecture planning</strong>: GPT-5.2, o1-preview, or Claude Sonnet 4.5</li><li><strong>Route scaffolding</strong>: Claude Haiku 4.5, Grok Code Fast 1, or GPT-4 Turbo</li><li><strong>Business logic</strong>: Claude Sonnet 4.5, GPT-4o, or GPT-5.1-Codex</li><li><strong>Input validation &amp; security</strong>: Claude Sonnet 4.5 or Claude Opus 4.5</li><li><strong>Tests</strong>: GPT-4o or Grok Code Fast 1</li><li><strong>Documentation</strong>: Claude Sonnet 4.5 or GPT-4o</li><li><strong>Performance optimization</strong>: GPT-5.2-Codex or GPT-5.1-Codex-Max</li></ol><p><strong>Estimated premium usage:</strong> ~8-12 requests (mix of 0x and 1x models)</p><h3 id="Scenario-2-Debugging-Production-Issue"><a href="#Scenario-2-Debugging-Production-Issue" class="headerlink" title="Scenario 2: Debugging Production Issue"></a>Scenario 2: Debugging Production Issue</h3><ol><li><strong>Initial investigation</strong>: GPT-5.2, GPT-4o, or Claude Sonnet 4.5</li><li><strong>Understanding stack traces</strong>: GPT-5.2 or o1-mini</li><li><strong>Quick fixes</strong>: GPT-4o, Grok Code Fast 1, or Claude Haiku 4.5</li><li><strong>Root cause analysis</strong>: Claude Sonnet 4.5, o1-preview, or Claude Opus 4.5</li><li><strong>Verification</strong>: Claude Sonnet 4.5 or GPT-4o</li></ol><p><strong>Estimated premium usage:</strong> ~4-7 requests (mix of 0x and 1x)</p><h3 id="Scenario-3-Rapid-Prototyping"><a href="#Scenario-3-Rapid-Prototyping" class="headerlink" title="Scenario 3: Rapid Prototyping"></a>Scenario 3: Rapid Prototyping</h3><ol><li><strong>Quick iterations</strong>: Claude Haiku 4.5, Grok Code Fast 1, or GPT-4 Turbo</li><li><strong>Core functionality</strong>: GPT-4o or Grok Code Fast 1</li><li><strong>Edge cases</strong>: Claude Sonnet 4 or o1-mini</li><li><strong>Polish</strong>: Claude Sonnet 4.5 or GPT-4o</li></ol><p><strong>Estimated premium usage:</strong> ~2-4 requests (mostly 0x and low multipliers)</p><h3 id="Scenario-4-Legacy-Code-Refactoring"><a href="#Scenario-4-Legacy-Code-Refactoring" class="headerlink" title="Scenario 4: Legacy Code Refactoring"></a>Scenario 4: Legacy Code Refactoring</h3><ol><li><strong>Understanding existing code</strong>: GPT-5.2, o1-preview, or Claude Opus 4.5</li><li><strong>Identifying improvements</strong>: Claude Sonnet 4.5 or o1-mini</li><li><strong>Implementing changes</strong>: Claude Sonnet 4.5, GPT-4o, or GPT-5.1-Codex</li><li><strong>Ensuring compatibility</strong>: Claude Sonnet 4.5 or GPT-5.2</li><li><strong>Migration tests</strong>: GPT-4o or Grok Code Fast 1</li></ol><p><strong>Estimated premium usage:</strong> ~7-11 requests (higher quality models)</p><h3 id="Scenario-5-High-Volume-Feature-Development"><a href="#Scenario-5-High-Volume-Feature-Development" class="headerlink" title="Scenario 5: High-Volume Feature Development"></a>Scenario 5: High-Volume Feature Development</h3><ol><li><strong>Boilerplate generation</strong>: Grok Code Fast 1 (0x) or Raptor mini (0x)</li><li><strong>Standard implementations</strong>: GPT-4o or Grok Code Fast 1 (0x)</li><li><strong>Complex components</strong>: Claude Sonnet 4 (1x) or GPT-5.1-Codex (1x)</li><li><strong>Code review</strong>: Claude Sonnet 4.5 (1x) or GPT-4o</li></ol><p><strong>Estimated premium usage:</strong> ~3-6 requests (optimized for volume with 0x models)</p><h3 id="Scenario-6-Algorithm-Design-amp-Optimization"><a href="#Scenario-6-Algorithm-Design-amp-Optimization" class="headerlink" title="Scenario 6: Algorithm Design &amp; Optimization"></a>Scenario 6: Algorithm Design &amp; Optimization</h3><ol><li><strong>Problem analysis</strong>: o1-preview or o1-mini</li><li><strong>Algorithm design</strong>: o1-preview or GPT-5.2</li><li><strong>Implementation</strong>: GPT-5.2-Codex or GPT-5.1-Codex-Max</li><li><strong>Optimization</strong>: GPT-5.2-Codex or o1-mini</li><li><strong>Testing</strong>: GPT-4o or Grok Code Fast 1</li></ol><p><strong>Estimated premium usage:</strong> ~6-10 requests (reasoning and code-specific models)</p><h2 id="Performance-Matrix"><a href="#Performance-Matrix" class="headerlink" title="Performance Matrix"></a>Performance Matrix</h2><table><thead><tr><th>Task Type</th><th>Speed</th><th>Quality</th><th>Efficiency</th><th>Recommended Models</th></tr></thead><tbody><tr><td>Boilerplate</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>Haiku 4.5, Grok, Raptor mini, GPT-4 Turbo</td></tr><tr><td>Feature Dev</td><td>⭐⭐⭐⭐</td><td>⭐⭐⭐⭐</td><td>⭐⭐⭐⭐</td><td>GPT-4o, Sonnet 4, o1-mini</td></tr><tr><td>Complex Logic</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>Sonnet 4.5, GPT-5.2, o1-preview</td></tr><tr><td>Code Review</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>Sonnet 4.5, GPT-4o</td></tr><tr><td>Debugging</td><td>⭐⭐⭐⭐</td><td>⭐⭐⭐⭐</td><td>⭐⭐⭐⭐</td><td>GPT-5.2, Sonnet 4.5, GPT-4o</td></tr><tr><td>Security</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>Sonnet 4.5, Opus 4.5</td></tr><tr><td>Documentation</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>Sonnet 4.5, GPT-4o</td></tr><tr><td>Prototyping</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>Haiku 4.5, Grok, GPT-4 Turbo</td></tr><tr><td>Reasoning</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>o1-preview, o1-mini, GPT-5</td></tr><tr><td>Code Optimization</td><td>⭐⭐⭐</td><td>⭐⭐⭐⭐⭐</td><td>⭐⭐⭐</td><td>GPT-5.2-Codex, GPT-5.1-Codex-Max</td></tr><tr><td>Multimodal</td><td>⭐⭐⭐⭐</td><td>⭐⭐⭐⭐</td><td>⭐⭐⭐⭐</td><td>Gemini 3 Flash, Gemini 3 Pro, GPT-4o</td></tr></tbody></table><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>The expanded model selection in GitHub Copilot for 2026 gives developers unprecedented control over their AI-assisted development workflow. With models ranging from unlimited usage (Grok at 0x, Raptor mini at 0x) to premium powerhouses (Claude Opus 4.5 at 3x), and specialized reasoning models (o1-preview, o1-mini), you can optimize for speed, quality, cost, or reasoning depth based on your specific needs.</p><h3 id="Quick-Reference-Guide"><a href="#Quick-Reference-Guide" class="headerlink" title="Quick Reference Guide:"></a>Quick Reference Guide:</h3><ul><li><strong>Need speed?</strong> → GPT-4 Turbo, Claude Haiku 4.5, or Grok Code Fast 1</li><li><strong>Daily coding?</strong> → Grok Code Fast 1 or Raptor mini (0x multiplier—unlimited!)</li><li><strong>Need quality?</strong> → Claude Sonnet 4.5 or GPT-4o</li><li><strong>Complex problems?</strong> → GPT-5.2, o1-preview, or Claude Opus 4.5</li><li><strong>Advanced reasoning?</strong> → o1-preview or o1-mini</li><li><strong>Security-critical?</strong> → Claude Sonnet 4.5 or Claude Opus 4.5</li><li><strong>Code optimization?</strong> → GPT-5.2-Codex or GPT-5.1-Codex-Max</li><li><strong>Budget-conscious?</strong> → Grok Code Fast 1 or Raptor mini (unlimited!)</li><li><strong>Prototyping?</strong> → GPT-4 Turbo, Claude Haiku 4.5, or Grok Code Fast 1</li><li><strong>Production code?</strong> → GPT-4o, Claude Sonnet 4.5, or GPT-5.2</li><li><strong>Balanced performance?</strong> → GPT-4o or Claude Sonnet 4</li><li><strong>Multimodal tasks?</strong> → Gemini 3 Flash, Gemini 3 Pro, or GPT-4o</li></ul><h3 id="Key-Takeaways"><a href="#Key-Takeaways" class="headerlink" title="Key Takeaways:"></a>Key Takeaways:</h3><ol><li><strong>Use 0x multiplier models as your default</strong> (Grok Code Fast 1, Raptor mini) for unlimited usage</li><li><strong>GPT-4o provides excellent balance</strong> for production development</li><li><strong>o1 models excel at reasoning</strong> tasks requiring deep analysis</li><li><strong>Low multiplier models</strong> (Haiku, Gemini Flash, Codex-Mini) are perfect for high-volume work</li><li><strong>Reserve premium models</strong> (Opus 4.5) for truly critical challenges</li><li><strong>GPT-5 series offers cutting-edge</strong> capabilities for innovative work</li><li><strong>Switch models mid-task</strong> based on complexity—don’t stick to one default</li><li><strong>Leverage specialized models</strong>—use Codex variants for code-heavy tasks, o1 for reasoning</li><li><strong>Consider multimodal capabilities</strong>—Gemini models support images</li><li><strong>Monitor your usage</strong>—understand multipliers to optimize premium request consumption</li></ol><h3 id="Action-Items"><a href="#Action-Items" class="headerlink" title="Action Items:"></a>Action Items:</h3><ol><li>Explore GPT-4o as your balanced production model</li><li>Try o1-preview for complex reasoning tasks</li><li>Set Grok Code Fast 1 or Raptor mini as default for unlimited usage</li><li>Set up quick-access shortcuts for your top 5-6 models</li><li>Document team preferences and decision criteria</li><li>Experiment with GPT-5 series for cutting-edge capabilities</li><li>Use o1-mini for efficient reasoning tasks</li><li>Track your premium usage patterns to optimize model selection</li><li>Test GPT-4 Turbo for high-velocity development</li><li>Leverage code-specific Codex variants for optimization work</li></ol><p>The key to success is flexibility: match the model to the task, not the other way around. Start with unlimited models (Grok, Raptor mini) for daily work, use efficient models (Haiku, Gemini Flash) for high-volume tasks, leverage reasoning models (o1-preview, o1-mini) for complex analysis, escalate to premium models when quality matters, and always be ready to switch based on the challenge at hand.</p><p>To see actual code samples for all these latest Copilot models, you can check out my Github repo <a href="https://github.com/awongCM/copilot-models-testing/tree/main/results">here</a>.  You can check out all the present models with their corresponding observations notes I have prepared them for so you can take note of their differences.</p><p>Hope these will be helpful for you.</p><p>Till next time, Happy Coding!</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_github_copilot_models_developers_guide.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;GitHub Copilot now supports multiple AI models from leading providers including OpenAI, Anthropic, Google, and xAI, giving developers unprecedented flexibility to choose the right tool for their specific coding scenarios. Understanding the strengths and trade-offs of each model can significantly improve your productivity and code quality. This guide breaks down when to use each model and provides practical tips for getting the most out of GitHub Copilot in 2026.&lt;/p&gt;</summary>
    
    
    
    
    <category term="ai, gihtub-copilot, large-language-models, productivity, developer-tools, ai-programming, software-development, tech-guide, software-engineering, developer-resources" scheme="http://awongcm.io/tags/ai-gihtub-copilot-large-language-models-productivity-developer-tools-ai-programming-software-development-tech-guide-software-engineering-developer-resources/"/>
    
  </entry>
  
  <entry>
    <title>Building Modern CI/CD Pipelines with GitHub Actions: A Complete Guide to Docker, LocalStack, and AWS Glue Testing</title>
    <link href="http://awongcm.io/blog/2026/01/17/building-modern-ci-cd-pipelines-with-github-actions-a-complete-guide-to-docker-localstack-and-aws-glue-testing/"/>
    <id>http://awongcm.io/blog/2026/01/17/building-modern-ci-cd-pipelines-with-github-actions-a-complete-guide-to-docker-localstack-and-aws-glue-testing/</id>
    <published>2026-01-17T00:46:10.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_building_github_actions_localstack_unit_tests.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>Since my last post on setting local AWS Glue using Docker, one of the potential extensions I mentioned is to setup Github Actions CI&#x2F;CD pipeline for it. In modern data engineering, testing cloud-native applications locally before deployment is very crucial for rapid iteration and cost efficiency. Thus, as part of this exploratory exercise, I want my CI&#x2F;CD pipeline to achieve the following outcomes(in scope):</p><ul><li>Automate testing of AWS Glue jobs in a local environment</li><li>Use Docker to containerize the Glue runtime</li><li>Leverage LocalStack to simulate AWS services locally</li><li>Integrate with GitHub Actions for continuous integration and delivery</li><li>Ensure code quality with SonarCloud and Codecov</li></ul><p>Not in scope for this pipeline:</p><ul><li>Terraform or infrastructure-as-code testing</li><li>Deployment to actual AWS environments</li><li>Advanced security hardening for enterprise compliance</li></ul><p>This guide walks you through building very robust (if not production-ready) CI&#x2F;CD pipeline using GitHub Actions that integrates Docker containerization, LocalStack for AWS service emulation, and comprehensive testing for AWS Glue jobs.</p><span id="more"></span><p>This proof-of-concept end-to-end solution was conducted by trial and error using my Copilot and Claude Sonnets to help me along the learning journey. I’m pretty satisfied with its findings and I hope you find it useful too.</p><p>By the end of this post, you’ll understand how to leverage GitHub’s ecosystem to create automated pipelines that test your data engineering workloads without incurring cloud costs during development.</p><h2 id="Why-This-Matters"><a href="#Why-This-Matters" class="headerlink" title="Why This Matters"></a>Why This Matters</h2><p>Traditional cloud development workflows often require:</p><ul><li>Deploying to actual AWS environments for testing</li><li>Incurring costs for every test run</li><li>Waiting for cloud resource provisioning</li><li>Managing multiple AWS accounts for dev&#x2F;test&#x2F;prod</li></ul><p><strong>The solution?</strong> A fully containerized CI&#x2F;CD pipeline that:</p><ul><li>Tests AWS Glue jobs locally using Docker</li><li>Simulates AWS services with LocalStack</li><li>Runs automated tests on every commit</li><li>Publishes Docker images to GitHub Container Registry</li><li>Integrates code quality checks with SonarCloud and Codecov</li></ul><h2 id="Architecture-Overview"><a href="#Architecture-Overview" class="headerlink" title="Architecture Overview"></a>Architecture Overview</h2><p>The pipeline implements a multi-stage workflow:</p><ol><li><strong>Build Stage</strong>: Creates Docker images with AWS Glue runtime</li><li><strong>Test Stage</strong>: Spins up LocalStack services and runs unit&#x2F;integration tests</li><li><strong>Quality Stage</strong>: Analyzes code coverage and quality metrics</li><li><strong>Publish Stage</strong>: Pushes validated images to GitHub Container Registry</li></ol><h2 id="The-Complete-GitHub-Actions-Workflow"><a href="#The-Complete-GitHub-Actions-Workflow" class="headerlink" title="The Complete GitHub Actions Workflow"></a>The Complete GitHub Actions Workflow</h2><p>Let’s break down the workflow file (<code>ci-pipeline.yml</code>) into digestible components.</p><h3 id="1-Workflow-Triggers-and-Permissions"><a href="#1-Workflow-Triggers-and-Permissions" class="headerlink" title="1. Workflow Triggers and Permissions"></a>1. Workflow Triggers and Permissions</h3><pre><code class="hljs yaml"><span class="hljs-attr">name:</span> <span class="hljs-string">Run</span> <span class="hljs-string">Tests</span> <span class="hljs-string">with</span> <span class="hljs-string">LocalStack,</span> <span class="hljs-string">Build</span> <span class="hljs-string">Docker</span> <span class="hljs-string">Image,</span> <span class="hljs-string">and</span> <span class="hljs-string">Upload</span> <span class="hljs-string">Coverage</span> <span class="hljs-string">to</span> <span class="hljs-string">SonarCloud</span> <span class="hljs-string">&amp;</span> <span class="hljs-string">Codecov</span><span class="hljs-attr">on:</span>  <span class="hljs-attr">push:</span>    <span class="hljs-attr">branches:</span> [<span class="hljs-string">main</span>, <span class="hljs-string">develop</span>]  <span class="hljs-attr">pull_request:</span>    <span class="hljs-attr">branches:</span> [<span class="hljs-string">main</span>, <span class="hljs-string">develop</span>]<span class="hljs-attr">permissions:</span>  <span class="hljs-attr">contents:</span> <span class="hljs-string">read</span>  <span class="hljs-attr">packages:</span> <span class="hljs-string">write</span></code></pre><p><strong>Key Points:</strong></p><ul><li>Triggers on pushes and PRs to <code>main</code> and <code>develop</code> branches</li><li><code>packages: write</code> permission enables pushing to GitHub Container Registry (GHCR)</li><li><code>contents: read</code> allows checkout access</li></ul><h3 id="2-LocalStack-Service-Configuration"><a href="#2-LocalStack-Service-Configuration" class="headerlink" title="2. LocalStack Service Configuration"></a>2. LocalStack Service Configuration</h3><pre><code class="hljs yaml"><span class="hljs-attr">services:</span>  <span class="hljs-attr">localstack:</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">localstack/localstack:latest</span>    <span class="hljs-attr">env:</span>      <span class="hljs-attr">SERVICES:</span> <span class="hljs-string">dynamodb,s3,cloudwatch,logs,iam,sts</span>      <span class="hljs-attr">DEFAULT_REGION:</span> <span class="hljs-string">us-east-1</span>      <span class="hljs-attr">AWS_ACCESS_KEY_ID:</span> <span class="hljs-string">test</span>      <span class="hljs-attr">AWS_SECRET_ACCESS_KEY:</span> <span class="hljs-string">test</span>      <span class="hljs-attr">DEBUG:</span> <span class="hljs-number">1</span>    <span class="hljs-attr">ports:</span>      <span class="hljs-bullet">-</span> <span class="hljs-number">4566</span><span class="hljs-string">:4566</span>    <span class="hljs-attr">options:</span> <span class="hljs-string">&gt;-</span><span class="hljs-string">      --health-cmd &quot;curl -f http://localhost:4566/_localstack/health || exit 1&quot;</span><span class="hljs-string">      --health-interval 10s</span><span class="hljs-string">      --health-timeout 5s</span><span class="hljs-string">      --health-retries 10</span></code></pre><p><strong>Why LocalStack?</strong></p><ul><li>Emulates AWS services locally without cloud costs</li><li>Provides S3, DynamoDB, CloudWatch, and IAM functionality</li><li>Health checks ensure services are ready before tests run</li><li>Port 4566 is LocalStack’s unified endpoint</li></ul><p><strong>Pro Tip:</strong> The health check configuration prevents race conditions where tests start before LocalStack is fully initialized.</p><h3 id="3-Disk-Space-Optimization"><a href="#3-Disk-Space-Optimization" class="headerlink" title="3. Disk Space Optimization"></a>3. Disk Space Optimization</h3><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Free</span> <span class="hljs-string">up</span> <span class="hljs-string">disk</span> <span class="hljs-string">space</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    # Remove unnecessary tools and SDKs</span><span class="hljs-string">    sudo rm -rf /usr/share/dotnet</span><span class="hljs-string">    sudo rm -rf /opt/ghc</span><span class="hljs-string">    sudo rm -rf /usr/local/share/boost</span><span class="hljs-string">    sudo rm -rf &quot;$AGENT_TOOLSDIRECTORY&quot;</span><span class="hljs-string"></span>    <span class="hljs-comment"># Clean apt package manager</span>    <span class="hljs-string">sudo</span> <span class="hljs-string">apt-get</span> <span class="hljs-string">clean</span>    <span class="hljs-string">sudo</span> <span class="hljs-string">apt-get</span> <span class="hljs-string">autoclean</span>    <span class="hljs-string">sudo</span> <span class="hljs-string">apt-get</span> <span class="hljs-string">autoremove</span> <span class="hljs-string">-y</span>    <span class="hljs-comment"># Aggressive Docker cleanup</span>    <span class="hljs-string">docker</span> <span class="hljs-string">system</span> <span class="hljs-string">prune</span> <span class="hljs-string">-af</span> <span class="hljs-string">--volumes</span>    <span class="hljs-string">docker</span> <span class="hljs-string">builder</span> <span class="hljs-string">prune</span> <span class="hljs-string">-af</span></code></pre><p><strong>Why This Matters:</strong></p><ul><li>GitHub Actions runners have limited disk space (~14GB free)</li><li>Docker images for Glue environments can be 2-3GB+</li><li>Build caches and intermediate layers consume additional space</li><li>Cleanup can free up 10-15GB, preventing build failures</li></ul><p><strong>Real-World Impact:</strong> Without this step, builds often fail with “no space left on device” errors.</p><h3 id="4-Docker-Build-and-Push-with-Caching"><a href="#4-Docker-Build-and-Push-with-Caching" class="headerlink" title="4. Docker Build and Push with Caching"></a>4. Docker Build and Push with Caching</h3><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Set</span> <span class="hljs-string">up</span> <span class="hljs-string">Docker</span> <span class="hljs-string">Buildx</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">docker/setup-buildx-action@v3</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Convert</span> <span class="hljs-string">repository</span> <span class="hljs-string">owner</span> <span class="hljs-string">to</span> <span class="hljs-string">lowercase</span>  <span class="hljs-attr">id:</span> <span class="hljs-string">repo</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    REPO_OWNER_LOWER=$(echo &quot;$&#123;&#123; github.repository_owner &#125;&#125;&quot; | tr &#x27;[:upper:]&#x27; &#x27;[:lower:]&#x27;)</span><span class="hljs-string">    REPO_NAME_LOWER=$(echo &quot;$&#123;&#123; github.event.repository.name &#125;&#125;&quot; | tr &#x27;[:upper:]&#x27; &#x27;[:lower:]&#x27;)</span><span class="hljs-string">    echo &quot;owner=$REPO_OWNER_LOWER&quot; &gt;&gt; $GITHUB_OUTPUT</span><span class="hljs-string">    echo &quot;name=$REPO_NAME_LOWER&quot; &gt;&gt; $GITHUB_OUTPUT</span><span class="hljs-string"></span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Log</span> <span class="hljs-string">in</span> <span class="hljs-string">to</span> <span class="hljs-string">GitHub</span> <span class="hljs-string">Container</span> <span class="hljs-string">Registry</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">docker/login-action@v3</span>  <span class="hljs-attr">with:</span>    <span class="hljs-attr">registry:</span> <span class="hljs-string">ghcr.io</span>    <span class="hljs-attr">username:</span> <span class="hljs-string">$&#123;&#123;</span> <span class="hljs-string">github.actor</span> <span class="hljs-string">&#125;&#125;</span>    <span class="hljs-attr">password:</span> <span class="hljs-string">$&#123;&#123;</span> <span class="hljs-string">secrets.GITHUB_TOKEN</span> <span class="hljs-string">&#125;&#125;</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Build</span> <span class="hljs-string">and</span> <span class="hljs-string">push</span> <span class="hljs-string">Glue</span> <span class="hljs-string">PySpark</span> <span class="hljs-string">Docker</span> <span class="hljs-string">image</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">docker/build-push-action@v5</span>  <span class="hljs-attr">with:</span>    <span class="hljs-attr">context:</span> <span class="hljs-string">.</span>    <span class="hljs-attr">file:</span> <span class="hljs-string">./Dockerfile</span>    <span class="hljs-attr">push:</span> <span class="hljs-literal">true</span>    <span class="hljs-attr">tags:</span> <span class="hljs-string">ghcr.io/$&#123;&#123;</span> <span class="hljs-string">steps.repo.outputs.owner</span> <span class="hljs-string">&#125;&#125;/$&#123;&#123;</span> <span class="hljs-string">steps.repo.outputs.name</span> <span class="hljs-string">&#125;&#125;/glue-pyspark:latest</span>    <span class="hljs-attr">cache-from:</span> <span class="hljs-string">type=registry,ref=ghcr.io/$&#123;&#123;</span> <span class="hljs-string">steps.repo.outputs.owner</span> <span class="hljs-string">&#125;&#125;/$&#123;&#123;</span> <span class="hljs-string">steps.repo.outputs.name</span> <span class="hljs-string">&#125;&#125;/glue-pyspark:buildcache</span>    <span class="hljs-attr">cache-to:</span> <span class="hljs-string">type=registry,ref=ghcr.io/$&#123;&#123;</span> <span class="hljs-string">steps.repo.outputs.owner</span> <span class="hljs-string">&#125;&#125;/$&#123;&#123;</span> <span class="hljs-string">steps.repo.outputs.name</span> <span class="hljs-string">&#125;&#125;/glue-pyspark:buildcache,mode=max</span></code></pre><p><strong>Advanced Techniques:</strong></p><ol><li><strong>Docker Buildx</strong>: Enables advanced build features and multi-platform support</li><li><strong>Lowercase Conversion</strong>: GHCR requires lowercase repository names (GitHub allows mixed case)</li><li><strong>Registry Caching</strong>: Dramatically speeds up builds by reusing layers from previous runs</li><li><strong>Automatic Authentication</strong>: Uses <code>GITHUB_TOKEN</code> for seamless GHCR access</li></ol><p><strong>Performance Impact:</strong> Registry caching can reduce build times from 15+ minutes to 2-3 minutes on subsequent runs.</p><h3 id="5-LocalStack-Health-Check-and-Initialization"><a href="#5-LocalStack-Health-Check-and-Initialization" class="headerlink" title="5. LocalStack Health Check and Initialization"></a>5. LocalStack Health Check and Initialization</h3><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Wait</span> <span class="hljs-string">for</span> <span class="hljs-string">LocalStack</span> <span class="hljs-string">to</span> <span class="hljs-string">be</span> <span class="hljs-string">ready</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    echo &quot;Waiting for LocalStack to be ready...&quot;</span><span class="hljs-string">    max_attempts=60</span><span class="hljs-string">    attempt=0</span><span class="hljs-string">    until curl -f -s http://localhost:4566/_localstack/health | grep -q &#x27;&quot;s3&quot;&#x27; || [ $attempt -eq $max_attempts ]; do</span><span class="hljs-string">      echo &quot;Attempt $((++attempt))/$max_attempts: Waiting for LocalStack...&quot;</span><span class="hljs-string">      if [ $((attempt % 10)) -eq 0 ]; then</span><span class="hljs-string">        echo &quot;Health check response:&quot;</span><span class="hljs-string">        curl -s http://localhost:4566/_localstack/health || echo &quot;No response&quot;</span><span class="hljs-string">      fi</span><span class="hljs-string">      sleep 2</span><span class="hljs-string">    done</span></code></pre><p><strong>Robust Error Handling:</strong></p><ul><li>Polls LocalStack health endpoint up to 60 times (2 minutes)</li><li>Provides diagnostic output every 10 attempts</li><li>Fails fast with detailed logs if LocalStack doesn’t start</li><li>Checks for specific service availability (S3 in this case)</li></ul><h3 id="6-AWS-Resource-Provisioning-in-LocalStack"><a href="#6-AWS-Resource-Provisioning-in-LocalStack" class="headerlink" title="6. AWS Resource Provisioning in LocalStack"></a>6. AWS Resource Provisioning in LocalStack</h3><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Configure</span> <span class="hljs-string">AWS</span> <span class="hljs-string">resources</span> <span class="hljs-string">in</span> <span class="hljs-string">LocalStack</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    docker run --rm \</span><span class="hljs-string">      --network host \</span><span class="hljs-string">      -e AWS_ACCESS_KEY_ID=test \</span><span class="hljs-string">      -e AWS_SECRET_ACCESS_KEY=test \</span><span class="hljs-string">      -e AWS_REGION=us-east-1 \</span><span class="hljs-string">      amazon/aws-cli:latest \</span><span class="hljs-string">      --endpoint-url=http://localhost:4566 s3 mb s3://bronze-bucket || true</span><span class="hljs-string"></span>    <span class="hljs-string">docker</span> <span class="hljs-string">run</span> <span class="hljs-string">--rm</span> <span class="hljs-string">\</span>      <span class="hljs-string">--network</span> <span class="hljs-string">host</span> <span class="hljs-string">\</span>      <span class="hljs-string">-e</span> <span class="hljs-string">AWS_ACCESS_KEY_ID=test</span> <span class="hljs-string">\</span>      <span class="hljs-string">-e</span> <span class="hljs-string">AWS_SECRET_ACCESS_KEY=test</span> <span class="hljs-string">\</span>      <span class="hljs-string">-e</span> <span class="hljs-string">AWS_REGION=us-east-1</span> <span class="hljs-string">\</span>      <span class="hljs-string">amazon/aws-cli:latest</span> <span class="hljs-string">\</span>      <span class="hljs-string">--endpoint-url=http://localhost:4566</span> <span class="hljs-string">dynamodb</span> <span class="hljs-string">create-table</span> <span class="hljs-string">\</span>      <span class="hljs-string">--table-name</span> <span class="hljs-string">gold_table_plain</span> <span class="hljs-string">\</span>      <span class="hljs-string">--attribute-definitions</span> <span class="hljs-string">AttributeName=id,AttributeType=N</span> <span class="hljs-string">\</span>      <span class="hljs-string">--key-schema</span> <span class="hljs-string">AttributeName=id,KeyType=HASH</span> <span class="hljs-string">\</span>      <span class="hljs-string">--provisioned-throughput</span> <span class="hljs-string">ReadCapacityUnits=5,WriteCapacityUnits=5</span> <span class="hljs-string">||</span> <span class="hljs-literal">true</span></code></pre><p><strong>Infrastructure as Code in CI:</strong></p><ul><li>Creates S3 buckets for data lake layers (bronze, iceberg)</li><li>Provisions DynamoDB tables for gold layer aggregations</li><li>Sets up CloudWatch log groups for Glue job monitoring</li><li>Uses <code>|| true</code> to handle idempotent operations gracefully</li></ul><p><strong>Design Pattern:</strong> This mirrors production infrastructure setup, ensuring test environments match production.</p><h3 id="7-Running-Tests-in-the-Glue-Container"><a href="#7-Running-Tests-in-the-Glue-Container" class="headerlink" title="7. Running Tests in the Glue Container"></a>7. Running Tests in the Glue Container</h3><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Run</span> <span class="hljs-string">tests</span> <span class="hljs-string">inside</span> <span class="hljs-string">Glue</span> <span class="hljs-string">container</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    docker run --rm \</span><span class="hljs-string">      --network host \</span><span class="hljs-string">      -v $&#123;&#123; github.workspace &#125;&#125;/plain:/app/plain \</span><span class="hljs-string">      -v $&#123;&#123; github.workspace &#125;&#125;/purchase_order:/app/purchase_order \</span><span class="hljs-string">      -v $&#123;&#123; github.workspace &#125;&#125;/tests:/app/tests \</span><span class="hljs-string">      -v $&#123;&#123; github.workspace &#125;&#125;/pytest.ini:/app/pytest.ini \</span><span class="hljs-string">      -v $&#123;&#123; github.workspace &#125;&#125;:/output \</span><span class="hljs-string">      -e AWS_ACCESS_KEY_ID=test \</span><span class="hljs-string">      -e AWS_SECRET_ACCESS_KEY=test \</span><span class="hljs-string">      -e AWS_REGION=us-east-1 \</span><span class="hljs-string">      my-glue-pyspark:latest \</span><span class="hljs-string">      &quot;poetry run pytest tests/ --cov=plain --cov=purchase_order --cov-report=term-missing --cov-report=xml:/output/coverage.xml --cov-report=html:/output/htmlcov -v&quot;</span></code></pre><p><strong>Container Testing Strategy:</strong></p><ol><li><strong>Volume Mounts</strong>: Source code and tests are mounted into the container</li><li><strong>Network Mode</strong>: <code>--network host</code> allows container to access LocalStack on localhost</li><li><strong>Output Persistence</strong>: Coverage reports are written back to GitHub workspace</li><li><strong>Poetry Integration</strong>: Uses Poetry for dependency management and test execution</li><li><strong>Multiple Coverage Formats</strong>: XML for SonarCloud&#x2F;Codecov, HTML for artifact viewing</li></ol><p><strong>Why This Works:</strong></p><ul><li>Tests run in the exact same environment as production Glue jobs</li><li>No “works on my machine” issues</li><li>Consistent Python version, dependencies, and Spark configuration</li></ul><h3 id="8-Code-Quality-Integration"><a href="#8-Code-Quality-Integration" class="headerlink" title="8. Code Quality Integration"></a>8. Code Quality Integration</h3><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">SonarCloud</span> <span class="hljs-string">Scan</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">SonarSource/sonarcloud-github-action@master</span>  <span class="hljs-attr">env:</span>    <span class="hljs-attr">GITHUB_TOKEN:</span> <span class="hljs-string">$&#123;&#123;</span> <span class="hljs-string">secrets.GITHUB_TOKEN</span> <span class="hljs-string">&#125;&#125;</span>    <span class="hljs-attr">SONAR_TOKEN:</span> <span class="hljs-string">$&#123;&#123;</span> <span class="hljs-string">secrets.SONAR_TOKEN</span> <span class="hljs-string">&#125;&#125;</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Upload</span> <span class="hljs-string">coverage</span> <span class="hljs-string">reports</span> <span class="hljs-string">to</span> <span class="hljs-string">Codecov</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">codecov/codecov-action@v4</span>  <span class="hljs-attr">if:</span> <span class="hljs-string">always()</span>  <span class="hljs-attr">with:</span>    <span class="hljs-attr">files:</span> <span class="hljs-string">./coverage.xml</span>    <span class="hljs-attr">flags:</span> <span class="hljs-string">unittests</span>    <span class="hljs-attr">name:</span> <span class="hljs-string">codecov-umbrella</span>    <span class="hljs-attr">fail_ci_if_error:</span> <span class="hljs-literal">false</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Upload</span> <span class="hljs-string">coverage</span> <span class="hljs-string">HTML</span> <span class="hljs-string">report</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">actions/upload-artifact@v4</span>  <span class="hljs-attr">if:</span> <span class="hljs-string">always()</span>  <span class="hljs-attr">with:</span>    <span class="hljs-attr">name:</span> <span class="hljs-string">coverage-report</span>    <span class="hljs-attr">path:</span> <span class="hljs-string">htmlcov/</span>    <span class="hljs-attr">retention-days:</span> <span class="hljs-number">7</span></code></pre><p><strong>Multi-Tool Quality Assurance:</strong></p><ol><li><strong>SonarCloud</strong>: Static code analysis, security vulnerabilities, code smells</li><li><strong>Codecov</strong>: Coverage tracking with historical trends and PR comments</li><li><strong>GitHub Artifacts</strong>: Downloadable HTML reports for detailed inspection</li></ol><p><strong>Pro Tip:</strong> <code>if: always()</code> ensures quality reports are uploaded even if tests fail, providing debugging information.</p><hr><h2 id="Workflow-Pipeline-Execution"><a href="#Workflow-Pipeline-Execution" class="headerlink" title="Workflow Pipeline Execution"></a>Workflow Pipeline Execution</h2><p>Once you’re done setting up the workflow pipeline above, you can commit and push it to your repository. GitHub Actions will automatically trigger the pipeline on pushes or pull requests to the specified branches. In this case, for mine, <code>main</code> and <code>develop</code>.</p><p>You can monitor the progress of each job in the “Actions” tab of your GitHub repository. Each stage will show logs, status, and any errors encountered during execution.</p><p>With this <code>ci-pipeline.yml</code> workflow, you will see only one job running all the steps sequentially and you can view its steps log in <a href="https://github.com/awongCM/docker_glue_pyspark_demo/actions/runs/21084727991">detail</a>.  But by good general practice, you should use the <code>ci-sequential.yml</code> workflow if you want to see each stage as separate jobs for better visibility and parallel execution - see it <a href="https://github.com/awongCM/docker_glue_pyspark_demo/actions/runs/21085490168">here</a>.</p><hr><h2 id="Enterprise-Considerations-Security-Compliance-and-Risk-Management"><a href="#Enterprise-Considerations-Security-Compliance-and-Risk-Management" class="headerlink" title="Enterprise Considerations: Security, Compliance, and Risk Management"></a>Enterprise Considerations: Security, Compliance, and Risk Management</h2><h3 id="The-Open-Source-vs-Corporate-Reality"><a href="#The-Open-Source-vs-Corporate-Reality" class="headerlink" title="The Open-Source vs. Corporate Reality"></a>The Open-Source vs. Corporate Reality</h3><p>While everything what I discovered and achieved here so far has been great exprience so far and this guide demonstrates a powerful open-source DevOps stack, <strong>enterprise adoption, however, requires careful evaluation</strong> of organizational constraints. What works seamlessly in exploratory projects may face significant hurdles in regulated corporate environments. Thus that leaves you plenty of critical questions and notes to ponder before adopting this stack wholesome.</p><h3 id="Critical-Evaluation-Framework"><a href="#Critical-Evaluation-Framework" class="headerlink" title="Critical Evaluation Framework"></a>Critical Evaluation Framework</h3><h4 id="1-GitHub-Actions-and-Third-Party-Runners"><a href="#1-GitHub-Actions-and-Third-Party-Runners" class="headerlink" title="1. GitHub Actions and Third-Party Runners"></a>1. <strong>GitHub Actions and Third-Party Runners</strong></h4><p><strong>Open-Source Approach:</strong></p><ul><li>Uses GitHub-hosted runners (shared infrastructure)</li><li>Free for public repositories</li><li>Minimal setup required</li></ul><p><strong>Enterprise Constraints:</strong></p><table><thead><tr><th>Risk Factor</th><th>Consideration</th><th>Mitigation Options</th></tr></thead><tbody><tr><td><strong>Data Residency</strong></td><td>Code and artifacts processed on GitHub’s infrastructure</td><td>Self-hosted runners in your VPC&#x2F;data center</td></tr><tr><td><strong>Compliance</strong></td><td>May violate SOC2, HIPAA, PCI-DSS requirements</td><td>GitHub Enterprise Server (on-premises)</td></tr><tr><td><strong>Network Security</strong></td><td>Runners access public internet</td><td>Private networking with GitHub Enterprise Cloud</td></tr><tr><td><strong>Audit Trails</strong></td><td>Limited visibility into runner environment</td><td>Enhanced audit logging with Enterprise features</td></tr></tbody></table><p><strong>Questions to Ask Your Security Team:</strong></p><ul><li>Can our source code leave our network perimeter?</li><li>What data classification levels are we handling?</li><li>Do we need SOC2 Type II compliance for our CI&#x2F;CD?</li><li>Are GitHub’s data centers in approved geographic regions?</li></ul><h4 id="2-LocalStack-Licensing-and-Support"><a href="#2-LocalStack-Licensing-and-Support" class="headerlink" title="2. LocalStack Licensing and Support"></a>2. <strong>LocalStack Licensing and Support</strong></h4><p><strong>Open-Source Approach:</strong></p><ul><li>Uses LocalStack Community Edition (free, limited features)</li><li>No official support or SLAs</li><li>Community-driven bug fixes</li></ul><p><strong>Enterprise Constraints:</strong></p><table><thead><tr><th>Aspect</th><th>Community Edition</th><th>Enterprise Considerations</th></tr></thead><tbody><tr><td><strong>License</strong></td><td>Apache 2.0 (permissive)</td><td>Legal review required for commercial use</td></tr><tr><td><strong>Features</strong></td><td>Basic AWS service emulation</td><td>Advanced services require Pro license ($50-500&#x2F;dev&#x2F;month)</td></tr><tr><td><strong>Support</strong></td><td>GitHub issues only</td><td>Enterprise support with SLAs may be mandated</td></tr><tr><td><strong>Security Updates</strong></td><td>Community-driven timeline</td><td>Compliance may require guaranteed patch schedules</td></tr><tr><td><strong>Accuracy</strong></td><td>Best-effort AWS parity</td><td>Production validation required for critical workloads</td></tr></tbody></table><p><strong>Red Flags for Enterprise:</strong></p><ul><li>Financial services: LocalStack may not meet regulatory testing requirements</li><li>Healthcare: PHI data cannot touch LocalStack without BAA (not available)</li><li>Government: FedRAMP&#x2F;IL5 compliance impossible with LocalStack</li></ul><p><strong>Alternative Approaches:</strong></p><ol><li><strong>AWS-native testing</strong>: Use dedicated AWS accounts with cost controls</li><li><strong>Moto library</strong>: Python-only AWS mocking (more limited but officially maintained)</li><li><strong>Hybrid approach</strong>: LocalStack for dev, real AWS for pre-production CI</li></ol><h4 id="3-Docker-and-Container-Registry-Security"><a href="#3-Docker-and-Container-Registry-Security" class="headerlink" title="3. Docker and Container Registry Security"></a>3. <strong>Docker and Container Registry Security</strong></h4><p><strong>Open-Source Approach:</strong></p><ul><li>Pulls base images from Docker Hub</li><li>Pushes to GitHub Container Registry (public or private)</li><li>Minimal image scanning</li></ul><p><strong>Enterprise Constraints:</strong></p><table><thead><tr><th>Security Concern</th><th>Risk Level</th><th>Enterprise Solution</th></tr></thead><tbody><tr><td><strong>Supply Chain Attacks</strong></td><td>HIGH</td><td>Private registries with approved base images only</td></tr><tr><td><strong>Image Vulnerabilities</strong></td><td>HIGH</td><td>Mandatory scanning (Trivy, Snyk, Prisma Cloud)</td></tr><tr><td><strong>Secrets in Images</strong></td><td>CRITICAL</td><td>Secret scanning + admission controllers</td></tr><tr><td><strong>Registry Access</strong></td><td>MEDIUM</td><td>SSO integration, RBAC, audit logging</td></tr><tr><td><strong>Image Provenance</strong></td><td>MEDIUM</td><td>Image signing with Sigstore&#x2F;Notary</td></tr></tbody></table><p><strong>Corporate Policies That May Block This Approach:</strong></p><ul><li><strong>No Docker Hub access</strong>: Many enterprises block external registries</li><li><strong>Approved base images only</strong>: Must use hardened, patched internal images</li><li><strong>Image signing required</strong>: Unsigned images cannot be deployed</li><li><strong>SBOM requirements</strong>: Software Bill of Materials for compliance</li></ul><p><strong>Adaptation Strategy:</strong></p><pre><code class="hljs yaml"><span class="hljs-comment"># Enterprise-compliant Docker build</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Build</span> <span class="hljs-string">with</span> <span class="hljs-string">approved</span> <span class="hljs-string">base</span> <span class="hljs-string">image</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    # Use internal registry with approved images</span><span class="hljs-string">    docker build \</span><span class="hljs-string">      --build-arg BASE_IMAGE=registry.company.com/approved/python:3.9-slim \</span><span class="hljs-string">      --label &quot;git.commit=$&#123;&#123; github.sha &#125;&#125;&quot; \</span><span class="hljs-string">      --label &quot;build.date=$(date -u +%Y-%m-%dT%H:%M:%SZ)&quot; \</span><span class="hljs-string">      -t registry.company.com/data-eng/glue-pyspark:$&#123;&#123; github.sha &#125;&#125; .</span><span class="hljs-string"></span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Scan</span> <span class="hljs-string">image</span> <span class="hljs-string">for</span> <span class="hljs-string">vulnerabilities</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    trivy image --severity HIGH,CRITICAL \</span><span class="hljs-string">      --exit-code 1 \</span><span class="hljs-string">      registry.company.com/data-eng/glue-pyspark:$&#123;&#123; github.sha &#125;&#125;</span><span class="hljs-string"></span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Sign</span> <span class="hljs-string">image</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    cosign sign --key cosign.key \</span><span class="hljs-string">      registry.company.com/data-eng/glue-pyspark:$&#123;&#123; github.sha &#125;&#125;</span></code></pre><h4 id="4-Third-Party-GitHub-Actions-and-Supply-Chain-Risk"><a href="#4-Third-Party-GitHub-Actions-and-Supply-Chain-Risk" class="headerlink" title="4. Third-Party GitHub Actions and Supply Chain Risk"></a>4. <strong>Third-Party GitHub Actions and Supply Chain Risk</strong></h4><p><strong>Open-Source Approach:</strong></p><ul><li>Uses community actions (<code>actions/checkout@v4</code>, <code>docker/build-push-action@v5</code>)</li><li>Trusts action maintainers implicitly</li><li>Auto-updates to latest versions</li></ul><p><strong>Enterprise Constraints:</strong></p><table><thead><tr><th>Risk</th><th>Impact</th><th>Mitigation</th></tr></thead><tbody><tr><td><strong>Malicious Actions</strong></td><td>Code injection, credential theft</td><td>Pin to commit SHAs, not tags</td></tr><tr><td><strong>Abandoned Actions</strong></td><td>Security vulnerabilities unpatched</td><td>Fork and maintain internally</td></tr><tr><td><strong>Compliance</strong></td><td>Actions not vetted by security team</td><td>Approved action registry</td></tr><tr><td><strong>Audit Requirements</strong></td><td>Cannot trace action provenance</td><td>SBOM for actions, vendoring</td></tr></tbody></table><p><strong>Secure Action Usage Pattern:</strong></p><pre><code class="hljs yaml"><span class="hljs-comment"># ❌ INSECURE: Uses mutable tag</span><span class="hljs-bullet">-</span> <span class="hljs-attr">uses:</span> <span class="hljs-string">actions/checkout@v4</span><span class="hljs-comment"># ✅ SECURE: Pinned to immutable commit SHA</span><span class="hljs-bullet">-</span> <span class="hljs-attr">uses:</span> <span class="hljs-string">actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11</span> <span class="hljs-comment"># v4.1.1</span>  <span class="hljs-comment"># Comment includes version for human readability</span></code></pre><p><strong>Corporate Alternatives:</strong></p><ul><li><strong>Self-hosted action runners</strong>: Host approved actions in internal registry</li><li><strong>Action vendoring</strong>: Copy actions into your repository</li><li><strong>Restricted marketplace</strong>: Only allow pre-approved actions</li></ul><h4 id="5-SonarCloud-and-Codecov-Data-Exfiltration-Concerns"><a href="#5-SonarCloud-and-Codecov-Data-Exfiltration-Concerns" class="headerlink" title="5. SonarCloud and Codecov: Data Exfiltration Concerns"></a>5. <strong>SonarCloud and Codecov: Data Exfiltration Concerns</strong></h4><p><strong>Open-Source Approach:</strong></p><ul><li>Sends source code to SonarCloud (third-party SaaS)</li><li>Uploads coverage data to Codecov (third-party SaaS)</li><li>Convenient integrations with GitHub</li></ul><p><strong>Enterprise Constraints:</strong></p><table><thead><tr><th>Service</th><th>Data Shared</th><th>Compliance Risk</th><th>Enterprise Alternative</th></tr></thead><tbody><tr><td><strong>SonarCloud</strong></td><td>Full source code, secrets (if leaked)</td><td>HIGH - IP theft, compliance violations</td><td>SonarQube self-hosted</td></tr><tr><td><strong>Codecov</strong></td><td>Coverage reports, file paths, metadata</td><td>MEDIUM - Reveals architecture</td><td>Self-hosted coverage tools</td></tr></tbody></table><p><strong>Real-World Incident:</strong></p><ul><li><strong>Codecov breach (2021)</strong>: Supply chain attack exposed customer credentials for months</li><li><strong>Impact</strong>: Many enterprises banned Codecov permanently</li></ul><p><strong>Secure Alternatives:</strong></p><pre><code class="hljs yaml"><span class="hljs-comment"># Self-hosted SonarQube</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">SonarQube</span> <span class="hljs-string">Scan</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    sonar-scanner \</span><span class="hljs-string">      -Dsonar.host.url=https://sonarqube.company.com \</span><span class="hljs-string">      -Dsonar.login=$&#123;&#123; secrets.SONAR_TOKEN_INTERNAL &#125;&#125;</span><span class="hljs-string"></span><span class="hljs-comment"># Internal coverage dashboard</span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Upload</span> <span class="hljs-string">coverage</span> <span class="hljs-string">to</span> <span class="hljs-string">internal</span> <span class="hljs-string">system</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    curl -X POST https://coverage.company.com/api/upload \</span><span class="hljs-string">      -H &quot;Authorization: Bearer $&#123;&#123; secrets.COVERAGE_TOKEN &#125;&#125;&quot; \</span><span class="hljs-string">      -F &quot;coverage=@coverage.xml&quot; \</span><span class="hljs-string">      -F &quot;project=$&#123;&#123; github.repository &#125;&#125;&quot; \</span><span class="hljs-string">      -F &quot;commit=$&#123;&#123; github.sha &#125;&#125;&quot;</span></code></pre><h4 id="6-Secrets-Management-and-Credential-Exposure"><a href="#6-Secrets-Management-and-Credential-Exposure" class="headerlink" title="6. Secrets Management and Credential Exposure"></a>6. <strong>Secrets Management and Credential Exposure</strong></h4><p><strong>Open-Source Approach:</strong></p><ul><li>Uses GitHub Secrets (encrypted at rest)</li><li>Secrets accessible to all workflows</li><li>Limited rotation capabilities</li></ul><p><strong>Enterprise Constraints:</strong></p><table><thead><tr><th>Requirement</th><th>GitHub Secrets</th><th>Enterprise Solution</th></tr></thead><tbody><tr><td><strong>Secrets Rotation</strong></td><td>Manual only</td><td>HashiCorp Vault, AWS Secrets Manager</td></tr><tr><td><strong>Audit Logging</strong></td><td>Limited</td><td>Comprehensive audit trails required</td></tr><tr><td><strong>Access Control</strong></td><td>Repository-level</td><td>Fine-grained RBAC per secret</td></tr><tr><td><strong>Compliance</strong></td><td>May not meet standards</td><td>FIPS 140-2 validated HSMs</td></tr><tr><td><strong>Just-in-Time Access</strong></td><td>Not supported</td><td>Dynamic credential generation</td></tr></tbody></table><p><strong>Enterprise Pattern:</strong></p><pre><code class="hljs yaml"><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Retrieve</span> <span class="hljs-string">secrets</span> <span class="hljs-string">from</span> <span class="hljs-string">Vault</span>  <span class="hljs-attr">id:</span> <span class="hljs-string">vault</span>  <span class="hljs-attr">uses:</span> <span class="hljs-string">hashicorp/vault-action@v2</span>  <span class="hljs-attr">with:</span>    <span class="hljs-attr">url:</span> <span class="hljs-string">https://vault.company.com</span>    <span class="hljs-attr">method:</span> <span class="hljs-string">jwt</span>    <span class="hljs-attr">role:</span> <span class="hljs-string">github-actions-role</span>    <span class="hljs-attr">secrets:</span> <span class="hljs-string">|</span><span class="hljs-string">      secret/data/aws/credentials access_key | AWS_ACCESS_KEY_ID ;</span><span class="hljs-string">      secret/data/aws/credentials secret_key | AWS_SECRET_ACCESS_KEY</span><span class="hljs-string"></span><span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Use</span> <span class="hljs-string">short-lived</span> <span class="hljs-string">credentials</span>  <span class="hljs-attr">run:</span> <span class="hljs-string">|</span><span class="hljs-string">    # Credentials automatically expire after job completion</span><span class="hljs-string">    aws s3 ls</span><span class="hljs-string"></span>  <span class="hljs-attr">env:</span>    <span class="hljs-attr">AWS_ACCESS_KEY_ID:</span> <span class="hljs-string">$&#123;&#123;</span> <span class="hljs-string">steps.vault.outputs.AWS_ACCESS_KEY_ID</span> <span class="hljs-string">&#125;&#125;</span>    <span class="hljs-attr">AWS_SECRET_ACCESS_KEY:</span> <span class="hljs-string">$&#123;&#123;</span> <span class="hljs-string">steps.vault.outputs.AWS_SECRET_ACCESS_KEY</span> <span class="hljs-string">&#125;&#125;</span></code></pre><h3 id="Decision-Matrix-Can-Your-Organization-Use-This-Stack"><a href="#Decision-Matrix-Can-Your-Organization-Use-This-Stack" class="headerlink" title="Decision Matrix: Can Your Organization Use This Stack?"></a>Decision Matrix: Can Your Organization Use This Stack?</h3><p>Use this matrix to assess feasibility:</p><table><thead><tr><th>Factor</th><th>Green Light ✅</th><th>Yellow Light ⚠️</th><th>Red Light 🛑</th></tr></thead><tbody><tr><td><strong>Industry</strong></td><td>Tech startups, SaaS</td><td>Professional services</td><td>Finance, Healthcare, Government</td></tr><tr><td><strong>Data Classification</strong></td><td>Public, Internal</td><td>Confidential</td><td>Restricted, PHI, PCI, PII</td></tr><tr><td><strong>Compliance</strong></td><td>None, ISO 27001</td><td>SOC2, GDPR</td><td>HIPAA, PCI-DSS, FedRAMP</td></tr><tr><td><strong>Security Posture</strong></td><td>Risk-tolerant</td><td>Risk-aware</td><td>Risk-averse</td></tr><tr><td><strong>Budget</strong></td><td>Limited</td><td>Moderate</td><td>Enterprise</td></tr><tr><td><strong>Team Size</strong></td><td>&lt;50 engineers</td><td>50-500 engineers</td><td>500+ engineers</td></tr></tbody></table><p><strong>Interpretation:</strong></p><ul><li><strong>All Green</strong>: Adopt this stack with minimal modifications</li><li><strong>Some Yellow</strong>: Requires adaptations (self-hosted runners, private registries)</li><li><strong>Any Red</strong>: Significant re-architecture needed or complete alternative approach</li></ul><h3 id="Hybrid-Approach-Balancing-Innovation-and-Compliance"><a href="#Hybrid-Approach-Balancing-Innovation-and-Compliance" class="headerlink" title="Hybrid Approach: Balancing Innovation and Compliance"></a>Hybrid Approach: Balancing Innovation and Compliance</h3><p>Most enterprises land somewhere in the middle. Consider this graduated adoption strategy:</p><h4 id="Phase-1-Proof-of-Concept-Open-Source-Stack"><a href="#Phase-1-Proof-of-Concept-Open-Source-Stack" class="headerlink" title="Phase 1: Proof of Concept (Open-Source Stack)"></a>Phase 1: Proof of Concept (Open-Source Stack)</h4><ul><li>Use for non-production, non-sensitive projects</li><li>Validate technical feasibility</li><li>Build team expertise</li><li><strong>Duration</strong>: 1-3 months</li></ul><h4 id="Phase-2-Security-Hardening-Hybrid"><a href="#Phase-2-Security-Hardening-Hybrid" class="headerlink" title="Phase 2: Security Hardening (Hybrid)"></a>Phase 2: Security Hardening (Hybrid)</h4><ul><li>Migrate to self-hosted GitHub Actions runners</li><li>Replace LocalStack with AWS dev accounts</li><li>Implement image scanning and signing</li><li>Use internal SonarQube instead of SonarCloud</li><li><strong>Duration</strong>: 3-6 months</li></ul><h4 id="Phase-3-Enterprise-Integration-Fully-Compliant"><a href="#Phase-3-Enterprise-Integration-Fully-Compliant" class="headerlink" title="Phase 3: Enterprise Integration (Fully Compliant)"></a>Phase 3: Enterprise Integration (Fully Compliant)</h4><ul><li>Integrate with corporate Vault&#x2F;secrets management</li><li>Implement RBAC and audit logging</li><li>Add compliance gates (policy-as-code with OPA)</li><li>Establish SLAs and support processes</li><li><strong>Duration</strong>: 6-12 months</li></ul><h3 id="Questions-to-Ask-Before-Adopting"><a href="#Questions-to-Ask-Before-Adopting" class="headerlink" title="Questions to Ask Before Adopting"></a>Questions to Ask Before Adopting</h3><p><strong>For Engineering Leadership:</strong></p><ol><li>What is our organization’s risk appetite for CI&#x2F;CD tooling?</li><li>Do we have budget for enterprise licenses (GitHub Enterprise, LocalStack Pro)?</li><li>Can we dedicate resources to maintain self-hosted infrastructure?</li><li>What is our timeline for compliance certification (SOC2, ISO, etc.)?</li></ol><p><strong>For Security Teams:</strong></p><ol><li>Can source code be processed outside our network perimeter?</li><li>What are our requirements for secrets management and rotation?</li><li>Do we need FIPS 140-2 validated cryptography?</li><li>What audit logging and retention requirements apply?</li></ol><p><strong>For Compliance Teams:</strong></p><ol><li>What data classification levels will this pipeline handle?</li><li>Are there geographic restrictions on data processing?</li><li>Do we need vendor security assessments (VSAs) for all third parties?</li><li>What is our incident response process for supply chain compromises?</li></ol><h3 id="The-Bottom-Line"><a href="#The-Bottom-Line" class="headerlink" title="The Bottom Line"></a>The Bottom Line</h3><p><strong>This open-source stack is ideal for:</strong></p><ul><li>Startups and scale-ups with minimal compliance requirements</li><li>Public open-source projects</li><li>Internal tools handling non-sensitive data</li><li>Proof-of-concept and learning environments</li><li>Teams with high risk tolerance and technical sophistication</li></ul><p><strong>This stack requires significant modification for:</strong></p><ul><li>Regulated industries (finance, healthcare, government)</li><li>Enterprises with strict data residency requirements</li><li>Organizations handling PII, PHI, or PCI data</li><li>Companies requiring SOC2 Type II or ISO 27001 certification</li><li>Risk-averse security cultures</li></ul><p><strong>The key insight</strong>: Open-source DevOps tools provide incredible velocity and developer experience, but <strong>enterprise adoption is not a copy-paste exercise</strong>. Successful implementation requires understanding your organization’s specific constraints and adapting accordingly.</p><hr><h2 id="Key-Takeaways-and-Best-Practices"><a href="#Key-Takeaways-and-Best-Practices" class="headerlink" title="Key Takeaways and Best Practices"></a>Key Takeaways and Best Practices</h2><h3 id="1-Use-Service-Containers-for-Dependencies"><a href="#1-Use-Service-Containers-for-Dependencies" class="headerlink" title="1. Use Service Containers for Dependencies"></a>1. <strong>Use Service Containers for Dependencies</strong></h3><p>GitHub Actions service containers provide isolated, ephemeral infrastructure perfect for testing AWS services with LocalStack.</p><h3 id="2-Implement-Robust-Health-Checks"><a href="#2-Implement-Robust-Health-Checks" class="headerlink" title="2. Implement Robust Health Checks"></a>2. <strong>Implement Robust Health Checks</strong></h3><p>Never assume services are ready immediately. Implement polling with timeouts and diagnostic logging.</p><h3 id="3-Optimize-Disk-Space-Proactively"><a href="#3-Optimize-Disk-Space-Proactively" class="headerlink" title="3. Optimize Disk Space Proactively"></a>3. <strong>Optimize Disk Space Proactively</strong></h3><p>Large Docker images require aggressive cleanup on GitHub runners. Monitor disk usage and clean early.</p><h3 id="4-Leverage-Registry-Caching"><a href="#4-Leverage-Registry-Caching" class="headerlink" title="4. Leverage Registry Caching"></a>4. <strong>Leverage Registry Caching</strong></h3><p>Docker layer caching can reduce build times by 80-90%. Always configure <code>cache-from</code> and <code>cache-to</code>.</p><h3 id="5-Test-in-Production-Like-Environments"><a href="#5-Test-in-Production-Like-Environments" class="headerlink" title="5. Test in Production-Like Environments"></a>5. <strong>Test in Production-Like Environments</strong></h3><p>Running tests inside the actual Glue container eliminates environment discrepancies.</p><h3 id="6-Integrate-Multiple-Quality-Tools"><a href="#6-Integrate-Multiple-Quality-Tools" class="headerlink" title="6. Integrate Multiple Quality Tools"></a>6. <strong>Integrate Multiple Quality Tools</strong></h3><p>Combine SonarCloud, Codecov, and GitHub Artifacts for comprehensive quality visibility.</p><h3 id="7-Make-Workflows-Idempotent"><a href="#7-Make-Workflows-Idempotent" class="headerlink" title="7. Make Workflows Idempotent"></a>7. <strong>Make Workflows Idempotent</strong></h3><p>Use <code>|| true</code> for resource creation commands that might already exist (especially in LocalStack).</p><h3 id="8-Separate-Concerns-with-Volume-Mounts"><a href="#8-Separate-Concerns-with-Volume-Mounts" class="headerlink" title="8. Separate Concerns with Volume Mounts"></a>8. <strong>Separate Concerns with Volume Mounts</strong></h3><p>Mount only necessary directories into containers to maintain clean separation and faster builds.</p><h3 id="9-Assess-Organizational-Constraints-Early"><a href="#9-Assess-Organizational-Constraints-Early" class="headerlink" title="9. Assess Organizational Constraints Early"></a>9. <strong>Assess Organizational Constraints Early</strong></h3><p>Evaluate security, compliance, and risk requirements before committing to open-source tooling.</p><h3 id="10-Plan-for-Graduated-Adoption"><a href="#10-Plan-for-Graduated-Adoption" class="headerlink" title="10. Plan for Graduated Adoption"></a>10. <strong>Plan for Graduated Adoption</strong></h3><p>Start with POCs, then progressively harden for enterprise requirements.</p><h2 id="Real-World-Benefits"><a href="#Real-World-Benefits" class="headerlink" title="Real-World Benefits"></a>Real-World Benefits</h2><p>Teams using this approach report:</p><ul><li><strong>90% reduction</strong> in AWS testing costs</li><li><strong>5x faster</strong> feedback loops (minutes vs. hours)</li><li><strong>Zero production incidents</strong> from environment mismatches</li><li><strong>Improved developer experience</strong> with local-first workflows</li></ul><p><strong>However</strong>, enterprise teams also report:</p><ul><li><strong>3-6 months</strong> to adapt for compliance requirements</li><li><strong>Significant investment</strong> in self-hosted infrastructure</li><li><strong>Ongoing maintenance</strong> burden for custom solutions</li></ul><h2 id="Advanced-Extensions"><a href="#Advanced-Extensions" class="headerlink" title="Advanced Extensions"></a>Advanced Extensions</h2><p>Consider these enhancements for production use:</p><ol><li><strong>Matrix Testing</strong>: Test against multiple Python&#x2F;Spark versions</li><li><strong>Parallel Execution</strong>: Split tests across multiple jobs</li><li><strong>Deployment Gates</strong>: Require quality thresholds before merging</li><li><strong>Terraform Integration</strong>: Validate infrastructure code in CI</li><li><strong>Performance Benchmarking</strong>: Track job execution times over commits</li><li><strong>Multi-Architecture Builds</strong>: Support ARM64 for Apple Silicon developers</li><li><strong>Policy-as-Code</strong>: Implement OPA for compliance enforcement</li><li><strong>SBOM Generation</strong>: Create Software Bill of Materials for supply chain security</li></ol><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>Modern CI&#x2F;CD pipelines should be fast, reliable, and cost-effective. By combining GitHub Actions, Docker, and LocalStack, you can build sophisticated testing workflows that rival enterprise CI systems—all within GitHub’s free tier for public repositories.</p><p><strong>The critical caveat</strong>: This open-source stack is a starting point, not a universal solution. <strong>Successful enterprise adoption requires understanding your organization’s security posture, compliance obligations, and risk tolerance.</strong> What works for a startup may be completely inappropriate for a regulated financial institution.</p><p>The workflows I demonstrated here provides a foundation for data engineering teams to iterate quickly, catch issues early, and deploy with confidence. Whether you’re building AWS Glue jobs, Lambda functions, or any cloud-native application, these patterns apply universally—<strong>but always through the lens of your organizational constraints</strong>.</p><p><strong>Remember</strong>: The best DevOps pipeline is one that balances developer velocity with organizational risk management. Don’t let perfect security become the enemy of good engineering, but equally, don’t let engineering convenience compromise critical compliance requirements.</p><p>In my next post, I will explore the items not in scope for this guide, including Terraform testing, deployment strategies, and advanced security hardening for enterprise compliance. One of them is about configuration of OIDC with GitHub Actions for secure, token-based authentication to AWS resources without long-lived credentials. This is important because enterprises are increasingly adopting OIDC to enhance security posture and reduce risk of credential leakage since short-lived tokens are used instead of static secrets.</p><p>Thanks for reading. Till then, Happy Coding!</p><p>For more information on my CI&#x2F;CD setup guide, please refer to the links below.</p><h2 id="Resources"><a href="#Resources" class="headerlink" title="Resources"></a>Resources</h2><ul><li><strong>Repository</strong>: <a href="https://github.com/awongCM/docker_glue_pyspark_demo">docker_glue_pyspark_demo</a></li><li><strong>Workflow File</strong>: <a href="https://github.com/awongCM/docker_glue_pyspark_demo/blob/main/.github/workflows/ci-pipeline.yml">ci-pipeline.yml</a></li><li><strong>Workflow File</strong>: <a href="https://github.com/awongCM/docker_glue_pyspark_demo/blob/main/.github/workflows/ci-sequential.yml">ci-sequential.yml</a></li><li><strong>LocalStack Documentation</strong>: <a href="https://docs.localstack.cloud/">docs.localstack.cloud</a></li><li><strong>GitHub Actions</strong>: <a href="https://docs.github.com/actions">docs.github.com&#x2F;actions</a></li><li><strong>Docker Buildx</strong>: <a href="https://docs.docker.com/reference/cli/docker/buildx/">docs.docker.com&#x2F;buildx</a></li><li><strong>GitHub Security Best Practices</strong>: <a href="https://docs.github.com/actions/security-guides">docs.github.com&#x2F;actions&#x2F;security-guides</a></li><li><strong>OWASP CI&#x2F;CD Security</strong>: <a href="https://owasp.org/www-project-devsecops-guideline">owasp.org&#x2F;www-project-devsecops-guideline</a></li></ul><hr><p><strong>Disclaimer</strong>: Again, I stressed this is only an exploratory exercise and proof-of-concept for learning purposes as well as sharing the knowledge with the community. It’s not by far the best and only approach to build CI&#x2F;CD pipelines for AWS Glue jobs. Always evaluate your organization’s specific needs and constraints before adopting new technologies. This guide reflects technical patterns and does not constitute legal or compliance advice. Always consult your organization’s security, legal, and compliance teams before adopting new tooling.</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_building_github_actions_localstack_unit_tests.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;Since my last post on setting local AWS Glue using Docker, one of the potential extensions I mentioned is to setup Github Actions CI&amp;#x2F;CD pipeline for it. In modern data engineering, testing cloud-native applications locally before deployment is very crucial for rapid iteration and cost efficiency. Thus, as part of this exploratory exercise, I want my CI&amp;#x2F;CD pipeline to achieve the following outcomes(in scope):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Automate testing of AWS Glue jobs in a local environment&lt;/li&gt;
&lt;li&gt;Use Docker to containerize the Glue runtime&lt;/li&gt;
&lt;li&gt;Leverage LocalStack to simulate AWS services locally&lt;/li&gt;
&lt;li&gt;Integrate with GitHub Actions for continuous integration and delivery&lt;/li&gt;
&lt;li&gt;Ensure code quality with SonarCloud and Codecov&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Not in scope for this pipeline:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Terraform or infrastructure-as-code testing&lt;/li&gt;
&lt;li&gt;Deployment to actual AWS environments&lt;/li&gt;
&lt;li&gt;Advanced security hardening for enterprise compliance&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This guide walks you through building very robust (if not production-ready) CI&amp;#x2F;CD pipeline using GitHub Actions that integrates Docker containerization, LocalStack for AWS service emulation, and comprehensive testing for AWS Glue jobs.&lt;/p&gt;</summary>
    
    
    
    
    <category term="ci-cd, github-actions, continuous-integration, continuous-delivery, aws, localstack, aws-glue, docker, integration-testing, unit-testing, sonarquloud" scheme="http://awongcm.io/tags/ci-cd-github-actions-continuous-integration-continuous-delivery-aws-localstack-aws-glue-docker-integration-testing-unit-testing-sonarquloud/"/>
    
  </entry>
  
  <entry>
    <title>The Future of the AI Era for Developers: What 2026 Really Means for Software Builders</title>
    <link href="http://awongcm.io/blog/2026/01/04/the-future-of-the-ai-era-for-developers-what-2026-really-means-for-software-builders/"/>
    <id>http://awongcm.io/blog/2026/01/04/the-future-of-the-ai-era-for-developers-what-2026-really-means-for-software-builders/</id>
    <published>2026-01-04T02:01:51.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_the_furture_of_ai_era_for_developers.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>First off.</p><p>Happy New Year to all! Every engineer of various specialisations (ie front-end, back-end, data, devops, cloud, mobile, etc) would come to realise by now how much AI has heavily shaped and redefined our ways of engineering practices since ChatGPT made major headlines back in late 2022.</p><p>More than 3 years on with plethora of evolving AI tools at engineers’ disposals , I arrive with this conclusion.</p><blockquote><p><em>“AI didn’t replace my love for software engineering — it gave it back to me.”</em></p></blockquote><p>In 2016 (and beyond), being a software developer or engineer no longer means fighting every line of code alone.</p><p>It means having an intelligent collaborator that helps you think, build, refactor, document, test, and ship — faster and with more confidence than ever before.</p><p>For me personally, this shift became real the moment I subscribed to <strong>GitHub Copilot Pro</strong> for my own pet projects.</p><p>I reopened application repositories that I hadn’t touched in years, especially starting off with this one as an <a href="https://github.com/awongCM/spaceinvadersdemo">example</a>. It’s a classic space invaders game written purely in Vanilla JS.</p><p>When looking into this repo for the first time, it came with my immediate reactions.</p><ul><li>Old ideas.</li><li>Half-finished tools.</li><li>Abandoned experiments.</li></ul><p>And suddenly.</p><p>Using AI — instead of feeling overwhelmed — I felt <strong>energized</strong>.</p><span id="more"></span><p>With AI assisting me, I wasn’t crawling through feature development anymore.<br>I was moving at <strong>100× or even 1000× the velocity</strong> I had before.</p><p>That feeling — breaking through the wall of hesitation, complexity, and unfinished code — is what the AI era truly brings to developers.</p><p>This article is about <strong>what AI really means for software builders</strong>: not a replacement, but rather a renaissance.</p><hr><h2 id="The-Old-World-Software-Development-Was-Becoming-Emotionally-Expensive"><a href="#The-Old-World-Software-Development-Was-Becoming-Emotionally-Expensive" class="headerlink" title="The Old World: Software Development Was Becoming Emotionally Expensive"></a>The Old World: Software Development Was Becoming Emotionally Expensive</h2><p>Before AI copilots, writing software wasn’t just technically hard — it was <strong>emotionally heavy</strong>.</p><p>You probably know the feeling:</p><ul><li>You open a repo you wrote two or XXX years ago</li><li>You don’t remember the design</li><li>The frameworks have changed</li><li>The tests are broken</li><li>The motivation is gone</li></ul><p>Starting again feels like climbing a mountain with no trail.</p><p>So projects stayed abandoned.<br>Ideas stayed unfinished.<br>And a lot of creative engineering energy was wasted.</p><p>AI changes that.</p><hr><h2 id="The-New-World-Developers-Now-Have-Cognitive-Exoskeletons"><a href="#The-New-World-Developers-Now-Have-Cognitive-Exoskeletons" class="headerlink" title="The New World: Developers Now Have Cognitive Exoskeletons"></a>The New World: Developers Now Have Cognitive Exoskeletons</h2><p>In 2026, developers don’t just write code — they <strong>co-create</strong> with AI.</p><p>Tools like:</p><ul><li>GitHub Copilot</li><li>Cursor</li><li>WindSurf</li><li>TabbyML</li><li>ChatGPT</li><li>Claude</li><li>Local LLMs</li></ul><p>act like <strong>cognitive exoskeletons</strong>.</p><p>They don’t replace your brain — they amplify it.</p><p>They:</p><ul><li>Remind you of syntax</li><li>Generate boilerplate</li><li>Infer missing logic</li><li>Suggest architectures</li><li>Explain legacy code</li><li>Write tests</li><li>Generate documentation</li></ul><p>Suddenly, you spend more time <strong>thinking</strong> and less time <strong>fighting tools</strong>.</p><p>That’s exactly what I experienced when revisiting my personal projects.</p><p>I could focus on:</p><blockquote><p>“What should this app become?”</p></blockquote><p>instead of</p><blockquote><p>“How do I wire this stupid API again?”</p></blockquote><hr><h2 id="Why-This-Changes-Everything-in-2026"><a href="#Why-This-Changes-Everything-in-2026" class="headerlink" title="Why This Changes Everything in 2026"></a>Why This Changes Everything in 2026</h2><p>The biggest change in 2026 is not just faster coding.</p><p>It is <strong>lower emotional friction</strong>.</p><p>Developers are now willing to:</p><ul><li>Revive old projects</li><li>Learn new languages</li><li>Try new frameworks</li><li>Build ambitious ideas</li></ul><p>Because AI removes the “activation energy” required to start.</p><p>Just like electric motors replaced hand cranks,<br>AI replaces the painful first 80% of development.</p><hr><h2 id="AI-Is-Not-Replacing-Developers-—-It-Is-Upgrading-Them"><a href="#AI-Is-Not-Replacing-Developers-—-It-Is-Upgrading-Them" class="headerlink" title="AI Is Not Replacing Developers — It Is Upgrading Them"></a>AI Is Not Replacing Developers — It Is Upgrading Them</h2><p>The biggest myth is that AI will replace programmers.</p><p>What it really replaces is:</p><ul><li>Typing boilerplate</li><li>Searching StackOverflow</li><li>Copy-pasting docs</li><li>Debugging trivial mistakes</li></ul><p>What remains is:</p><ul><li>System design</li><li>Architecture</li><li>Business logic</li><li>Data modeling</li><li>Security</li><li>Performance</li><li>Ethics</li><li>Product thinking</li></ul><p>AI is not making developers irrelevant.<br>It is making <strong>great developers unstoppable</strong>.</p><p>This is applicable for all regardless of their tenure in the game of software development,include junior developers, which I explain more in later</p><hr><h2 id="From-Code-Writers-to-System-Designers"><a href="#From-Code-Writers-to-System-Designers" class="headerlink" title="From Code Writers to System Designers"></a>From Code Writers to System Designers</h2><p>In 2026, the job of a developer looks more like:</p><blockquote><p>Designing, guiding, and orchestrating intelligent systems</p></blockquote><p>rather than:</p><blockquote><p>Manually typing every line</p></blockquote><p>We are becoming:</p><ul><li>Architects</li><li>Curators</li><li>Reviewers</li><li>System thinkers</li></ul><p>AI produces code.<br>Developers decide <strong>what code should exist</strong>.</p><hr><h2 id="Why-This-Feels-So-Motivating"><a href="#Why-This-Feels-So-Motivating" class="headerlink" title="Why This Feels So Motivating"></a>Why This Feels So Motivating</h2><p>This is why my Copilot experience was so powerful.</p><p>It removed:</p><ul><li>The guilt of unfinished projects</li><li>The fear of restarting</li><li>The pain of remembering old code</li></ul><p>And replaced it with:</p><ul><li>Momentum</li><li>Flow</li><li>Curiosity</li><li>Creative joy</li></ul><p>That feeling — <strong>wanting to build again</strong> — is the real revolution.</p><p>Not automation.<br>Not speed.<br>But <strong>renewed love for engineering</strong>.</p><hr><h2 id="The-Developers-Who-Will-Thrive-in-2026"><a href="#The-Developers-Who-Will-Thrive-in-2026" class="headerlink" title="The Developers Who Will Thrive in 2026"></a>The Developers Who Will Thrive in 2026</h2><p>The winners of the AI era won’t be the ones who type fastest.</p><p>They will be the ones who:</p><ul><li>Ask better questions</li><li>Design better systems</li><li>Understand data</li><li>Know how to structure problems</li><li>Can evaluate AI output critically</li></ul><p>AI raises the floor.<br>But it also raises the ceiling.</p><hr><h2 id="For-the-Aspiring-Junior-Developers-in-the-AI-Era"><a href="#For-the-Aspiring-Junior-Developers-in-the-AI-Era" class="headerlink" title="For the Aspiring Junior Developers in the AI Era"></a>For the Aspiring Junior Developers in the AI Era</h2><p>If you’re just starting your software development journey in 2026, you’re not at risk of being replaced.</p><p>You’re at the <strong>greatest advantage in the history of programming</strong>.</p><p>Here’s why:</p><p>The old guard learned to code the hard way — grinding through syntax errors, manually reading documentation, spending hours debugging obscure issues, and building everything from scratch.</p><p>They earned their 10,000 hours through <strong>sheer friction</strong>.</p><p>You get to earn yours through <strong>deliberate practice at 100× speed</strong>.</p><hr><h3 id="AI-Amplifies-Learning-Not-Just-Coding"><a href="#AI-Amplifies-Learning-Not-Just-Coding" class="headerlink" title="AI Amplifies Learning, Not Just Coding"></a>AI Amplifies Learning, Not Just Coding</h3><p>When you’re learning with AI as your copilot, you’re not just writing code faster.</p><p>You’re:</p><ul><li>Getting instant feedback on your mistakes</li><li>Seeing multiple approaches to the same problem</li><li>Understanding patterns in real-time</li><li>Building more projects in less time</li><li>Experimenting without fear of getting stuck</li></ul><p>Every iteration you complete with AI is a <strong>high-quality learning loop</strong>.</p><p>The seniors spent weeks building their first CRUD app.<br>You can build five in a weekend — and actually understand what you’re doing.</p><hr><h3 id="Quality-Over-Grinding"><a href="#Quality-Over-Grinding" class="headerlink" title="Quality Over Grinding"></a>Quality Over Grinding</h3><p>The 10,000-hour rule still applies.</p><p>But in the AI era, those hours are <strong>more intentional, more focused, and more productive</strong> than ever before.</p><p>You’re not wasting time on:</p><ul><li>Typos</li><li>Missing semicolons</li><li>Forgotten imports</li><li>Boilerplate setup</li></ul><p>You’re spending time on:</p><ul><li>System design</li><li>Problem-solving</li><li>Architecture decisions</li><li>Code quality</li><li>Real-world patterns</li></ul><p>That’s the kind of practice the old guard never had access to.</p><hr><h3 id="You’re-Not-Cheating-—-You’re-Learning-Smarter"><a href="#You’re-Not-Cheating-—-You’re-Learning-Smarter" class="headerlink" title="You’re Not Cheating — You’re Learning Smarter"></a>You’re Not Cheating — You’re Learning Smarter</h3><p>Some will say using AI is “cheating” or that you won’t learn the fundamentals.</p><p>Ignore them.</p><p>Instead opt to have an open and growth mindset.</p><p>Using AI to learn is like using a calculator to learn math.<br>It doesn’t replace understanding — it <strong>accelerates it</strong>.</p><p>You still need to know:</p><ul><li>Why the code works</li><li>When to use which pattern</li><li>How to debug when AI gets it wrong</li><li>What good architecture looks like</li></ul><p>But you get to those insights <strong>faster and with more confidence</strong>.</p><hr><h3 id="The-Junior-Developer-Advantage"><a href="#The-Junior-Developer-Advantage" class="headerlink" title="The Junior Developer Advantage"></a>The Junior Developer Advantage</h3><p>Here’s the truth that experienced developers won’t tell you:</p><p><strong>You have less to unlearn.</strong></p><p>You don’t have muscle memory fighting against AI suggestions.<br>You don’t have ego tied to “doing it the hard way.”<br>You don’t have decades of habits built in a pre-AI world.</p><p>You get to build your entire development practice <strong>natively with AI</strong>.</p><p>That’s not a disadvantage.<br>That’s a <strong>superpower</strong>.</p><hr><h3 id="Embrace-It-Fully"><a href="#Embrace-It-Fully" class="headerlink" title="Embrace It Fully"></a>Embrace It Fully</h3><p>If you’re a junior developer in 2026:</p><ol><li><strong>Use AI without guilt</strong> — it’s a tool, not a crutch</li><li><strong>Build relentlessly</strong> — ship more projects than any previous generation could</li><li><strong>Ask better questions</strong> — AI rewards curiosity and precision</li><li><strong>Learn fundamentals faster</strong> — use AI to explore concepts deeply, not just copy code</li><li><strong>Stay hungry</strong> — the ceiling is higher than ever before</li></ol><p>The developers who started in 2026 with AI will be <strong>unstoppable by 2030</strong>.</p><p>Not because they skipped the hard work.</p><p>But because they did <strong>higher-quality work, faster, with more creativity</strong> than any generation before them.</p><p>That’s the real future of software development that lies ahead for them.</p><p>But.</p><p>What about the senior developers?</p><hr><h2 id="For-Senior-Developers-Your-Experience-Just-Became-Your-Superpower"><a href="#For-Senior-Developers-Your-Experience-Just-Became-Your-Superpower" class="headerlink" title="For Senior Developers: Your Experience Just Became Your Superpower"></a>For Senior Developers: Your Experience Just Became Your Superpower</h2><p>If you’ve been building software for 10, 15, or 20+ years, AI doesn’t diminish your expertise.</p><p>It <strong>multiplies it exponentially</strong>.</p><p>Here’s what changes for you in 2026:</p><hr><h3 id="You-Finally-Have-Leverage"><a href="#You-Finally-Have-Leverage" class="headerlink" title="You Finally Have Leverage"></a>You Finally Have Leverage</h3><p>You’ve spent years accumulating knowledge:</p><ul><li>Design patterns</li><li>System architecture</li><li>Performance optimization</li><li>Security best practices</li><li>Team dynamics</li><li>Technical debt management</li></ul><p>But until now, that knowledge was bottlenecked by <strong>your typing speed and available hours</strong>.</p><p>AI removes that bottleneck.</p><p>Now your expertise can flow directly into:</p><ul><li>Multiple codebases simultaneously</li><li>Rapid prototyping of complex systems</li><li>Exploring architectural alternatives in minutes</li><li>Refactoring legacy systems you’ve been dreading</li></ul><p>You’re no longer constrained by implementation time.<br>You’re <strong>only constrained by your vision</strong>.</p><hr><h3 id="From-Implementer-to-Architect-at-Scale"><a href="#From-Implementer-to-Architect-at-Scale" class="headerlink" title="From Implementer to Architect-at-Scale"></a>From Implementer to Architect-at-Scale</h3><p>As a senior developer, you know <strong>what good looks like</strong>.</p><p>You can:</p><ul><li>Spot code smells instantly</li><li>Recognize anti-patterns</li><li>Predict where systems will break</li><li>Design for scale, security, and maintainability</li></ul><p>AI can generate code.<br>But it can’t replace <strong>20 years of battle scars and wisdom</strong>.</p><p>Your role becomes:</p><ul><li>Setting the architectural direction</li><li>Reviewing and refining AI output</li><li>Making high-level design decisions</li><li>Mentoring teams on best practices</li><li>Ensuring quality and consistency</li></ul><p>You’re not writing every line anymore.<br>You’re <strong>orchestrating systems at a level that was previously impossible</strong>.</p><hr><h3 id="The-Side-Projects-You-Never-Had-Time-For"><a href="#The-Side-Projects-You-Never-Had-Time-For" class="headerlink" title="The Side Projects You Never Had Time For"></a>The Side Projects You Never Had Time For</h3><p>Remember all those ideas you shelved because you didn’t have time?</p><p>The open-source library you wanted to build.<br>The SaaS product you sketched on a napkin.<br>The framework you knew the world needed.</p><p>AI gives you back that time.</p><p>As a senior developer with AI:</p><ul><li>You can build in weekends what used to take months</li><li>You can maintain multiple projects without burning out</li><li>You can explore new languages and frameworks fearlessly</li><li>You can finally ship the ideas that have been waiting in your head</li></ul><p>Your constraint is no longer <strong>time or energy</strong>.<br>It’s <strong>choosing which ideas to pursue</strong>.</p><hr><h3 id="Learning-New-Stacks-Without-Starting-Over"><a href="#Learning-New-Stacks-Without-Starting-Over" class="headerlink" title="Learning New Stacks Without Starting Over"></a>Learning New Stacks Without Starting Over</h3><p>You’ve mastered your stack.</p><p>But what about:</p><ul><li>That new framework everyone’s talking about?</li><li>The language you’ve been curious about?</li><li>The cloud platform you haven’t touched?</li></ul><p>In the old world, learning meant:</p><ul><li>Reading docs for hours</li><li>Fighting setup issues</li><li>Writing hello-world apps</li><li>Slowly building muscle memory</li></ul><p>With AI, you can:</p><ul><li>Jump into a new ecosystem immediately</li><li>Leverage your existing knowledge while AI handles syntax</li><li>Build real projects from day one</li><li>Focus on concepts, not memorization</li></ul><p>Your <strong>transferable knowledge</strong> becomes even more valuable.<br>AI handles the surface-level differences.</p><hr><h3 id="Mentorship-at-Scale"><a href="#Mentorship-at-Scale" class="headerlink" title="Mentorship at Scale"></a>Mentorship at Scale</h3><p>As a senior developer, you’ve probably mentored juniors.</p><p>But you could only help:</p><ul><li>One person at a time</li><li>During work hours</li><li>On problems you directly encountered</li></ul><p>With AI, your mentorship scales:</p><ul><li>Document your architectural decisions — AI helps juniors understand them</li><li>Write high-level specs — AI helps teams implement them</li><li>Create templates and patterns — AI propagates them across codebases</li><li>Review code faster — AI flags issues, you provide wisdom</li></ul><p>You’re not just teaching individuals.<br>You’re <strong>encoding your expertise into systems</strong>.</p><hr><h3 id="The-Competitive-Edge"><a href="#The-Competitive-Edge" class="headerlink" title="The Competitive Edge"></a>The Competitive Edge</h3><p>Here’s the reality:</p><p>A senior developer with AI is <strong>10× more productive</strong> than a senior developer without it.</p><p>And a senior developer with AI is <strong>100× more valuable</strong> than a junior with AI.</p><p>Because:</p><ul><li>Juniors know how to use the tool</li><li>Seniors know <strong>what to build and why</strong></li></ul><p>Your years of experience in:</p><ul><li>Navigating organizational complexity</li><li>Making trade-off decisions</li><li>Understanding business context</li><li>Debugging production disasters</li><li>Leading technical initiatives</li></ul><p>are <strong>irreplaceable</strong>.</p><p>AI amplifies your judgment.<br>It doesn’t replace it.</p><hr><h3 id="Don’t-Fight-It-—-Lead-With-It"><a href="#Don’t-Fight-It-—-Lead-With-It" class="headerlink" title="Don’t Fight It — Lead With It"></a>Don’t Fight It — Lead With It</h3><p>Some senior developers resist AI because:</p><ul><li>“I already know how to code”</li><li>“I don’t need help”</li><li>“This feels like cheating”</li></ul><p>But the developers who embrace AI in 2026 will:</p><ul><li>Ship more ambitious projects</li><li>Solve harder problems</li><li>Have more creative freedom</li><li>Stay relevant and energized</li></ul><p>The choice is simple:</p><p>Be a senior developer who types fast.<br>Or be a senior developer who <strong>builds the future</strong>.</p><hr><h3 id="Your-Best-Work-Is-Still-Ahead"><a href="#Your-Best-Work-Is-Still-Ahead" class="headerlink" title="Your Best Work Is Still Ahead"></a>Your Best Work Is Still Ahead</h3><p>If you’re a senior developer in 2026, this is your moment.</p><p>You have:</p><ul><li>The experience to know what matters</li><li>The judgment to evaluate AI output</li><li>The vision to design great systems</li><li>The tools to finally build at the speed of thought</li></ul><p>AI doesn’t make you obsolete.</p><p>It makes you <strong>unstoppable</strong>.</p><hr><h3 id="Before-We-Wrap-Up"><a href="#Before-We-Wrap-Up" class="headerlink" title="Before We Wrap Up"></a>Before We Wrap Up</h3><p>Remember what I said earlier about my space invaders project in this post?</p><p>I asked Copilot to revitalize it with modern tooling.</p><p>Here’ what it gave me in minutes:</p><p>From a legacy Vanilla JS app:</p><img src="/images/legacy-space-invaders.png" class="center" width="500" height="500" title="Legacy Vanilla JS Space Invaders Legacy Vanilla JS Space Invaders"><p><em><p style="text-align: center;">Legacy Vanilla JS Space Invaders</p></em></p><p>to a Vite + Typescript + TailwindCSS app:</p><img src="/images/modern-space-invaders.png" class="center" width="500" height="500" title="Vite + Typescript + TailwindCSS Space Invaders Vite + Typescript + TailwindCSS Space Invaders"><p><em><p style="text-align: center;">Vite + Typescript + TailwindCSS Space Invaders</p></em></p><p>Can’t spot the difference?</p><p>On the surface, it looks the same. But under the hood, you can see the implementation detail has been modernized with best sofware practices (thanks to Copilot) via my Github repo. One for <a href="https://github.com/awongCM/spaceinvadersdemo/tree/main/legacy">legacy</a>, and the other for <a href="https://github.com/awongCM/spaceinvadersdemo/tree/main/modern">modern</a>.</p><p>You can even check the actual working demo of two versions running side by side <a href="https://space-invaders-demo.onrender.com/">here</a>. It is also showcased on my portfolio page of this blog site <a href="https://www.awongcm.io/portfolio">here</a>.</p><p>For your curiosity, on my entire space invaders repo, you can check out on my <a href="https://github.com/awongCM/spaceinvadersdemo">GitHub repo here</a></p><p>With that, I can hardly wait to see what I can accomplish next with AI on my other 60 plus personal projects that I have shelved away for years. (Yes, I have that many side projects lying around… 😅)</p><hr><h2 id="Final-Thought"><a href="#Final-Thought" class="headerlink" title="Final Thought"></a>Final Thought</h2><p>The future of AI for developers is not about replacing us.</p><p>It’s about finally giving us the freedom to:</p><ul><li>Build what we imagine</li><li>Explore without fear</li><li>And fall back in love with software engineering</li></ul><p>2026 is not the end of coding.</p><p>It is the <strong>golden age of builders</strong>.</p><hr><p>Till then, I wish you a great start of Happy New Year of Coding ahead.</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_the_furture_of_ai_era_for_developers.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;First off.&lt;/p&gt;
&lt;p&gt;Happy New Year to all! Every engineer of various specialisations (ie front-end, back-end, data, devops, cloud, mobile, etc) would come to realise by now how much AI has heavily shaped and redefined our ways of engineering practices since ChatGPT made major headlines back in late 2022.&lt;/p&gt;
&lt;p&gt;More than 3 years on with plethora of evolving AI tools at engineers’ disposals , I arrive with this conclusion.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;“AI didn’t replace my love for software engineering — it gave it back to me.”&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;In 2016 (and beyond), being a software developer or engineer no longer means fighting every line of code alone.&lt;/p&gt;
&lt;p&gt;It means having an intelligent collaborator that helps you think, build, refactor, document, test, and ship — faster and with more confidence than ever before.&lt;/p&gt;
&lt;p&gt;For me personally, this shift became real the moment I subscribed to &lt;strong&gt;GitHub Copilot Pro&lt;/strong&gt; for my own pet projects.&lt;/p&gt;
&lt;p&gt;I reopened application repositories that I hadn’t touched in years, especially starting off with this one as an &lt;a href=&quot;https://github.com/awongCM/spaceinvadersdemo&quot;&gt;example&lt;/a&gt;. It’s a classic space invaders game written purely in Vanilla JS.&lt;/p&gt;
&lt;p&gt;When looking into this repo for the first time, it came with my immediate reactions.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Old ideas.&lt;/li&gt;
&lt;li&gt;Half-finished tools.&lt;/li&gt;
&lt;li&gt;Abandoned experiments.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;And suddenly.&lt;/p&gt;
&lt;p&gt;Using AI — instead of feeling overwhelmed — I felt &lt;strong&gt;energized&lt;/strong&gt;.&lt;/p&gt;</summary>
    
    
    
    
    <category term="ai, software development, developers, future tech, programming, copilot, github copilot" scheme="http://awongcm.io/tags/ai-software-development-developers-future-tech-programming-copilot-github-copilot/"/>
    
  </entry>
  
  <entry>
    <title>Beyond Prompt Engineering: From Code Coverage to Code Confidence, Master Unit Tests with GitHub Copilot Agent</title>
    <link href="http://awongcm.io/blog/2025/12/16/beyond-prompt-engineering-from-code-coverage-to-code-confidence-master-unit-tests-with-github-copilot-agent/"/>
    <id>http://awongcm.io/blog/2025/12/16/beyond-prompt-engineering-from-code-coverage-to-code-confidence-master-unit-tests-with-github-copilot-agent/</id>
    <published>2025-12-16T10:14:54.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_copilot_unit_test_coverage_code_confidence.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>As we bring 2025 to a close, software engineers like myself find themselves immersed in all things AI from every direction—left, right, and dead center: Medium blogs, AI forums like Rundown AI, AI meetups, etc.—all in a constant state of flux. The message couldn’t be clearer: ignoring AI is no longer an option. Those who resist embracing these tools risk falling drastically behind in an industry moving at lightning speed. The greatest folly isn’t experimenting with AI and making mistakes — it’s refusing to adapt at all.</p><p>But here’s the critical insight most engineers miss: <strong>it’s not about using AI; it’s about using AI systematically.</strong></p><span id="more"></span><p>Using one example, in my last post, writing comprehensive unit tests for data‑engineering pipelines isn’t just about achieving high coverage numbers—it’s about building confidence that your transformations work correctly under all conditions. GitHub Copilot Agent transforms this challenge from a tedious manual process into a systematic, AI‑assisted workflow that drives both coverage and quality.</p><h2 id="The-Challenge-Testing-AWS-Glue-Pipelines"><a href="#The-Challenge-Testing-AWS-Glue-Pipelines" class="headerlink" title="The Challenge: Testing AWS Glue Pipelines"></a>The Challenge: Testing AWS Glue Pipelines</h2><ol><li><strong>Environment Dependencies</strong> – Tests require <code>awsglue</code> modules only available in specific Docker containers.</li><li><strong>Complex Data Transformations</strong> – Bronze, Silver, and Gold layer pipelines contain intricate business logic.</li><li><strong>Integration Points</strong> – S3 interactions, DynamicFrames, Spark sessions, and AWS service mocking.</li><li><strong>Scale</strong> – Multiple pipelines (Plain and Purchase Order) across three transformation layers.</li></ol><p>Traditional approaches often result in inconsistent test quality, missing edge cases, poor coverage of critical paths, and unreliable refactoring.</p><h2 id="The-Solution-Structured-Copilot-Agent-Methodology"><a href="#The-Solution-Structured-Copilot-Agent-Methodology" class="headerlink" title="The Solution: Structured Copilot Agent Methodology"></a>The Solution: Structured Copilot Agent Methodology</h2><h3 id="1-Create-a-Reusable-Prompt-Template"><a href="#1-Create-a-Reusable-Prompt-Template" class="headerlink" title="1. Create a Reusable Prompt Template"></a>1. Create a Reusable Prompt Template</h3><p>Store this file as <strong><code>.github/prompts/unit-tester.prompt.md</code></strong>:</p><pre><code class="hljs markdown"><span class="hljs-section"># Unit Test Generation Guidelines (Python)</span>You are a unit test generator assistant for Python code.  Strictly follow these rules when generating tests:<span class="hljs-section">## Test Structure</span>Use the AAA (Arrange‑Act‑Assert) pattern for structuring tests:<span class="hljs-bullet">1.</span> <span class="hljs-strong">**Arrange**</span> – Set up test data, fixtures, and preconditions.<span class="hljs-bullet">2.</span> <span class="hljs-strong">**Act**</span> – Execute the function/method under test.<span class="hljs-bullet">3.</span> <span class="hljs-strong">**Assert**</span> – Verify results and side effects.<span class="hljs-section">## Naming Convention</span><span class="hljs-bullet">-</span> Name tests as <span class="hljs-code">`should_expected_behavior_when_state_under_test`</span>.<span class="hljs-bullet">-</span> Use clear, descriptive names that document the test&#x27;s purpose.<span class="hljs-bullet">-</span> For pytest, use snake<span class="hljs-emphasis">_case function names.</span><span class="hljs-emphasis"></span><span class="hljs-emphasis">## Best Practices</span><span class="hljs-emphasis"></span><span class="hljs-emphasis">1. Test one behavior per test.</span><span class="hljs-emphasis">2. Use meaningful, representative test data.</span><span class="hljs-emphasis">3. Handle expected exceptions using `pytest.raises`.</span><span class="hljs-emphasis">4. Add comments and a docstring per test.</span><span class="hljs-emphasis">5. Include negative tests and edge cases (empty strings, None, large values).</span><span class="hljs-emphasis">6. Prefer pure functions and deterministic outcomes for unit tests.</span><span class="hljs-emphasis">7. Use fixtures (`@pytest.fixture`) for shared setup instead of global state.</span><span class="hljs-emphasis">8. Mock external systems (I/O, network, AWS) using `pytest-mock`.</span><span class="hljs-emphasis">9. Add docstrings to test functions.</span><span class="hljs-emphasis">10. Add negative tests</span><span class="hljs-emphasis">11. Add integration tests for critical paths</span><span class="hljs-emphasis">12. Add edge cases</span><span class="hljs-emphasis"></span><span class="hljs-emphasis">## Additional Rules</span><span class="hljs-emphasis"></span><span class="hljs-emphasis">1. Revalidate and think step‑by‑step.</span><span class="hljs-emphasis">2. Always follow the AAA pattern.</span><span class="hljs-emphasis">3. Keep assertions focused and specific.</span><span class="hljs-emphasis">4. Isolate unit tests from environment and external services.</span><span class="hljs-emphasis">5. When testing PySpark/Glue code, prefer isolating transformation logic into small functions that can be tested without cluster dependencies; use a local SparkSession only when necessary.</span></code></pre><h3 id="2-Establish-Test-Infrastructure"><a href="#2-Establish-Test-Infrastructure" class="headerlink" title="2. Establish Test Infrastructure"></a>2. Establish Test Infrastructure</h3><p><strong>Docker Configuration (<code>docker-compose.yaml</code>)</strong></p><pre><code class="hljs yaml"><span class="hljs-attr">volumes:</span>  <span class="hljs-bullet">-</span> <span class="hljs-string">./tests:/app/tests</span>  <span class="hljs-bullet">-</span> <span class="hljs-string">./scripts:/app/scripts</span>  <span class="hljs-bullet">-</span> <span class="hljs-string">./pytest.ini:/app/pytest.ini</span>  <span class="hljs-bullet">-</span> <span class="hljs-string">./pyproject.toml:/app/pyproject.toml</span>  <span class="hljs-bullet">-</span> <span class="hljs-string">./poetry.lock:/app/poetry.lock</span></code></pre><p><strong>Test Runner Script (<code>scripts/run-tests-docker.bash</code>)</strong></p><pre><code class="hljs bash"><span class="hljs-meta">#!/usr/bin/env bash</span><span class="hljs-comment"># Usage: ./scripts/run-tests-docker.bash &lt;command&gt;</span><span class="hljs-comment"># Commands: all, plain, po, bronze, silver, gold, fast, coverage, shell, clean</span><span class="hljs-built_in">set</span> -eCMD=<span class="hljs-variable">$1</span><span class="hljs-keyword">case</span> <span class="hljs-variable">$CMD</span> <span class="hljs-keyword">in</span>  all)      poetry run pytest -v --cov=. --cov-report=html ;;  plain)    poetry run pytest -v tests/plain ;;  po)       poetry run pytest -v tests/purchase_order ;;  bronze)   poetry run pytest -v tests/*/test_bronze_job.py ;;  silver)   poetry run pytest -v tests/*/test_silver_job.py ;;  gold)     poetry run pytest -v tests/*/test_gold_job.py ;;  fast)     poetry run pytest -v --maxfail=1 ;;  coverage) poetry run pytest -v --cov=. --cov-report=html ;;  shell)    /bin/bash ;;  clean)    <span class="hljs-built_in">rm</span> -rf .pytest_cache htmlcov .coverage ;;  *) <span class="hljs-built_in">echo</span> <span class="hljs-string">&quot;Unknown command: <span class="hljs-variable">$CMD</span>&quot;</span> &amp;&amp; <span class="hljs-built_in">exit</span> 1 ;;<span class="hljs-keyword">esac</span></code></pre><p><strong>Pytest Configuration (<code>pytest.ini</code>)</strong></p><pre><code class="hljs ini"><span class="hljs-section">[pytest]</span><span class="hljs-attr">testpaths</span> = tests<span class="hljs-attr">python_files</span> = test_*.py<span class="hljs-attr">python_classes</span> = Test*<span class="hljs-attr">python_functions</span> = test_*<span class="hljs-attr">addopts</span> = -v --cov=. --cov-report=html --cov-report=term</code></pre><h3 id="3-Use-Copilot-Agent-with-Context"><a href="#3-Use-Copilot-Agent-with-Context" class="headerlink" title="3. Use Copilot Agent with Context"></a>3. Use Copilot Agent with Context</h3><p>When generating tests, provide Copilot Agent with:</p><ul><li><strong>Your Prompt Template</strong> (<code>@.github/prompts/unit-tester.prompt.md</code>)</li><li><strong>The Code to Test</strong> (e.g., <code>plain/bronze_job.py</code>, <code>purchase_order/bronze_job.py</code> etc)</li><li><strong>Shared Fixtures</strong> (<code>tests/conftest.py</code>)</li></ul><p><strong>Example Prompt</strong></p><pre><code class="hljs plaintext">@unit-tester.prompt.md Generate comprehensive unit tests for plain/bronze_job.pyfocusing on:- Data ingestion from S3- Schema validation- Error handling for malformed data- Logging functionality- Integration with GlueContext</code></pre><h3 id="4-Organize-Tests-by-Layer-and-Concern"><a href="#4-Organize-Tests-by-Layer-and-Concern" class="headerlink" title="4. Organize Tests by Layer and Concern"></a>4. Organize Tests by Layer and Concern</h3><pre><code class="hljs plaintext">tests/├── conftest.py                    # Shared fixtures├── plain/│   ├── test_bronze_job.py         # 4 test classes, ~12 tests│   ├── test_silver_job.py         # 4 test classes, ~15 tests│   └── test_gold_job.py           # 4 test classes, ~14 tests└── purchase_order/    ├── test_bronze_job.py         # 4 test classes, ~10 tests    ├── test_silver_job.py         # 4 test classes, ~17 tests    └── test_gold_job.py           # Similar structure</code></pre><h3 id="5-Example-AI‑Generated-Test-Class"><a href="#5-Example-AI‑Generated-Test-Class" class="headerlink" title="5. Example AI‑Generated Test Class"></a>5. Example AI‑Generated Test Class</h3><pre><code class="hljs python"><span class="hljs-keyword">class</span> <span class="hljs-title class_">TestBronzeJobDataIngestion</span>:    <span class="hljs-string">&quot;&quot;&quot;Test data ingestion functionality for bronze layer.&quot;&quot;&quot;</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">test_should_read_data_from_s3_when_valid_path_provided</span>(<span class="hljs-params"></span><span class="hljs-params">        self, spark_session, mocker</span><span class="hljs-params">    </span>):        <span class="hljs-string">&quot;&quot;&quot;Arrange‑Act‑Assert: Valid S3 path should successfully read data.&quot;&quot;&quot;</span>        <span class="hljs-comment"># Arrange</span>        mock_glue_context = mocker.Mock()        s3_path = <span class="hljs-string">&quot;s3://bucket/data/&quot;</span>        expected_count = <span class="hljs-number">100</span>        <span class="hljs-comment"># Mock the DynamicFrame creation</span>        mock_dynamic_frame = mocker.Mock()        mock_dynamic_frame.count.return_value = expected_count        mock_glue_context.create_dynamic_frame.from_options.return_value = (            mock_dynamic_frame        )        <span class="hljs-comment"># Act</span>        result = read_bronze_data(mock_glue_context, s3_path)        <span class="hljs-comment"># Assert</span>        <span class="hljs-keyword">assert</span> result.count() == expected_count        mock_glue_context.create_dynamic_frame.from_options.assert_called_once()    <span class="hljs-keyword">def</span> <span class="hljs-title function_">test_should_raise_error_when_s3_path_is_empty</span>(<span class="hljs-params">self, mocker</span>):        <span class="hljs-string">&quot;&quot;&quot;Arrange‑Act‑Assert: Empty S3 path should raise ValueError.&quot;&quot;&quot;</span>        <span class="hljs-comment"># Arrange</span>        mock_glue_context = mocker.Mock()        invalid_path = <span class="hljs-string">&quot;&quot;</span>        <span class="hljs-comment"># Act &amp; Assert</span>        <span class="hljs-keyword">with</span> pytest.raises(ValueError, <span class="hljs-keyword">match</span>=<span class="hljs-string">&quot;S3 path cannot be empty&quot;</span>):            read_bronze_data(mock_glue_context, invalid_path)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">test_should_handle_missing_files_when_path_not_found</span>(<span class="hljs-params"></span><span class="hljs-params">        self, spark_session, mocker</span><span class="hljs-params">    </span>):        <span class="hljs-string">&quot;&quot;&quot;Arrange‑Act‑Assert: Missing S3 path should raise FileNotFoundError.&quot;&quot;&quot;</span>        <span class="hljs-comment"># Arrange</span>        mock_glue_context = mocker.Mock()        mock_glue_context.create_dynamic_frame.from_options.side_effect = (            FileNotFoundError(<span class="hljs-string">&quot;Path not found&quot;</span>)        )        s3_path = <span class="hljs-string">&quot;s3://bucket/nonexistent/&quot;</span>        <span class="hljs-comment"># Act &amp; Assert</span>        <span class="hljs-keyword">with</span> pytest.raises(FileNotFoundError):            read_bronze_data(mock_glue_context, s3_path)</code></pre><h2 id="Real-Results-Coverage-and-Quality-Metrics"><a href="#Real-Results-Coverage-and-Quality-Metrics" class="headerlink" title="Real Results: Coverage and Quality Metrics"></a>Real Results: Coverage and Quality Metrics</h2><h3 id="Coverage-Achieved"><a href="#Coverage-Achieved" class="headerlink" title="Coverage Achieved"></a>Coverage Achieved</h3><table><thead><tr><th>Name</th><th>Stmts</th><th>Miss</th><th>Cover</th><th>Missing</th></tr></thead><tbody><tr><td>plain&#x2F;bronze_job.py</td><td>45</td><td>6</td><td>95%</td><td>105‑114, 135</td></tr><tr><td>plain&#x2F;silver_job.py</td><td>36</td><td>1</td><td>97%</td><td>105</td></tr><tr><td>plain&#x2F;gold_job.py</td><td>47</td><td>1</td><td>98%</td><td>165</td></tr><tr><td>purchase_order&#x2F;bronze_job.py</td><td>42</td><td>6</td><td>86%</td><td>149-158, 175</td></tr><tr><td>purchase_order&#x2F;silver_job.py</td><td>42</td><td>1</td><td>98%</td><td>166</td></tr><tr><td>purchase_order&#x2F;gold_job.py</td><td>44</td><td>1</td><td>98%</td><td>152</td></tr><tr><td><strong>TOTAL</strong></td><td>252</td><td>16</td><td>94%</td><td></td></tr></tbody></table><h3 id="Test-Suite-Statistics"><a href="#Test-Suite-Statistics" class="headerlink" title="Test Suite Statistics"></a>Test Suite Statistics</h3><ol><li><strong>Plain Pipeline</strong> – 41 tests across 3 layers</li><li><strong>Purchase Order Pipeline</strong> – 44 tests across 3 layers</li><li><strong>Total</strong> – 85+ unit tests (plus integration suite)</li><li><strong>Execution Time</strong> – under 30 seconds for the full suite</li></ol><h3 id="Quality-Improvements"><a href="#Quality-Improvements" class="headerlink" title="Quality Improvements"></a>Quality Improvements</h3><ul><li>Consistent AAA structure across all tests</li><li>Clear, descriptive naming (<code>should_*_when_*</code>)</li><li>Comprehensive edge‑case coverage (null values, empty data, malformed input)</li><li>Full mocking of external dependencies (S3, AWS services)</li><li>Detailed docstrings for every test</li></ul><h2 id="Key-Strategies-That-Worked"><a href="#Key-Strategies-That-Worked" class="headerlink" title="Key Strategies That Worked"></a>Key Strategies That Worked</h2><ol><li><strong>Prompt Engineering as Code</strong> – Storing the prompt template in version control guarantees uniform standards.</li><li><strong>Iterative Refinement</strong> – Initial generations are generic; refined prompts yield high‑quality tests.</li><li><strong>Context‑Aware Generation</strong> – Supplying fixtures, naming conventions, and business rules guides Copilot to produce relevant tests.</li><li><strong>Docker‑First Testing</strong> – Guarantees a reproducible environment where <code>awsglue</code> modules are available.</li><li><strong>Layered Test Strategy</strong> – Bronze focuses on ingestion, Silver on transformations, Gold on aggregations; integration tests validate end‑to‑end flow.</li></ol><h2 id="Practical-Workflow"><a href="#Practical-Workflow" class="headerlink" title="Practical Workflow"></a>Practical Workflow</h2><ol><li><strong>Define Your Standards</strong> – Create <code>.github/prompts/unit-tester.prompt.md</code>.</li><li><strong>Set Up Infrastructure</strong> – Docker volumes, pytest config, shared fixtures (<code>conftest.py</code>).</li><li><strong>Generate Tests with Copilot</strong> – Use the prompt template and explicit requirements.</li><li><strong>Review and Refine</strong> – Run tests, examine coverage, ask Copilot to fill gaps.</li><li><strong>Iterate on Uncovered Code</strong> – Target specific uncovered lines with new prompts.</li></ol><h2 id="Lessons-Learned"><a href="#Lessons-Learned" class="headerlink" title="Lessons Learned"></a>Lessons Learned</h2><ul><li>Structured prompts dramatically improve test consistency.</li><li>Docker eliminates environment‑related failures.</li><li>Incremental, layer‑by‑layer testing is more manageable than tackling the whole project at once.</li><li>Coverage reports guide focused test generation.</li></ul><h2 id="Advanced-Techniques"><a href="#Advanced-Techniques" class="headerlink" title="Advanced Techniques"></a>Advanced Techniques</h2><ol><li><strong>Parametrized Test Generation</strong> – Produce data‑driven tests with <code>@pytest.mark.parametrize</code>.</li><li><strong>Property‑Based Testing</strong> – Use <code>hypothesis</code> to validate invariants across arbitrary inputs.</li><li><strong>Integration Test Generation</strong> – Write end‑to‑end tests that run the full Bronze→Silver→Gold pipeline inside Docker.</li></ol><h2 id="Measuring-Success"><a href="#Measuring-Success" class="headerlink" title="Measuring Success"></a>Measuring Success</h2><table><thead><tr><th>Metric</th><th>Before Copilot</th><th>After Copilot</th></tr></thead><tbody><tr><td>Test Coverage</td><td>35 %</td><td>95 %</td></tr><tr><td>Time to Write Tests (per module)</td><td>2‑3 h</td><td>30‑45 min</td></tr><tr><td>Bugs Detected Pre‑Production</td><td>0 (undetected)</td><td>12 (caught)</td></tr><tr><td>Team Confidence</td><td>Low</td><td>High</td></tr><tr><td>ROI (time saved)</td><td>–</td><td>~60 % reduction</td></tr></tbody></table><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>GitHub Copilot Agent isn’t merely a code‑completion tool—it’s a disciplined partner that turns unit‑test generation into a repeatable, high‑quality process. By combining a clear prompt template, Docker‑first infrastructure, and systematic coverage analysis, you achieve real <strong>code confidence</strong>, not just high coverage numbers. The workflow scales from a single module to an entire data‑engineering codebase, delivering faster development cycles, fewer production bugs, and a stronger testing culture.</p><p>For more information about my actual copilot-driven unit and integration tests implementation, you can find it from my commits <a href="https://github.com/awongCM/docker_glue_pyspark_demo/commit/4c3f970955bba88c5e2705d05c70665730e46682">here</a>, or you can read more from my <a href="https://github.com/awongCM/docker_glue_pyspark_demo/blob/main/TEST_QUICK_REFERENCE.md">testing readme</a> doc as well.</p><hr><p><em>Till next time, Happy Coding, and wish you a happy and safe Xmas Holidays ahead🎄🎅🤶🧑‍🎄!</em></p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_copilot_unit_test_coverage_code_confidence.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;As we bring 2025 to a close, software engineers like myself find themselves immersed in all things AI from every direction—left, right, and dead center: Medium blogs, AI forums like Rundown AI, AI meetups, etc.—all in a constant state of flux. The message couldn’t be clearer: ignoring AI is no longer an option. Those who resist embracing these tools risk falling drastically behind in an industry moving at lightning speed. The greatest folly isn’t experimenting with AI and making mistakes — it’s refusing to adapt at all.&lt;/p&gt;
&lt;p&gt;But here’s the critical insight most engineers miss: &lt;strong&gt;it’s not about using AI; it’s about using AI systematically.&lt;/strong&gt;&lt;/p&gt;</summary>
    
    
    
    
    <category term="python pyspark unit-test code-coverage github copilot code-confidence" scheme="http://awongcm.io/tags/python-pyspark-unit-test-code-coverage-github-copilot-code-confidence/"/>
    
  </entry>
  
  <entry>
    <title>Setting up Local AWS Glue Development Environment with Docker</title>
    <link href="http://awongcm.io/blog/2025/11/22/setting-up-local-aws-glue-development-environment-with-docker/"/>
    <id>http://awongcm.io/blog/2025/11/22/setting-up-local-aws-glue-development-environment-with-docker/</id>
    <published>2025-11-22T02:09:38.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_data_engineering_local_setup_development.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>If you’re like me, you’ve probably experienced the frustration of developing any AWS applications in the cloud especially such as lambda, cloudfront, S3, etc using tools like SAM, CloudFormation etc.. Now that I’ve been delving into the world of data engineering, AWS Glue jobs is one of the main AWS data engineering tooling offerings, the frustrations in having to push Glue jobs directly in the cloud—waiting for job runs, nitty gritty AWS Glue binaries setup, dealing with slow feedback loops, and watching your AWS bill creep up with every test iteration. I’ve been there, and it’s not fun.</p><p>That’s why I built this local development environment. After experimenting with different approaches, I’ve put together a Docker-based setup that lets me develop and test Glue jobs on my laptop before deploying to AWS. The best part? It includes all the real-world components I actually use in actual cloud envs: Kafka for streaming data, Iceberg for my data lake tables, and LocalStack to simulate AWS services.</p><span id="more"></span><p>In this guide, I’ll walk you through my setup step-by-step. Whether you’re working on batch ETL jobs or real-time streaming pipelines, this environment will save you time, money, and a lot of headaches.</p><h2 id="What-We’re-Building"><a href="#What-We’re-Building" class="headerlink" title="What We’re Building"></a>What We’re Building</h2><p>Here’s what I’ve included in my local stack:</p><ol><li><strong>AWS Glue 4.0</strong> - The core ETL engine, just like in production</li><li><strong>JupyterLab</strong> - For interactive development and quick experiments</li><li><strong>Confluent Kafka</strong> - Because most of my pipelines involve streaming data</li><li><strong>Apache Iceberg</strong> - My go-to table format for ACID guarantees and time travel</li><li><strong>LocalStack</strong> - To mock S3 and DynamoDB without touching AWS</li></ol><h2 id="Before-You-Start"><a href="#Before-You-Start" class="headerlink" title="Before You Start"></a>Before You Start</h2><p>Make sure you have these installed on your machine:</p><ol><li>Docker Desktop (20.10+) with at least 8GB RAM allocated</li><li>Docker Compose v2.0+</li><li>Basic familiarity with AWS Glue and Spark</li></ol><h2 id="My-Project-Structure"><a href="#My-Project-Structure" class="headerlink" title="My Project Structure"></a>My Project Structure</h2><p>Here’s how I organize my demo projects:</p><pre><code class="hljs plaintext">docker-glue-pyspark-demo/├── docker-compose.yml├── Dockerfile├── .env├── scripts/│   ├── start-containers.sh│   └── shutdown-cointainers.sh├── plain/│   ├── bronze_job.py│   ├── silver_job.py│   └── gold_job.py├── terraform│   ├── main.tf│   └── providers.tf├── poetry.lock├── pyproject.toml│└── notebooks/</code></pre><h2 id="Step-1-Docker-Compose-Setup"><a href="#Step-1-Docker-Compose-Setup" class="headerlink" title="Step 1: Docker Compose Setup"></a>Step 1: Docker Compose Setup</h2><p>Create <code>docker-compose.yml</code>:</p><pre><code class="hljs yaml"><span class="hljs-attr">services:</span>  <span class="hljs-attr">glue-pyspark:</span>    <span class="hljs-attr">build:</span>      <span class="hljs-attr">context:</span> <span class="hljs-string">.</span>      <span class="hljs-attr">dockerfile:</span> <span class="hljs-string">Dockerfile</span>    <span class="hljs-attr">container_name:</span> <span class="hljs-string">glue-pyspark-poc</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">my-glue-pyspark</span>    <span class="hljs-attr">volumes:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">./plain:/app/plain</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">./purchase_order:/app/purchase_order</span>    <span class="hljs-attr">ports:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">&quot;18080:18080&quot;</span>    <span class="hljs-attr">environment:</span>      <span class="hljs-attr">AWS_ACCESS_KEY_ID:</span> <span class="hljs-string">&quot;test&quot;</span>      <span class="hljs-attr">AWS_SECRET_ACCESS_KEY:</span> <span class="hljs-string">&quot;test&quot;</span>      <span class="hljs-attr">AWS_REGION:</span> <span class="hljs-string">&quot;us-east-1&quot;</span>    <span class="hljs-attr">stdin_open:</span> <span class="hljs-literal">true</span> <span class="hljs-comment"># Keep STDIN open</span>    <span class="hljs-attr">tty:</span> <span class="hljs-literal">true</span>    <span class="hljs-attr">depends_on:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">kafka</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">iceberg</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">localstack</span>    <span class="hljs-attr">networks:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">localstack</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">kafka-net</span>  <span class="hljs-attr">jupyterlab:</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">my-glue-pyspark</span>    <span class="hljs-attr">container_name:</span> <span class="hljs-string">jupterlab-poc</span>    <span class="hljs-attr">environment:</span>      <span class="hljs-attr">AWS_ACCESS_KEY_ID:</span> <span class="hljs-string">&quot;test&quot;</span>      <span class="hljs-attr">AWS_SECRET_ACCESS_KEY:</span> <span class="hljs-string">&quot;test&quot;</span>      <span class="hljs-attr">AWS_REGION:</span> <span class="hljs-string">&quot;us-east-1&quot;</span>      <span class="hljs-attr">DISABLE_SSL:</span> <span class="hljs-literal">true</span>    <span class="hljs-attr">command:</span>      [        <span class="hljs-string">&quot;/home/glue_user/jupyter/jupyter_start.sh&quot;</span>,        <span class="hljs-string">&quot;--ip=0.0.0.0&quot;</span>,        <span class="hljs-string">&quot;--no-browser&quot;</span>,        <span class="hljs-string">&quot;--allow-root&quot;</span>,        <span class="hljs-string">&quot;--NotebookApp.token=test&quot;</span>,        <span class="hljs-string">&quot;--NotebookApp.allow_origin=*&quot;</span>,        <span class="hljs-string">&quot;--NotebookApp.base_url=http://localhost:8888&quot;</span>,        <span class="hljs-string">&quot;--NotebookApp.token_expire_in=36&quot;</span>,      ]    <span class="hljs-attr">ports:</span>      <span class="hljs-bullet">-</span> <span class="hljs-number">8888</span><span class="hljs-string">:8888</span>    <span class="hljs-attr">depends_on:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">glue-pyspark</span>    <span class="hljs-attr">volumes_from:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">glue-pyspark</span>    <span class="hljs-attr">networks:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">localstack</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">kafka-net</span>  <span class="hljs-attr">localstack:</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">localstack/localstack</span>    <span class="hljs-attr">environment:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">SERVICES=dynamodb,s3,iam</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">DEFAULT_REGION=us-east-1</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">AWS_ACCESS_KEY_ID=test</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">AWS_SECRET_ACCESS_KEY=test</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">DEBUG=1</span>    <span class="hljs-attr">ports:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">&quot;4566:4566&quot;</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">&quot;4510-4559:4510-4559&quot;</span>    <span class="hljs-attr">networks:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">localstack</span>  <span class="hljs-attr">kafka:</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">confluentinc/cp-kafka:7.4.0</span>    <span class="hljs-attr">ports:</span>      <span class="hljs-bullet">-</span> <span class="hljs-number">29092</span><span class="hljs-string">:29092</span>      <span class="hljs-bullet">-</span> <span class="hljs-number">9092</span><span class="hljs-string">:9092</span>    <span class="hljs-attr">environment:</span>      <span class="hljs-attr">KAFKA_BROKER_ID:</span> <span class="hljs-number">1</span>      <span class="hljs-attr">KAFKA_ZOOKEEPER_CONNECT:</span> <span class="hljs-string">zookeeper:2181</span>      <span class="hljs-attr">KAFKA_ADVERTISED_LISTENERS:</span> <span class="hljs-string">PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092</span>      <span class="hljs-attr">KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:</span> <span class="hljs-string">PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT</span>      <span class="hljs-attr">KAFKA_INTER_BROKER_LISTENER_NAME:</span> <span class="hljs-string">PLAINTEXT</span>      <span class="hljs-attr">KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR:</span> <span class="hljs-number">1</span>      <span class="hljs-attr">KAFKA_LISTENERS:</span> <span class="hljs-string">PLAINTEXT://0.0.0.0:29092,PLAINTEXT_HOST://0.0.0.0:9092</span>    <span class="hljs-attr">depends_on:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">zookeeper</span>    <span class="hljs-attr">networks:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">kafka-net</span>  <span class="hljs-attr">zookeeper:</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">confluentinc/cp-zookeeper:latest</span>    <span class="hljs-attr">ports:</span>      <span class="hljs-bullet">-</span> <span class="hljs-number">2181</span><span class="hljs-string">:2181</span>    <span class="hljs-attr">environment:</span>      <span class="hljs-attr">ZOOKEEPER_CLIENT_PORT:</span> <span class="hljs-number">2181</span>      <span class="hljs-attr">ZOOKEEPER_TICK_TIME:</span> <span class="hljs-number">2000</span>    <span class="hljs-attr">networks:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">kafka-net</span>  <span class="hljs-attr">iceberg:</span>    <span class="hljs-attr">image:</span> <span class="hljs-string">tabulario/spark-iceberg:latest</span>    <span class="hljs-attr">networks:</span>      <span class="hljs-bullet">-</span> <span class="hljs-string">kafka-net</span><span class="hljs-attr">networks:</span>  <span class="hljs-attr">localstack:</span>    <span class="hljs-attr">driver:</span> <span class="hljs-string">bridge</span>  <span class="hljs-attr">kafka-net:</span></code></pre><h2 id="Step-3-Custom-Glue-Dockerfile"><a href="#Step-3-Custom-Glue-Dockerfile" class="headerlink" title="Step 3: Custom Glue Dockerfile"></a>Step 3: Custom Glue Dockerfile</h2><p>Create <code>Dockerfile</code>:</p><pre><code class="hljs dockerfile"><span class="hljs-comment"># https://stackoverflow.com/a/74598849</span><span class="hljs-keyword">RUN</span><span class="language-bash"> <span class="hljs-built_in">rm</span> /home/glue_user/spark/conf/hive-site.xml</span><span class="hljs-comment"># Removed as they conflict with structured streaming jars below</span><span class="hljs-keyword">RUN</span><span class="language-bash"> <span class="hljs-built_in">rm</span> /home/glue_user/spark/jars/org.apache.commons_commons-pool2-2.6.2.jar</span><span class="hljs-keyword">RUN</span><span class="language-bash"> <span class="hljs-built_in">rm</span> /home/glue_user/spark/jars/org.apache.kafka_kafka-clients-2.6.0.jar</span><span class="hljs-keyword">RUN</span><span class="language-bash"> <span class="hljs-built_in">rm</span> /home/glue_user/spark/jars/org.spark-project.spark_unused-1.0.0.jar</span>.........<span class="hljs-comment"># Install additional dependencies</span><span class="hljs-comment"># Upgrade pip to the latest version</span><span class="hljs-keyword">RUN</span><span class="language-bash"> pip3 install --upgrade pip</span><span class="hljs-comment"># Install Jupyterlab</span><span class="hljs-keyword">RUN</span><span class="language-bash"> pip3 install jupyterlab</span><span class="hljs-comment"># Expose Jupyter port</span><span class="hljs-keyword">EXPOSE</span> <span class="hljs-number">8888</span><span class="hljs-comment"># Install Poetry for dependency management</span><span class="hljs-keyword">RUN</span><span class="language-bash"> pip3 install poetry==1.8.5</span><span class="hljs-keyword">COPY</span><span class="language-bash"> pyproject.toml poetry.lock* /app/</span><span class="hljs-keyword">COPY</span><span class="language-bash"> plain /app/plain</span><span class="hljs-comment"># Disable Poetry&#x27;s virtual environment creation, install project dependencies via poetry</span><span class="hljs-keyword">RUN</span><span class="language-bash"> poetry install</span><span class="hljs-comment"># Run the container - non-interactive</span><span class="hljs-keyword">ENTRYPOINT</span><span class="language-bash"> [ <span class="hljs-string">&quot;/bin/bash&quot;</span>, <span class="hljs-string">&quot;-l&quot;</span>, <span class="hljs-string">&quot;-c&quot;</span> ]</span><span class="hljs-keyword">CMD</span><span class="language-bash"> [<span class="hljs-string">&quot;/bin/bash&quot;</span>]</span></code></pre><h2 id="Step-4-Initialize-LocalStack-Resources"><a href="#Step-4-Initialize-LocalStack-Resources" class="headerlink" title="Step 4: Initialize LocalStack Resources"></a>Step 4: Initialize LocalStack Resources</h2><p>Navigate to terraform and run <code>tflocal apply --auto-approve</code>:</p><p>You notice the following locally emulated AWS services below:</p><pre><code class="hljs tf">resource &quot;aws_s3_bucket&quot; &quot;bronze-bucket&quot; &#123;  bucket = &quot;bronze-bucket&quot;  acl    = &quot;private&quot;&#125;resource &quot;aws_s3_bucket&quot; &quot;iceberg&quot; &#123;  bucket = &quot;iceberg&quot;  acl    = &quot;private&quot;&#125;resource &quot;aws_dynamodb_table&quot; &quot;dynamo_db_table_gold_table_plain&quot; &#123;  name         = &quot;gold_table_plain&quot;  billing_mode = &quot;PAY_PER_REQUEST&quot;  hash_key     = &quot;id&quot;  attribute &#123;    name = &quot;id&quot;    type = &quot;N&quot;  &#125;&#125;</code></pre><h2 id="Step-5-Generated-Kafka-Topics-Events-for-Data-ingestion"><a href="#Step-5-Generated-Kafka-Topics-Events-for-Data-ingestion" class="headerlink" title="Step 5: Generated Kafka Topics Events for Data ingestion"></a>Step 5: Generated Kafka Topics Events for Data ingestion</h2><p>Create <code>scripts/generate_plain_events.bash</code>:</p><pre><code class="hljs bash"><span class="hljs-meta">#!/bin/bash</span><span class="hljs-built_in">set</span> -ex  <span class="hljs-comment"># Exit on error, print commands</span><span class="hljs-comment"># Kafka settings</span>TOPIC=<span class="hljs-string">&quot;plain-topic&quot;</span>BOOTSTRAP_SERVERS=<span class="hljs-string">&quot;kafka:9092&quot;</span>CONTAINER_NAME=<span class="hljs-string">&quot;docker_glue_pyspark_demo-kafka-1&quot;</span>  <span class="hljs-comment"># Replace with your actual container name</span><span class="hljs-comment"># Function to generate a random JSON event</span><span class="hljs-function"><span class="hljs-title">generate_event</span></span>() &#123;  <span class="hljs-built_in">id</span>=$((RANDOM % <span class="hljs-number">100</span> + <span class="hljs-number">1</span>))  name=$(<span class="hljs-built_in">cat</span> /usr/share/dict/words | <span class="hljs-built_in">shuf</span> -n 1 | awk <span class="hljs-string">&#x27;&#123;print toupper(substr($0,1,1)) tolower(substr($0,2))&#125;&#x27;</span>)  amount=$((RANDOM % <span class="hljs-number">201</span>))  <span class="hljs-built_in">echo</span> <span class="hljs-string">&quot;&#123;\&quot;id\&quot;: <span class="hljs-variable">$id</span>, \&quot;name\&quot;: \&quot;<span class="hljs-variable">$name</span>\&quot;, \&quot;amount\&quot;: <span class="hljs-variable">$amount</span>&#125;&quot;</span>&#125;......<span class="hljs-comment"># Generate and send events for the specified number of iterations</span><span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> $(<span class="hljs-built_in">seq</span> 1 <span class="hljs-variable">$iterations</span>); <span class="hljs-keyword">do</span>  event_data=$(generate_event)  <span class="hljs-built_in">echo</span> <span class="hljs-string">&quot;Sending event: <span class="hljs-variable">$event_data</span>&quot;</span>  <span class="hljs-comment"># Execute the kafka-console-producer command inside the Docker container</span>  docker <span class="hljs-built_in">exec</span> -i <span class="hljs-string">&quot;<span class="hljs-variable">$CONTAINER_NAME</span>&quot;</span> \    kafka-console-producer --topic <span class="hljs-string">&quot;<span class="hljs-variable">$TOPIC</span>&quot;</span> --bootstrap-server <span class="hljs-string">&quot;<span class="hljs-variable">$BOOTSTRAP_SERVERS</span>&quot;</span> \    &lt;&lt;&lt; <span class="hljs-string">&quot;<span class="hljs-variable">$event_data</span>&quot;</span>  <span class="hljs-built_in">echo</span> <span class="hljs-string">&quot;Event sent (exit code: $?)&quot;</span>  <span class="hljs-built_in">sleep</span> 1  <span class="hljs-comment"># Adjust the sleep interval as needed</span><span class="hljs-keyword">done</span></code></pre><h2 id="Step-6-Kafka-Streaming-Job-Bronze-Layer"><a href="#Step-6-Kafka-Streaming-Job-Bronze-Layer" class="headerlink" title="Step 6: Kafka Streaming Job (Bronze Layer)"></a>Step 6: Kafka Streaming Job (Bronze Layer)</h2><p>Create <code>plain/bronze_job.py</code>:</p><pre><code class="hljs python"><span class="hljs-keyword">from</span> pyspark.sql <span class="hljs-keyword">import</span> SparkSession<span class="hljs-keyword">from</span> pyspark.sql.functions <span class="hljs-keyword">import</span> from_json, col<span class="hljs-keyword">from</span> pyspark.sql.types <span class="hljs-keyword">import</span> StructType, StructField, StringType, IntegerType<span class="hljs-keyword">import</span> boto3<span class="hljs-keyword">import</span> time<span class="hljs-keyword">import</span> jsonspark = SparkSession.builder \        .appName(<span class="hljs-string">&quot;Bronze Layer Job&quot;</span>) \        .master(<span class="hljs-string">&quot;local[*]&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.endpoint&quot;</span>, <span class="hljs-string">&quot;http://localstack:4566&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.access.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.secret.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.impl&quot;</span>, <span class="hljs-string">&quot;org.apache.hadoop.fs.s3a.S3AFileSystem&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.path.style.access&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.multiobjectdelete.enable&quot;</span>, <span class="hljs-string">&quot;false&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.fast.upload&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.fast.upload.buffer&quot;</span>, <span class="hljs-string">&quot;disk&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.endpoint.region&quot;</span>, <span class="hljs-string">&quot;us-east-1&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.aws.credentials.provider&quot;</span>, <span class="hljs-string">&quot;org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider&quot;</span>) \        .getOrCreate()<span class="hljs-comment"># # Set S3 endpoint to point to LocalStack</span>s3_endpoint_url = <span class="hljs-string">&quot;http://localstack:4566&quot;</span><span class="hljs-comment"># Kafka broker address and topic</span>kafka_bootstrap_servers = <span class="hljs-string">&quot;kafka:29092&quot;</span>kafka_topic = <span class="hljs-string">&quot;plain-topic&quot;</span>bucket_name = <span class="hljs-string">&quot;bronze-bucket&quot;</span>output_s3_bucket = <span class="hljs-string">f&#x27;s3a://<span class="hljs-subst">&#123;bucket_name&#125;</span>/plain&#x27;</span><span class="hljs-comment"># Read from Kafka</span>df = spark \    .readStream \    .<span class="hljs-built_in">format</span>(<span class="hljs-string">&quot;kafka&quot;</span>) \    .option(<span class="hljs-string">&quot;kafka.bootstrap.servers&quot;</span>, kafka_bootstrap_servers) \    .option(<span class="hljs-string">&quot;subscribe&quot;</span>, kafka_topic) \    .option(<span class="hljs-string">&quot;startingOffsets&quot;</span>, <span class="hljs-string">&quot;earliest&quot;</span>) \    .load()<span class="hljs-comment"># Extract the value and parse the JSON</span>parsed_df = df.selectExpr(<span class="hljs-string">&quot;CAST(value AS STRING) as value&quot;</span>) \    .select(from_json(col(<span class="hljs-string">&quot;value&quot;</span>), schema).alias(<span class="hljs-string">&quot;data&quot;</span>)) \    .select(<span class="hljs-string">&quot;data.*&quot;</span>)query = parsed_df.writeStream \        .foreachBatch(process_batch) \        .<span class="hljs-built_in">format</span>(<span class="hljs-string">&quot;parquet&quot;</span>) \        .option(<span class="hljs-string">&quot;path&quot;</span>,output_s3_bucket) \        .option(<span class="hljs-string">&quot;checkpointLocation&quot;</span>, <span class="hljs-string">&quot;/tmp/spark_checkpoints/bronze_layer/plain&quot;</span>) \        .trigger(processingTime=<span class="hljs-string">&quot;500 milliseconds&quot;</span>) \        .outputMode(<span class="hljs-string">&quot;append&quot;</span>) \        .start()query.awaitTermination()</code></pre><h2 id="Step-7-Batch-ETL-Job-Kafka-to-Icerberg-Silver-Layer"><a href="#Step-7-Batch-ETL-Job-Kafka-to-Icerberg-Silver-Layer" class="headerlink" title="Step 7: Batch ETL Job - Kafka to Icerberg (Silver Layer)"></a>Step 7: Batch ETL Job - Kafka to Icerberg (Silver Layer)</h2><p>Create <code>plain/silver_job.py</code>:</p><pre><code class="hljs python"><span class="hljs-keyword">from</span> pyspark.sql <span class="hljs-keyword">import</span> SparkSessionbucket_name = <span class="hljs-string">&quot;bronze-bucket&quot;</span>output_s3_bucket = <span class="hljs-string">f&#x27;s3a://<span class="hljs-subst">&#123;bucket_name&#125;</span>/plain&#x27;</span>s3_endpoint_url = <span class="hljs-string">&quot;http://localstack:4566&quot;</span>namespace_catalog = <span class="hljs-string">&quot;local_catalog&quot;</span>catalog_name = <span class="hljs-string">&quot;local_catalog_plain&quot;</span>table_name = <span class="hljs-string">&quot;silver_table&quot;</span>full_table_name = <span class="hljs-string">f&quot;`<span class="hljs-subst">&#123;namespace_catalog&#125;</span>`.`<span class="hljs-subst">&#123;catalog_name&#125;</span>`.`<span class="hljs-subst">&#123;table_name&#125;</span>`&quot;</span>spark = SparkSession.builder \        .appName(<span class="hljs-string">&quot;Silver Layer Job&quot;</span>) \        .master(<span class="hljs-string">&quot;local[*]&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.extensions&quot;</span>, <span class="hljs-string">&quot;org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog&quot;</span>, <span class="hljs-string">&quot;org.apache.iceberg.spark.SparkCatalog&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.type&quot;</span>, <span class="hljs-string">&quot;hadoop&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.warehouse&quot;</span>, <span class="hljs-string">&quot;s3a://iceberg/warehouse&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.endpoint&quot;</span>, <span class="hljs-string">&quot;http://localstack:4566&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.access.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.secret.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.connection.ssl.enabled&quot;</span>, <span class="hljs-string">&quot;false&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.path.style.access&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \ <span class="hljs-comment"># Create the database if it doesn&#x27;t exist using PySpark SQL</span>create_database_if_not_exists(spark, catalog_name)<span class="hljs-comment"># Create the table if it doesn&#x27;t exist using PySpark SQL</span>create_table_if_not_exists(spark, namespace_catalog, catalog_name, table_name) <span class="hljs-comment"># Read from S3 (Bronze)</span>bronze_path = output_s3_bucketdf = spark.read.<span class="hljs-built_in">format</span>(<span class="hljs-string">&quot;parquet&quot;</span>).load(bronze_path)<span class="hljs-comment"># Perform data cleaning and enrichment</span>cleaned_df = df.<span class="hljs-built_in">filter</span>(<span class="hljs-string">&quot;amount &lt; 100&quot;</span>)<span class="hljs-comment"># Write to Iceberg (Silver)</span>cleaned_df.writeTo(full_table_name).append()<span class="hljs-comment"># Show the Silver Iceberg table records</span>message = spark.table(full_table_name).show()spark.stop()</code></pre><h2 id="Step-8-Batch-ETL-Job-Iceberg-to-Dynamo-Gold-Layer"><a href="#Step-8-Batch-ETL-Job-Iceberg-to-Dynamo-Gold-Layer" class="headerlink" title="Step 8: Batch ETL Job - Iceberg to Dynamo (Gold Layer)"></a>Step 8: Batch ETL Job - Iceberg to Dynamo (Gold Layer)</h2><p>Create <code>plain/gold_job.py</code>:</p><pre><code class="hljs python"><span class="hljs-keyword">from</span> pyspark.sql <span class="hljs-keyword">import</span> SparkSession<span class="hljs-keyword">from</span> botocore.exceptions <span class="hljs-keyword">import</span> ClientError<span class="hljs-keyword">import</span> time<span class="hljs-keyword">import</span> json<span class="hljs-comment"># # Set S3 endpoint to point to LocalStack</span>s3_endpoint_url = <span class="hljs-string">&quot;http://localstack:4566&quot;</span>namespace_catalog = <span class="hljs-string">&quot;local_catalog&quot;</span>catalog_name = <span class="hljs-string">&quot;local_catalog_plain&quot;</span>table_name = <span class="hljs-string">&quot;silver_table&quot;</span>full_table_name = <span class="hljs-string">f&quot;`<span class="hljs-subst">&#123;namespace_catalog&#125;</span>`.`<span class="hljs-subst">&#123;catalog_name&#125;</span>`.`<span class="hljs-subst">&#123;table_name&#125;</span>`&quot;</span><span class="hljs-comment"># Initialize Spark session</span>spark = SparkSession.builder \    .appName(<span class="hljs-string">&quot;Gold Layer Job&quot;</span>) \    .master(<span class="hljs-string">&quot;local[*]&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.extensions&quot;</span>, <span class="hljs-string">&quot;org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog&quot;</span>, <span class="hljs-string">&quot;org.apache.iceberg.spark.SparkCatalog&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.type&quot;</span>, <span class="hljs-string">&quot;hadoop&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.warehouse&quot;</span>, <span class="hljs-string">&quot;s3a://iceberg/warehouse&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.endpoint&quot;</span>, <span class="hljs-string">&quot;http://localstack:4566&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.access.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.secret.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.connection.ssl.enabled&quot;</span>, <span class="hljs-string">&quot;false&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.path.style.access&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.endpoint&quot;</span>, <span class="hljs-string">&quot;http://localstack:4566&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.access.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.secret.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.impl&quot;</span>, <span class="hljs-string">&quot;org.apache.hadoop.fs.s3a.S3AFileSystem&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.multiobjectdelete.enable&quot;</span>, <span class="hljs-string">&quot;false&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.fast.upload&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.fast.upload.buffer&quot;</span>, <span class="hljs-string">&quot;disk&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.endpoint.region&quot;</span>, <span class="hljs-string">&quot;us-east-1&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.path.style.access&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \    .config(<span class="hljs-string">&quot;spark.hadoop.fs.s3a.aws.credentials.provider&quot;</span>, <span class="hljs-string">&quot;org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider&quot;</span>) \    .getOrCreate()<span class="hljs-comment"># Read from Iceberg (Silver)</span>df = spark.read.<span class="hljs-built_in">format</span>(<span class="hljs-string">&quot;iceberg&quot;</span>).load(full_table_name)<span class="hljs-comment"># Define the table name</span>dynamo_table_name = <span class="hljs-string">&quot;gold_table_plain&quot;</span>dynamodb = boto3.client(<span class="hljs-string">&quot;dynamodb&quot;</span>, endpoint_url=s3_endpoint_url,region_name=<span class="hljs-string">&quot;us-east-1&quot;</span>)data = df.collect()<span class="hljs-keyword">for</span> row <span class="hljs-keyword">in</span> data:    item = &#123;        <span class="hljs-string">&#x27;id&#x27;</span>: &#123;<span class="hljs-string">&#x27;N&#x27;</span>: <span class="hljs-built_in">str</span>(row[<span class="hljs-string">&#x27;id&#x27;</span>])&#125;,        <span class="hljs-string">&#x27;name&#x27;</span>: &#123;<span class="hljs-string">&#x27;S&#x27;</span>: row[<span class="hljs-string">&#x27;name&#x27;</span>]&#125;,        <span class="hljs-string">&#x27;amount&#x27;</span>: &#123;<span class="hljs-string">&#x27;N&#x27;</span>: <span class="hljs-built_in">str</span>(row[<span class="hljs-string">&#x27;amount&#x27;</span>])&#125;    &#125;    dynamodb.put_item(TableName=dynamo_table_name, Item=item)    log_logging_events(<span class="hljs-string">f&quot;Inserted <span class="hljs-subst">&#123;json.dumps(item)&#125;</span> items into <span class="hljs-subst">&#123;dynamo_table_name&#125;</span>.&quot;</span>, logs_client)    <span class="hljs-built_in">print</span>(<span class="hljs-string">f&quot;Inserted <span class="hljs-subst">&#123;<span class="hljs-built_in">len</span>(data)&#125;</span> items into <span class="hljs-subst">&#123;dynamo_table_name&#125;</span>.&quot;</span>)spark.stop()</code></pre><h2 id="Step-9-Launch-the-Environment"><a href="#Step-9-Launch-the-Environment" class="headerlink" title="Step 9: Launch the Environment"></a>Step 9: Launch the Environment</h2><p>Start all services:</p><p>Run <code>scripts/start-containers.bash</code></p><pre><code class="hljs bash">docker-compose up --build -d</code></pre><p>Stop all resources when done:</p><p>Run <code>scripts/shutdown-containers.bash</code></p><pre><code class="hljs bash">docker-compose down</code></pre><h2 id="Step-9-Access-JupyterLab"><a href="#Step-9-Access-JupyterLab" class="headerlink" title="Step 9: Access JupyterLab"></a>Step 9: Access JupyterLab</h2><p>Navigate to <code>http://localhost:8888</code> and you will see the Jupyter Notebook interface there.</p><h2 id="Step-10-Verify-the-complete-Setup"><a href="#Step-10-Verify-the-complete-Setup" class="headerlink" title="Step 10: Verify the complete Setup"></a>Step 10: Verify the complete Setup</h2><h3 id="Verify-Localstack-S3-and-DynamoDB-resources"><a href="#Verify-Localstack-S3-and-DynamoDB-resources" class="headerlink" title="Verify Localstack S3 and DynamoDB resources"></a>Verify Localstack S3 and DynamoDB resources</h3><p>Localstack comes with UI Desktop version for devs to navigate and confirm data is available locally and visually once their etl applications finished running.</p><p>You will see something like this</p><img src="/images/localstack_s3_resources.png" class="center" width="800" height="600" title="LocalStack S3 Resources"><p><em><p style="text-align: center;">LocalStack S3 Resources</p></em></p><img src="/images/localstack_dynamo_resources.png" class="center" width="800" height="600" title="LocalStack Dynamo Resources"><p><em><p style="text-align: center;">LocalStack Dynamo Resources</p></em></p><p>You can download them <a href="https://docs.localstack.cloud/aws/capabilities/web-app/localstack-desktop/">here</a></p><h3 id="Query-Iceberg-table"><a href="#Query-Iceberg-table" class="headerlink" title="Query Iceberg table"></a>Query Iceberg table</h3><p>In JupyterLab:</p><p>Write some jupyter notebook code to inspect the iceberg table data. They will be saved under <code>notebooks</code> folder.</p><pre><code class="hljs python"><span class="hljs-keyword">from</span> pyspark.sql <span class="hljs-keyword">import</span> SparkSession<span class="hljs-comment"># Initialize Spark session</span>spark = SparkSession.builder \        .appName(<span class="hljs-string">&quot;Silver Layer Job&quot;</span>) \        .master(<span class="hljs-string">&quot;local[*]&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.extensions&quot;</span>, <span class="hljs-string">&quot;org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog&quot;</span>, <span class="hljs-string">&quot;org.apache.iceberg.spark.SparkCatalog&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.type&quot;</span>, <span class="hljs-string">&quot;hadoop&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.warehouse&quot;</span>, <span class="hljs-string">&quot;s3a://iceberg/warehouse&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.endpoint&quot;</span>, <span class="hljs-string">&quot;http://localstack:4566&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.access.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.secret.key&quot;</span>, <span class="hljs-string">&quot;test&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.connection.ssl.enabled&quot;</span>, <span class="hljs-string">&quot;false&quot;</span>) \        .config(<span class="hljs-string">&quot;spark.sql.catalog.local_catalog.hadoop.fs.s3a.path.style.access&quot;</span>, <span class="hljs-string">&quot;true&quot;</span>) \        .getOrCreate()df  = spark.read.<span class="hljs-built_in">format</span>(<span class="hljs-string">&quot;iceberg&quot;</span>).load(<span class="hljs-string">&quot;local_catalog.local_catalog_plain.silver_table&quot;</span>)df.show(truncate=<span class="hljs-literal">False</span>)<span class="hljs-built_in">print</span>(<span class="hljs-string">f&quot;Number of rows in the table: <span class="hljs-subst">&#123;df.count()&#125;</span>&quot;</span>)</code></pre><h2 id="Pro-Tips"><a href="#Pro-Tips" class="headerlink" title="Pro Tips"></a>Pro Tips</h2><h3 id="1-Increase-Performance"><a href="#1-Increase-Performance" class="headerlink" title="1. Increase Performance"></a>1. Increase Performance</h3><p>Add to <code>docker-compose.yml</code>:</p><pre><code class="hljs yaml"><span class="hljs-attr">glue:</span>  <span class="hljs-attr">environment:</span>    <span class="hljs-bullet">-</span> <span class="hljs-string">SPARK_DRIVER_MEMORY=4g</span>    <span class="hljs-bullet">-</span> <span class="hljs-string">SPARK_EXECUTOR_MEMORY=4g</span></code></pre><h3 id="2-Debug-Spark-Jobs"><a href="#2-Debug-Spark-Jobs" class="headerlink" title="2. Debug Spark Jobs"></a>2. Debug Spark Jobs</h3><p>Access Spark UI at <code>http://localhost:4040</code> while jobs are running.</p><h2 id="Common-Issues"><a href="#Common-Issues" class="headerlink" title="Common Issues"></a>Common Issues</h2><p><strong>LocalStack connection fails</strong>: Use service names (<code>localstack</code>, <code>kafka</code>) not <code>localhost</code> in container code.</p><p><strong>Kafka not ready</strong>: Wait 30-60 seconds after <code>docker-compose up</code> before creating topics.</p><p><strong>Memory errors</strong>: Increase Docker memory allocation to 8GB+ in settings.</p><h2 id="Wrapping-Up"><a href="#Wrapping-Up" class="headerlink" title="Wrapping Up"></a>Wrapping Up</h2><p>This setup has transformed my development workflow. I can now:</p><ol><li>✅ Test Glue jobs locally in seconds</li><li>✅ Debug streaming pipelines without AWS costs</li><li>✅ Experiment with Iceberg features safely</li><li>✅ Catch bugs before production deployment</li></ol><p>The complete code is available in my <a href="https://github.com/awongCM/docker_glue_pyspark_demo">docker_glue_pyspark_demo</a> repository.</p><h2 id="What’s-Next"><a href="#What’s-Next" class="headerlink" title="What’s Next?"></a>What’s Next?</h2><p>Here are some potential enhancements this local setup can be extended:</p><ol><li>Add data quality checks with dbt&#x2F;Great Expectations.</li><li>Implement unit tests and extend tests coverage for these.</li><li>Set up CI&#x2F;CD pipelines using Github Actions for job deployment, unit test, peformance etc.</li><li>Explore Iceberg’s time travel capabilities, etc</li></ol><p>Feel free to fork the repo and adapt it to your needs.</p><p>Till next time, Happy coding!</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_data_engineering_local_setup_development.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;If you’re like me, you’ve probably experienced the frustration of developing any AWS applications in the cloud especially such as lambda, cloudfront, S3, etc using tools like SAM, CloudFormation etc.. Now that I’ve been delving into the world of data engineering, AWS Glue jobs is one of the main AWS data engineering tooling offerings, the frustrations in having to push Glue jobs directly in the cloud—waiting for job runs, nitty gritty AWS Glue binaries setup, dealing with slow feedback loops, and watching your AWS bill creep up with every test iteration. I’ve been there, and it’s not fun.&lt;/p&gt;
&lt;p&gt;That’s why I built this local development environment. After experimenting with different approaches, I’ve put together a Docker-based setup that lets me develop and test Glue jobs on my laptop before deploying to AWS. The best part? It includes all the real-world components I actually use in actual cloud envs: Kafka for streaming data, Iceberg for my data lake tables, and LocalStack to simulate AWS services.&lt;/p&gt;</summary>
    
    
    
    
    <category term="python pyspark aws cloud localstack apache-iceberg apache-kafka" scheme="http://awongcm.io/tags/python-pyspark-aws-cloud-localstack-apache-iceberg-apache-kafka/"/>
    
  </entry>
  
  <entry>
    <title>Tips and Tricks for Working with Arrays of Objects in Javascript and Typescript</title>
    <link href="http://awongcm.io/blog/2025/05/03/tips-and-tricks-for-working-with-arrays-of-objects-in-javascript-and-typescript/"/>
    <id>http://awongcm.io/blog/2025/05/03/tips-and-tricks-for-working-with-arrays-of-objects-in-javascript-and-typescript/</id>
    <published>2025-05-03T07:24:05.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_tips_tricks_array_objects_javascript_typescript.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction:"></a>Introduction:</h2><p>Since my new types of interesting engineering work have been mostly tied with backend and data engineering as of late, I’ve been earger to do a check in on what’s been happening lately in the web browser space and how much the world of web and front end development has changed since stepping away little over 18 months at the time of writing, particularly when working and manipulating with arrays of objects.</p><p>Sure enough, modern JavaScript and TypeScript, powered by the latest ECMAScript features, are loaded with powerful tools for working with arrays of objects. They enable concise, readable, and efficient code, making it simpler to manage, search, filter, and transform data structures. In this blog post, I’ll cover actionable tips and tricks to level up your expertise in handling arrays of objects, starting with some refresher basics up to something more advanced takes.</p><span id="more"></span><h2 id="1-Iterate-Like-a-Pro-Using-map-forEach-and-Optional-Chaining"><a href="#1-Iterate-Like-a-Pro-Using-map-forEach-and-Optional-Chaining" class="headerlink" title="1. Iterate Like a Pro Using .map, .forEach, and Optional Chaining"></a><strong>1. Iterate Like a Pro Using <code>.map</code>, <code>.forEach</code>, and Optional Chaining</strong></h2><p>When working with arrays of objects, you’ll often need to transform the elements or interact with their properties. The <code>.map</code> method is great for returning a new array, while <code>.forEach</code> is ideal for side effects like logging or modifying values.</p><h3 id="Example"><a href="#Example" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-comment">// Array of objects</span><span class="hljs-keyword">const</span> users = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Alice&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">28</span> &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Bob&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">34</span> &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">3</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Charlie&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">25</span> &#125;,];<span class="hljs-comment">// Transform with .map</span><span class="hljs-keyword">const</span> userNames = users.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">name</span>); <span class="hljs-comment">// [&#x27;Alice&#x27;, &#x27;Bob&#x27;, &#x27;Charlie&#x27;]</span><span class="hljs-comment">// Optional chaining to avoid undefined errors</span><span class="hljs-keyword">const</span> userAges = users.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">age</span>?.<span class="hljs-title function_">toString</span>() || <span class="hljs-string">&quot;Age unavailable&quot;</span>); <span class="hljs-comment">// [&#x27;28&#x27;, &#x27;34&#x27;, &#x27;25&#x27;]</span><span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(userNames);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(userAges);</code></pre><h2 id="2-Perform-Efficient-Search-with-find-and-some"><a href="#2-Perform-Efficient-Search-with-find-and-some" class="headerlink" title="2. Perform Efficient Search with .find and .some"></a><strong>2. Perform Efficient Search with <code>.find</code> and <code>.some</code></strong></h2><p>Finding a specific object or verifying the presence of a condition is effortless through <code>.find</code> and <code>.some</code>.</p><h3 id="Example-1"><a href="#Example-1" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-comment">// Find specific user by ID</span><span class="hljs-keyword">const</span> user = users.<span class="hljs-title function_">find</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">id</span> === <span class="hljs-number">2</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(user); <span class="hljs-comment">// &#123; id: 2, name: &#x27;Bob&#x27;, age: 34 &#125;</span><span class="hljs-comment">// Check if any user meets a condition</span><span class="hljs-keyword">const</span> hasOlderUser = users.<span class="hljs-title function_">some</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">age</span> &gt; <span class="hljs-number">30</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(hasOlderUser); <span class="hljs-comment">// true</span></code></pre><h2 id="3-Filter-Arrays-Easily-with-filter"><a href="#3-Filter-Arrays-Easily-with-filter" class="headerlink" title="3. Filter Arrays Easily with .filter"></a><strong>3. Filter Arrays Easily with <code>.filter</code></strong></h2><p>When you need to work with specific subsets, .filter is your best friend.</p><h3 id="Example-2"><a href="#Example-2" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-comment">// Filter users by age condition</span><span class="hljs-keyword">const</span> youngUsers = users.<span class="hljs-title function_">filter</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">age</span> &lt; <span class="hljs-number">30</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(youngUsers); <span class="hljs-comment">// [&#123; id: 1, name: &#x27;Alice&#x27;, age: 28 &#125;, &#123; id: 3, name: &#x27;Charlie&#x27;, age: 25 &#125;]</span></code></pre><h2 id="4-Sort-Objects-with-sort"><a href="#4-Sort-Objects-with-sort" class="headerlink" title="4. Sort Objects with .sort"></a><strong>4. Sort Objects with <code>.sort</code></strong></h2><p>Sorting an array based on specific object properties is straightforward with <code>.sort</code>.</p><h3 id="Example-3"><a href="#Example-3" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> sortedByAge = users.<span class="hljs-title function_">sort</span>(<span class="hljs-function">(<span class="hljs-params">a, b</span>) =&gt;</span> a.<span class="hljs-property">age</span> - b.<span class="hljs-property">age</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(sortedByAge);<span class="hljs-comment">// [&#123; id: 3, name: &#x27;Charlie&#x27;, age: 25 &#125;, &#123; id: 1, name: &#x27;Alice&#x27;, age: 28 &#125;, &#123; id: 2, name: &#x27;Bob&#x27;, age: 34 &#125;]</span></code></pre><p>Use <code>.slice()</code> to prevent mutating the original array:</p><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> sortedImmutable = users.<span class="hljs-title function_">slice</span>().<span class="hljs-title function_">sort</span>(<span class="hljs-function">(<span class="hljs-params">a, b</span>) =&gt;</span> a.<span class="hljs-property">age</span> - b.<span class="hljs-property">age</span>);</code></pre><h2 id="5-Group-Objects-with-reduce"><a href="#5-Group-Objects-with-reduce" class="headerlink" title="5. Group Objects with .reduce"></a><strong>5. Group Objects with <code>.reduce</code></strong></h2><p><code>.reduce</code> isn’t just for summing numbers—it can group objects based on properties.</p><h3 id="Example-4"><a href="#Example-4" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> usersByAge = users.<span class="hljs-title function_">reduce</span>(<span class="hljs-function">(<span class="hljs-params">acc, user</span>) =&gt;</span> &#123;  <span class="hljs-keyword">const</span> ageGroup = user.<span class="hljs-property">age</span> &lt; <span class="hljs-number">30</span> ? <span class="hljs-string">&quot;under30&quot;</span> : <span class="hljs-string">&quot;over30&quot;</span>;  (acc[ageGroup] ||= []).<span class="hljs-title function_">push</span>(user);  <span class="hljs-keyword">return</span> acc;&#125;, &#123;&#125; <span class="hljs-keyword">as</span> <span class="hljs-title class_">Record</span>&lt;<span class="hljs-built_in">string</span>, <span class="hljs-keyword">typeof</span> users&gt;);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(usersByAge);<span class="hljs-comment">// &#123;</span><span class="hljs-comment">//   under30: [</span><span class="hljs-comment">//     &#123; id: 1, name: &#x27;Alice&#x27;, age: 28 &#125;,</span><span class="hljs-comment">//     &#123; id: 3, name: &#x27;Charlie&#x27;, age: 25 &#125;</span><span class="hljs-comment">//   ],</span><span class="hljs-comment">//   over30: [</span><span class="hljs-comment">//     &#123; id: 2, name: &#x27;Bob&#x27;, age: 34 &#125;</span><span class="hljs-comment">//   ]</span><span class="hljs-comment">// &#125;</span></code></pre><h2 id="6-Remove-Duplicate-Objects-Using-Sets-and-map"><a href="#6-Remove-Duplicate-Objects-Using-Sets-and-map" class="headerlink" title="6. Remove Duplicate Objects Using Sets and .map"></a><strong>6. Remove Duplicate Objects Using Sets and <code>.map</code></strong></h2><p>Deduplicating arrays of objects based on specific properties can be achieved using Sets and <code>.map.</code></p><h3 id="Example-5"><a href="#Example-5" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> uniqueUsers = <span class="hljs-title class_">Array</span>.<span class="hljs-title function_">from</span>(  <span class="hljs-keyword">new</span> <span class="hljs-title class_">Map</span>(users.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> [user.<span class="hljs-property">id</span>, user])).<span class="hljs-title function_">values</span>());<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(uniqueUsers);<span class="hljs-comment">// Deduplicates users based on their `id` property</span></code></pre><h2 id="7-Combine-Multiple-Arrays-with-Spread-Syntax"><a href="#7-Combine-Multiple-Arrays-with-Spread-Syntax" class="headerlink" title="7. Combine Multiple Arrays with Spread Syntax"></a><strong>7. Combine Multiple Arrays with Spread Syntax</strong></h2><p>The spread operator (<code>...</code>) is an easy way to merge arrays of objects.</p><h3 id="Example-6"><a href="#Example-6" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> moreUsers = [&#123; <span class="hljs-attr">id</span>: <span class="hljs-number">4</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;David&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">32</span> &#125;];<span class="hljs-keyword">const</span> allUsers = [...users, ...moreUsers];<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(allUsers);<span class="hljs-comment">// Merged array of objects</span></code></pre><h2 id="8-TypeScript-Bonus-Tips-Leverage-Interfaces-and-Generics"><a href="#8-TypeScript-Bonus-Tips-Leverage-Interfaces-and-Generics" class="headerlink" title="8. TypeScript Bonus Tips: Leverage Interfaces and Generics"></a><strong>8. TypeScript Bonus Tips: Leverage Interfaces and Generics</strong></h2><p>TypeScript allows you to ensure type safety when working with arrays of objects. Use interfaces or types for clarity.</p><h3 id="Example-7"><a href="#Example-7" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">interface</span> <span class="hljs-title class_">User</span> &#123;  <span class="hljs-attr">id</span>: <span class="hljs-built_in">number</span>;  <span class="hljs-attr">name</span>: <span class="hljs-built_in">string</span>;  <span class="hljs-attr">age</span>: <span class="hljs-built_in">number</span>;&#125;<span class="hljs-keyword">const</span> <span class="hljs-attr">typedUsers</span>: <span class="hljs-title class_">User</span>[] = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Alice&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">28</span> &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Bob&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">34</span> &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">3</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Charlie&quot;</span>, <span class="hljs-attr">age</span>: <span class="hljs-number">25</span> &#125;,];<span class="hljs-comment">// Access with proper autocomplete and safety</span>typedUsers.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">name</span>);</code></pre><p>Generics in functions can add flexibility:</p><pre><code class="hljs typescript"><span class="hljs-keyword">function</span> getById&lt;T <span class="hljs-keyword">extends</span> &#123; <span class="hljs-attr">id</span>: <span class="hljs-built_in">number</span> &#125;&gt;(  <span class="hljs-attr">array</span>: T[],  <span class="hljs-attr">id</span>: <span class="hljs-built_in">number</span>): T | <span class="hljs-literal">undefined</span> &#123;  <span class="hljs-keyword">return</span> array.<span class="hljs-title function_">find</span>(<span class="hljs-function">(<span class="hljs-params">item</span>) =&gt;</span> item.<span class="hljs-property">id</span> === id);&#125;<span class="hljs-keyword">const</span> user = <span class="hljs-title function_">getById</span>(typedUsers, <span class="hljs-number">2</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(user);</code></pre><h2 id="9-Merge-or-Transform-Properties-Using-the-Spread-Operator"><a href="#9-Merge-or-Transform-Properties-Using-the-Spread-Operator" class="headerlink" title="9. Merge or Transform Properties Using the Spread Operator"></a><strong>9. Merge or Transform Properties Using the Spread Operator</strong></h2><p>The spread operator can be used to transform properties while keeping the original object intact.</p><h3 id="Example-8"><a href="#Example-8" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-comment">// Update a single object while maintaining immutability</span><span class="hljs-keyword">const</span> updatedUsers = users.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span>  user.<span class="hljs-property">id</span> === <span class="hljs-number">2</span> ? &#123; ...user, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Robert&quot;</span> &#125; : user);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(updatedUsers);</code></pre><h2 id="10-Handle-Large-Arrays-with-flatMap"><a href="#10-Handle-Large-Arrays-with-flatMap" class="headerlink" title="10. Handle Large Arrays with .flatMap"></a><strong>10. Handle Large Arrays with <code>.flatMap</code></strong></h2><p>When an array of objects includes nested arrays, .flatMap is perfect for flattening arrays while applying map function.</p><h3 id="Example-9"><a href="#Example-9" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> userActivities = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">activities</span>: [<span class="hljs-string">&quot;running&quot;</span>, <span class="hljs-string">&quot;cycling&quot;</span>] &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">activities</span>: [<span class="hljs-string">&quot;swimming&quot;</span>] &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">3</span>, <span class="hljs-attr">activities</span>: [<span class="hljs-string">&quot;hiking&quot;</span>, <span class="hljs-string">&quot;climbing&quot;</span>] &#125;,];<span class="hljs-keyword">const</span> flattenedActivities = userActivities.<span class="hljs-title function_">flatMap</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> user.<span class="hljs-property">activities</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(flattenedActivities); <span class="hljs-comment">// [&#x27;running&#x27;, &#x27;cycling&#x27;, &#x27;swimming&#x27;, &#x27;hiking&#x27;, &#x27;climbing&#x27;]</span></code></pre><h2 id="11-Handle-Arrays-with-Nested-Objects"><a href="#11-Handle-Arrays-with-Nested-Objects" class="headerlink" title="11. Handle Arrays with Nested Objects"></a><strong>11. Handle Arrays with Nested Objects</strong></h2><p>An object may contain nested objects or nested arrays of related entities.</p><h3 id="Example-10"><a href="#Example-10" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">interface</span> <span class="hljs-title class_">Employee</span> &#123;  <span class="hljs-attr">id</span>: <span class="hljs-built_in">number</span>;  <span class="hljs-attr">name</span>: <span class="hljs-built_in">string</span>;  <span class="hljs-attr">department</span>: &#123;    <span class="hljs-attr">name</span>: <span class="hljs-built_in">string</span>;    <span class="hljs-attr">manager</span>: <span class="hljs-built_in">string</span>;  &#125;;&#125;<span class="hljs-keyword">const</span> <span class="hljs-attr">employees</span>: <span class="hljs-title class_">Employee</span>[] = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Alice&quot;</span>, <span class="hljs-attr">department</span>: &#123; <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Engineering&quot;</span>, <span class="hljs-attr">manager</span>: <span class="hljs-string">&quot;Bob&quot;</span> &#125; &#125;,  &#123;    <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>,    <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Charlie&quot;</span>,    <span class="hljs-attr">department</span>: &#123; <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Marketing&quot;</span>, <span class="hljs-attr">manager</span>: <span class="hljs-string">&quot;John&quot;</span> &#125;,  &#125;,];</code></pre><h3 id="How-to-Handle"><a href="#How-to-Handle" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Use optional chaining (?.) to safely access nested properties:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> deptNames = employees.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">emp</span>) =&gt;</span> emp.<span class="hljs-property">department</span>?.<span class="hljs-property">name</span>); <span class="hljs-comment">// [&#x27;Engineering&#x27;, &#x27;Marketing&#x27;]</span></code></pre><ul><li>Flatten or traverse nested structures using recursion or .flatMap() if necessary.</li></ul><h2 id="12-Handle-Deeply-Nested-Arrays"><a href="#12-Handle-Deeply-Nested-Arrays" class="headerlink" title="12. Handle Deeply Nested Arrays"></a><strong>12. Handle Deeply Nested Arrays</strong></h2><p>Arrays inside objects, which themselves contain more arrays.</p><h3 id="Example-11"><a href="#Example-11" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">interface</span> <span class="hljs-title class_">Team</span> &#123;  <span class="hljs-attr">name</span>: <span class="hljs-built_in">string</span>;  <span class="hljs-attr">members</span>: &#123; <span class="hljs-attr">id</span>: <span class="hljs-built_in">number</span>; <span class="hljs-attr">name</span>: <span class="hljs-built_in">string</span> &#125;[];&#125;<span class="hljs-keyword">const</span> <span class="hljs-attr">teams</span>: <span class="hljs-title class_">Team</span>[] = [  &#123;    <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Frontend&quot;</span>,    <span class="hljs-attr">members</span>: [      &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Alice&quot;</span> &#125;,      &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Bob&quot;</span> &#125;,    ],  &#125;,  &#123; <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Backend&quot;</span>, <span class="hljs-attr">members</span>: [&#123; <span class="hljs-attr">id</span>: <span class="hljs-number">3</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Charlie&quot;</span> &#125;] &#125;,];</code></pre><h3 id="How-to-Handle-1"><a href="#How-to-Handle-1" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Flatten Nested Arrays using <code>.flatMap()</code> or recursion:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> allMembers = teams.<span class="hljs-title function_">flatMap</span>(<span class="hljs-function">(<span class="hljs-params">team</span>) =&gt;</span> team.<span class="hljs-property">members</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(allMembers);<span class="hljs-comment">// [&#123; id: 1, name: &#x27;Alice&#x27; &#125;, &#123; id: 2, name: &#x27;Bob&#x27; &#125;, &#123; id: 3, name: &#x27;Charlie&#x27; &#125;]</span></code></pre><h2 id="13-Handle-Arrays-Containing-Mixed-Types"><a href="#13-Handle-Arrays-Containing-Mixed-Types" class="headerlink" title="13. Handle Arrays Containing Mixed Types"></a><strong>13. Handle Arrays Containing Mixed Types</strong></h2><p>Objects in the array may follow different structures (discriminated union).</p><h3 id="Example-12"><a href="#Example-12" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">type</span> <span class="hljs-title class_">Payment</span> =  | &#123; <span class="hljs-attr">type</span>: <span class="hljs-string">&quot;card&quot;</span>; <span class="hljs-attr">cardNumber</span>: <span class="hljs-built_in">string</span> &#125;  | &#123; <span class="hljs-attr">type</span>: <span class="hljs-string">&quot;paypal&quot;</span>; <span class="hljs-attr">email</span>: <span class="hljs-built_in">string</span> &#125;;<span class="hljs-keyword">const</span> <span class="hljs-attr">payments</span>: <span class="hljs-title class_">Payment</span>[] = [  &#123; <span class="hljs-attr">type</span>: <span class="hljs-string">&quot;card&quot;</span>, <span class="hljs-attr">cardNumber</span>: <span class="hljs-string">&quot;1234 5678 9012 3456&quot;</span> &#125;,  &#123; <span class="hljs-attr">type</span>: <span class="hljs-string">&quot;paypal&quot;</span>, <span class="hljs-attr">email</span>: <span class="hljs-string">&quot;user@example.com&quot;</span> &#125;,];</code></pre><h3 id="How-to-Handle-2"><a href="#How-to-Handle-2" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Use TypeScript discriminated unions or runtime checks:</li></ul><pre><code class="hljs typescript">payments.<span class="hljs-title function_">forEach</span>(<span class="hljs-function">(<span class="hljs-params">payment</span>) =&gt;</span> &#123;  <span class="hljs-keyword">if</span> (payment.<span class="hljs-property">type</span> === <span class="hljs-string">&quot;card&quot;</span>) &#123;    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">&quot;Card Payment: &quot;</span>, payment.<span class="hljs-property">cardNumber</span>);  &#125; <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> (payment.<span class="hljs-property">type</span> === <span class="hljs-string">&quot;paypal&quot;</span>) &#123;    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">&quot;PayPal Email: &quot;</span>, payment.<span class="hljs-property">email</span>);  &#125;&#125;);</code></pre><h2 id="14-Handle-Associative-Arrays-Key-x2F-Value-Maps"><a href="#14-Handle-Associative-Arrays-Key-x2F-Value-Maps" class="headerlink" title="14. Handle Associative Arrays (Key&#x2F;Value Maps)"></a><strong>14. Handle Associative Arrays (Key&#x2F;Value Maps)</strong></h2><p>Objects in the array may act as pseudo-dictionary.</p><h3 id="Example-13"><a href="#Example-13" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> keyValuePairs = [  &#123; <span class="hljs-attr">key</span>: <span class="hljs-string">&quot;username&quot;</span>, <span class="hljs-attr">value</span>: <span class="hljs-string">&quot;Alice&quot;</span> &#125;,  &#123; <span class="hljs-attr">key</span>: <span class="hljs-string">&quot;email&quot;</span>, <span class="hljs-attr">value</span>: <span class="hljs-string">&quot;alice@example.com&quot;</span> &#125;,];</code></pre><h3 id="How-to-Handle-3"><a href="#How-to-Handle-3" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Convert to a normal object for fast lookup:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> dictionary = keyValuePairs.<span class="hljs-title function_">reduce</span>(<span class="hljs-function">(<span class="hljs-params">acc, pair</span>) =&gt;</span> &#123;  acc[pair.<span class="hljs-property">key</span>] = pair.<span class="hljs-property">value</span>;  <span class="hljs-keyword">return</span> acc;&#125;, &#123;&#125;);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(dictionary); <span class="hljs-comment">// &#123; username: &#x27;Alice&#x27;, email: &#x27;alice@example.com&#x27; &#125;</span></code></pre><h2 id="15-Explode-Object’s-enumerated-Key-x2F-Value-pairs-into-Arrays-of-data"><a href="#15-Explode-Object’s-enumerated-Key-x2F-Value-pairs-into-Arrays-of-data" class="headerlink" title="15. Explode Object’s enumerated Key&#x2F;Value pairs into Arrays of data"></a><strong>15. Explode Object’s enumerated Key&#x2F;Value pairs into Arrays of data</strong></h2><p>Object’s enumerated key&#x2F;value properties can be exploded into 3 types of arrays ie</p><ul><li>Array of Object’s individual Key&#x2F;Value Pairs</li><li>Array of Object’s Keys Only</li><li>Array of Object’s Values Only</li></ul><h3 id="Example-14"><a href="#Example-14" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> someObjectWithKeyValuePairs = &#123;  <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Bob&quot;</span>,  <span class="hljs-attr">age</span>: <span class="hljs-string">&quot;44&quot;</span>,  <span class="hljs-attr">address</span>: <span class="hljs-string">&quot;Address 123&quot;</span>,&#125;;</code></pre><h3 id="How-to-Handle-4"><a href="#How-to-Handle-4" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Split object into array of object’s individual key&#x2F;value pairs:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> arrOfObjectKeyValuePairs = <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">entries</span>(someObjectWithKeyValuePairs).<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">[key, value]</span>) =&gt;</span> (&#123;  [key] : value&#125;))<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(arrOfObjectKeyValuePairs): <span class="hljs-comment">// [&#123;name: &quot;Bob&quot;&#125;, &#123;&quot;age&quot;: &quot;44&quot;&#125;, &#123;&quot;address&quot;:&quot;Address 123&quot;&#125;]</span></code></pre><ul><li>Split object into array of object’s keys only:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> arrOfObjectKeysOnly = <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">keys</span>(arrOfObjectKeyValuePairs);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(arrOfObjectKeysOnly); <span class="hljs-comment">// [&quot;name&quot;, &quot;age&quot;, &quot;address&quot;]</span></code></pre><ul><li>Split object into array of object’s values only:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> arrOfObjectValuesOnly = <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">values</span>(arrOfObjectKeyValuePairs);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(arrOfObjectValuesOnly); <span class="hljs-comment">// [&quot;Bob&quot;, &quot;44&quot;, &quot;Address 123&quot;]</span></code></pre><h2 id="16-Enumerate-Array-of-Object’s-Key-x2F-Value-pairs-into-Array-of-either-Keys-or-Values-only"><a href="#16-Enumerate-Array-of-Object’s-Key-x2F-Value-pairs-into-Array-of-either-Keys-or-Values-only" class="headerlink" title="16. Enumerate Array of Object’s Key&#x2F;Value pairs into Array of either Keys or Values only"></a><strong>16. Enumerate Array of Object’s Key&#x2F;Value pairs into Array of either Keys or Values only</strong></h2><h3 id="Example-15"><a href="#Example-15" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> arrayOfObjectsKeyValuePair = [  &#123; <span class="hljs-attr">integer</span>: <span class="hljs-number">1</span> &#125;,  &#123; <span class="hljs-attr">double</span>: <span class="hljs-number">1.2</span> &#125;,  &#123; <span class="hljs-attr">floating</span>: <span class="hljs-number">123e5</span> &#125;,];</code></pre><h3 id="How-to-Handle-5"><a href="#How-to-Handle-5" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Fetch only the objects’ keys into new array. To do this, we iterate each item of array, explode item’s keys into array. Since the same array has only one key item, we can only fetch it on the zero-index.</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> arrOfKeysOnly = arrayOfObjectsKeyValuePair.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">item</span>) =&gt;</span> <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">keys</span>(item)[<span class="hljs-number">0</span>])<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(arrOfKeysOnly): <span class="hljs-comment">// [&quot;integer&quot;, &quot;double&quot;, &quot;floating&quot;]</span></code></pre><ul><li>Fetch only the objects’ values into new array. To do this, we iterate each item of array, explode item’s value only into array. Since the same array has only one key item, we can only fetch it on the zero-index.</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> arrOfValuesOnly = arrayOfObjectsKeyValuePair.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">item</span>) =&gt;</span> <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">values</span>(item)[<span class="hljs-number">0</span>])<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(arrOValuesOnly): <span class="hljs-comment">// [1, 1.2, 123e5]</span></code></pre><p><strong>NB: Note this approach looks similar approach #14 and #15, but we reuse them for spliting array of objects into boths arrays of keys and values only respectively</strong></p><h2 id="17-Handle-Circular-References"><a href="#17-Handle-Circular-References" class="headerlink" title="17. Handle Circular References"></a><strong>17. Handle Circular References</strong></h2><p>Objects in the array may reference each other, creating circular dependencies.</p><h3 id="Example-16"><a href="#Example-16" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> parent = &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Parent&quot;</span> &#125;;<span class="hljs-keyword">const</span> child = &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Child&quot;</span>, parent &#125;;parent.<span class="hljs-property">child</span> = child; <span class="hljs-comment">// Circular reference!</span><span class="hljs-keyword">const</span> family = [parent, child];</code></pre><h3 id="How-to-Handle-6"><a href="#How-to-Handle-6" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Serialize circular references (e.g., using flatted or a custom replacer):</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">import</span> &#123; stringify &#125; <span class="hljs-keyword">from</span> <span class="hljs-string">&quot;flatted&quot;</span>;<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-title function_">stringify</span>(family));</code></pre><h2 id="18-Objects-with-Dynamic-Properties"><a href="#18-Objects-with-Dynamic-Properties" class="headerlink" title="18. Objects with Dynamic Properties"></a><strong>18. Objects with Dynamic Properties</strong></h2><p>Each object may contain dynamic keys that are unpredictable.</p><h3 id="Example-17"><a href="#Example-17" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> dynamicObjects = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">meta</span>: &#123; <span class="hljs-attr">created</span>: <span class="hljs-string">&quot;202-01-01&quot;</span>, <span class="hljs-attr">updated</span>: <span class="hljs-string">&quot;202-01-10&quot;</span> &#125; &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">meta</span>: &#123; <span class="hljs-attr">created</span>: <span class="hljs-string">&quot;202-05-01&quot;</span>, <span class="hljs-attr">deprecated</span>: <span class="hljs-literal">true</span> &#125; &#125;,];</code></pre><h3 id="How-to-Handle-7"><a href="#How-to-Handle-7" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Use dot-notation, optional chaining, or dynamic property access:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> createdDates = dynamicObjects.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">obj</span>) =&gt;</span> obj.<span class="hljs-property">meta</span>?.<span class="hljs-property">created</span>);</code></pre><h2 id="19-Complex-Relationships-Between-Objects"><a href="#19-Complex-Relationships-Between-Objects" class="headerlink" title="19. Complex Relationships Between Objects"></a><strong>19. Complex Relationships Between Objects</strong></h2><p>Objects reference or relate to other objects (relational data).</p><h3 id="Example-18"><a href="#Example-18" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> users = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Alice&quot;</span>, <span class="hljs-attr">friendIds</span>: [<span class="hljs-number">2</span>, <span class="hljs-number">3</span>] &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Bob&quot;</span>, <span class="hljs-attr">friendIds</span>: [<span class="hljs-number">1</span>] &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">3</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Charlie&quot;</span>, <span class="hljs-attr">friendIds</span>: [<span class="hljs-number">1</span>] &#125;,];</code></pre><h3 id="How-to-Handle-8"><a href="#How-to-Handle-8" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Lookup relationships by ID:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> userMap = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Map</span>(users.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">user</span>) =&gt;</span> [user.<span class="hljs-property">id</span>, user]));<span class="hljs-keyword">const</span> <span class="hljs-title function_">getFriends</span> = (<span class="hljs-params"><span class="hljs-attr">userId</span>: <span class="hljs-built_in">number</span></span>) =&gt; &#123;  <span class="hljs-keyword">const</span> user = userMap.<span class="hljs-title function_">get</span>(userId);  <span class="hljs-keyword">return</span> user?.<span class="hljs-property">friendIds</span>.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">id</span>) =&gt;</span> userMap.<span class="hljs-title function_">get</span>(id));&#125;;<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-title function_">getFriends</span>(<span class="hljs-number">1</span>));<span class="hljs-comment">// [&#123; id: 2, name: &#x27;Bob&#x27; &#125;, &#123; id: 3, name: &#x27;Charlie&#x27; &#125;]</span></code></pre><h2 id="20-API-Pagination-and-Chunked-Data"><a href="#20-API-Pagination-and-Chunked-Data" class="headerlink" title="20. API Pagination and Chunked Data"></a><strong>20. API Pagination and Chunked Data</strong></h2><p>Arrays retrieved from APIs are paginated or chunked into parts.</p><h3 id="Example-19"><a href="#Example-19" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> paginatedResponse = &#123;  <span class="hljs-attr">data</span>: [    &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Alice&quot;</span> &#125;,    &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Bob&quot;</span> &#125;,  ],  <span class="hljs-attr">meta</span>: &#123; <span class="hljs-attr">page</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">totalPages</span>: <span class="hljs-number">5</span> &#125;,&#125;;</code></pre><h3 id="How-to-Handle-9"><a href="#How-to-Handle-9" class="headerlink" title="How to Handle:"></a><strong>How to Handle:</strong></h3><ul><li>Concatenate paginated data:</li></ul><pre><code class="hljs typescript"><span class="hljs-keyword">let</span> allUsers = [];<span class="hljs-keyword">for</span> (<span class="hljs-keyword">let</span> page = <span class="hljs-number">1</span>; page &lt;= paginatedResponse.<span class="hljs-property">meta</span>.<span class="hljs-property">totalPages</span>; page++) &#123;  <span class="hljs-keyword">const</span> response = <span class="hljs-title function_">fetchData</span>(page); <span class="hljs-comment">// Simulated API call</span>  allUsers = allUsers.<span class="hljs-title function_">concat</span>(response.<span class="hljs-property">data</span>);&#125;</code></pre><h2 id="21-Arrays-with-Computed-Properties"><a href="#21-Arrays-with-Computed-Properties" class="headerlink" title="21. Arrays with Computed Properties"></a><strong>21. Arrays with Computed Properties</strong></h2><p>Objects may include computed or derived properties.</p><h3 id="Example-20"><a href="#Example-20" class="headerlink" title="Example:"></a><strong>Example:</strong></h3><pre><code class="hljs typescript"><span class="hljs-keyword">const</span> products = [  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Laptop&quot;</span>, <span class="hljs-attr">price</span>: <span class="hljs-number">1000</span> &#125;,  &#123; <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>, <span class="hljs-attr">name</span>: <span class="hljs-string">&quot;Phone&quot;</span>, <span class="hljs-attr">price</span>: <span class="hljs-number">500</span> &#125;,];<span class="hljs-keyword">const</span> productsWithTax = products.<span class="hljs-title function_">map</span>(<span class="hljs-function">(<span class="hljs-params">product</span>) =&gt;</span> (&#123;  ...product,  <span class="hljs-attr">priceWithTax</span>: product.<span class="hljs-property">price</span> * <span class="hljs-number">1.1</span>, <span class="hljs-comment">// Add10% tax</span>&#125;));</code></pre><h2 id="Final-Thoughts"><a href="#Final-Thoughts" class="headerlink" title="Final Thoughts"></a><strong>Final Thoughts</strong></h2><p>Handling arrays of objects in JavaScript and TypeScript has never been easier with ECMAScript’s robust methods along TypeScript’s type safety features. Leveraging these modern tools helps me to write concise, maintainable, and efficient code once you have recognised numerous patterns of array data manipulation problems you’re going to deal with over and over again. Along with having solid fundamentals on Javascript arrays and primitive object types at its core gived you ultimate confidence to write succinct and clear Javascript array handling code where possible. Start incorporating these tips and tricks into your next project! It’s been fun learning experience.</p><p>Till next time, Happy Coding!</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_tips_tricks_array_objects_javascript_typescript.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction:&quot;&gt;&lt;/a&gt;Introduction:&lt;/h2&gt;&lt;p&gt;Since my new types of interesting engineering work have been mostly tied with backend and data engineering as of late, I’ve been earger to do a check in on what’s been happening lately in the web browser space and how much the world of web and front end development has changed since stepping away little over 18 months at the time of writing, particularly when working and manipulating with arrays of objects.&lt;/p&gt;
&lt;p&gt;Sure enough, modern JavaScript and TypeScript, powered by the latest ECMAScript features, are loaded with powerful tools for working with arrays of objects. They enable concise, readable, and efficient code, making it simpler to manage, search, filter, and transform data structures. In this blog post, I’ll cover actionable tips and tricks to level up your expertise in handling arrays of objects, starting with some refresher basics up to something more advanced takes.&lt;/p&gt;</summary>
    
    
    
    
    <category term="javascript typescript objects arrays arrayofobjects data-structures" scheme="http://awongcm.io/tags/javascript-typescript-objects-arrays-arrayofobjects-data-structures/"/>
    
  </entry>
  
  <entry>
    <title>How My Ultra-Wide Curved Monitor Supercharged My Productivity as a Software Engineer</title>
    <link href="http://awongcm.io/blog/2025/01/05/how-my-ultra-wide-curved-monitor-supercharged-my-productivity-as-a-software-engineer/"/>
    <id>http://awongcm.io/blog/2025/01/05/how-my-ultra-wide-curved-monitor-supercharged-my-productivity-as-a-software-engineer/</id>
    <published>2025-01-05T05:48:10.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_ultra_wide_curved_monitor.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction:"></a>Introduction:</h2><p>First off - have a Happy New Year of 2025 everyone! :)</p><p>What a busy and challenging year of 2024 it’s been.</p><p>Looking back, I’ve been wondering what can I look forward in uplifting my personal productivity when in comes to staying lazer-focus, maintain strong context aware of code repositories without losing to context switching, which I have been struggling to achieve for some considerable time.</p><p>Especially when it comes to having the decent monitor screen size as my external display to check out code, read code, write code, submit PRs, review PRs etc.</p><p>Hence the purpose of this writing up this blog post - describing my recent experience of installing ultra-wide curved monitor for my WFH setup.</p><span id="more"></span><p>As a software engineer, multitasking across several applications and coding environments is part of my daily routine. However, the constant switching between windows and tabs was a productivity killer. It’s been constant struggle for me over the course of my years of programming career. That’s when I decided to give an ultra-wide monitor a try. In this post, I’ll share how this piece of technology has transformed my workflow and significantly boosted my productivity.</p><img src="/images/old_external_monitor_wfh_setup.png" class="center" width="500" height="500" title="My old external LG 22" alt="monitor WFH setup My old external LG 22"><p><em><p style="text-align: center;">My old external LG 22” monitor WFH setup</p></em></p><img src="/images/new_external_ultra_wide_curved_monitor_wfh_setup_1.png" class="center" width="500" height="500" title="My new ultra wide curved monitor improved WFH setup My new ultra wide curved monitor improved WFH setup"><p><em><p style="text-align: center;">My new ultra wide curved monitor improved WFH setup</p></em></p><h2 id="First-Impressions"><a href="#First-Impressions" class="headerlink" title="First Impressions:"></a>First Impressions:</h2><p>When I first set up my 34” ultra-wide curved monitor during the new year holidays, the sheer size was overwhelming, yet exciting. The expansive screen real estate was unlike anything I had worked with before. From the get-go, the potential for enhancing my coding environment was clear. The crisp resolution and the ability to have multiple windows fully visible side-by-side was a game changer. See the photos before and after the ultra-wide monitor setup.</p><h2 id="Major-Benefits"><a href="#Major-Benefits" class="headerlink" title="Major Benefits:"></a>Major Benefits:</h2><p>The key benefits I gained from this setup are identified in following sections:</p><h3 id="Code-Management-amp-Visibility"><a href="#Code-Management-amp-Visibility" class="headerlink" title="Code Management &amp; Visibility:"></a>Code Management &amp; Visibility:</h3><ul><li>More Code on Screen: The most obvious benefit is the increased screen real estate. Engineers can view significantly more lines of code simultaneously, reducing the need for constant scrolling and improving context. This is particularly helpful when working with large codebases or multiple files.</li><li>Side-by-Side Comparisons: Easily compare different versions of a file, or view related files (e.g., header and implementation) side-by-side without overlapping windows. This simplifies debugging and code reviews.</li><li>Improved Debugging: Debugging is easier when you can see more of the call stack, variables, and watch expressions at the same time</li><li>Wider View of Version Control Systems: See more branches, commits, and diffs in Git or other version control systems without constant resizing or scrolling.</li></ul><img src="/images/new_external_ultra_wide_curved_monitor_wfh_setup_2.png" class="center" width="500" height="500" title="My new ultra wide curved monitor with my VSCode editor expansive horizontal view My new ultra wide curved monitor with my VSCode editor expansive horizontal view"><p><em><p style="text-align: center;">My new ultra wide curved monitor with my VSCode editor expansive horizontal view</p></em></p><h3 id="Multitasking-amp-Workflow-Efficiency"><a href="#Multitasking-amp-Workflow-Efficiency" class="headerlink" title="Multitasking &amp; Workflow Efficiency:"></a>Multitasking &amp; Workflow Efficiency:</h3><ul><li>Reduced Window Clutter: Consolidate multiple windows onto a single screen, reducing the need to constantly switch between applications or minimize&#x2F;maximize windows. This minimizes distractions and improves focus thus eliminated the need for alt-tabbing through different windows, allowing for a smoother workflow and less cognitive load.</li><li>Enhanced Multitasking: Run multiple applications (IDE, browser, communication tools, etc.) simultaneously without them overlapping or feeling cramped. This allows for quicker task switching and a more streamlined workflow.</li><li>Virtual Desktops: While not unique to ultrawides (such as Deskpad for eg), they become even more powerful with the extra space. Each virtual desktop can hold more windows and applications, allowing for better organization and separation of different projects or tasks.</li><li>Improved Workflow with Integrated Development Environments (IDEs): Many IDEs are designed to take advantage of widescreen displays, allowing for better organization of panels, tool windows, and the code editor itself.</li></ul><h3 id="Improved-Code-Readability-and-Debugging"><a href="#Improved-Code-Readability-and-Debugging" class="headerlink" title="Improved Code Readability and Debugging:"></a>Improved Code Readability and Debugging:</h3><p>One of the immediate advantages I noticed was in code readability. Being able to open multiple files side by side without line wraps meant less scrolling and easier navigation through complex codebases. Debugging became less of a chore because I could see more code at once, making it easier to trace function calls and variable usage across files.</p><h3 id="Ergonomics-amp-Comfort"><a href="#Ergonomics-amp-Comfort" class="headerlink" title="Ergonomics &amp; Comfort:"></a>Ergonomics &amp; Comfort:</h3><ul><li>Less Neck Strain: With more information displayed horizontally, users may experience less need to turn their heads back and forth, potentially reducing neck strain.</li><li>Better Organization: The wider screen allows for a more organized and visually appealing workspace, which can contribute to a more comfortable and productive work environment.</li><li>Less Eye Strain: Especially with ultra wide monitors that are curved, they are designed to match the curvature of your eye’s natural peripheral vision so I never need to strain my eyeball view from one extreme corner to another extreme corner of the monitor. In addition, modern ultra-wide monitors come with eye care options that helps to reduce the amount of blue light being emitted as you stare through the night so you should be able to sleep better after its long-continuous use.</li><li>Higher HDR and Refresh rates: As a bonus, features like these makes things considerably more responsive, particularly if I’m scrolling through web pages or documents. Monitors with higher refresh rates also often have better motion handling, which helps text and images remain clear even if you’re quickly scrolling through spreadsheets, web pages, and more.</li></ul><h3 id="Better-Collaboration-and-Communication"><a href="#Better-Collaboration-and-Communication" class="headerlink" title="Better Collaboration and Communication:"></a>Better Collaboration and Communication:</h3><p>During remote pair programming sessions, the ultra-wide monitor allowed me to share my screen without compromising on which parts of my workspace were visible to my colleagues. This made collaboration more effective and communication clearer, as we could all see the same level of detail.</p><h2 id="Specific-Scenarios"><a href="#Specific-Scenarios" class="headerlink" title="Specific Scenarios:"></a>Specific Scenarios:</h2><p>There are two things that sprung to my mind that would find having ultra wide monitors would be beneficial:</p><ol><li>I remembered working on a particularly challenging project involving numerous legacy&#x2F;new microservices I had developed and maintained at banking enterprise-wide banking organisation. Looking back, if I had begun to invest in purchasing my ultra-wide monitor earlier on, I would have been able to display each service’s logs side by side and grabbed much wider view of the interdependencies between microservices.<br>It would have granted me with better clarity and stronger system understanding - horizontal-wise(as per my increased screen real estate comment earlier).<br>This setup would have been crucial in helping me quickly identify and resolve issues that were affecting multiple services, significantly cutting down the debugging time potentially.</li><li>With modern IDEs and internet browsers these days, at the time of writing, we are slowly seeing more AI tools embedded into these such as MS CoPilot, TabbyML, Github CoPilot etc as part of viewing panels to interact that’s supposedly to drive our producitivty much further.<br>However, with normal built-in retina monitors on your laptops such Macbook Pro like mine, you will have reduced window viewing space for them consequently. Thus you will run into this constant battle in readjusting panels layout - for they always fight for your peripheral and central visions within view.<br>Having external monitor does certainly alleviate this pain.. but with ultra-wide type monitors, they will go even further in greatly reducing this pain down to minor nuisance. Again, all thanks to the increased screen real estate you gained from.</li></ol><h2 id="Tools-and-Setup"><a href="#Tools-and-Setup" class="headerlink" title="Tools and Setup:"></a>Tools and Setup:</h2><p>To make the most out of my ultra-wide curved monitor, for Macbooks, I use a window management tool called Reactangle that allows me to easily snap windows into organized tiles. This tool maximizes my screen usage without the need to manually resize windows constantly. Additionally, adjusting the monitor’s resolution and scaling settings was key to finding the perfect balance between readability and screen real estate.<br>Having said that though - the tool is only possible if I have full admin rights to the Macbook, which in this case, it’s my personal Macbook. For work Macbooks, this is not possible due to company sofware usage restrictions policies so you have to consider alternate window management tools your company provides that may offer similar experience as popular open source tools like Rectangle - if possible.</p><h2 id="Challenges-and-Considerations"><a href="#Challenges-and-Considerations" class="headerlink" title="Challenges and Considerations:"></a>Challenges and Considerations:</h2><p>Adapting to an ultra-wide monitor wasn’t without challenges. Initially, the vast screen space was overwhelming, and I found my eyes tiring quickly. However, by adjusting the monitor’s brightness and using proper desk lighting, I was able to create a comfortable setup. It’s important to consider the ergonomics of using a large monitor to avoid strain.</p><p>In saying that, however, there are some potential downsides you have to consider as follows:</p><ul><li>Cost: Ultrawide monitors are generally more expensive than the standard widescreen monitors. Ultra wide monitors can be priced either under $1000 up or above slightly depending on the make and model brand you’re after. For even wider ultra wide monitors say up to 49”, they can go up to $2000 and beyond. For such high price points, they’re considered to be a more luxury item and I personally don’t find them any practical use for my daily personal work productity. You might as well get yourself an ultra wide entertainment TV for IMO. Thus you need to consider a budget for a good buy. I got my 34” Sony ViewInfinity S6 series for special half price during the Boxing day sale of 2024 so it was a good sale offer!</li><li>Desk Space: They do require a larger desk area, especially for anything larger than the baseline standard of 27”. For my 34”, my present WFH desk I bought from Amazon about 3 years ago is about 180 cm in width so it’s more than suffice to cover.</li><li>Eye Strain: Some users may experience eye strain if they have to constantly move their eyes across the wide screen. Proper viewing distance and breaks are important. Having said that, I mentioned eye care option in my above points so that help to reduce things greatly.</li><li>Compatibility: Not all applications are optimized for ultrawide resolutions, which can sometimes lead to scaling issues or distorted UI elements. So be prepared in getting used to adjusting resolution sizes when connecting to devices such as laptops and desktops machines with their pre-built resolution sizes in order the monitors to adjust and abide by them.</li></ul><p>With that said, here’s a Youtuber programmer who laments his regrets for buying super ultra-wide monitor as a luxury item purchase. Just to give you some perspective as an FYI.</p><iframe width="560" height="315" src="https://www.youtube.com/embed/2fzXqSU858I?si=QXiZ4AywHEh3Hbkt" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion:"></a>Conclusion:</h2><p>Switching to an ultra-wide monitor has been one of the best decisions for enhancing my productivity as a software engineer. The ability to view multiple applications at once, enhanced readability of code, and improved collaboration are just a few of the benefits. If you find yourself juggling multiple windows and applications, an ultra-wide monitor might just be the upgrade you need.</p><p>Hope my comprehensive guide in setting up ultra-wide curved monitor has been superful for you in your software development journey.</p><p>Till then, Happy Coding for 2025 and beyond!</p><p><strong>PS</strong>: Do you know that ultra wide monitors come with built-in KVM switches these days? What that means simply is using one keyboard and one mouse, you can take control of multiple computing devices connected to one screen, depending on how many devices the make and model brand is willing to support. You can use the toggle menu to switch, say between Macbook OS&#x2F;Windows 10&#x2F;11&#x2F;Ubuntu Linux, or however you may want to arrange the switch between your favourite OSes on the same screen.<br>Even better - you can also split the ultra-wide screen in half and view both of your favourite OSes at the same time - just like what I have done for my setup. All I need to do is switch the bluetooth peripheral device connectivity between two OSes and I’m all set! (see photos below).</p><img src="/images/new_external_ultra_wide_curved_monitor_wfh_setup_3.png" class="center" width="500" height="500" title="My new ultra wide curved monitor with split screen between Windows 10 and Mac Sanoma OS My new ultra wide curved monitor with split screen between Windows 10 and Mac Sanoma OS"><p><em><p style="text-align: center;">My new ultra wide curved monitor with split screen between Windows 10 and Mac Sanoma OS</p></em></p><img src="/images/new_external_ultra_wide_curved_monitor_wfh_setup_4.png" class="center" width="500" height="500" title="Close up view of split screen between Windows 10 and Mac Sanoma OS Close up view of split screen between Windows 10 and Mac Sanoma OS"><p><em><p style="text-align: center;">Close up view of split screen between Windows 10 and Mac Sanoma OS</p></em></p><p>Amazing isn’t it? Having ultra wide monitor with KVM switch feature gives me the very good reason to not leave my “abandoned” Windows 7(now upgraded to Windows 10)laptop unused for several years since I moved on to using Macbook a lot profesionally. I couldn’t find the space for my WFH setup prior to getting the ultra wide monitor.. but now it’s not longer sitting like an old furniture just to collect dust over time. I can have the best of both worlds, side by side, moving forth.</p><p>Have you used an ultra-wide monitor in your coding setup? What has been your experience? Share your thoughts and any tips you might have in the comments below!</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_ultra_wide_curved_monitor.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction:&quot;&gt;&lt;/a&gt;Introduction:&lt;/h2&gt;&lt;p&gt;First off - have a Happy New Year of 2025 everyone! :)&lt;/p&gt;
&lt;p&gt;What a busy and challenging year of 2024 it’s been.&lt;/p&gt;
&lt;p&gt;Looking back, I’ve been wondering what can I look forward in uplifting my personal productivity when in comes to staying lazer-focus, maintain strong context aware of code repositories without losing to context switching, which I have been struggling to achieve for some considerable time.&lt;/p&gt;
&lt;p&gt;Especially when it comes to having the decent monitor screen size as my external display to check out code, read code, write code, submit PRs, review PRs etc.&lt;/p&gt;
&lt;p&gt;Hence the purpose of this writing up this blog post - describing my recent experience of installing ultra-wide curved monitor for my WFH setup.&lt;/p&gt;</summary>
    
    
    
    
    <category term="ultra-wide-curved-monitor developer-productivity hobbyist-project continuous-improvements wfh-hacks better-code-reviews" scheme="http://awongcm.io/tags/ultra-wide-curved-monitor-developer-productivity-hobbyist-project-continuous-improvements-wfh-hacks-better-code-reviews/"/>
    
  </entry>
  
  <entry>
    <title>Bye Bye Heroku, Hello Render! Migrating My Personal Apps from Heroku to Render: A Comprehensive Guide</title>
    <link href="http://awongcm.io/blog/2024/07/14/bye-bye-heroku-hello-render-migrating-my-personal-apps-from-heroku-to-render-a-comprehensive-guide/"/>
    <id>http://awongcm.io/blog/2024/07/14/bye-bye-heroku-hello-render-migrating-my-personal-apps-from-heroku-to-render-a-comprehensive-guide/</id>
    <published>2024-07-14T11:18:04.000Z</published>
    <updated>2026-03-30T12:13:51.447Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_migrated_heroku_to_render.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>In the early days of my software development career, I relied on Heroku to host all of my personal apps without incurring any costs. However, over the years, Heroku has gradually shifted from a free model to a subscription-based service. As a software developer, I’m always on the lookout for platforms that offer better performance, scalability, and ease of use for hosting my applications while continuing to manage my projects on budget. After careful consideration and evaluation, I’ve decided to migrate all of my personal apps from Heroku to Render. In this guide, I’ll walk you through the migration process and explain the reasons behind my decision.</p><span id="more"></span><h2 id="Why-Migrate-to-Render"><a href="#Why-Migrate-to-Render" class="headerlink" title="Why Migrate to Render?"></a>Why Migrate to Render?</h2><h3 id="1-Simplicity-and-Ease-of-Use"><a href="#1-Simplicity-and-Ease-of-Use" class="headerlink" title="1. Simplicity and Ease of Use"></a>1. <strong>Simplicity and Ease of Use</strong></h3><p>Render offers a streamlined and intuitive platform for deploying and managing applications. Its simple interface and straightforward setup make it easy for developers to get up and running quickly without the complexity of managing infrastructure.</p><h3 id="2-Automatic-Scaling-and-High-Availability"><a href="#2-Automatic-Scaling-and-High-Availability" class="headerlink" title="2. Automatic Scaling and High Availability"></a>2. <strong>Automatic Scaling and High Availability</strong></h3><p>Render automatically scales applications based on traffic patterns, ensuring optimal performance and reliability without manual intervention. With built-in redundancy and failover mechanisms, Render provides high availability and uptime guarantees, even during peak traffic.</p><h3 id="3-Transparent-Pricing"><a href="#3-Transparent-Pricing" class="headerlink" title="3. Transparent Pricing"></a>3. <strong>Transparent Pricing</strong></h3><p>Render’s transparent pricing model provides predictable and affordable pricing for hosting applications. With no hidden fees or complex pricing tiers, I have full visibility into my hosting costs, allowing me to budget effectively and avoid unexpected charges.</p><h3 id="4-Seamless-Integration-with-Modern-Technologies"><a href="#4-Seamless-Integration-with-Modern-Technologies" class="headerlink" title="4. Seamless Integration with Modern Technologies"></a>4. <strong>Seamless Integration with Modern Technologies</strong></h3><p>Like Heroku, Render supports a wide range of programming languages, frameworks, and databases, making it easy to deploy applications built with modern technologies. Whether I’m working with Node.js, Python, Ruby, Go, or Docker containers, Render has me covered.</p><h3 id="5-Excellent-Documentation-and-Support"><a href="#5-Excellent-Documentation-and-Support" class="headerlink" title="5. Excellent Documentation and Support"></a>5. <strong>Excellent Documentation and Support</strong></h3><p>Render offers comprehensive documentation and responsive support to assist developers at every stage of the deployment process. From getting started guides to troubleshooting tips, Render’s support resources are invaluable for ensuring a smooth migration experience.</p><h2 id="Migration-Guide"><a href="#Migration-Guide" class="headerlink" title="Migration Guide"></a>Migration Guide</h2><h3 id="1-Create-a-Render-Account"><a href="#1-Create-a-Render-Account" class="headerlink" title="1. Create a Render Account"></a>1. <strong>Create a Render Account</strong></h3><ul><li>Firstly, visit the <a href="https://render.com/">Render website</a> and sign up for a free account.</li><li>Verify your email address and complete the account setup process.</li></ul><h3 id="2-Import-Your-Applications"><a href="#2-Import-Your-Applications" class="headerlink" title="2. Import Your Applications"></a>2. <strong>Import Your Applications</strong></h3><ul><li>Secondly, navigate to the Render dashboard and click on the “Create New” button.</li><li>Select the type of application you want to deploy (e.g., Web Service, Static Site, Docker Service) and follow the prompts to import your codebase.</li></ul><p>For my case, I had about 8 personal projects to migrate, 6 are static site (written in Javascript&#x2F;React plus my tech blog written in Hexo framework and my web portfolio powered by Pug&#x2F;CSS&#x2F;Javascript), one Ruby on Rails API service and one NodeJS smtp service. Prior to the writing this, I found that I had to learn this concept from Render.com documentation pages about BluePrint yaml files. Fundamentally, they are just your IaC much akin similar to Ansible, AWS Cloudformation, Chef, Terraform etc, but are more simpler and easier to construct without having to memorise several configuration parameters at great length. See the upcoming notes on point <code>3</code>.</p><h3 id="3-Configure-Your-Environment"><a href="#3-Configure-Your-Environment" class="headerlink" title="3. Configure Your Environment"></a>3. <strong>Configure Your Environment</strong></h3><ul><li>Customize your deployment settings, including environment variables, build commands, and routing rules, to match your application’s requirements.</li><li>Take advantage of Render’s integrations with GitHub, GitLab, and Bitbucket for seamless CI&#x2F;CD workflows.</li></ul><p>As mentioned from point <code>2</code>, to migrate Ruby on Rails API service as a web service for eg, I had to setup configure its Blueprint yaml config file. It’s usually comes as <code>render.yaml</code> by default (see below).</p><pre><code class="hljs yaml"><span class="hljs-attr">services:</span>  <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">post-it-notes-board-api</span>    <span class="hljs-attr">type:</span> <span class="hljs-string">web</span>    <span class="hljs-attr">env:</span> <span class="hljs-string">ruby</span>    <span class="hljs-attr">buildCommand:</span> <span class="hljs-string">bin/render-build.sh</span>    <span class="hljs-attr">startCommand:</span> <span class="hljs-string">bundle</span> <span class="hljs-string">exec</span> <span class="hljs-string">puma</span> <span class="hljs-string">-C</span> <span class="hljs-string">config/puma.rb</span>    <span class="hljs-attr">envVars:</span>      <span class="hljs-bullet">-</span> <span class="hljs-attr">key:</span> <span class="hljs-string">SECRET_KEY_BASE</span>        <span class="hljs-attr">generateValue:</span> <span class="hljs-literal">true</span>      <span class="hljs-bullet">-</span> <span class="hljs-attr">key:</span> <span class="hljs-string">DATABASE_URL</span>        <span class="hljs-attr">fromDatabase:</span>          <span class="hljs-attr">name:</span> <span class="hljs-string">post-it-notes-board-db</span>          <span class="hljs-attr">property:</span> <span class="hljs-string">connectionString</span>      <span class="hljs-bullet">-</span> <span class="hljs-attr">key:</span> <span class="hljs-string">REDIS_URL</span> <span class="hljs-comment"># this must match the name of the environment variable used in production.rb</span>        <span class="hljs-attr">fromService:</span>          <span class="hljs-attr">type:</span> <span class="hljs-string">redis</span>          <span class="hljs-attr">name:</span> <span class="hljs-string">post-it-notes-board-api-redis</span>          <span class="hljs-attr">property:</span> <span class="hljs-string">connectionString</span>  <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">post-it-notes-board-api-redis</span>    <span class="hljs-attr">type:</span> <span class="hljs-string">redis</span>    <span class="hljs-attr">ipAllowList:</span> [] <span class="hljs-comment"># only allow connections from services in this Render accoun</span>    <span class="hljs-attr">maxmemoryPolicy:</span> <span class="hljs-string">allkeys-lfu</span><span class="hljs-attr">databases:</span>  <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">post-it-notes-board-db</span>    <span class="hljs-attr">ipAllowList:</span> [] <span class="hljs-comment"># only allow connections from services in this Render account</span></code></pre><p>I needed the Blueprint yaml file for this web service is because I knew my RAILS API service would be interconnected with Redis as well as Postgres DB services in order to be fully functional full stack API app.  </p><p>Looking at the YAML file content above, I found this is actually far more simplistic and incredibly easy to comprehend in setting up compared to working with AWS Cloudformation I had for years where you are obliged to know all nooks and cranies IaC’s parameters to setup your own infrastructure at such length. But with this, I only need to know few basic IaC Render provides you and that’s suffice enough to get your app hosted and up running with very minimal development.</p><p>For more info, on how to interpret and setup this yaml file, look up this page to understand more - <a href="https://docs.render.com/blueprint-spec">https://docs.render.com/blueprint-spec</a></p><h3 id="4-Deploy-Your-Applications"><a href="#4-Deploy-Your-Applications" class="headerlink" title="4. Deploy Your Applications"></a>4. <strong>Deploy Your Applications</strong></h3><ul><li>Trigger a deployment of your applications by pushing your code to the Render Git repository or using the Render CLI.</li><li>Monitor the deployment process in the Render dashboard and review the deployment logs for any errors or warnings.</li></ul><img src="/images/deployed_apps_render.png" class="center" width="750" height="750" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">My list of personal apps hosted at Render</p></em></p><h3 id="5-Test-and-Verify"><a href="#5-Test-and-Verify" class="headerlink" title="5. Test and Verify"></a>5. <strong>Test and Verify</strong></h3><ul><li>Once your applications are deployed, test them thoroughly to ensure that they’re functioning correctly on Render’s platform.</li><li>Use Render’s built-in monitoring and logging features to track performance metrics and diagnose any issues that arise.</li></ul><h3 id="6-Update-DNS-Records"><a href="#6-Update-DNS-Records" class="headerlink" title="6. Update DNS Records"></a>6. <strong>Update DNS Records</strong></h3><ul><li>If you’re using a custom domain with Heroku, update your DNS records to point to Render’s servers to ensure that traffic is directed to your new deployments.</li><li>Follow Render’s documentation for detailed instructions on updating DNS records for your domain provider.</li></ul><p>Fortunately, for my case, I don’t have that many to begin with so the DNS record transitioning was super breezy when following this <a href="https://docs.render.com/configure-namecheap-dns">link</a> to configure Namecheap DNS hosting.</p><h3 id="7-Monitor-and-Optimize"><a href="#7-Monitor-and-Optimize" class="headerlink" title="7. Monitor and Optimize"></a>7. <strong>Monitor and Optimize</strong></h3><ul><li>Continuously monitor your applications’ performance and resource usage on Render to identify areas for optimization and improvement.</li><li>Take advantage of Render’s scaling and caching features to optimize performance and reduce costs over time.</li></ul><p>That’s pretty much it!</p><p>With all that said and done, why not considering popular cloud choices like AWS, Azure, etc?</p><p>That’s a fair question. </p><p>I had originally put some hard thought into this.. But honestly speaking, using my extensive experience in building software solutions with major cloud provider players out there is not going to be the best use case for it just simply because the inherent complex features of it do not hugely outweigh the benefits for my small to medium-sized apps where they don’t require high end capability of scalabitity and function customizations etc.  AWS, Azure etc are good for building enterprise-grade applications by default, especially if you know your application scalability needs upfront already. And I mentioned earlier that I want to have my apps running in the cloud with manageable shoetring or free budget to maintain the cost as the primary goal, Render would be perfect choice for it. I can always migrate them to AWS if I do decided to level up their scalablity once I do gain some traction for these.  </p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>Migrating my personal apps from Heroku to Render was not only a strategic decision, but also an personal one based on the platform’s superior performance, scalability, and ease of use. As much as I’d love Heroku for all the important things I wanted to do in making my hobbyist projects fun to begin with, I’ve come to a point where I have to say my difficult goodbyes to it after receiving this email newsletter from Heroku less than 2 year ago..</p><img src="/images/heroku_free_plan_discontinued_2022_letter_part1.png" class="center" width="750" height="750" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Letter Notification to end Heroku free plans - Part 1</p></em></p><img src="/images/heroku_free_plan_discontinued_2022_letter_part2.png" class="center" width="750" height="750" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Letter Notification to end Heroku free plans - Part 2</p></em></p><p>Upon reading this newsletter, I went further to discover the true reason behind their tough decision to discontinue their free plan <a href="https://techcrunch.com/2022/08/25/heroku-announces-plans-to-eliminate-free-plans-blaming-fraud-and-abuse/">here</a>, I can start to empathise them to chose putting the nail to the free-plan coffin.</p><p>With Render, I have confidence that my applications will remain highly available, performant, and cost-effective, allowing me to focus on building and improving my software projects.</p><p>If you’re considering migrating your applications to Render, I highly recommend taking the leap. With Render’s powerful platform and comprehensive support, you’ll be well-equipped to take your applications to the next level before I say my parting farewells to Heroku after staying with them close to 8 years before their discontinued free-plan was announced. Thanks for the fun memories. 🥲🥲</p><p>Till then, Happy coding on Render!</p><p><strong>PS:</strong> On one slight note, after migrating my RAILS API app to Render, I learned Render had 30-day limit expiry when running free PostgresSQL DB service ie <code>post-it-notes-board-db</code> once deployed. Once 30 day limit is up, you need to upgrade DB service from Free tier to Starter tier package where I will be billed around $7USD&#x2F;month for running DB service. Since I wasn’t keen to make the commitment to sign this up immediately after slowly all of my apps to Render free tier option already, I decided to opt DB service out of Render and configure DB service elsewhere using Supabase when reading this <a href="https://supabase.com/docs/guides/getting-started/quickstarts/ruby-on-rails">guide</a>. After creating my SupaBase account, the setup was incrediby straighforward. I just need to configure <code>DATABASE_URL</code>env var to point to an actual Supabase PostgresDb instance and I can run db rails migrate commands to seed data on that DB service as if it was my own local db service! The DB transitioning was also incredibly painless! You should definitely checkout <a href="https://supabase.com/">Supabase</a> as well. It’s build for small hobbyist projects for small budget in mind.</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_migrated_heroku_to_render.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;In the early days of my software development career, I relied on Heroku to host all of my personal apps without incurring any costs. However, over the years, Heroku has gradually shifted from a free model to a subscription-based service. As a software developer, I’m always on the lookout for platforms that offer better performance, scalability, and ease of use for hosting my applications while continuing to manage my projects on budget. After careful consideration and evaluation, I’ve decided to migrate all of my personal apps from Heroku to Render. In this guide, I’ll walk you through the migration process and explain the reasons behind my decision.&lt;/p&gt;</summary>
    
    
    
    
    <category term="cloud-appllication-development application-migration hobbyist-project free-tier javascript reactjs ruby subscription-based heroku render" scheme="http://awongcm.io/tags/cloud-appllication-development-application-migration-hobbyist-project-free-tier-javascript-reactjs-ruby-subscription-based-heroku-render/"/>
    
  </entry>
  
  <entry>
    <title>Demystifying Common Design Patterns in Modern Software Development</title>
    <link href="http://awongcm.io/blog/2024/04/21/demystifying-common-design-patterns-in-modern-software-development/"/>
    <id>http://awongcm.io/blog/2024/04/21/demystifying-common-design-patterns-in-modern-software-development/</id>
    <published>2024-04-21T02:22:42.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_design_patterns.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>In my previous <a href="http://www.awongcm.io/blog/2019/11/17/most-common-and-useful-design-patterns-you-should-be-aware-of-as-a-javascript-developer/">post</a>, I mentioned the usefulness of apply software patterns for scalable Javascript applications so far. I thought of revisiting this again because after working as a software engineer&#x2F;developer for a while, I always found myself that design patterns are heavily applied across all projects (outside of Javascript domain) I have seen. Those projects were anything around Java, PHP, Python, C++, etc, thus I found there’s a common notion that design patterns are ubiquitous when applying to solve interesting and challenging software problems in our modern times.</p><span id="more"></span><p>To recap, design patterns are essential tools in a software developer’s toolkit, offering proven solutions to recurring problems in software design. In this blog post, we’ll explore some of more common design patterns, their use cases, and how they enhance the structure and flexibility of modern software applications.</p><h2 id="Table-of-Contents"><a href="#Table-of-Contents" class="headerlink" title="Table of Contents"></a>Table of Contents</h2><ol><li><a href="#What-are-Design-Patterns">What are Design Patterns?</a></li><li><a href="#Creational-Patterns">Creational Patterns</a><ul><li><a href="#Singleton-Pattern">Singleton Pattern</a></li><li><a href="#Factory-Method-Pattern">Factory Method Pattern</a></li><li><a href="#Abstract-Factory-Pattern">Abstract Factory Pattern</a></li></ul></li><li><a href="#Structural-Patterns">Structural Patterns</a><ul><li><a href="#Adapter-Pattern">Adapter Pattern</a></li><li><a href="#Decorator-Pattern">Decorator Pattern</a></li><li><a href="#Facade-Pattern">Facade Pattern</a></li></ul></li><li><a href="#Behavioral-Patterns">Behavioral Patterns</a><ul><li><a href="#Observer-Pattern">Observer Pattern</a></li><li><a href="#Strategy-Pattern">Strategy Pattern</a></li><li><a href="#Command-Pattern">Command Pattern</a></li></ul></li><li><a href="#Use-Cases-and-Examples">Use Cases and Examples</a></li><li><a href="#What-does-this-all-mean">What does this all mean?</a></li><li><a href="#Conclusion">Conclusion</a></li></ol><h2 id="What-are-Design-Patterns"><a href="#What-are-Design-Patterns" class="headerlink" title="What are Design Patterns?"></a>What are Design Patterns?</h2><p>Design patterns are fundamentally general reusable solutions to common problems encountered in software design. They represent best practices and provide a clear, effective way to solve design issues that aim to accomplish 3 main objectives. They are to:</p><ul><li>Promote code reusability</li><li>Promote ease of maintainability</li><li>Promote flexibility for future changes or enhancements</li></ul><h2 id="Creational-Patterns"><a href="#Creational-Patterns" class="headerlink" title="Creational Patterns"></a>Creational Patterns</h2><p>Creational patterns are concerned with the creation of objects. They provide various mechanisms for object creation, allowing developers to decouple the instantiation process from the actual implementation of the objects.</p><h3 id="Singleton-Pattern"><a href="#Singleton-Pattern" class="headerlink" title="Singleton Pattern"></a>Singleton Pattern</h3><p>The Singleton pattern ensures that a class has only one instance and provides a global point of access to that instance. This pattern is useful when you want to restrict the instantiation of a class to a single object, such as managing a shared resource or controlling access to a global configuration.</p><h3 id="Factory-Method-Pattern"><a href="#Factory-Method-Pattern" class="headerlink" title="Factory Method Pattern"></a>Factory Method Pattern</h3><p>The Factory Method pattern defines an interface for creating objects but allows subclasses to alter the type of objects that will be created. This pattern is useful when you want to delegate the creation of objects to subclasses, providing a way to create objects without specifying their exact class.</p><h3 id="Abstract-Factory-Pattern"><a href="#Abstract-Factory-Pattern" class="headerlink" title="Abstract Factory Pattern"></a>Abstract Factory Pattern</h3><p>The Abstract Factory pattern provides an interface for creating families of related or dependent objects without specifying their concrete classes. This pattern is useful when you want to ensure that the created objects are compatible with each other and belong to the same family.</p><h2 id="Structural-Patterns"><a href="#Structural-Patterns" class="headerlink" title="Structural Patterns"></a>Structural Patterns</h2><p>Structural patterns are concerned with the composition of classes or objects to form larger structures. They focus on simplifying the structure of a system and promoting flexibility and reusability.</p><h3 id="Adapter-Pattern"><a href="#Adapter-Pattern" class="headerlink" title="Adapter Pattern"></a>Adapter Pattern</h3><p>The Adapter pattern allows incompatible interfaces to work together by wrapping an existing class with a new interface. This pattern is useful when integrating new features or components into an existing system without modifying its existing codebase.</p><h3 id="Decorator-Pattern"><a href="#Decorator-Pattern" class="headerlink" title="Decorator Pattern"></a>Decorator Pattern</h3><p>The Decorator pattern attaches additional responsibilities to an object dynamically. This pattern is useful for extending the functionalities of classes in a flexible and reusable way without modifying their code.</p><h3 id="Facade-Pattern"><a href="#Facade-Pattern" class="headerlink" title="Facade Pattern"></a>Facade Pattern</h3><p>The Facade pattern provides a unified interface to a set of interfaces in a subsystem. This pattern is useful when simplifying complex systems and presenting a simplified interface to the client, hiding the complexities of the subsystem.</p><h2 id="Behavioral-Patterns"><a href="#Behavioral-Patterns" class="headerlink" title="Behavioral Patterns"></a>Behavioral Patterns</h2><p>Behavioral patterns are concerned with the interaction and communication between objects. They focus on defining how objects collaborate to achieve common goals and behaviors.</p><h3 id="Observer-Pattern"><a href="#Observer-Pattern" class="headerlink" title="Observer Pattern"></a>Observer Pattern</h3><p>The Observer pattern defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. This pattern is useful when you want to establish a loosely coupled communication between objects, allowing them to communicate and synchronize changes efficiently.</p><h3 id="Strategy-Pattern"><a href="#Strategy-Pattern" class="headerlink" title="Strategy Pattern"></a>Strategy Pattern</h3><p>The Strategy pattern defines a family of algorithms, encapsulates each one, and makes them interchangeable. This pattern is useful when you want to allow clients to choose algorithms from a family dynamically, providing a way to switch between different algorithms at runtime.</p><h3 id="Command-Pattern"><a href="#Command-Pattern" class="headerlink" title="Command Pattern"></a>Command Pattern</h3><p>The Command pattern encapsulates a request as an object, thereby allowing for parameterization of clients with different requests, queuing of requests, and logging of the parameters. This pattern is useful for decoupling the sender and receiver of a request, providing a way to decouple command execution from command invocation.</p><h2 id="Use-Cases-and-Examples"><a href="#Use-Cases-and-Examples" class="headerlink" title="Use Cases and Examples"></a>Use Cases and Examples</h2><p>In this section, we’ll explore real-world use cases for each design pattern, providing examples and insights into how they can be applied effectively in modern software development. We can achieve these using in any programming languages we like. For the intents and purposes of this blog, I’m going to pick Python as a language of choice because it’s a beginner friendly language to understand.</p><h3 id="Creational-Patterns-1"><a href="#Creational-Patterns-1" class="headerlink" title="Creational Patterns"></a>Creational Patterns</h3><h4 id="Singleton-Pattern-1"><a href="#Singleton-Pattern-1" class="headerlink" title="Singleton Pattern"></a>Singleton Pattern</h4><p><strong>Use Case:</strong> Logging Systems<br>In a logging system, you may want to ensure that there’s only one instance of the logger throughout your application to maintain a single log file and prevent multiple instances from causing conflicts.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">class</span> <span class="hljs-title class_">Logger</span>:    _instance = <span class="hljs-literal">None</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__new__</span>(<span class="hljs-params">cls</span>):        <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> cls._instance:            cls._instance = <span class="hljs-built_in">super</span>().__new__(cls)        <span class="hljs-keyword">return</span> cls._instance<span class="hljs-comment"># Usage</span>logger1 = Logger()logger2 = Logger()<span class="hljs-built_in">print</span>(logger1 <span class="hljs-keyword">is</span> logger2)  <span class="hljs-comment"># Output: True</span></code></pre><h4 id="Factory-Method-Pattern-1"><a href="#Factory-Method-Pattern-1" class="headerlink" title="Factory Method Pattern"></a>Factory Method Pattern</h4><p><strong>Use Case:</strong> Document Creation<br>In a document editor application, different types of documents (e.g., text documents, spreadsheets) may need to be created. Using a factory method pattern, you can define an interface for creating documents and let subclasses decide which type of document to instantiate.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">from</span> abc <span class="hljs-keyword">import</span> ABC, abstractmethod<span class="hljs-keyword">class</span> <span class="hljs-title class_">Document</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">TextDocument</span>(<span class="hljs-title class_ inherited__">Document</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;Text Document Created&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">SpreadsheetDocument</span>(<span class="hljs-title class_ inherited__">Document</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;Spreadsheet Document Created&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">DocumentFactory</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create_document</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">TextDocumentFactory</span>(<span class="hljs-title class_ inherited__">DocumentFactory</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create_document</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> TextDocument()<span class="hljs-keyword">class</span> <span class="hljs-title class_">SpreadsheetDocumentFactory</span>(<span class="hljs-title class_ inherited__">DocumentFactory</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create_document</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> SpreadsheetDocument()<span class="hljs-comment"># Usage</span>text_document_factory = TextDocumentFactory()text_document = text_document_factory.create_document()<span class="hljs-built_in">print</span>(text_document.create())  <span class="hljs-comment"># Output: Text Document Created</span>spreadsheet_document_factory = SpreadsheetDocumentFactory()spreadsheet_document = spreadsheet_document_factory.create_document()<span class="hljs-built_in">print</span>(spreadsheet_document.create())  <span class="hljs-comment"># Output: Spreadsheet Document Created</span></code></pre><h4 id="Abstract-Factory-Method-Pattern"><a href="#Abstract-Factory-Method-Pattern" class="headerlink" title="Abstract Factory Method Pattern"></a>Abstract Factory Method Pattern</h4><p><strong>Use Case:</strong> GUI Toolkit<br>In a GUI toolkit, you may need to create different types of UI elements (e.g., buttons, text fields) for different operating systems. Using an abstract factory pattern, you can create families of related UI elements without specifying their concrete classes.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">from</span> abc <span class="hljs-keyword">import</span> ABC, abstractmethod<span class="hljs-keyword">class</span> <span class="hljs-title class_">Button</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">paint</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">LinuxButton</span>(<span class="hljs-title class_ inherited__">Button</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">paint</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;Rendering a Linux Button&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">WindowsButton</span>(<span class="hljs-title class_ inherited__">Button</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">paint</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;Rendering a Windows Button&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">GUIFactory</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create_button</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">LinuxGUIFactory</span>(<span class="hljs-title class_ inherited__">GUIFactory</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create_button</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> LinuxButton()<span class="hljs-keyword">class</span> <span class="hljs-title class_">WindowsGUIFactory</span>(<span class="hljs-title class_ inherited__">GUIFactory</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">create_button</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> WindowsButton()<span class="hljs-comment"># Usage</span>linux_factory = LinuxGUIFactory()linux_button = linux_factory.create_button()<span class="hljs-built_in">print</span>(linux_button.paint())  <span class="hljs-comment"># Output: Rendering a Linux Button</span>windows_factory = WindowsGUIFactory()windows_button = windows_factory.create_button()<span class="hljs-built_in">print</span>(windows_button.paint())  <span class="hljs-comment"># Output: Rendering a Windows Button</span></code></pre><h3 id="Structural-Patterns-1"><a href="#Structural-Patterns-1" class="headerlink" title="Structural Patterns"></a>Structural Patterns</h3><h4 id="Adapter-Pattern-1"><a href="#Adapter-Pattern-1" class="headerlink" title="Adapter Pattern"></a>Adapter Pattern</h4><p><strong>Use Case:</strong> Legacy System Integration<br>When integrating a new system with existing legacy systems, the adapter pattern can be used to adapt the interface of the new system to match the interface expected by the legacy system.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">class</span> <span class="hljs-title class_">Target</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">request</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;Target: The default target&#x27;s behavior.&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Adaptee</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">specific_request</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;.eetpadA eht fo roivaheb laicepS&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Adapter</span>(<span class="hljs-title class_ inherited__">Target</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, adaptee: Adaptee</span>):        <span class="hljs-variable language_">self</span>.adaptee = adaptee    <span class="hljs-keyword">def</span> <span class="hljs-title function_">request</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        <span class="hljs-keyword">return</span> <span class="hljs-string">f&quot;Adapter: (TRANSLATED) <span class="hljs-subst">&#123;self.adaptee.specific_request()[::-<span class="hljs-number">1</span>]&#125;</span>&quot;</span><span class="hljs-comment"># Usage</span>adaptee = Adaptee()adapter = Adapter(adaptee)<span class="hljs-built_in">print</span>(adapter.request())  <span class="hljs-comment"># Output: Adapter: (TRANSLATED) Special behavior of the Adaptee.</span></code></pre><h4 id="Decorator-Pattern-1"><a href="#Decorator-Pattern-1" class="headerlink" title="Decorator Pattern"></a>Decorator Pattern</h4><p><strong>Use Case:</strong> Text Formatting<br>In a text editor application, the decorator pattern can be used to add additional formatting options to text, such as bold, italic, or underline, without modifying the original text class.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">from</span> abc <span class="hljs-keyword">import</span> ABC, abstractmethod<span class="hljs-keyword">class</span> <span class="hljs-title class_">TextComponent</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">render</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Text</span>(<span class="hljs-title class_ inherited__">TextComponent</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, content</span>):        <span class="hljs-variable language_">self</span>._content = content    <span class="hljs-keyword">def</span> <span class="hljs-title function_">render</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">f&quot;Content: <span class="hljs-subst">&#123;self._content&#125;</span>&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">TextDecorator</span>(<span class="hljs-title class_ inherited__">TextComponent</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, text_component: TextComponent</span>):        <span class="hljs-variable language_">self</span>._text_component = text_component    <span class="hljs-keyword">def</span> <span class="hljs-title function_">render</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-variable language_">self</span>._text_component.render()<span class="hljs-keyword">class</span> <span class="hljs-title class_">BoldDecorator</span>(<span class="hljs-title class_ inherited__">TextDecorator</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">render</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">f&quot;&lt;b&gt;<span class="hljs-subst">&#123;self._text_component.render()&#125;</span>&lt;/b&gt;&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">ItalicDecorator</span>(<span class="hljs-title class_ inherited__">TextDecorator</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">render</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">f&quot;&lt;i&gt;<span class="hljs-subst">&#123;self._text_component.render()&#125;</span>&lt;/i&gt;&quot;</span><span class="hljs-comment"># Usage</span>text = Text(<span class="hljs-string">&quot;Hello, world!&quot;</span>)bold_text = BoldDecorator(text)italic_text = ItalicDecorator(text)<span class="hljs-built_in">print</span>(text.render())  <span class="hljs-comment"># Output: Content: Hello, world!</span><span class="hljs-built_in">print</span>(bold_text.render())  <span class="hljs-comment"># Output: &lt;b&gt;Content: Hello, world!&lt;/b&gt;</span><span class="hljs-built_in">print</span>(italic_text.render())  <span class="hljs-comment"># Output: &lt;i&gt;Content: Hello, world!&lt;/i&gt;</span></code></pre><h4 id="Facade-Pattern-1"><a href="#Facade-Pattern-1" class="headerlink" title="Facade Pattern"></a>Facade Pattern</h4><p><strong>Use Case:</strong> Complex Subsystem Simplification<br>In a computer system, the facade pattern can be used to provide a simplified interface to a complex subsystem, hiding its internal details and providing a single entry point for client interaction.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">class</span> <span class="hljs-title class_">SubsystemA</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">operation_a1</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;SubsystemA: Ready!&quot;</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">operation_a2</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;SubsystemA: Go!&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">SubsystemB</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">operation_b1</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;SubsystemB: Fire!&quot;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Facade</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, subsystem_a: SubsystemA, subsystem_b: SubsystemB</span>):        <span class="hljs-variable language_">self</span>._subsystem_a = subsystem_a        <span class="hljs-variable language_">self</span>._subsystem_b = subsystem_b    <span class="hljs-keyword">def</span> <span class="hljs-title function_">operation</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-built_in">str</span>:        results = []        results.append(<span class="hljs-string">&quot;Facade initializes subsystems:&quot;</span>)        results.append(<span class="hljs-variable language_">self</span>._subsystem_a.operation_a1())        results.append(<span class="hljs-variable language_">self</span>._subsystem_b.operation_b1())        results.append(<span class="hljs-string">&quot;Facade orders subsystems to perform the action:&quot;</span>)        results.append(<span class="hljs-variable language_">self</span>._subsystem_a.operation_a2())        <span class="hljs-keyword">return</span> <span class="hljs-string">&quot;\n&quot;</span>.join(results)<span class="hljs-comment"># Usage</span>subsystem_a = SubsystemA()subsystem_b = SubsystemB()facade = Facade(subsystem_a, subsystem_b)<span class="hljs-built_in">print</span>(facade.operation())</code></pre><h3 id="Behavioral-Patterns-1"><a href="#Behavioral-Patterns-1" class="headerlink" title="Behavioral Patterns"></a>Behavioral Patterns</h3><h4 id="Observer-Pattern-1"><a href="#Observer-Pattern-1" class="headerlink" title="Observer Pattern"></a>Observer Pattern</h4><p><strong>Use Case:</strong> Event Handling<br>In a graphical user interface (GUI) framework, the observer pattern can be used to implement event handling mechanisms, where multiple objects (observers) are notified when a particular event occurs.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">from</span> abc <span class="hljs-keyword">import</span> ABC, abstractmethod<span class="hljs-keyword">class</span> <span class="hljs-title class_">Subject</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">attach</span>(<span class="hljs-params">self, observer: <span class="hljs-string">&#x27;Observer&#x27;</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">detach</span>(<span class="hljs-params">self, observer: <span class="hljs-string">&#x27;Observer&#x27;</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">notify</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">ConcreteSubject</span>(<span class="hljs-title class_ inherited__">Subject</span>):    _state: <span class="hljs-built_in">int</span> = <span class="hljs-literal">None</span>    _observers: <span class="hljs-type">List</span>[<span class="hljs-string">&#x27;Observer&#x27;</span>] = []    <span class="hljs-keyword">def</span> <span class="hljs-title function_">attach</span>(<span class="hljs-params">self, observer: <span class="hljs-string">&#x27;Observer&#x27;</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;Subject: Attached an observer.&quot;</span>)        <span class="hljs-variable language_">self</span>._observers.append(observer)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">detach</span>(<span class="hljs-params">self, observer: <span class="hljs-string">&#x27;Observer&#x27;</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._observers.remove(observer)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">notify</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;Subject: Notifying observers...&quot;</span>)        <span class="hljs-keyword">for</span> observer <span class="hljs-keyword">in</span> <span class="hljs-variable language_">self</span>._observers:            observer.update(<span class="hljs-variable language_">self</span>)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">some_business_logic</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;\nSubject: I&#x27;m doing something important.&quot;</span>)        <span class="hljs-variable language_">self</span>._state = <span class="hljs-number">5</span>        <span class="hljs-built_in">print</span>(<span class="hljs-string">f&quot;Subject: My state has just changed to: <span class="hljs-subst">&#123;self._state&#125;</span>&quot;</span>)        <span class="hljs-variable language_">self</span>.notify()<span class="hljs-keyword">class</span> <span class="hljs-title class_">Observer</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">update</span>(<span class="hljs-params">self, subject: Subject</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">ConcreteObserverA</span>(<span class="hljs-title class_ inherited__">Observer</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">update</span>(<span class="hljs-params">self, subject: Subject</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">if</span> subject._state &lt; <span class="hljs-number">3</span>:            <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;ConcreteObserverA: Reacted to the event&quot;</span>)<span class="hljs-keyword">class</span> <span class="hljs-title class_">ConcreteObserverB</span>(<span class="hljs-title class_ inherited__">Observer</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">update</span>(<span class="hljs-params">self, subject: Subject</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">if</span> subject._state == <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> subject._state &gt;= <span class="hljs-number">2</span>:            <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;ConcreteObserverB: Reacted to the event&quot;</span>)<span class="hljs-comment"># Usage</span>subject = ConcreteSubject()observer_a = ConcreteObserverA()subject.attach(observer_a)observer_b = ConcreteObserverB()subject.attach(observer_b)subject.some_business_logic()subject.some_business_logic()</code></pre><h4 id="Strategy-Pattern-1"><a href="#Strategy-Pattern-1" class="headerlink" title="Strategy Pattern"></a>Strategy Pattern</h4><p><strong>Use Case:</strong> Sorting Algorithms<br>In a sorting algorithm library, the strategy pattern can be used to encapsulate various sorting algorithms (e.g., bubble sort, quicksort) into separate strategy classes, allowing clients to choose the desired sorting strategy dynamically.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">from</span> abc <span class="hljs-keyword">import</span> ABC, abstractmethod<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> <span class="hljs-type">List</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Strategy</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute</span>(<span class="hljs-params">self, data: <span class="hljs-type">List</span>[<span class="hljs-built_in">int</span>]</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">BubbleSortStrategy</span>(<span class="hljs-title class_ inherited__">Strategy</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute</span>(<span class="hljs-params">self, data: <span class="hljs-type">List</span>[<span class="hljs-built_in">int</span>]</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">f&quot;Bubble Sort: Sorting <span class="hljs-subst">&#123;data&#125;</span>&quot;</span>)        <span class="hljs-comment"># Implement bubble sort algorithm</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">QuickSortStrategy</span>(<span class="hljs-title class_ inherited__">Strategy</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute</span>(<span class="hljs-params">self, data: <span class="hljs-type">List</span>[<span class="hljs-built_in">int</span>]</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">f&quot;Quick Sort: Sorting <span class="hljs-subst">&#123;data&#125;</span>&quot;</span>)        <span class="hljs-comment"># Implement quicksort algorithm</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Context</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, strategy: Strategy</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._strategy = strategy    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute_strategy</span>(<span class="hljs-params">self, data: <span class="hljs-type">List</span>[<span class="hljs-built_in">int</span>]</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._strategy.execute(data)<span class="hljs-comment"># Usage</span>bubble_sort_strategy = BubbleSortStrategy()context = Context(bubble_sort_strategy)context.execute_strategy([<span class="hljs-number">3</span>, <span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">5</span>, <span class="hljs-number">4</span>])  <span class="hljs-comment"># Output: Bubble Sort: Sorting [3, 1, 2, 5, 4]</span>quick_sort_strategy = QuickSortStrategy()context = Context(quick_sort_strategy)context.execute_strategy([<span class="hljs-number">3</span>, <span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">5</span>, <span class="hljs-number">4</span>])  <span class="hljs-comment"># Output: Quick Sort: Sorting [3, 1, 2, 5, 4]</span></code></pre><h4 id="Command-Pattern-1"><a href="#Command-Pattern-1" class="headerlink" title="Command Pattern"></a>Command Pattern</h4><p><strong>Use Case:</strong> Remote Control<br>In a remote control application, the command pattern can be used to encapsulate various actions (e.g., turning on&#x2F;off a TV, adjusting volume) into separate command objects, allowing clients to execute these actions dynamically.</p><p><strong>Example:</strong></p><pre><code class="hljs python"><span class="hljs-keyword">from</span> abc <span class="hljs-keyword">import</span> ABC, abstractmethod<span class="hljs-keyword">class</span> <span class="hljs-title class_">Command</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">TurnOnCommand</span>(<span class="hljs-title class_ inherited__">Command</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, device: <span class="hljs-string">&#x27;Device&#x27;</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._device = device    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._device.turn_on()<span class="hljs-keyword">class</span> <span class="hljs-title class_">TurnOffCommand</span>(<span class="hljs-title class_ inherited__">Command</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, device: <span class="hljs-string">&#x27;Device&#x27;</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._device = device    <span class="hljs-keyword">def</span> <span class="hljs-title function_">execute</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._device.turn_off()<span class="hljs-keyword">class</span> <span class="hljs-title class_">Device</span>(<span class="hljs-title class_ inherited__">ABC</span>):<span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">turn_on</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-meta">    @abstractmethod</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">turn_off</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">pass</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">TV</span>(<span class="hljs-title class_ inherited__">Device</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">turn_on</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;Turning on the TV...&quot;</span>)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">turn_off</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;Turning off the TV...&quot;</span>)<span class="hljs-keyword">class</span> <span class="hljs-title class_">RemoteControl</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._commands = &#123;&#125;    <span class="hljs-keyword">def</span> <span class="hljs-title function_">register_command</span>(<span class="hljs-params">self, command_name: <span class="hljs-built_in">str</span>, command: Command</span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-variable language_">self</span>._commands[command_name] = command    <span class="hljs-keyword">def</span> <span class="hljs-title function_">press_button</span>(<span class="hljs-params">self, command_name: <span class="hljs-built_in">str</span></span>) -&gt; <span class="hljs-literal">None</span>:        <span class="hljs-keyword">if</span> command_name <span class="hljs-keyword">in</span> <span class="hljs-variable language_">self</span>._commands:            <span class="hljs-variable language_">self</span>._commands[command_name].execute()        <span class="hljs-keyword">else</span>:            <span class="hljs-built_in">print</span>(<span class="hljs-string">&quot;Unknown command.&quot;</span>)<span class="hljs-comment"># Usage</span>remote_control = RemoteControl()tv = TV()turn_on_command = TurnOnCommand(tv)turn_off_command = TurnOffCommand(tv)remote_control.register_command(<span class="hljs-string">&quot;turn_on_tv&quot;</span>, turn_on_command)remote_control.register_command(<span class="hljs-string">&quot;turn_off_tv&quot;</span>, turn_off_command)remote_control.press_button(<span class="hljs-string">&quot;turn_on_tv&quot;</span>)  <span class="hljs-comment"># Output: Turning on the TV...</span>remote_control.press_button(<span class="hljs-string">&quot;turn_off_tv&quot;</span>)  <span class="hljs-comment"># Output: Turning off the TV...</span></code></pre><h2 id="What-does-this-all-mean"><a href="#What-does-this-all-mean" class="headerlink" title="What does this all mean?"></a>What does this all mean?</h2><p>If you notice in the above patterns and use case examples so far, they are geared towards traditional object-oriented languages like Python, Java or similar. This is because design patterns were invented to solve object-oriented software application many many years ago circa in the mid 1990s, which was popularised by the software engineering book: <a href="https://en.wikipedia.org/wiki/Design_Patterns"><em>Design Patterns: Elements of Reusable Object-Oriented Software</em></a>. You can find out more by either googling or ask ChatGPT or similar to recite history behind the inception of such a book. I won’t bore you the details of their motivation in this post, other than offering you the refresher you are going to face a lot of these concept usages over the course of your programming career <em>many times</em> no matter how new or seasoned you are in mastering these.</p><p>What all this means to you, if you’re coming from web or front-end development background like ReactJS&#x2F;Javascript. You can expect to apply the patterns in the following ways (which may have been covered in my previous post already.)</p><h3 id="Singleton-Pattern-2"><a href="#Singleton-Pattern-2" class="headerlink" title="Singleton Pattern"></a>Singleton Pattern</h3><ul><li><strong>Use Case</strong>: Managing Global State<ul><li>In ReactJS applications, you might need to manage global state, such as user authentication status or theme settings. You can use the Singleton pattern to ensure there’s only one instance of your state management object (e.g., Redux store) throughout your application.</li></ul></li></ul><h3 id="Factory-Method-Pattern-2"><a href="#Factory-Method-Pattern-2" class="headerlink" title="Factory Method Pattern"></a>Factory Method Pattern</h3><ul><li><strong>Use Case</strong>: Component Creation<ul><li>In ReactJS, you often create components dynamically based on certain conditions or data. You can use the Factory Method pattern to define an interface for creating React components and allow subclasses or factory functions to decide which type of component to create based on input parameters.</li></ul></li></ul><h3 id="Abstract-Factory-Pattern-1"><a href="#Abstract-Factory-Pattern-1" class="headerlink" title="Abstract Factory Pattern"></a>Abstract Factory Pattern</h3><ul><li><strong>Use Case</strong>: UI Component Libraries<ul><li>When building complex user interfaces in ReactJS, you might use UI component libraries like Material-UI or Ant Design. These libraries often provide abstract factory mechanisms for creating different types of UI components (e.g., buttons, inputs) with consistent styles and behaviors across the application.</li></ul></li></ul><h3 id="Adapter-Pattern-2"><a href="#Adapter-Pattern-2" class="headerlink" title="Adapter Pattern"></a>Adapter Pattern</h3><ul><li><strong>Use Case</strong>: Data Integration<ul><li>When fetching data from APIs in a ReactJS application, you may need to adapt the format of the API response to match the expected format of your components. You can use the Adapter pattern to create adapter functions or components that transform API data into a format that’s compatible with your components.</li></ul></li></ul><h3 id="Decorator-Pattern-2"><a href="#Decorator-Pattern-2" class="headerlink" title="Decorator Pattern"></a>Decorator Pattern</h3><ul><li><strong>Use Case</strong>: Higher-Order Components (HOCs)<ul><li>In ReactJS, Higher-Order Components (HOCs) are functions that take a component and return a new component with additional functionality. This is akin to the Decorator pattern, where you enhance the behavior of a component without modifying its underlying implementation.</li></ul></li></ul><h3 id="Facade-Pattern-2"><a href="#Facade-Pattern-2" class="headerlink" title="Facade Pattern"></a>Facade Pattern</h3><ul><li><strong>Use Case</strong>: Complex State Management<ul><li>React applications often involve complex state management logic, especially in large-scale applications. You can use the Facade pattern to create a facade or wrapper around your state management solution (e.g., Redux, Context API) to provide a simpler interface for interacting with the application state.</li></ul></li></ul><h3 id="Observer-Pattern-2"><a href="#Observer-Pattern-2" class="headerlink" title="Observer Pattern"></a>Observer Pattern</h3><ul><li><strong>Use Case</strong>: Event Handling<ul><li>ReactJS applications often handle user interactions and events (e.g., button clicks, form submissions). You can use the Observer pattern to implement event handling mechanisms where components subscribe to specific events and are notified when those events occur.</li></ul></li></ul><h3 id="Strategy-Pattern-2"><a href="#Strategy-Pattern-2" class="headerlink" title="Strategy Pattern"></a>Strategy Pattern</h3><ul><li><strong>Use Case</strong>: Conditional Rendering<ul><li>React components often need to render different content or UI elements based on certain conditions or user inputs. You can use the Strategy pattern to encapsulate different rendering strategies (e.g., conditional rendering logic) and dynamically choose the appropriate strategy based on runtime conditions.</li></ul></li></ul><h3 id="Command-Pattern-2"><a href="#Command-Pattern-2" class="headerlink" title="Command Pattern"></a>Command Pattern</h3><ul><li><strong>Use Case</strong>: User Actions and Undo&#x2F;Redo<ul><li>In interactive applications built with ReactJS, users perform various actions (e.g., editing text, moving elements). You can use the Command pattern to encapsulate user actions as command objects, allowing you to implement features like undo&#x2F;redo functionality by maintaining a history of executed commands.</li></ul></li></ul><p>Then what about people coming from non-object-oriented programming backgrounds? They are still applicable but if not for all.</p><h3 id="Functional-Programming-Languages"><a href="#Functional-Programming-Languages" class="headerlink" title="Functional Programming Languages:"></a>Functional Programming Languages:</h3><ul><li>Functional programming languages like Haskell, Lisp, and Scala rely heavily on functions as first-class citizens. In these languages, design patterns may be expressed differently but still serve the same purpose of solving common software design problems.</li><li>For example, the Strategy pattern can be implemented using higher-order functions or function composition. Instead of defining classes, you can define functions that encapsulate different algorithms and pass them as arguments to other functions.</li></ul><h3 id="Procedural-Programming-Languages"><a href="#Procedural-Programming-Languages" class="headerlink" title="Procedural Programming Languages:"></a>Procedural Programming Languages:</h3><ul><li>In procedural programming languages like C, design patterns can be implemented using procedural techniques such as function pointers, structs, and modules.</li><li>For example, the Singleton pattern can be implemented using static variables within functions to ensure that only one instance of a resource is created.</li></ul><h3 id="Declarative-Programming-Languages"><a href="#Declarative-Programming-Languages" class="headerlink" title="Declarative Programming Languages:"></a>Declarative Programming Languages:</h3><ul><li>Declarative programming languages like SQL and Prolog focus on expressing what should be done rather than how to do it. Design patterns in these languages may involve expressing common query patterns or rule-based systems.</li><li>For example, in SQL, common design patterns include the use of JOINs, subqueries, and views to structure and manipulate data efficiently.</li></ul><h3 id="Scripting-Languages"><a href="#Scripting-Languages" class="headerlink" title="Scripting Languages:"></a>Scripting Languages:</h3><ul><li>Scripting languages like JavaScript and Ruby support both object-oriented and functional programming paradigms. Design patterns in these languages may involve a combination of object-oriented techniques, higher-order functions, and closures.</li><li>For example, the Observer pattern can be implemented using event listeners in JavaScript to respond to changes in state or user interactions.</li></ul><h3 id="Aspect-Oriented-Programming-Languages"><a href="#Aspect-Oriented-Programming-Languages" class="headerlink" title="Aspect-Oriented Programming Languages:"></a>Aspect-Oriented Programming Languages:</h3><ul><li>Aspect-oriented programming languages like AspectJ allow developers to modularize cross-cutting concerns such as logging, security, and transaction management. Design patterns in these languages may involve defining aspects and weaving them into the codebase.</li><li>For example, the Decorator pattern can be implemented using aspect-oriented techniques to add logging or caching behavior to methods without modifying their code.</li></ul><h3 id="Concurrency-Oriented-Programming-Languages"><a href="#Concurrency-Oriented-Programming-Languages" class="headerlink" title="Concurrency-Oriented Programming Languages:"></a>Concurrency-Oriented Programming Languages:</h3><ul><li>Concurrency-oriented programming languages like Erlang and Go focus on building scalable and fault-tolerant concurrent systems. Design patterns in these languages may involve patterns for message passing, supervision, and error handling.</li><li>For example, the Actor model in Erlang provides a pattern for building concurrent and distributed systems using lightweight processes that communicate via message passing.</li></ul><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>As you can see, having the core understanding of design patterns and knowing when to apply in your projects can significantly improve the quality and maintainability of your software. By incorporating these proven solutions into your development practices, you’ll be better equipped to tackle complex design challenges and build robust, scalable applications.</p><p>Depending on where you are in your career journey, some of these patterns may not look very obvious to you yet. Maybe you’re a graduate or junior developer or mid developer striving to becoming a senior developer and beyond, all this will only make sense to you over time as you build up your programming craft and expertise in your projects. Or perhaps you’re a seasoned developer or senior developer who may have accomplished some if not all of these in your career, it’s a good refresher to come here and revisit them over and over again. Quite often once you’ve developed your domain-knowledge expertise at this level, you’ve have grown wiser and knowledgable when to apply these patterns in your next projects (or not) with great confidence. The key here is about work programming experience as well as having a life-long learning mindset.</p><p>Before I wrap this up, as a disclaimer note, all the Python sample codes I’ve provided here so far are by no means prescriptive and that you should not use them across in production environments - at all! The idea behind in illustrating these fundamentals so that you can grasp the mental model behind them and look to apply these patterns in your future projects when you see them fit. Understand and analyse the problems first; determine design patterns fit for them; apply them when necessarily.</p><p>Here’s to your happy learning and future software project success!</p><p>Till then, Happy coding!</p><p><strong>PS:</strong> For more software design patterns to learn, you can check this link out - <a href="https://refactoring.guru/design-patterns">Refactoring Guru</a>. It’s got great learning material (and content is always fresh) on software design patterns to solidify my understanding of best software engineering practices.</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_design_patterns.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;In my previous &lt;a href=&quot;http://www.awongcm.io/blog/2019/11/17/most-common-and-useful-design-patterns-you-should-be-aware-of-as-a-javascript-developer/&quot;&gt;post&lt;/a&gt;, I mentioned the usefulness of apply software patterns for scalable Javascript applications so far. I thought of revisiting this again because after working as a software engineer&amp;#x2F;developer for a while, I always found myself that design patterns are heavily applied across all projects (outside of Javascript domain) I have seen. Those projects were anything around Java, PHP, Python, C++, etc, thus I found there’s a common notion that design patterns are ubiquitous when applying to solve interesting and challenging software problems in our modern times.&lt;/p&gt;</summary>
    
    
    
    
    <category term="design-patterns software-development object-oriented-programming python javascript" scheme="http://awongcm.io/tags/design-patterns-software-development-object-oriented-programming-python-javascript/"/>
    
  </entry>
  
  <entry>
    <title>Setting Up New MacBook Pro M2 for Local Development</title>
    <link href="http://awongcm.io/blog/2024/02/04/setting-up-new-macBook-pro-m2-for-local-development/"/>
    <id>http://awongcm.io/blog/2024/02/04/setting-up-new-macBook-pro-m2-for-local-development/</id>
    <published>2024-02-04T01:18:49.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_macbook_pro_m2_setup.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>Happy Belated New Year of 2024!</p><p>With the arrival of my shiny new MacBook Pro M2, I will explain In this guide how I setup my machine for a seamless software development experience.</p><p>I would love to keep my current Macbook Pro Intel Core i5 2017 edition to continue using it.. But, over the years, the CPU and memory requirements have gone up for more modern full-stack development using Docker, updated Chrome extensions, or upgrade browsers, my limited 8GB RAM and 250GB of SSD storage started to feel small for me to do any decent software development tasks these days.  If I knew Apple is incredibly stringent to make memory and storage computing parts non-upgradeable after the purchase is made, I could have turned back the time to request upfront upgrade for these back then, and not to feel the terrible waiting grind for the systems resources to free itself whenever I start up Chrome browser or Docker background process, which ended up to seem like an eternal wait… ⏰⏱</p><p>So, forcibly speaking, I have to start moving from my old mac, and say hello to my new Macbook Pro M2!</p><span id="more"></span><p>With all that rant of mine being said, let’s get straight into it!</p><h2 id="Table-of-Contents"><a href="#Table-of-Contents" class="headerlink" title="Table of Contents"></a>Table of Contents</h2><ol><li><a href="#Initial-Setup">Initial Setup</a></li><li><a href="#Homebrew-for-Package-Management">Homebrew for Package Management</a></li><li><a href="#Configuring-Terminal-for-Best-CLI-Developer-Experience">Configuring Terminal for Best CLI Developer Experience</a></li><li><a href="#Choosing-an-IDE">Choosing an IDE</a></li><li><a href="#Containerization-with-Docker">Containerization with Docker</a></li><li><a href="#Database-Setup">Database Setup</a></li><li><a href="#Customizing-Your-macbook-defaults-Environment">Customizing Your macbook.defaults Environment</a></li><li><a href="#Conclusion">Conclusion</a></li></ol><h2 id="Initial-Setup"><a href="#Initial-Setup" class="headerlink" title="Initial Setup"></a>Initial Setup</h2><p>Start by going through the initial macOS setup process. Make sure your system is updated to the latest version to ensure compatibility with the latest development tools.</p><p>Verify the system preferences. ie <code>Apple Icon -&gt; System Settings -&gt; General -&gt; About</code></p><pre><code class="hljs vim">Name: &lt;The name of your Macbook Pro M2&gt;Chip: Apple M2Memory: <span class="hljs-number">16</span> GBSerial Number: &lt;The unique serial <span class="hljs-keyword">number</span> of your machine&gt;macOS: macOS Sonoma Version <span class="hljs-number">14</span>+</code></pre><p>Prior to writing this post, my machine default macOS was macOS Ventura after placing order overseas. To make the software update,  <code>Apple Icon -&gt; System Settings -&gt; General -&gt; Software Update -&gt; Click Update Now</code> and you should be good to go.</p><h2 id="Homebrew-for-Package-Management"><a href="#Homebrew-for-Package-Management" class="headerlink" title="Homebrew for Package Management"></a>Homebrew for Package Management</h2><p><a href="https://brew.sh/">Homebrew</a> is a powerful package manager for macOS. Install it by running the following command in your terminal:</p><pre><code class="hljs bash">/bin/bash -c <span class="hljs-string">&quot;<span class="hljs-subst">$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)</span>&quot;</span></code></pre><p>After this, assuming you use ZSH terminal like I do, you configure the homebrew shell settings to natively run well in ZSH terminal as below:</p><pre><code class="hljs bash">(<span class="hljs-built_in">echo</span>; <span class="hljs-built_in">echo</span> <span class="hljs-string">&#x27;eval &quot;$(/opt/homebrew/bin/brew shellenv)&quot;&#x27;</span>) &gt;&gt; ~/.zprofile</code></pre><p>Do <code>brew update</code> to make sure the latest brew updates.</p><p>At this point, I want to bring all of your favourite software development tools from my old macbook to my new one.  The obvious solution I thought would be to download them all, extract them and drag&#x2F;move to the applications folder, but this is actually very boring and repetitive to do.  It’s such a manual process to open&#x2F;drag&#x2F;drop everywhere in the screen to this.  Thanks to brew, I don’t have to do this anymore.</p><p>Instead, we use <code>brew</code> for that.</p><p>Let’s say I want to download Google Chrome browser.  What I do is rather to pass in the <code>cask</code> flag follow by application name I want to download which is <code>google-chrome</code></p><p>With all that said, you run the following</p><pre><code class="hljs bash">brew install --cask google-chrome</code></pre><p>And that’s it!</p><p>The incredible thing with this one-liner command is that it downloads from the HomeBrew package site (if it’s available) for the app, installs and moves to the Applications folder for you - without having necessarily to lift any heavy mouse-clicking for the complete the same action.  That’s the quick win for your simple automation like that. </p><p>With it I can start write up a bash script to add other software downloads as I require like so</p><pre><code class="hljs bash"><span class="hljs-meta">#!/bin/.bash</span>brew install --cask firefoxbrew install --cask visual-studio-codebrew install --cask iterm2<span class="hljs-comment"># etc, etc</span></code></pre><p>or if you like to DRY up the brew install, you can.  You simply use <code>\</code> keystroke like so.</p><pre><code class="hljs bash">brew install --cask firefox \visual-studio-code \iterm2 \<span class="hljs-comment"># etc, etc</span></code></pre><p>It helps to run the brew install actions in parallel.  But bear in mind. With this you may run network time-out issue thus some apps you may not able to complete the download times on time. You have to redo the brew install step all over again.  So I rather step away from using  this for running many things at once.  It’s a matter of preference.</p><p>For some tooling. in nature, they’re more terminal or CLI based commands, then using <code>cask</code> wouldn’t make sense.  We just simply omit it all together for eg.</p><pre><code class="hljs bash">brew install gitbrew install python3brew install openjdkbrew install apache-kafka<span class="hljs-comment"># etc, etc</span></code></pre><p>And we’re good to go from here.</p><h2 id="Configuring-Terminal-for-Best-CLI-Developer-Experience"><a href="#Configuring-Terminal-for-Best-CLI-Developer-Experience" class="headerlink" title="Configuring Terminal for Best CLI Developer Experience"></a>Configuring Terminal for Best CLI Developer Experience</h2><p>I’ve used ZSH for years thus I found it to be my second-nature when navigating the developer folder and system resources in my Iterm2 terminal. I also have mentioned about configuring brew for the ZSH profile earlier in this article hence that’s where I’m going with this.</p><p>To start, I do the following installation:</p><pre><code class="hljs bash">sh -c <span class="hljs-string">&quot;<span class="hljs-subst">$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)</span>&quot;</span></code></pre><p>Once you installed and configured it, I wanted to enhance my terminal experience further by executing these other 3 tools set at my disposal, which I found them incredibly useful.</p><pre><code class="hljs bash">git <span class="hljs-built_in">clone</span> https://github.com/zsh-users/zsh-autosuggestions <span class="hljs-variable">$&#123;ZSH_CUSTOM:-~/.oh-my-zsh/custom&#125;</span>/plugins/zsh-autosuggestions <span class="hljs-comment"># --&gt; ZSH Autosuggestions</span>git <span class="hljs-built_in">clone</span> https://github.com/zsh-users/zsh-syntax-highlighting.git <span class="hljs-variable">$&#123;ZSH_CUSTOM:-~/.oh-my-zsh/custom&#125;</span>/plugins/zsh-syntax-highlighting <span class="hljs-comment"># --&gt; Super Syntax Highlighting</span>git <span class="hljs-built_in">clone</span> https://github.com/romkatv/powerlevel10k.git <span class="hljs-variable">$ZSH_CUSTOM</span>/themes/powerlevel10k <span class="hljs-comment"># --&gt; PowerLevel10k colour scheme theming over Iterm2</span></code></pre><p>To find out more on their configuration options (and others), you can follow the Medium <a href="https://medium.com/crypticcrazeforcs/customising-oh-my-zsh-in-macos-6d4e24ef983">article</a> for some more setup guidance.</p><h2 id="Choosing-an-IDE"><a href="#Choosing-an-IDE" class="headerlink" title="Choosing an IDE"></a>Choosing an IDE</h2><p>As my primary languages I use these days as of late are ReactJS, TypeScript, Python, Java and NodeJS (though there are others that I can program sufficiently well), Visual Studio Code has been be my primary choice of IDE (though I’m slowly mastering VIM as it’s a CLI-based code editor experience, which is also pretty good)</p><p>But overtime, I got more involved in building backend systems development more in my later years of career, I got more accustomed to using IntelliJ and PyCharm Community Editions lately.  These are incredibly good as they’re more fully integrated developer experience where as Visual Studio Code is only a code editor on its own.  You have to either download the extensions plugin or create the plugins yourself from scratch to match up similar experience as these IDE products.</p><p>We can download all of these using brew install cask cli options as stated earlier in the article.</p><h2 id="Containerization-with-Docker"><a href="#Containerization-with-Docker" class="headerlink" title="Containerization with Docker"></a>Containerization with Docker</h2><p>Using Docker for Mac is quite a beast when comes to CPU&#x2F;memory resources it needs.  It always have a high start-up cost of my old Macbook machine every time I load up the mac screen thus have a huge impact on my machine’s overall performance running docker containers locally.</p><p>Thankfully, with Macbook M2 Pro machine having a bigger RAM and SDD storage, I have less to worry.  Moreover,  thanks for its ARM based architecture and the recent Docker community have been opening up their high adoption to have Docker images to run on ARM-based chipsets more than ever before.</p><p>With latest releases coming from macOS Sonoma. they will come in time in improving the docker virtualisation memory resources usage and things will get better from here.</p><p>Docker for Mac can installed using the brew install cask command as well.</p><h2 id="Database-Setup"><a href="#Database-Setup" class="headerlink" title="Database Setup"></a>Database Setup</h2><p>With databases, you have popular choices such as Redis, Mysql, Postgresql, etc.  Everything can be downloaded via brew command as well.  Again, without the <code>cask</code> flag.</p><p>Be mindful - because their cli-based services which you start and stop these services using <code>brew services start/stop &lt;database-service-name&gt;</code> , you need a proper database client like DBeaver, DataGrip or similar to interact with them.  </p><p>They can be downloaded via brew install cask.</p><h2 id="Customizing-Your-macbook-defaults-Environment"><a href="#Customizing-Your-macbook-defaults-Environment" class="headerlink" title="Customizing Your macbook.defaults Environment"></a>Customizing Your macbook.defaults Environment</h2><p>Last but not the least, I want to overwrite my new Macbook M2 Pro defaults behaviour as part of my goal of enhancing my local development experience on machine further.</p><p>I found links like <a href="https://macos-defaults.com/">macos-defaults</a>  or <a href="https://www.defaults-write.com/">defaults-write</a> are incredibly helpful with this.</p><p>With these, I found that the most obvious ones I want to overwrite is the Dock as I want to recreate same feeling on my old Macbook settings.  The settings are to minimize docker by default but with on mouse hover event when to autohide, along with  genie mini effect when opening&#x2F;minimising desktop screens.</p><p>They are setup as below:</p><pre><code class="hljs bash">defaults write com.apple.dock <span class="hljs-string">&quot;tilesize&quot;</span> -int <span class="hljs-string">&quot;24&quot;</span> &amp;&amp; killall Dockdefaults write com.apple.dock largesize -int <span class="hljs-string">&quot;128&quot;</span> &amp;&amp; killall Dockdefaults write com.apple.dock <span class="hljs-string">&quot;mineffect&quot;</span> -string <span class="hljs-string">&quot;genie&quot;</span> &amp;&amp; killall Dockdefaults write com.apple.dock <span class="hljs-string">&quot;autohide&quot;</span> -bool <span class="hljs-string">&quot;true&quot;</span> &amp;&amp; killall Dockdefaults write com.apple.dock <span class="hljs-string">&quot;autohide-time-modifier&quot;</span> -<span class="hljs-built_in">float</span> <span class="hljs-string">&quot;0.45&quot;</span> &amp;&amp; killall Dockdefaults write com.apple.dock <span class="hljs-string">&quot;autohide-delay&quot;</span> -<span class="hljs-built_in">float</span> <span class="hljs-string">&quot;0.2&quot;</span> &amp;&amp; killall Dock</code></pre><p>The next one is Finder where on Finder, the default display behaviour of not showing the folder paths and status bar when navigating always been the major pain point for me.  I don’t like seeing myself getting lost every time I want to find something that I want.</p><p>So to sort that out, I do the following:</p><pre><code class="hljs bash">defaults write com.apple.finder ShowPathBar -bool <span class="hljs-literal">true</span> &amp;&amp; killall Finderdefaults write com.apple.finder ShowStatusBar -bool <span class="hljs-literal">true</span> &amp;&amp; killall Finder</code></pre><p>And sometimes it would useful to toggle on&#x2F;off hidden system files fro view on Macbook when I want to verify my local development settings, so I use the following:</p><pre><code class="hljs bash">defaults write com.apple.finder <span class="hljs-string">&quot;AppleShowAllFiles&quot;</span> -bool <span class="hljs-string">&quot;false&quot;</span> &amp;&amp; killall Finder <span class="hljs-comment"># to hide system files</span>defaults write com.apple.finder <span class="hljs-string">&quot;AppleShowAllFiles&quot;</span> -bool <span class="hljs-string">&quot;true&quot;</span> &amp;&amp; killall Finder <span class="hljs-comment"># to show system files</span></code></pre><p>For more <code>macbook.defaults</code> override customisations you desire, you can follow above links I provided earlier.</p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>That’s it!  I’ve shown you all the cool tricks you can use to speed up things up as much as possible with Bre, macbook.defaults ,etc.  You can also find from my Gist profile <a href="https://gist.github.com/awongCM/2e754ade92dbfe1d90a516c71833f3f3">here</a> as if you need reference on the CLI commands I use for local development. </p><p>That’s the complete setup guide for my new Macbook M2 Pro for development as a start.</p><p>The next steps is transfer all of your Macbook software licences like MS Office, Alfred, Bartender etc from the old to the new.  But you can do that in your pace at any time, if you’re not in the hurry to rush transferring them across.</p><p>Hope that’s been incredibly helpful for you.</p><p>Here’s to the new exciting chapter of software development experience for the new year of 2024 to come!</p><p>Till then, Happy Coding! 👩‍💻👨‍💻💪🖥</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_macbook_pro_m2_setup.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;Introduction&quot;&gt;&lt;a href=&quot;#Introduction&quot; class=&quot;headerlink&quot; title=&quot;Introduction&quot;&gt;&lt;/a&gt;Introduction&lt;/h2&gt;&lt;p&gt;Happy Belated New Year of 2024!&lt;/p&gt;
&lt;p&gt;With the arrival of my shiny new MacBook Pro M2, I will explain In this guide how I setup my machine for a seamless software development experience.&lt;/p&gt;
&lt;p&gt;I would love to keep my current Macbook Pro Intel Core i5 2017 edition to continue using it.. But, over the years, the CPU and memory requirements have gone up for more modern full-stack development using Docker, updated Chrome extensions, or upgrade browsers, my limited 8GB RAM and 250GB of SSD storage started to feel small for me to do any decent software development tasks these days.  If I knew Apple is incredibly stringent to make memory and storage computing parts non-upgradeable after the purchase is made, I could have turned back the time to request upfront upgrade for these back then, and not to feel the terrible waiting grind for the systems resources to free itself whenever I start up Chrome browser or Docker background process, which ended up to seem like an eternal wait… ⏰⏱&lt;/p&gt;
&lt;p&gt;So, forcibly speaking, I have to start moving from my old mac, and say hello to my new Macbook Pro M2!&lt;/p&gt;</summary>
    
    
    
    
    <category term="macbook-pro m2-chip software-development apple-silicon m1-chip local-development" scheme="http://awongcm.io/tags/macbook-pro-m2-chip-software-development-apple-silicon-m1-chip-local-development/"/>
    
  </entry>
  
  <entry>
    <title>Elevating Software Development: Unleashing the Power of Apple MacBook Pro M2 (2023)</title>
    <link href="http://awongcm.io/blog/2023/12/10/elevating-software-development-unleashing-the-power-of-apple-macBook-pro-m2-2023/"/>
    <id>http://awongcm.io/blog/2023/12/10/elevating-software-development-unleashing-the-power-of-apple-macBook-pro-m2-2023/</id>
    <published>2023-12-10T06:00:28.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_elevating_software_development_macbook_m2_power.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>Before I start making my blog post intro, I have to say, Wow! It’s been a while since I made my post here. It’s almost the entire year of 2023 that’s about to draw its last curtain before 2024 is coming around the corner.</p><p>Since my last post earlier in the year, I mentioned my wedding plans mid-year of 2023 and now that’s all done and dusted. Naturally, the next thing to do after my wedding was to book our honeymoon to tour all the fun places around Malaysia and Indonesia towards the end of October 2023.</p><p>Here’s one of the best photo trips we took during our initial stay in KL.</p><p>The magnificent sight of Petronas Twin Towers during the night.</p><span id="more"></span><img src="/images/honeymoon_petronas_twin_towers_2.png" class="center" width="500" height="500" title="Petronas Twin Towers Honeymoon at Night Petronas Twin Towers Honeymoon at Night"><p>Here’s me and my wife took the selfie shot of the Petronas Twin Towers itself.</p><img src="/images/honeymoon_petronas_twin_towers_1.png" class="center" width="500" height="500" title="Petronas Twin Towers Honeymoon with me and my wifey Petronas Twin Towers Honeymoon with me and my wifey"><p>One of the best highlights of our trip throughout the honeymoon experience so far, was before we finally arrived home in Sydney in late November.</p><p>Before making this trip, I made plans of wanting to place an order for the Apple MacBook Pro M2 machine when contacting a PC store in Kuching, my home city of Sarawak, Malaysia. I was hoping to have the machine ready by the time I arrive in Kuching on our second leg of the honeymoon journey as I can bring it physically back home to Sydney with me.</p><p>But due to some unforeseen circumstances, the logistics behind the delivery of a new custom-made Macbook M2 Pro machine was not within my control and under my preferred scheduled delivery, I had to organise the DHL overseas delivery by not only the package payment of $350 AUD but also had to pay the import tax duty of $375 AUD on top of it as well! It’s certainly awful to spend a lot of that money on top of purchasing a MacBook M2 Pro machine for a good price, (which worked out I’m better off saving $800 AUD than the local retail price in Sydney on average). Sigh!</p><p>In spite of all that, the item did arrive safely at my local newsagency collection centre, I wasted no time to pick it up as a major sign of relief and ecstasy!</p><p>With all that bridge under troubled waters now over, I can get straight into the business of mentioning my thoughts on the best developer experience using MacBook M2 Pro device specs. I couldn’t be any happier to have this fine machine spec as my early Xmas present! 😊🥰🎄🎁</p><p>Without further ado.</p><h2 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h2><p>Whether you’re a seasoned developer or just starting on your coding journey, the right tools can make a significant difference in your productivity and overall experience. In this blog post, we’ll explore why the Apple MacBook Pro M2 is an exciting choice for software developers seeking to elevate their skills and workflow.</p><h2 id="The-Power-of-Apple-Silicon"><a href="#The-Power-of-Apple-Silicon" class="headerlink" title="The Power of Apple Silicon"></a>The Power of Apple Silicon</h2><p>Prior to having my new Macbook, my present Macbook Pro 2017 edition comes with 2.3 GHz Dual-Core Intel Core i5, 8GB RAM, 256 GB storage space and 1GB graphic memory. Over years of using this spec, I have noticed with modern apps getting far more advanced for development needs in terms of CPU, storage and memory performance, they’re getting far too slow for modern-day development these days. Especially, the majority of the boot and run time resources are spent on running local docker containers and browser multi tabs like Chrome, Firefox, Safari etc on a constant basis, they should put a lot of strain on the limited system resources I got.</p><p>Apple’s transition to its custom M1 and now M2 silicon has revolutionized the computing landscape. The MacBook Pro M2, powered by this cutting-edge architecture, brings unparalleled performance and efficiency to the table.</p><img src="/images/macbook_pro_m2_1.png" class="center" width="600" height="600" title="Apple Macbook M2 Pro Box Package - Different Angle Apple Macbook M2 Pro Box Package - Different Angle"><img src="/images/macbook_pro_m2_2.png" class="center" width="600" height="600" title="Apple Macbook M2 Pro Box Package - Different Angle Apple Macbook M2 Pro Box Package - Different Angle"><img src="/images/macbook_pro_m2_3.png" class="center" width="600" height="600" title="Apple Macbook M2 Pro System Screen Apple Macbook M2 Pro System Screen"><p>With my above unpackaged item, my new machine has the following specs:</p><ul><li>13-inch</li><li>Apple M2 Pro, 8-core CPU, 10-core GPU, 16 Core Neural Engine</li><li>16 GB RAM</li><li>512 GB SSD</li><li>macOS Ventura</li></ul><h3 id="1-Blazing-Fast-Speeds"><a href="#1-Blazing-Fast-Speeds" class="headerlink" title="1. Blazing Fast Speeds"></a><strong>1. Blazing Fast Speeds</strong></h3><p>The M2 chip’s multi-core performance ensures that your software compiles and runs swiftly, significantly reducing development cycles. Experience the thrill of near-instantaneous response times as you code, test, and debug with ease.</p><p>The M2 chip is an upgraded version of the M1 chip, which was Apple’s first attempt at designing its own ARM-based silicon. The M2 is the latest and most efficient Apple Silicon, with an 18% faster CPU, a 35% faster GPU and a 40% faster neural engine than previous generations, as well as 50% more memory bandwidth.</p><h3 id="2-Efficient-Power-Consumption"><a href="#2-Efficient-Power-Consumption" class="headerlink" title="2. Efficient Power Consumption"></a><strong>2. Efficient Power Consumption</strong></h3><p>Enjoy extended battery life without compromising performance. The M2 chip excels in energy efficiency, allowing you to code on the go without constantly worrying about running out of battery power.</p><h2 id="Exceptional-Development-Environment"><a href="#Exceptional-Development-Environment" class="headerlink" title="Exceptional Development Environment"></a>Exceptional Development Environment</h2><p>A great development environment is crucial for any software engineer. The MacBook Pro M2 excels in providing an environment that fosters productivity and creativity.</p><h3 id="1-Retina-Display-for-Clarity"><a href="#1-Retina-Display-for-Clarity" class="headerlink" title="1. Retina Display for Clarity"></a><strong>1. Retina Display for Clarity</strong></h3><p>The MacBook Pro’s Retina display offers crystal-clear visuals, making code easier to read and reducing eye strain during long coding sessions. The high resolution ensures that every detail of your project is displayed with precision.</p><h3 id="2-Touch-Bar-for-Intuitive-Controls"><a href="#2-Touch-Bar-for-Intuitive-Controls" class="headerlink" title="2. Touch Bar for Intuitive Controls"></a><strong>2. Touch Bar for Intuitive Controls</strong></h3><p>Take advantage of the Touch Bar, a dynamic and context-aware input device, to streamline your workflow. Customize it to access your favourite commands, making repetitive tasks a breeze.</p><h2 id="Seamless-Integration-with-Developer-Tools"><a href="#Seamless-Integration-with-Developer-Tools" class="headerlink" title="Seamless Integration with Developer Tools"></a>Seamless Integration with Developer Tools</h2><p>Apple’s ecosystem is known for its seamless integration, and the MacBook Pro M2 is no exception.</p><h3 id="1-Xcode-for-iOS-and-macOS-Development"><a href="#1-Xcode-for-iOS-and-macOS-Development" class="headerlink" title="1. Xcode for iOS and macOS Development"></a><strong>1. Xcode for iOS and macOS Development</strong></h3><p>If you’re into iOS or macOS development, the MacBook Pro M2 is a dream machine. Compile and run your apps faster than ever, and leverage the efficiency of Xcode to enhance your development workflow. But, if native mobile development is not one of your forte skills, you can certainly give open-source platforms such as Expo Go to build native app experiences using your front-end skills like React&#x2F;JS a go. MacBook M2 Pro will certainly give you a major edge to take your native development to the next level.</p><h3 id="2-Homebrew-for-Package-Management"><a href="#2-Homebrew-for-Package-Management" class="headerlink" title="2. Homebrew for Package Management"></a><strong>2. Homebrew for Package Management</strong></h3><p>Easily install and manage development tools with Homebrew. The MacBook Pro M2 ensures compatibility and smooth performance for the vast array of packages available.</p><h2 id="Future-Proofing-Your-Skill-Set"><a href="#Future-Proofing-Your-Skill-Set" class="headerlink" title="Future-Proofing Your Skill Set"></a>Future-Proofing Your Skill Set</h2><p>Investing in the MacBook Pro M2 is not just about the present; it’s about future-proofing your development capabilities.</p><h3 id="1-Ongoing-Software-Support"><a href="#1-Ongoing-Software-Support" class="headerlink" title="1. Ongoing Software Support"></a><strong>1. Ongoing Software Support</strong></h3><p>Apple’s commitment to software updates ensures that your MacBook Pro M2 remains current with the latest features and improvements, keeping you at the forefront of technological advancements.</p><h3 id="2-Expanding-App-Ecosystem"><a href="#2-Expanding-App-Ecosystem" class="headerlink" title="2. Expanding App Ecosystem"></a><strong>2. Expanding App Ecosystem</strong></h3><p>As the Apple ecosystem grows, so do the opportunities for developers. Take advantage of the expanding App Store and tap into new markets and tools with your innovative applications.</p><h3 id="3-More-Bare-metal-Processing-Power-for-Local-Development"><a href="#3-More-Bare-metal-Processing-Power-for-Local-Development" class="headerlink" title="3.  More Bare-metal Processing Power for Local Development"></a><strong>3.  More Bare-metal Processing Power for Local Development</strong></h3><p>As we’re living in the age of cloud computing and cloud development services, while it’s true the core advantage of such things is to provide vast scalability, cost savings, increased performance etc as your app grows at a particular scale point in the long run, I’d find sometimes this is not always the case in the short run, especially when you just starting out, Local development would clearly outshine here because they are extremely fast, easy and free to do, Developers don’t have to pay for cloud resources upfront in the beginning, even an Internet is not necessarily required. This is especially true when you just want to explore distributed streaming technology locally such as Apache Kafka for eg. With MacBook M2 Pro specs in your hands, you can untap its raw power to spin enough of its local resources to develop and validate your app functionality as much as you want until you’re ready to move on to cloud services and be financially confident to afford micro&#x2F;macro transactions for their services diligently. Before M1&#x2F;M2 existence, this is fairly difficult to accomplish.</p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>In conclusion, upgrading to the Apple MacBook Pro M2 is a game-changer for developers. From the incredible speed and efficiency of the M2 chip to the seamless integration with powerful developer tools, this machine is designed to elevate your software development experience. Make the leap and unlock the next level in your coding journey.</p><p>Till next time, have a Merry Safe Xmas Holidays.</p><p>See you in the new year, Happy Coding!</p><p><em>PS: This article is by no means a complete guide to my new Mac setup for development. There are a lot of guides online dev bloggers have built in their own certain ways. Each one of them is unique to their own personal development tastes. I would write up a separate blog post in the future for it on my own approach.</em></p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_elevating_software_development_macbook_m2_power.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Before I start making my blog post intro, I have to say, Wow! It’s been a while since I made my post here. It’s almost the entire year of 2023 that’s about to draw its last curtain before 2024 is coming around the corner.&lt;/p&gt;
&lt;p&gt;Since my last post earlier in the year, I mentioned my wedding plans mid-year of 2023 and now that’s all done and dusted. Naturally, the next thing to do after my wedding was to book our honeymoon to tour all the fun places around Malaysia and Indonesia towards the end of October 2023.&lt;/p&gt;
&lt;p&gt;Here’s one of the best photo trips we took during our initial stay in KL.&lt;/p&gt;
&lt;p&gt;The magnificent sight of Petronas Twin Towers during the night.&lt;/p&gt;</summary>
    
    
    
    
    <category term="macbook-pro m2-chip software-development apple-silicon m1-chip local-development" scheme="http://awongcm.io/tags/macbook-pro-m2-chip-software-development-apple-silicon-m1-chip-local-development/"/>
    
  </entry>
  
  <entry>
    <title>How to work with EventEmitters for unit testing in NodeJS</title>
    <link href="http://awongcm.io/blog/2023/01/23/how-to-work-with-eventemitters-for-unit-testing-in-nodeJS/"/>
    <id>http://awongcm.io/blog/2023/01/23/how-to-work-with-eventemitters-for-unit-testing-in-nodeJS/</id>
    <published>2023-01-23T06:46:41.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_unit_test_event_emitters_nodejs.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>Happy New Year! Here’s to my first blog post of 2023.</p><p>Since my last post, I have finally managed to get a number of errands to accomplish before the end of 2022. One of the most important items was getting my father’s ashes finally sent homebound back to Malaysia in September for his eternal rest. It was great to be back home and get a good holiday stint around my hometown of Sibu, Sarawak as well as the capital city, Kuching where I haven’t been for a long time to just wind down and relax, along with the wedding preparations underway downunder.</p><p>Now, that’s all out of the way, it’s time to get back into my world of blogging once more!</p><p>Last year, I faced a deceptively complex but interesting solution design problem when working with NodeJS event-driven programming model, <code>EventEmitter</code> and how we should go about writing up appropriate unit testing when listeners were subscribing to certain events being emitted.</p><span id="more"></span><p>If you recalled your EventEmitter basics, EventEmitter is one of the core NodeJS modules that facilitates communication interaction between objects when running in asynchronous event-driven architecture. For any objects to inherit this module, the easiest thing is to write them <code>extends</code> like so.</p><pre><code class="hljs javascript"><span class="hljs-keyword">class</span> <span class="hljs-title class_">FooBar</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">EventEmitter</span> &#123;  <span class="hljs-title function_">constructor</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-comment">// some object properties to go here..</span>  &#125;&#125;</code></pre><p>By doing this, the same object is now inheriting a lot of EventEmitter’s methods. </p><p>Mainly, they are:</p><ul><li>Emitting name events such as <code>emit</code></li><li>Registering and unregistering listener functions such as <code>on</code> and <code>off</code></li></ul><p>In its simplest concept, emitter objects that emit named events will cause the previously registered listeners to be called upon. This programming model works basically like in pub&#x2F;sub model more or less.</p><p>With that in mind, we can now look into this problem that I came across in one of my projects last year.</p><pre><code class="hljs javascript"><span class="hljs-keyword">class</span> <span class="hljs-title class_">CronService</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">EventEmitter</span> &#123;  <span class="hljs-title function_">constructor</span>(<span class="hljs-params">configExpression, runSomeCronJobFunc, logger</span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">configExpression</span> = configExpression;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">logger</span> = logger;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">runSomeCronJobFunc</span> = runSomeCronJobFunc;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">isStarted</span> = <span class="hljs-literal">false</span>;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">isProcessing</span> = <span class="hljs-literal">false</span>;  &#125;  <span class="hljs-keyword">async</span> <span class="hljs-title function_">start</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">isStarted</span> = <span class="hljs-literal">true</span>;    <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">performCronJob</span>();  &#125;  <span class="hljs-keyword">async</span> <span class="hljs-title function_">stop</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">isStarted</span> = <span class="hljs-literal">false</span>;  &#125;  <span class="hljs-keyword">async</span> <span class="hljs-title function_">performCronJob</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-keyword">if</span>(<span class="hljs-variable language_">this</span>.<span class="hljs-property">isProcessing</span>) &#123;      <span class="hljs-variable language_">this</span>.<span class="hljs-property">logger</span>.<span class="hljs-title function_">info</span>(<span class="hljs-function">()=&gt;</span> <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">info</span>(<span class="hljs-string">&#x27;job is still running&#x27;</span>));      <span class="hljs-keyword">return</span>;    &#125;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">isProcessing</span> = <span class="hljs-literal">true</span>;    <span class="hljs-keyword">try</span> &#123;      <span class="hljs-keyword">await</span> <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">runSomeCronJobFunc</span>();      <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">emit</span>(<span class="hljs-string">&#x27;finished&#x27;</span>, <span class="hljs-literal">null</span>)    &#125; <span class="hljs-keyword">catch</span>(error) &#123;      <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">warn</span>(<span class="hljs-function">()=&gt;</span> <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">warn</span>(<span class="hljs-string">&#x27;error encountered; job ended abruptly&#x27;</span>));      <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">emit</span>(<span class="hljs-string">&#x27;finished&#x27;</span>, error);    &#125;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">isProcessing</span> = <span class="hljs-literal">false</span>  &#125;  <span class="hljs-title function_">getStatus</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-keyword">return</span> &#123;      <span class="hljs-attr">isStarted</span>: <span class="hljs-variable language_">this</span>.<span class="hljs-property">isStarted</span>;      <span class="hljs-attr">isProcessing</span>: <span class="hljs-variable language_">this</span>.<span class="hljs-property">isProcessing</span>;s    &#125;  &#125;&#125;</code></pre><p>Here, we have a <code>CronService</code> whose primary task is to kick off the cron job that executes at a certain interval we provide. Thus CronService uses <code>configExpression</code> for checking the cron interval expression and <code>runSomeCronJobFunc</code> as our main task placed in cron service for execution, along with the other auxiliary parameters in checking where in the state of the cron service being run <code>isStarted</code> and <code>isProcessing</code>.</p><p>The main area of interest to look is the <code>performCronJob</code> function block where our CronService emits two <code>finished</code> events. What does this block say?<br>In our project requirement, we say, in our <code>try/catch</code> block,</p><ol><li>We <code>await this.runSomeCronJobFunc</code> and expect the cron job task to execute into completion, we will emit <code>finished</code> to signify the job’s completed without errors.</li><li>Or if we encountered any errors during the execution of the <code>runSomeCronJobFunc</code> , we capture the error and we will still emit <code>finished</code> to signify the job is also completed - but with errors this time.</li></ol><p>For either of the two outcomes above, we will flag <code>isProcessing</code> to be false; it will not be marked as true if the current job is still in the middle of execution and we don’t want the next instance of cronjob service to kick off another one until it’s completed.</p><p>So, why is it that we emit <code>finished</code> instead of <code>error</code> one may ask? That’s because, in our solution design, we have a clear requirement any failed&#x2F;incomplete cron job occurred is to be treated as a complete task so that at the next configExpression cycle we want to kick off the same CronService again. Our goal for CronService is to run batch job processing tasks at regular intervals throughout the day regardless if they were completed <em>successfully or not.</em></p><p>That’s the context for our design rationale.</p><p>With that out of the day, we now come to the important part of the question - who are the listeners to these finished events and how do we thoroughly test the EventEmitters are working correctly based on the conditions above?</p><p>To start, we write our hypothetical unit test file here.</p><pre><code class="hljs javascript"><span class="hljs-comment">// our assertion utilities</span><span class="hljs-keyword">const</span> &#123;assertThat, is, not, equalTo&#125; = <span class="hljs-built_in">require</span>(<span class="hljs-string">&#x27;hamjest&#x27;</span>);<span class="hljs-keyword">const</span> sinon = <span class="hljs-built_in">require</span>(<span class="hljs-string">&quot;sinon&quot;</span>);<span class="hljs-comment">// setup mocks and test data</span><span class="hljs-keyword">const</span> logger = <span class="hljs-keyword">new</span> <span class="hljs-title function_">someMockLogger</span>();<span class="hljs-keyword">const</span> <span class="hljs-variable constant_">HOURLY_INTERVAL</span> = <span class="hljs-number">1000</span> * <span class="hljs-number">60</span> * <span class="hljs-number">60</span>;<span class="hljs-comment">// cron to run hourly</span><span class="hljs-keyword">const</span> cronConfigExpression = <span class="hljs-string">&quot;0 * * * *&quot;</span>;<span class="hljs-keyword">let</span> someCronTaskPerformedCounter;<span class="hljs-keyword">const</span> <span class="hljs-title function_">performCronTask</span> = (<span class="hljs-params"></span>) =&gt; &#123;  someCronTaskPerformedCounter++;&#125;<span class="hljs-title function_">describe</span>((<span class="hljs-string">&#x27;CronService&#x27;</span>), <span class="hljs-function">() =&gt;</span> &#123;  <span class="hljs-keyword">let</span> cronService, clock;  <span class="hljs-title function_">beforeEach</span>(<span class="hljs-function">()=&gt;</span> &#123;    cronService = <span class="hljs-literal">null</span>;    clock = sinon.<span class="hljs-title function_">useFakeTimers</span>();  &#125;);  <span class="hljs-title function_">afterEach</span>(<span class="hljs-function">()=&gt;</span> &#123;    sinon.<span class="hljs-title function_">restore</span>();    <span class="hljs-keyword">if</span>(cronService !== <span class="hljs-literal">null</span>) &#123;      <span class="hljs-keyword">await</span> cronService.<span class="hljs-title function_">stop</span>();    &#125;  &#125;);  <span class="hljs-comment">/// More unit test blocks to follow shortly....</span>&#125;)</code></pre><p>What did we just write in the above?</p><ol><li>First, we add some assertion utilities into the mix using <a href="https://www.npmjs.com/package/hamjest">hamjest</a> and <a href="https://sinonjs.org/">sinon</a>.</li><li>We setup the mock and test data we need to pass&#x2F;inject for the CronService constructor to work, ie logger, cronConfigExpression, HOURLY_INTERVAL, etc.</li><li>We create <code>performCronTask</code> function call to pass as a callback parameter, that stores the total count of the cron task that gets performed ie <code>someCronTaskPerformedCounter</code> .</li><li>We setup the <code>describe</code> test block for cronService, outlining the <code>beforeEach</code> and <code>afterEach</code> callbacks because we want to reset the cronService instantiation and SinonJS clock’s stub timer as well for each unit test we run (in this case, we’re doing one for the purpose of this demo);</li></ol><p>Once your baseline unit testing structure is underway, we’re getting into the nitty-gritty of things.</p><pre><code class="hljs javascript"><span class="hljs-title function_">it</span>(<span class="hljs-string">&quot;should run every hour (0 * * * *)&quot;</span>, <span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> (<span class="hljs-params"></span>) &#123;  cronService = <span class="hljs-keyword">new</span> <span class="hljs-title class_">CronService</span>(    performSomeCronTask,    cronConfigExpression,    logger  );  <span class="hljs-title function_">assertThat</span>(someCronTaskPerformedCounter, <span class="hljs-title function_">is</span>(<span class="hljs-number">0</span>));  <span class="hljs-keyword">await</span> cronService.<span class="hljs-title function_">start</span>();  clock.<span class="hljs-title function_">tick</span>(<span class="hljs-variable constant_">HOURLY_INTERVAL</span>);  <span class="hljs-comment">// This won&#x27;t work!</span>  <span class="hljs-title function_">assertThat</span>(someCronTaskPerformedCounter, <span class="hljs-title function_">is</span>(<span class="hljs-number">1</span>));&#125;);</code></pre><p>Here, our test should expect the cronService to run at hourly intervals meaning once the current cronjob gets to kick off, it’ll go for an hour (not in actual real-time hour interval thanks for SinonJS timebending’s <code>clock.tick</code> API ), and expects it’s still being processed ie <code>isProcessing</code> is true until it runs into completion by the exact hour.</p><p>So towards the end of running time, we first thought we could assert someCronTaskPerformedCounter to naturally increment to 1, isn’t it? But no, it’s not!</p><p>It would still stated at 0. No change here.</p><p>How on earth is it that possible when we’re not running things in real-time as everything is running under controlled environments?</p><p>But remember in the earlier <code>performCronJob</code> block of <code>try/catch</code>.</p><pre><code class="hljs javascript"><span class="hljs-keyword">async</span> <span class="hljs-title function_">performCronJob</span>(<span class="hljs-params"></span>) &#123;  <span class="hljs-keyword">try</span> &#123;    <span class="hljs-keyword">await</span> <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">runSomeCronJobFunc</span>();    <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">emit</span>(<span class="hljs-string">&#x27;finished&#x27;</span>, <span class="hljs-literal">null</span>)  &#125; <span class="hljs-keyword">catch</span>(error) &#123;    <span class="hljs-comment">// boo!</span>  &#125;&#125;</code></pre><p><code>runSomeCronJobFunc</code> inherently becomes a Promise and it will pause at this execution level until the same Promise gets settled (be rejected or fulfilled), which makes sense why our someCronTaskPerformedCounter assertion didn’t work because we expect it to complete <em>prematurely</em> too early during its running interval!</p><p>Thus we ask ourselves - what on earth should we ever come up with an assertion that says the <code>runSomeCronJobFunc</code> Promise will get fulfilled at some point without executing the rest of the test cases since everything will be running synchronously after that? We cannot expect to block the testing runtime because that will suspend the entire executing thread of the JS runtime space. Nothing will run at all!</p><p>How can we use the <code>finished</code> emitters signal for the unit test to be aware something has occurred downstream of events when multiple of cronService instances could be fired at some regular interval in a sequential manner?</p><p>The solution? 🤔</p><p>We create a new Promise wrapper over our promised assertions that gets fulfilled when the <code>finished</code> emitter gets triggered.</p><pre><code class="hljs javascript"><span class="hljs-comment">// our main ingredient</span><span class="hljs-keyword">const</span> <span class="hljs-title function_">cronServiceFinished</span> = (<span class="hljs-params">cronService, runAssertions</span>) =&gt; &#123;  <span class="hljs-keyword">return</span> <span class="hljs-keyword">new</span> <span class="hljs-title class_">Promise</span>(<span class="hljs-function">(<span class="hljs-params">resolve, reject</span>) =&gt;</span> &#123;    cronService.<span class="hljs-title function_">on</span>(<span class="hljs-string">&#x27;finished&#x27;</span>, <span class="hljs-function">(<span class="hljs-params">error</span>) =&gt;</span> &#123;      <span class="hljs-title function_">runAssertions</span>(error).<span class="hljs-title function_">then</span>(resolve, reject);    &#125;  &#125;)&#125;</code></pre><p>If you look at this closely, it makes sense right?</p><p>We are saying - as we are awaiting on <code>runSomeCronJobFunc</code> to complete, we then have to await <code>cronServiceFinished</code> until the <code>finished</code> emitters get triggered. Once it’s triggered, we use its EventEmitter callback signature for the emitted data (which in this case it’s <code>null</code> in the above example) to perform our assertions rules we see fit, which in itself is also a Promise!</p><p>With that in mind, we can now rewrite our unit test into the following</p><pre><code class="hljs javascript"><span class="hljs-title function_">it</span>(<span class="hljs-string">&quot;should run every hour&quot;</span>, <span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> (<span class="hljs-params"></span>) &#123;  cronService = <span class="hljs-keyword">new</span> <span class="hljs-title class_">CronService</span>(    performSomeCronTask,    cronConfigExpression,    logger  );  <span class="hljs-title function_">assertThat</span>(someCronTaskPerformedCounter, <span class="hljs-title function_">is</span>(<span class="hljs-number">0</span>));  <span class="hljs-keyword">await</span> cronService.<span class="hljs-title function_">start</span>();  clock.<span class="hljs-title function_">tick</span>(<span class="hljs-variable constant_">HOURLY_INTERVAL</span>);  <span class="hljs-keyword">const</span> <span class="hljs-title function_">successAssertions</span> = <span class="hljs-keyword">async</span> (<span class="hljs-params">error</span>) =&gt; &#123;    <span class="hljs-title function_">assertThat</span>(error, <span class="hljs-title function_">is</span>(<span class="hljs-literal">null</span>));    <span class="hljs-title function_">assertThat</span>(someCronTaskPerformedCounter, <span class="hljs-title function_">is</span>(<span class="hljs-number">1</span>));  &#125;;  <span class="hljs-comment">// Now this works because it respects the async/await execution flow of the main cron task at hand.</span>  <span class="hljs-keyword">await</span> <span class="hljs-title function_">cronServiceFinished</span>(cronService, successAssertions);&#125;);</code></pre><p>That’s it!</p><p>Now we can make use of this to extend for another use case to verify cron job is still processing below:</p><pre><code class="hljs javascript"><span class="hljs-title function_">it</span>(<span class="hljs-string">&quot;verifies the cron service is processing when started&quot;</span>, <span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> (<span class="hljs-params"></span>) &#123;  cronService = <span class="hljs-keyword">new</span> <span class="hljs-title class_">CronService</span>(    performSomeCronTask,    cronConfigExpression,    logger  );  <span class="hljs-title function_">assertThat</span>(someCronTaskPerformedCounter, <span class="hljs-title function_">is</span>(<span class="hljs-number">0</span>));  <span class="hljs-keyword">const</span> <span class="hljs-title function_">successAssertions</span> = <span class="hljs-keyword">async</span> (<span class="hljs-params">error</span>) =&gt; &#123;    <span class="hljs-title function_">assertThat</span>(error, <span class="hljs-title function_">is</span>(<span class="hljs-literal">null</span>));    <span class="hljs-title function_">assertThat</span>(someCronTaskPerformedCounter, <span class="hljs-title function_">is</span>(<span class="hljs-number">1</span>));  &#125;;  <span class="hljs-keyword">await</span> cronService.<span class="hljs-title function_">start</span>();  clock.<span class="hljs-title function_">tick</span>(<span class="hljs-variable constant_">HOURLY_INTERVAL</span>);  <span class="hljs-keyword">const</span> status = cronService.<span class="hljs-title function_">getStatus</span>();  <span class="hljs-title function_">assertThat</span>(status.<span class="hljs-property">isProcessing</span>, <span class="hljs-title function_">is</span>(<span class="hljs-literal">true</span>));  <span class="hljs-keyword">await</span> <span class="hljs-title function_">cronServiceFinished</span>(cronService, successAssertions);&#125;);</code></pre><p>Now how do we handle when the cronjob service encountered an error during its processing time with <code>finished</code> emitter triggered with error data?</p><p>We do the following:</p><pre><code class="hljs javascript"><span class="hljs-title function_">it</span>(<span class="hljs-string">&quot;verifies the cron service is processing when started&quot;</span>, <span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> (<span class="hljs-params"></span>) &#123;  performSomeCronTaskWithErrorStub = sinon.<span class="hljs-title function_">stub</span>().<span class="hljs-title function_">rejects</span>(<span class="hljs-keyword">new</span> <span class="hljs-title class_">Error</span>(<span class="hljs-string">&quot;boo!&quot;</span>));  cronService = <span class="hljs-keyword">new</span> <span class="hljs-title class_">CronService</span>(    performSomeCronTaskWithErrorStub,    cronConfigExpression,    logger  );  <span class="hljs-title function_">assertThat</span>(performSomeCronTaskWithErrorStub.<span class="hljs-property">notCalled</span>, <span class="hljs-title function_">is</span>(<span class="hljs-literal">true</span>));  <span class="hljs-keyword">const</span> <span class="hljs-title function_">failledAssertions</span> = <span class="hljs-keyword">async</span> (<span class="hljs-params">error</span>) =&gt; &#123;    <span class="hljs-title function_">assertThat</span>(error, <span class="hljs-title function_">is</span>(<span class="hljs-title function_">not</span>(<span class="hljs-title function_">equalTo</span>(<span class="hljs-literal">null</span>))));    <span class="hljs-title function_">assertThat</span>(error.<span class="hljs-property">message</span>, <span class="hljs-title function_">is</span>(<span class="hljs-title function_">equalTo</span>(<span class="hljs-string">&quot;boo!&quot;</span>)));    <span class="hljs-title function_">assertThat</span>(performSomeCronTaskWithErrorStub.<span class="hljs-property">calledOnce</span>, <span class="hljs-title function_">is</span>(<span class="hljs-literal">true</span>));  &#125;;  <span class="hljs-keyword">await</span> cronService.<span class="hljs-title function_">start</span>();  clock.<span class="hljs-title function_">tick</span>(<span class="hljs-variable constant_">HOURLY_INTERVAL</span>);  <span class="hljs-keyword">await</span> <span class="hljs-title function_">cronServiceFinished</span>(cronService, failedAssertions);  <span class="hljs-keyword">const</span> status = cronService.<span class="hljs-title function_">getStatus</span>();  <span class="hljs-title function_">assertThat</span>(status.<span class="hljs-property">isProcessing</span>, <span class="hljs-title function_">is</span>(<span class="hljs-literal">false</span>));&#125;);</code></pre><p>Here, we have created a mock cron job task comes with a stubbed error.</p><p><code>performSomeCronTaskWithErrorStub = sinon.stub().rejects(new Error(&quot;boo!&quot;));</code></p><p>By using <code>sinon.stub</code>, we can make use of the spy method calls <code>calledOnce</code> or <code>notCalled</code> to inspect whether the internal states of the same function was called once or not at all respectivwely - hence for their named convenience methods.</p><p>Then our success assertions are now replaced with failure assertions that expect the <code>error</code> not to be null along with the generic error message that comes with it, and the <code>performSomeCronTaskWithErrorStub</code> was definitely called once.</p><p>When errors are encountered during its processing, we said to <code>finished</code> the cron job and thus marked the same instance state <code>isProcessing</code> to false as per our requirement.</p><p>So, there you have it.</p><p>This is how you want to write your unit test cases when dealing with EventEmitters in the event system flow where our unit test cases subscribe to these events and execute the assertions reliably from there. Especially when dealing with asynchronous operations like these, the use of Promise wrappers becomes incredibly handy when dealing with unexpected racing conditions between test cases running at different times to the finishing lines.</p><p>Hope you like it and you find this very useful.</p><p>Till next time, Happy Coding!</p><p><strong>PS: If you want to find out how do Hamejest and SinonJS used in detail, you can find their resources here.</strong></p><ul><li><a href="https://www.npmjs.com/package/hamjest">Hamejest</a></li><li><a href="https://sinonjs.org/releases/v15/spies/">SinonJS</a></li></ul>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_unit_test_event_emitters_nodejs.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Happy New Year! Here’s to my first blog post of 2023.&lt;/p&gt;
&lt;p&gt;Since my last post, I have finally managed to get a number of errands to accomplish before the end of 2022. One of the most important items was getting my father’s ashes finally sent homebound back to Malaysia in September for his eternal rest. It was great to be back home and get a good holiday stint around my hometown of Sibu, Sarawak as well as the capital city, Kuching where I haven’t been for a long time to just wind down and relax, along with the wedding preparations underway downunder.&lt;/p&gt;
&lt;p&gt;Now, that’s all out of the way, it’s time to get back into my world of blogging once more!&lt;/p&gt;
&lt;p&gt;Last year, I faced a deceptively complex but interesting solution design problem when working with NodeJS event-driven programming model, &lt;code&gt;EventEmitter&lt;/code&gt; and how we should go about writing up appropriate unit testing when listeners were subscribing to certain events being emitted.&lt;/p&gt;</summary>
    
    
    
    
    <category term="event-emitters nodejs promise unit-testing sinon" scheme="http://awongcm.io/tags/event-emitters-nodejs-promise-unit-testing-sinon/"/>
    
  </entry>
  
  <entry>
    <title>Migrating Octopress to Hexo</title>
    <link href="http://awongcm.io/blog/2022/05/09/migrating-octopress-to-hexo/"/>
    <id>http://awongcm.io/blog/2022/05/09/migrating-octopress-to-hexo/</id>
    <published>2022-05-09T03:53:34.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_migrating_octopress_to_hexo.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>We’re edging towards halfway of 2022.. And boy. Look how time has gotten passed by.</p><p>I must admit I haven’t had the chance to sit down and write up another good tech blog post. A lot is happening in motion since the start of the year 2022 from my recent engagement to my fiancee of 3 and half years; busy with my day job as a contractor for a bank; finally getting my passport renewal done in Canberra (after almost 2 years of nationwide lockdown) as part of my travel plans to Malaysia so I can send my father’s ashes for his final resting place; along with talking plans having my fiancee and me moving together later in the year and preparing my next career technical certifications to take on, etc…</p><p>The list could go on.</p><p>Nevertheless, still, I’m glad to take the time to write about what I have accomplished recently.</p><p>That is…</p><p>I’ve decided to migrate my Octopress site to Hexo! 📦🚚</p><span id="more"></span><p>Presently, it may not look like much to you, but If you haven’t noticed any difference between the old and my new blog platform, is that I have updated my favicon on the page in the top left-hand corner of the browser. The site is using Hexo favicon.  In my old one, previously, it was Octopress (obviously).   </p><p>Secondly, all of my code snippet’s syntax highlighting show here has been modified a bit. They just do not show any line numbers of code anymorem whereas  Octopress comes with them, by default.  </p><p>Lastly, my footer note section has been updated with a new description, from <code>Andy Wong - Generated using Octo theme from Octopress</code> to <code>Andy Wong - Generated using Octo theme(modified) for Hexo</code>.</p><p>These are the key noticeable differences.</p><h3 id="Why-am-I-making-the-switch"><a href="#Why-am-I-making-the-switch" class="headerlink" title="Why am I making the switch?"></a>Why am I making the switch?</h3><ol><li><strong>No longer supported</strong>. -  I first got my Octopress version 2 installed back in late 2014 and it’s no longer being updated.  Later, Octopress 3 was introduced in 2015 but then only lasted up to 2016 thus it was short-lived. Knowing that after writing my blog post content for 7 years, I am no longer getting any more updates and support from the thriving community it once was.</li><li><strong>Performance Issues</strong>. - When using Octopress 2, it’s taking a lot more than 25 seconds or so to generate all the website pages and content (which is about 35 articles at the time of writing). Even worse, each time I updated a single blog post with a minor typo or additional content lines, it takes the same amount of time when I first execute rake preview to regenerate. With Hexo, I get everything done under <em><strong>5.2</strong></em> seconds.  That’s incredbily fast!</li><li>For the <strong>reason #1</strong> and <strong>reason #2</strong>, blogging is no longer as fun as it used to be. When the community has long stopped caring&#x2F;supporting because it couldn’t gather enough interest from the online community, I started to cast my doubt on their long-term commitment to the project and pondered the risk of being left behind and not able to use better static site generation tooling outside in the market.  Thus, I need affirmative action on how to mitigate this.</li></ol><h3 id="Reasons-for-migrating-to-Hexo"><a href="#Reasons-for-migrating-to-Hexo" class="headerlink" title="Reasons for migrating to Hexo"></a>Reasons for migrating to Hexo</h3><p>The new CMS tool, <a href="https://hexo.io/">Hexo</a>, written by the Chinese development community, became the good choice for me after careful evaluations of open-source static generation tools out there such as Hugo, Gatsby, Next.js etc.  The deciding factor simply boils down to this:</p><ol><li><strong>Seamless Content migration</strong> - Hexo published a support <a href="https://hexo.io/docs/migration#Octopress">page</a> on how to migrate from a common platform such as Octopress to the Hexo platform. It’s a very straightforward process as both Hexo and Octopress share the same blogging workflow methodology, thus it is a major time-saver for me with minimal hassles when I’m pressed for time.</li><li><strong>Syntax Highlighting</strong> - retains all the same features that Octopress has, but with a touch of plugin config changes and everything becomes straight forward.</li><li><strong>Disqus Support</strong> - for the same reason as #1</li><li><strong>Familiarity of blog post generation tool chain</strong>. Thus the learning curve is really low thus I never find it painfully difficult to do this myself.</li><li><strong>Theme Support</strong> - there’s a number of good themes I could use but I decide to work on the base <code>octo</code> theme provided, modified for my own taste and liking.</li><li><strong>NodeJS Language Familiarity</strong> - I personally and professionally work with NodeJS&#x2F;JS for years now thus I’m very accustomed to the ecosystem tooling.</li><li><strong>Community support</strong> - there’s about 36,000+ Github stars on it so I’m guessing its much better thriving community to lean onto vs Octopress counterpart thus they have plenty of support to keep going inspite of other heavyweights competition from Gatsby or Next.js.</li></ol><h3 id="How-did-I-accomplish-it"><a href="#How-did-I-accomplish-it" class="headerlink" title="How did I accomplish it?"></a>How did I accomplish it?</h3><p>To my surprise, the whole thing did not take very long. At most, it took only 3-4 days to make this all happen.</p><p>I have to consider the following list of steps to work it out.</p><ol><li>Created a new Hexo git repo locally.</li><li>Published my first blog post or page locally (verify that it works) - <code>hexo generate &amp;&amp; hexo server</code>.</li><li>Configured deployment Heroku plugin for Hexo.</li><li>Started looking into migrating Octopress to Hexo (as mentioned earlier).</li><li>Installed and modified pre-existing theme I used for my old blog onto the new one.</li><li>Published all the Octopress blogging locally, verify all the blog posts content are there.</li><li>Once satisfied, test deploying the published content to the Heroku platform.</li></ol><p>And voila! 👏</p><p>Once I was happy my Heroku landing site is coming all together, I can start take my old Octopress site offline for maintenance on 27&#x2F;04&#x2F;2022 for few hours, and updated the DNS for both my <a href="https://www.namecheap.com/support/knowledgebase/article.aspx/9737/2208/pointing-a-domain-to-the-heroku-app/?gclid=CjwKCAjwsJ6TBhAIEiwAfl4TWDIirrOL6f_kI7ZP_PfEMZarTA0QyiAnmEG9dSwcMOrF4IPAu0tp2xoCx8IQAvD_BwE%5D">Namecheap DNS provider</a> and <a href="https://devcenter.heroku.com/articles/custom-domains">Heroku Customer Domains</a>. </p><p>You’ve landed on my blog post to witness all that new shiny armor of my blog post pages - for all its good glory.</p><p>If you’re curious how I manage to pull all of this off, I could explain in the following.</p><p>The critical changes I had to do are its basic <code>octo</code> theme I found via Hexo Theme Plugin search.</p><p>As my former Octo theme, codenamed BoldAndBlue, does not come available via Hexo Theme Plugin compared to the Octopress counterpart, I decided to take necessary ‘’hacking” steps to re-skin the theme that I want.</p><p>Here’s the following list of such changes</p><ol><li><p>Sidebar EJS for Twitter <code>sidebar-twitter.ejs</code> does not exist so I had to build from scratch and port the original functionality from the old site.</p></li><li><p>Default CSS Theme was ugly to be honest on the surface, so I had to port the entire raw-compiled SASS-to-CSS files from the old site.  My Hexo theme does not come with SASS support by default.  I don’t really believe there’s any value to maintain SASS files as I rarely need to tailor my web site pages for the past 7 years. So I’d leave that as it is.</p></li><li><p>Google Fonts Typography was ugly thus Google Web Fonts <code>Nato</code> and <code>Open Sans Serif</code> were brought back here. </p></li><li><p>Sidebar EJS for Github options were modified <code>sidebar-github.ejs</code>  to cater extra Github config options, for eg</p><pre><code class="hljs javascript"><span class="hljs-keyword">const</span> options = &#123;   <span class="hljs-attr">user</span>: <span class="hljs-string">&#x27;&lt;%=theme.sidebars.github.profile%&gt;&#x27;</span>,   <span class="hljs-attr">count</span>: <span class="hljs-string">&#x27;&lt;%=theme.sidebars.github.repo_count%&gt;&#x27;</span>,   <span class="hljs-attr">skip_forks</span>: <span class="hljs-string">&#x27;&lt;%=theme.sidebars.github.skip_forks%&gt;&#x27;</span>&#125;;</code></pre><p>which the current Hexo doesn’t come with such configurations.</p></li><li><p>Individual post date time display information was missing ie <code>new Date(page.date).toLocaleString()</code> along with formatting options.</p></li><li><p>Config.yaml settings; the following properties had to be altered:</p><ul><li>site </li><li>url</li><li>writing’s new_post_name</li><li>theme</li><li>deploy</li></ul></li><li><p>Theme’s config.yaml settings; the following properties had to be altered.</p><ul><li>site</li><li>menu</li><li>social</li><li>sidebars</li><li>disqus</li><li>google analytics id</li><li>atom - <code>npm install hexo-generator-feed</code></li><li>stylesheets - installed HightlightJS’s Solarized Dark CSS Theme</li></ul></li><li><p>Tags&#x2F;categories were missing&#x2F; and archives post headings to do not have properly urls.</p></li><li><p>Favicon changes</p></li><li><p>Disqus comments hyperlink next to post’s actual published time.</p><pre><code class="hljs handlebars"><span class="language-xml">// - post EJS template</span><span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">time</span> <span class="hljs-attr">datetime</span>=<span class="hljs-string">&quot;&lt;%=page.date.toJSON()%&gt;&quot;</span> <span class="hljs-attr">data-updated</span>=<span class="hljs-string">&quot;true&quot;</span>&gt;</span> ...<span class="hljs-tag">&lt;/<span class="hljs-name">time</span>&gt;</span></span><span class="language-xml">// - disqus comments hyperlink</span><span class="language-xml">&lt;% if (theme.disqus.enabled) &#123; %&gt;</span><span class="language-xml">   | <span class="hljs-tag">&lt;<span class="hljs-name">a</span> <span class="hljs-attr">href</span>=<span class="hljs-string">&quot;#disqus_thread&quot;</span></span></span><span class="hljs-tag"><span class="language-xml">   <span class="hljs-attr">data-disqus-identifier</span>=<span class="hljs-string">&quot;&lt;%=theme.site.url%&gt;&quot;</span>&gt;</span>Comments<span class="hljs-tag">&lt;/<span class="hljs-name">a</span>&gt;</span></span><span class="language-xml">&lt;% &#125; %&gt;</span></code></pre></li><li><p>Home page missing EJS tags, such as <code>page.ejs</code>, etc.</p></li></ol><h3 id="Concluding-thoughts"><a href="#Concluding-thoughts" class="headerlink" title="Concluding thoughts"></a>Concluding thoughts</h3><p>Overall,  I’m happy with the experience it comes with.  After performing the steps, I can start to concentrate better on writing content without having to deal with the inefficiencies that Octopress has suffered over the years since the community last supported in 2015, thanks to the influx of other competing static site tool generators to choose from.  I can only hope the same for Hexo as they still do recent releases in the past two years at the time this blog post is written.  So, touch wood, Hexo will be around for X-number of years to come.</p><p>Octopress was being good to me over the years and I remembered Ruby static site generators were incredibly popular during 2014&#x2F;2015 as web frameworks were incredibly high on demand before other popular programming languages like PHP, Javascript, Python, NodeJS etc came along.</p><p>I remembered the initial excitement of writing my <a href="http://www.awongcm.io/blog/2015/04/03/my-awesome-post/">first</a> tech blog post for the world to read back then.</p><p>Thus I want to feel what it feels like to launch my blog pages, for the second time - several years later!</p><p>Here’s to my future blogging days ahead, no matter what the future changes may bring.</p><p>Till then, Happy Coding! 👨‍💻💻⌨️</p><p><strong>PS:</strong> I want to give to this blog post link that ultimately helped to make Hexo transition smoothly in  the first place! 😉😊  -  <a href="https://gangmax.me/blog/2019/12/16/From-Octopress-to-Hexo/">https://gangmax.me/blog/2019/12/16/From-Octopress-to-Hexo/</a></p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_migrating_octopress_to_hexo.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;We’re edging towards halfway of 2022.. And boy. Look how time has gotten passed by.&lt;/p&gt;
&lt;p&gt;I must admit I haven’t had the chance to sit down and write up another good tech blog post. A lot is happening in motion since the start of the year 2022 from my recent engagement to my fiancee of 3 and half years; busy with my day job as a contractor for a bank; finally getting my passport renewal done in Canberra (after almost 2 years of nationwide lockdown) as part of my travel plans to Malaysia so I can send my father’s ashes for his final resting place; along with talking plans having my fiancee and me moving together later in the year and preparing my next career technical certifications to take on, etc…&lt;/p&gt;
&lt;p&gt;The list could go on.&lt;/p&gt;
&lt;p&gt;Nevertheless, still, I’m glad to take the time to write about what I have accomplished recently.&lt;/p&gt;
&lt;p&gt;That is…&lt;/p&gt;
&lt;p&gt;I’ve decided to migrate my Octopress site to Hexo! 📦🚚&lt;/p&gt;</summary>
    
    
    
    
    <category term="ruby" scheme="http://awongcm.io/tags/ruby/"/>
    
    <category term="javascript" scheme="http://awongcm.io/tags/javascript/"/>
    
    <category term="nodejs" scheme="http://awongcm.io/tags/nodejs/"/>
    
    <category term="static-site-generator" scheme="http://awongcm.io/tags/static-site-generator/"/>
    
    <category term="blog" scheme="http://awongcm.io/tags/blog/"/>
    
    <category term="migration" scheme="http://awongcm.io/tags/migration/"/>
    
  </entry>
  
  <entry>
    <title>Snyk - For your web security and dependencies vulnerabilities health check on Github repos</title>
    <link href="http://awongcm.io/blog/2021/09/14/snyk-for-your-web-security-and-dependencies-vulnerabilities-health-check-on-github-repos/"/>
    <id>http://awongcm.io/blog/2021/09/14/snyk-for-your-web-security-and-dependencies-vulnerabilities-health-check-on-github-repos/</id>
    <published>2021-09-14T18:47:06.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_snyk_for_web_dependencies_vulnerabilities_health_checks.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>If you’re like me, and you’ve built several Github repositories over time as part of developer learning fun projects, there’s probably a good chance these same repositories will not get maintained on some form of regular basis.  </p><p>The reasons for this to happen can be wide-ranging - everything from being too busy at work, changing interests for different software frameworks, lifestyle priorities changes, changing career responsibilities etc, etc - there’s too many to list here.  </p><p>They took the best of our coding life as months or years go by.  </p><p>As a consequence, your Github repositories do become quickly stale over time.   </p><p>This ‘staleness’ - come with old dependencies that do not get updated, which at worse, could be leaking software security issues over time.   </p><p>I have over 50 repositories in my Github profile and there’s no way that I could keep track of knowing which repos have more security holes to address after another, let alone have to fix up security vulnerabilities dependencies one PR at a time on my own… 😨  </p><p>Thus I need to find a way to auto-manage all these repositories without lifting a finger (much)  </p><p>Without further ado, I found this useful toolchain from Github marketplace - Snyk.  </p><span id="more"></span><img src="/images/snyk_logo.png" class="center" width="500" height="500" title="Synk Logo Synk Logo"><p>Synk is an open-source tool that helps developers track, find and fix security vulnerabilities across all of your Github projects.  </p><p>According to this <a href="https://github.com/marketplace/snyk">page</a>, as it describes as -</p><ol><li>Finds vulnerabilities by scanning your repos using its transitive database to match their vulnerabilities behind them</li><li>Fix their vulnerabilities by auto-generating a new PR that will be recommended the patches around those vulnerabilities hole. </li><li>Prevents vulnerabilities by running their suite of tests on those same PRs and prevent the vulnerabilities from merging to your PR </li><li>It offers continuous monitoring on Github repos so you can respond quickly as you need on a daily basis.</li></ol><p>This all sounds pretty awesome!  </p><p>This is the type of thing I will like to include as my software toolchain to know where my repos are going wrong or not. So I don’t have to manually install things myself.  </p><p>What a great boon to keep my repos and clean and fresh!  </p><p>Best of all - it is absolutely free! You can do this on unlimited Github repos you have in your portfolio. It comes with its own special CI&#x2F;CD pipeline as well.  </p><p>To start with,  </p><ol><li>You open up the enable tool-chain from Marketplace.</li><li>Snyk scanner pipeline is monitoring all of your Github repos.</li><li>Then you will get the Snyk emails to do look at them.</li></ol><p>Once you have all that setup, you will eventually arrive at this dashboard screen.  </p><img src="/images/snyk_dashboard.png" class="center" width="500" height="500" title="Synk Dashboard Synk Dashboard"><p>There it is - my one-stop landing page to monitor and assess the state of my Github repos’ dependencies health check on security vulnerabilities.  </p><p>Like all things of security audit and vulnerability checks as you would do with anti-virus and anti-Malwarebytes check, we have to make our own informed choices whether we should triage and resolve these vulnerabilities urgently or not, depending on their level of severity as well the broadness of their impacts to their entire health state of the app’s core functionality.  </p><p>Once you decide when you want to resolve these vulnerabilities with ease, Snyk’s give you this option to create open PR to one of your nominated repos, and it will self-generate the best security patch algorithm for you and will apply these patches should you decide to accept their security recommendation by merging.  </p><p>For eg, I recently did one of the security patches not long ago on one of my old Node repos, which I haven’t touched for some considerable time.</p><img src="/images/snyk_github_1.png" class="center" width="500" height="500" title="Snyk Merged PR Synk Merged PR"><p>If I open up the merged PR.  </p><img src="/images/snyk_github_2.png" class="center" width="500" height="500" title="Snyk package management update Synk package management update - screenshot 1"><p>Here you can see a table generated by Snyk outlining explanations of which files suffered security vulnerabilities and came up with recommended fixes for these.  </p><p>If you click on the ‘’Files Change’ tab, you notice the following:  </p><img src="/images/snyk_github_3.png" class="center" width="500" height="500" title="Snyk package management update Synk package management update - screenshot 2"><p>It forewarns that these are affected npm libaries that need to be upgraded at their latest working version whose security patches will be applied.  </p><p>That’s it!  </p><p>All of the thought of manually checking and verifying package libraries’ vulnerabilities by myself not became a major headache to deal with.</p><p>I have the available tools that help to automate that workflow process for me - without doing any heavy finger lifting!  </p><p>This is precisely what I love about it. 😎  </p><p>Go ahead, reap and sow the power of free open source software that’s built by the community that rests on the shoulder of giants!🤞🤞👨‍💻👩‍💻  </p><p>Till next time, Happy Coding!</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_snyk_for_web_dependencies_vulnerabilities_health_checks.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;If you’re like me, and you’ve built several Github repositories over time as part of developer learning fun projects, there’s probably a good chance these same repositories will not get maintained on some form of regular basis.  &lt;/p&gt;
&lt;p&gt;The reasons for this to happen can be wide-ranging - everything from being too busy at work, changing interests for different software frameworks, lifestyle priorities changes, changing career responsibilities etc, etc - there’s too many to list here.  &lt;/p&gt;
&lt;p&gt;They took the best of our coding life as months or years go by.  &lt;/p&gt;
&lt;p&gt;As a consequence, your Github repositories do become quickly stale over time.   &lt;/p&gt;
&lt;p&gt;This ‘staleness’ - come with old dependencies that do not get updated, which at worse, could be leaking software security issues over time.   &lt;/p&gt;
&lt;p&gt;I have over 50 repositories in my Github profile and there’s no way that I could keep track of knowing which repos have more security holes to address after another, let alone have to fix up security vulnerabilities dependencies one PR at a time on my own… 😨  &lt;/p&gt;
&lt;p&gt;Thus I need to find a way to auto-manage all these repositories without lifting a finger (much)  &lt;/p&gt;
&lt;p&gt;Without further ado, I found this useful toolchain from Github marketplace - Snyk.  &lt;/p&gt;</summary>
    
    
    
    
    <category term="github" scheme="http://awongcm.io/tags/github/"/>
    
    <category term="security" scheme="http://awongcm.io/tags/security/"/>
    
    <category term="dependencies-management" scheme="http://awongcm.io/tags/dependencies-management/"/>
    
    <category term="libraries" scheme="http://awongcm.io/tags/libraries/"/>
    
    <category term="package-management" scheme="http://awongcm.io/tags/package-management/"/>
    
  </entry>
  
  <entry>
    <title>Differences between Django vs Flask/Falcon</title>
    <link href="http://awongcm.io/blog/2021/06/09/differences-between-django-vs-flask-slash-falcon/"/>
    <id>http://awongcm.io/blog/2021/06/09/differences-between-django-vs-flask-slash-falcon/</id>
    <published>2021-06-09T19:07:44.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_django_flask_falcon.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>It’s been some months since 2021 has begun (and we’re not where near safe coming out of the pandemic woods just yet ). I’ve been itching to come and sit down to write my first blog post for the year.</p><p>What better way to start out writing ins-and-outs on certain Python web frameworks.</p><img src="/images/django_vs_flask_falcon_cropped.png" class="" title="Django vs Flask-Falcon Django vs Flask-Falcon"><p>I’ve been working on Python web projects for some time and I’m here to offer my rants&#x2F;thoughts when working between Django and Flask&#x2F;Falcon and outline the comparisons between these two.</p><p>Let’s start with the ones I’m most familiar with.</p><p><em>NB - Before I begin, this article assumes the reader either has a good understanding or good working knowledge of MVC software patterns as more web frameworks are built around this pattern.  If you don’t know what that is, you may want to visit this <a href="https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller">Wikipedia</a> page as a refresher course, before proceed reading.</em></p><span id="more"></span><h2 id="Flask-x2F-Falcon"><a href="#Flask-x2F-Falcon" class="headerlink" title="Flask&#x2F;Falcon"></a>Flask&#x2F;Falcon</h2><p>First of the rank is Flask. 🌶</p><p>Flask’s software design philosophy is to help developers to focus on code design simplicity first.  What it means by that is it’s a framework that does not come with a lot of ‘bells and whistle’ you need in building the web app such as scaling web server, incorporating templating engine language, authentication system, middleware libraries, service logic and database layers.  </p><p>You get to handpick a number of these yourselves to decide how your app should take form.</p><p>From project experience, I used a number of well-known libraries that are popular amongst the Python developer community.  They include things like:</p><ul><li>SQLAlchemy - popular ORM for Python</li><li>Alembic - database migration engine written Python</li><li>Gunicorn -  a popular web server for running Python web apps</li><li>Flask-related libraries ie file upload, admin logon etc.</li><li>Jinja templating engine</li><li>Marshmallow - simplified object serialisation&#x2F;deserialisation libraries</li></ul><p>So without further ado, let’s start with its basic layout</p><h5 id="Basic-Layout"><a href="#Basic-Layout" class="headerlink" title="Basic Layout"></a>Basic Layout</h5><pre><code class="hljs python"><span class="hljs-comment">## It&#x27;s Flasking time!</span><span class="hljs-keyword">from</span> flask <span class="hljs-keyword">import</span> Flaskapp = Flask(__name__)app.route(<span class="hljs-string">&#x27;/&#x27;</span>)<span class="hljs-keyword">def</span> <span class="hljs-title function_">hello_world</span>():    <span class="hljs-keyword">return</span> <span class="hljs-string">&#x27;Hello, World!&#x27;</span></code></pre><p>We start with importing the Flask package and made its object instantiation to kick off the application.  </p><p>With <code>app</code>,  you can start to create route system to have a home landing page to render the home page’s content page by making using of its route decorator functions.</p><h5 id="Folder-structure"><a href="#Folder-structure" class="headerlink" title="Folder structure"></a>Folder structure</h5><pre><code class="hljs plaintext">app.pyconfig.pyrequirements.txtstatictemplates</code></pre><p>In the above, we have its basic folder structure that comes with minimalists set of files.  Usually, it comes with the base app file, the <code>config.py</code> setup file, the <code>requirements.txt</code> for managing app dependencies along with <code>static</code> and <code>templates</code> folders, where static contains all the front end libraries like JQuery&#x2F;JS, CSS and one HTML file. While templates containing all of your Jinja templating files.</p><h5 id="View-Template"><a href="#View-Template" class="headerlink" title="View Template"></a>View Template</h5><pre><code class="hljs python"><span class="hljs-keyword">from</span> flask <span class="hljs-keyword">import</span> render_templateapp.route(<span class="hljs-string">&#x27;/hello/&#x27;</span>)<span class="hljs-keyword">def</span> <span class="hljs-title function_">hello</span>(<span class="hljs-params">name=<span class="hljs-literal">None</span></span>):    <span class="hljs-keyword">return</span> render_template(<span class="hljs-string">&#x27;hello.html&#x27;</span>, name=name)</code></pre><p>On the V-side of MVC pattern,   we use import<code>render_template</code>  function and using the same function, we make the function call to render particular a JINJA template file (which normally resides at root of the application folder by default), based on the current url route where we viewing the template from.</p><p>#####Database models</p><pre><code class="hljs python"><span class="hljs-comment"># Using Flask-SQLAlchemy</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">User</span>(db.Model):    <span class="hljs-built_in">id</span> = db.Column(db.Integer, primary_key=<span class="hljs-literal">True</span>)    username = db.Column(db.String(<span class="hljs-number">80</span>), unique=<span class="hljs-literal">True</span>, nullable=<span class="hljs-literal">False</span>)    email = db.Column(db.String(<span class="hljs-number">120</span>), unique=<span class="hljs-literal">True</span>, nullable=<span class="hljs-literal">False</span>)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__repr__</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">&#x27;&lt;User %r&gt;&#x27;</span> % <span class="hljs-variable language_">self</span>.username</code></pre><p>On the M-side of MVC pattern,  we use the more popular ORM library Flask-SQLAlchemy ()which is a wrapper on top of SQLAlchemy) that assists in define our entity models, along with using datatypes for our model’s attributes which are easy to understand and follow.</p><h5 id="Routes"><a href="#Routes" class="headerlink" title="Routes"></a>Routes</h5><pre><code class="hljs python">app.route(<span class="hljs-string">&#x27;/&#x27;</span>)<span class="hljs-keyword">def</span> <span class="hljs-title function_">index</span>():    <span class="hljs-keyword">return</span> <span class="hljs-string">&#x27;Index Page&#x27;</span><span class="hljs-meta">@app.route(<span class="hljs-params"><span class="hljs-string">&#x27;/hello&#x27;</span></span>)</span><span class="hljs-keyword">def</span> <span class="hljs-title function_">hello</span>():    <span class="hljs-keyword">return</span> <span class="hljs-string">&#x27;Hello, World&#x27;</span></code></pre><p>On the C-side of MVC pattern,  we use route decorators to tell Flask which url segments will be rerouted to the appropriate controller which are responsible for handling such requests.</p><p>Nothing new here.</p><h5 id="Forms-Handling"><a href="#Forms-Handling" class="headerlink" title="Forms Handling"></a>Forms Handling</h5><pre><code class="hljs python"><span class="hljs-keyword">from</span> flask <span class="hljs-keyword">import</span> Flask, render_template,requestapp = Flask(__name__)<span class="hljs-meta">@app.route(<span class="hljs-params"><span class="hljs-string">&#x27;/send&#x27;</span>,methods = [<span class="hljs-string">&#x27;GET&#x27;</span>,<span class="hljs-string">&#x27;POST&#x27;</span>]</span>)</span><span class="hljs-keyword">def</span> <span class="hljs-title function_">send</span>():    <span class="hljs-keyword">if</span> request.method == <span class="hljs-string">&#x27;POST&#x27;</span>:        age = request.form[<span class="hljs-string">&#x27;age&#x27;</span>]        <span class="hljs-keyword">return</span> render_template(<span class="hljs-string">&#x27;age.html&#x27;</span>,age=age)    <span class="hljs-keyword">return</span> render_template(<span class="hljs-string">&#x27;index.html&#x27;</span>)<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">&#x27;__main__&#x27;</span>:app.run()</code></pre><p>When working on the pure SSR(server side rendering) apps, we import our <code>render_template</code>  and <code>request</code> combo to tell Flask that we’re going to be dealing with form Jinja Template.</p><h5 id="Unit-Tests"><a href="#Unit-Tests" class="headerlink" title="Unit Tests"></a>Unit Tests</h5><pre><code class="hljs python"><span class="hljs-keyword">import</span> os<span class="hljs-keyword">import</span> tempfile<span class="hljs-keyword">import</span> pytest<span class="hljs-keyword">from</span> flaskr <span class="hljs-keyword">import</span> create_app<span class="hljs-keyword">from</span> flaskr <span class="hljs-keyword">import</span> flaskr.app <span class="hljs-keyword">as</span> flaskr_app<span class="hljs-meta">@pytest.fixture</span><span class="hljs-keyword">def</span> <span class="hljs-title function_">client</span>():    db_fd, flaskr_app.config[<span class="hljs-string">&#x27;DATABASE&#x27;</span>] = tempfile.mkstemp()    flaskr_app.config[<span class="hljs-string">&#x27;TESTING&#x27;</span>] = <span class="hljs-literal">True</span>    <span class="hljs-keyword">with</span> flaskr_app.test_client() <span class="hljs-keyword">as</span> client:        <span class="hljs-keyword">with</span> flaskr_app.app_context():            flaskr.init_db()        <span class="hljs-keyword">yield</span> client    os.close(db_fd)    os.unlink(flaskr_app.config[<span class="hljs-string">&#x27;DATABASE&#x27;</span>])</code></pre><p>When doing unit testing for Flask app, we can incorporate some of the popular Python testing frameworks like Unittest or Pytest without much difficulty.  Like, in the example above, we have the Python test file here that verifies the application configuration and initialising a new database.  Note that everything here is setup a pytest fixture thus we treat this as an individual test module.  Then, we can run execute the <code>pytest</code> command and we still get tests results feedback on Flask’s design implementation.  </p><p>Simple.  Not much different to our other types of unit and integration test suite</p><p>That’s it for Flask!</p><p>Next, we have Falcon. 🦅</p><p>Like Flask, Falcon is also a micro framework.  But it focuses purely on building REST-ful API-driven applications, therefore thus views and routes concepts are non-existent here. It’s considered a minimalist framework that does not come with plenty of dependencies and a heavy amount of abstractions you have to come to grip with.</p><p><em>NB - You can actually build APIs with Flask as well by the way.  Flask, by no means, it’s not built entirely just for SSR apps only.  You can download the Flask-RESTful library to achieve the minimal API - you can see the working sample code <a href="https://flask-restful.readthedocs.io/en/latest/quickstart.html">here</a></em> </p><h5 id="Basic-Layout-1"><a href="#Basic-Layout-1" class="headerlink" title="Basic Layout"></a>Basic Layout</h5><pre><code class="hljs python"><span class="hljs-keyword">from</span> wsgiref.simple_server <span class="hljs-keyword">import</span> make_server<span class="hljs-keyword">import</span> falconapp = falcon.App()</code></pre><p>To start, we make the falcon import to start the app.  No much different to Flask’s counterpart.</p><h5 id="Folder-structure-1"><a href="#Folder-structure-1" class="headerlink" title="Folder structure"></a>Folder structure</h5><pre><code class="hljs plaintext">api├── .venv└── api    ├── __init__.py    └── app.py</code></pre><p>We have the above folder structure.  Notice we don’t have any HTML rendering templates to view, compared to Flask.</p><h5 id="API-Resource-Design"><a href="#API-Resource-Design" class="headerlink" title="API Resource Design"></a>API Resource Design</h5><pre><code class="hljs python"><span class="hljs-comment"># Our resource definition</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">MediaResource</span>:    <span class="hljs-keyword">def</span> <span class="hljs-title function_">on_get</span>(<span class="hljs-params">self, req, resp</span>):        <span class="hljs-string">&quot;&quot;&quot;Handles GET requests&quot;&quot;&quot;</span>        resp.status = falcon.HTTP_200  <span class="hljs-comment"># This is the default status</span>        resp.content_type = falcon.MEDIA_TEXT  <span class="hljs-comment"># Default is JSON, so override</span>        resp.text = (<span class="hljs-string">&#x27;\nTwo things awe me most, the starry sky &#x27;</span>                     <span class="hljs-string">&#x27;above me and the moral law within me.\n&#x27;</span>                     <span class="hljs-string">&#x27;\n&#x27;</span>                     <span class="hljs-string">&#x27;    ~ Immanuel Kant\n\n&#x27;</span>)<span class="hljs-comment"># Our app instance</span>app = falcon.App()<span class="hljs-comment"># We have resources are presented as long-living instances</span>media = MediaResource()<span class="hljs-comment"># App will handled &#x27;media&#x27; requests here</span>app.add_route(<span class="hljs-string">&#x27;/media&#x27;</span>, media)</code></pre><p>Next, we start writing out our resource definition for our APIs.  </p><p>As part of Falcon’s design philosophy, it adheres to a lot of REST architectural styles, thus they ‘guide you in mapping resources and state manipulations to actual HTTP verbs.</p><p>Thus in the above example,  we declared our <code>MediaResource</code> that handles  any <code>get</code> requests coming in and produces the appropriate responses for them by loading the correct dependencies by its own dependency injector out of the box.</p><p>Simple nice and clean!</p><p>#####Database models</p><pre><code class="hljs python"><span class="hljs-comment"># Using SQLAlchemy </span><span class="hljs-keyword">class</span> <span class="hljs-title class_">User</span>(db.Model):    <span class="hljs-built_in">id</span> = db.Column(db.Integer, primary_key=<span class="hljs-literal">True</span>)    username = db.Column(db.String(<span class="hljs-number">80</span>), unique=<span class="hljs-literal">True</span>, nullable=<span class="hljs-literal">False</span>)    email = db.Column(db.String(<span class="hljs-number">120</span>), unique=<span class="hljs-literal">True</span>, nullable=<span class="hljs-literal">False</span>)    <span class="hljs-keyword">def</span> <span class="hljs-title function_">__repr__</span>(<span class="hljs-params">self</span>):        <span class="hljs-keyword">return</span> <span class="hljs-string">&#x27;&lt;User %r&gt;&#x27;</span> % <span class="hljs-variable language_">self</span>.username</code></pre><p>Again nothing different here.  I personally use SQLAlchemy ORM library and it works well when building your models to support your data-rich RESTful applications.</p><h5 id="Unit-tests"><a href="#Unit-tests" class="headerlink" title="Unit tests"></a>Unit tests</h5><pre><code class="hljs python"><span class="hljs-comment"># -----------------------------------------------------------------</span><span class="hljs-comment"># unittest</span><span class="hljs-comment"># -----------------------------------------------------------------</span><span class="hljs-keyword">from</span> falcon <span class="hljs-keyword">import</span> testing<span class="hljs-keyword">import</span> myapp<span class="hljs-keyword">class</span> <span class="hljs-title class_">MyTestCase</span>(testing.TestCase):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">setUp</span>(<span class="hljs-params">self</span>):        <span class="hljs-built_in">super</span>(MyTestCase, <span class="hljs-variable language_">self</span>).setUp()        <span class="hljs-variable language_">self</span>.app = myapp.create()<span class="hljs-keyword">class</span> <span class="hljs-title class_">TestMyApp</span>(<span class="hljs-title class_ inherited__">MyTestCase</span>):    <span class="hljs-keyword">def</span> <span class="hljs-title function_">test_get_message</span>(<span class="hljs-params">self</span>):        doc = &#123;<span class="hljs-string">&#x27;message&#x27;</span>: <span class="hljs-string">&#x27;Hello world!&#x27;</span>&#125;        result = <span class="hljs-variable language_">self</span>.simulate_get(<span class="hljs-string">&#x27;/messages/42&#x27;</span>)        <span class="hljs-variable language_">self</span>.assertEqual(result.json, doc)<span class="hljs-comment"># -----------------------------------------------------------------</span><span class="hljs-comment"># pytest</span><span class="hljs-comment"># -----------------------------------------------------------------</span><span class="hljs-keyword">from</span> falcon <span class="hljs-keyword">import</span> testing<span class="hljs-keyword">import</span> pytest<span class="hljs-keyword">import</span> myapp<span class="hljs-meta">@pytest.fixture()</span><span class="hljs-keyword">def</span> <span class="hljs-title function_">client</span>():    <span class="hljs-keyword">return</span> testing.TestClient(myapp.create())<span class="hljs-keyword">def</span> <span class="hljs-title function_">test_get_message</span>(<span class="hljs-params">client</span>):    doc = &#123;<span class="hljs-string">&#x27;message&#x27;</span>: <span class="hljs-string">&#x27;Hello world!&#x27;</span>&#125;    result = client.simulate_get(<span class="hljs-string">&#x27;/messages/42&#x27;</span>)    <span class="hljs-keyword">assert</span> result.json == doc</code></pre><p>When comes to building unit tests for Falcon, it’s relatively straightforward.  </p><p>You can add a test framework of your choice ie Unittest or Pytest without any difficulty to the framework.  You will achieve the same goals as Flask’ unit test configuration counterpart.  The only difference between the two is that Falcon comes with many testing utility functions that will better support functional testing in Falcon such as <code>simulate_*</code> method semantics.  These are, essentially, to support all of your request&#x2F;response test cycles across all the common HTTP verbs and actions. just like the <code>client.simulate_get</code> example above. </p><p>Now that we got those two out of the way with a firm understanding of how these minimal frameworks are going to work, let’s go and check out the Django side of the fence ⚠️ ⚠️⚠️ ⚠️ </p><h2 id="Django"><a href="#Django" class="headerlink" title="Django"></a>Django</h2><p>Recently worked on Django project, which is a Python fully-fledged MVC framework that has been grazing around the community for a very long time.</p><p>To recap, MVC (as stated earlier in this post),  is one of the oldest (and most typical) software design patterns any veteran would tell for years they have been using since the Internet era was born 20 years plus ago.</p><p>From this design pattern, you would expect to see the Python code written for each layer of componentised design of web applications or software system.</p><p>To start, you begin with</p><p><code>django-admin startproject myhomepage</code></p><h5 id="Application-Scaffolding-Folder-structure"><a href="#Application-Scaffolding-Folder-structure" class="headerlink" title="Application Scaffolding - Folder structure"></a>Application Scaffolding - Folder structure</h5><pre><code class="hljs python">myhomepage/    manage.py    myhomepage/        __init__.py        settings.py        urls.py        asgi.py        wsgi.py</code></pre><p>After running the django admin commands, you will be greeted with the above Django basic folder structure.  You start with the container project folder named <code>myhomepage</code> , which is the name you made in the first step.  Then, we have <code>manage.py</code> which is the utility Python file that lets you interact with the Django app via the CLI commands.</p><p>We see that there’s subfolder name that is named <code>myhomepage</code> again here but this folder will be marked as the Python package that used to contain all the core logic files which encapsulate everything about the Django app itself.</p><p>Within this folder, you’re greeted with the following:</p><ul><li><code>__init__.py</code> - an empty file to tell Python that this is considered as a Python package.</li><li><code>settings.py</code> - the settings file for the Django Project.</li><li><code>urls.py</code> - where you can write up all of your URLs declaration of the Django-powered website</li><li><code>asgi.py</code> - entry point for setting ASGI webserver to serve the app.</li><li><code>wsgi.py</code> - entry point for setting WSGI webserver to serve the app.</li></ul><p>To understand the difference between ASGI and WSGI webserver interface you can read more about it here on this <a href="https://medium.com/analytics-vidhya/difference-between-wsgi-and-asgi-807158ed1d4c">post</a> if you’re curious.</p><h5 id="Routes-x2F-Views-x2F-Templates"><a href="#Routes-x2F-Views-x2F-Templates" class="headerlink" title="Routes&#x2F;Views&#x2F;Templates"></a>Routes&#x2F;Views&#x2F;Templates</h5><pre><code class="hljs python"><span class="hljs-comment">## Routes - urls.py</span><span class="hljs-keyword">from</span> django.urls <span class="hljs-keyword">import</span> path<span class="hljs-keyword">from</span> . <span class="hljs-keyword">import</span> viewsurlpatterns = [    path(<span class="hljs-string">&#x27;&#x27;</span>, views.index, name=<span class="hljs-string">&#x27;index&#x27;</span>),    path(<span class="hljs-string">&#x27;&lt;int:question_id&gt;/&#x27;</span>, views.detail, name=<span class="hljs-string">&#x27;detail&#x27;</span>),    path(<span class="hljs-string">&#x27;&lt;int:question_id&gt;/results/&#x27;</span>, views.results, name=<span class="hljs-string">&#x27;results&#x27;</span>),    path(<span class="hljs-string">&#x27;&lt;int:question_id&gt;/vote/&#x27;</span>, views.vote, name=<span class="hljs-string">&#x27;vote&#x27;</span>),]<span class="hljs-comment">## Views - views.py</span><span class="hljs-keyword">from</span> django.http <span class="hljs-keyword">import</span> HttpResponse<span class="hljs-keyword">from</span> django.template <span class="hljs-keyword">import</span> loader<span class="hljs-keyword">from</span> .models <span class="hljs-keyword">import</span> Question<span class="hljs-keyword">def</span> <span class="hljs-title function_">index</span>(<span class="hljs-params">request</span>):    latest_question_list = Question.objects.order_by(<span class="hljs-string">&#x27;-pub_date&#x27;</span>)[:<span class="hljs-number">5</span>]    template = loader.get_template(<span class="hljs-string">&#x27;myhomepage/index.html&#x27;</span>)    context = &#123;<span class="hljs-string">&#x27;latest_question_list&#x27;</span>: latest_question_list,&#125;    output = template.render(context, request)    <span class="hljs-keyword">return</span> HttpResponse(output)    ........</code></pre><p>When comes to building routes and views,  Django has this plain concept of views where it is represented as Python function that is responsible for coordinating business logic and expects an actual HTML template that will bind the output of the business logic to render the page.  To view the actual rendered page on the browser, Django uses the URL declaration file <code>urls.py</code> where we can place all of our well-defined URL namespace routes for each rendered template output we specified for eg</p><p><code>path(&#39;main/&#39;, views.index, name=&#39;index&#39;)</code></p><p>To view the rendered ‘index’ page, we configured the URL path as <code>homepage</code> and then hook it up with our exported views’ <code>index</code> function.  Thus in the URL, you would see the content renderded via <a href="http://localhost/main/">http://localhost/main/</a>. </p><p>With this, we expected to have a big laundry list of views&#x2F;templates to route and render via the <code>urlpatterns</code> list.</p><h5 id="Database-models"><a href="#Database-models" class="headerlink" title="Database models"></a>Database models</h5><pre><code class="hljs python"><span class="hljs-keyword">from</span> django.db <span class="hljs-keyword">import</span> models<span class="hljs-keyword">class</span> <span class="hljs-title class_">Question</span>(models.Model):    question_text = models.CharField(max_length=<span class="hljs-number">200</span>)    pub_date = models.DateTimeField(<span class="hljs-string">&#x27;date published&#x27;</span>)<span class="hljs-keyword">class</span> <span class="hljs-title class_">Choice</span>(models.Model):    question = models.ForeignKey(Question, on_delete=models.CASCADE)    choice_text = models.CharField(max_length=<span class="hljs-number">200</span>)    votes = models.IntegerField(default=<span class="hljs-number">0</span>)</code></pre><p>When it comes to building up database models under the hood, Django comes bundled with its own ORM framework.  Like the SQLAlchemy counterpart, it also comes with the Pythonical way of interacting with relational databases such as creating, querying and manipulating tables inserts and updates.  Also, the semantics for describing the db data types  are slightly different to the SQLAlchemy as they are bounded to the django.db models having exposed API function calls to make.</p><p>When it comes to building APIs, compared to Flask&#x2F;Falcon, we have the external toolkit package that handles this well.  It’s called the <a href="https://www.django-rest-framework.org/">djangorestframework</a>.  The toolkit is built specifically for handling API-driven architecture systems as Django itself is a fully-fledged MVC framework for lots of server-side driven web applications which are traditionally tightly coupled together.</p><p>To start with, you do the following.</p><h5 id="Installation"><a href="#Installation" class="headerlink" title="Installation"></a>Installation</h5><pre><code class="hljs bash">pip install djangorestframework</code></pre><p>When creating the app,  you do the following</p><pre><code class="hljs bash">django-admin startproject myapp<span class="hljs-built_in">cd</span> myappdjango-admin startapp tutorial<span class="hljs-built_in">cd</span> ..</code></pre><p>And you will notice the following app folder structure.</p><pre><code class="hljs python">myapp/    db.sqlite3    manage.py    tutorial/    migrations        __init__.py        admin.py        apps.py        models.py        tests.py        views.py    myapp/    __init__.py    asgi.py    settings.py    urls.py    wsgi.py</code></pre><p>Again nothing different to the previous Django app setup.</p><p>In your <code>settings.py</code> file, you add the following:</p><pre><code class="hljs python">INSTALLED_APPS = [....    <span class="hljs-string">&#x27;rest_framework&#x27;</span>,]</code></pre><p>Once you’re done with this initial setup, you can begin to outlay your API architecture design look like so below</p><h5 id="API-Resource-Design-Basics"><a href="#API-Resource-Design-Basics" class="headerlink" title="API Resource Design Basics"></a>API Resource Design Basics</h5><pre><code class="hljs python"><span class="hljs-comment">#### Step 1 - Serializers Layout #####</span><span class="hljs-keyword">from</span> myapp.tutorial.models <span class="hljs-keyword">import</span> Question, Choice<span class="hljs-keyword">from</span> rest_framework <span class="hljs-keyword">import</span> serializers<span class="hljs-keyword">class</span> <span class="hljs-title class_">QuestionSerializer</span>(serializers.HyperlinkedModelSerializer):    <span class="hljs-keyword">class</span> <span class="hljs-title class_">Meta</span>:        model = Question        fields = [<span class="hljs-string">&#x27;url&#x27;</span>, <span class="hljs-string">&#x27;question_text&#x27;</span>, <span class="hljs-string">&#x27;pub_date&#x27;</span>]<span class="hljs-keyword">class</span> <span class="hljs-title class_">ChoiceSerializer</span>(serializers.HyperlinkedModelSerializer):    <span class="hljs-keyword">class</span> <span class="hljs-title class_">Meta</span>:        model = Choice        fields = [<span class="hljs-string">&#x27;url&#x27;</span>, <span class="hljs-string">&#x27;question&#x27;</span>, <span class="hljs-string">&#x27;choice_text&#x27;</span>, <span class="hljs-string">&#x27;votes&#x27;</span>]<span class="hljs-comment">#### end Step 1 ####</span><span class="hljs-comment">#### Step 2 - Views Layout ####</span><span class="hljs-keyword">from</span> myapp.tutorial.models <span class="hljs-keyword">import</span> Question, Choice<span class="hljs-keyword">from</span> rest_framework <span class="hljs-keyword">import</span> viewsets<span class="hljs-keyword">from</span> rest_framework <span class="hljs-keyword">import</span> permissions<span class="hljs-keyword">from</span> myapp.tutorial.serializers <span class="hljs-keyword">import</span> QuestionSerializer, ChoiceSerializer<span class="hljs-keyword">class</span> <span class="hljs-title class_">QuestionViewSet</span>(viewsets.ModelViewSet):    <span class="hljs-string">&quot;&quot;&quot;</span><span class="hljs-string">    API endpoint that allows users to be viewed or edited.</span><span class="hljs-string">    &quot;&quot;&quot;</span>    queryset = Question.objects.<span class="hljs-built_in">all</span>().order_by(<span class="hljs-string">&#x27;pub_date&#x27;</span>)    serializer_class = QuestionSerializer    permission_classes = [permissions.IsAuthenticated]<span class="hljs-keyword">class</span> <span class="hljs-title class_">ChoiceViewSet</span>(viewsets.ModelViewSet):    <span class="hljs-string">&quot;&quot;&quot;</span><span class="hljs-string">    API endpoint that allows groups to be viewed or edited.</span><span class="hljs-string">    &quot;&quot;&quot;</span>    queryset = Choice.objects.<span class="hljs-built_in">all</span>()    serializer_class = ChoiceSerializer    permission_classes = [permissions.IsAuthenticated]<span class="hljs-comment">#### end Step 2 ####</span><span class="hljs-comment">### Step 3 - URLs setup ###</span><span class="hljs-keyword">from</span> django.urls <span class="hljs-keyword">import</span> include, path<span class="hljs-keyword">from</span> rest_framework <span class="hljs-keyword">import</span> routers<span class="hljs-keyword">from</span> myapp.tutorial <span class="hljs-keyword">import</span> viewsrouter = routers.DefaultRouter()router.register(<span class="hljs-string">r&#x27;questions&#x27;</span>, views.QuestionViewSet)router.register(<span class="hljs-string">r&#x27;choices&#x27;</span>, views.ChoiceViewSet)registered_urls = router.urls<span class="hljs-comment"># Wire up our API using automatic URL routing.</span><span class="hljs-comment"># Additionally, we include login URLs for the browsable API.</span>urlpatterns = [    path(<span class="hljs-string">&#x27;&#x27;</span>, include(registered_urls)),    path(<span class="hljs-string">&#x27;api-auth/&#x27;</span>, include(<span class="hljs-string">&#x27;rest_framework.urls&#x27;</span>, namespace=<span class="hljs-string">&#x27;rest_framework&#x27;</span>))]<span class="hljs-comment">### end Step 3 ###</span><span class="hljs-comment">### Step 4 - Unit testing ###</span><span class="hljs-keyword">from</span> rest_framework.test <span class="hljs-keyword">import</span> APIRequestFactory<span class="hljs-comment"># Using the standard RequestFactory API to create a form POST request</span>factory = APIRequestFactory()view = Question.as_view()request = factory.post(<span class="hljs-string">&#x27;/question/&#x27;</span>, &#123;<span class="hljs-string">&#x27;title&#x27;</span>: <span class="hljs-string">&#x27;new question&#x27;</span>&#125;)response = view(request, pk=<span class="hljs-string">&#x27;4&#x27;</span>)response.render()<span class="hljs-variable language_">self</span>.assertEqual(response.content, <span class="hljs-string">&quot;some_response_data&quot;</span>)<span class="hljs-comment">### end Step 4 ###</span></code></pre><p>With the above, Django REST introduces few concepts here we have to learn here.</p><ul><li>Serializers</li><li>Viewsets</li><li>Hyperlinked APIs</li><li>Routers</li><li>Unit Testing</li></ul><p>In <strong>step 1</strong> - Django’s way of serializing JSON response data against models is by injecting one of its own serializer classes, such as <code>HyperlinkedModelSerializer</code>, to the object constructor.  This will instruct the class to interpret a number of complex data types such as querysets and model instances.  Then we can access Metadata options to bind a number of fields of Model into response output as well as controlling its output structure as well.</p><p>In <strong>step 2</strong> - when designing out your API resource endpoints that will take care of the logic  of your serializable responses from above, we have to understand Viewset which is described simply as a class-based view that responsible for handling all the common HTTP actions&#x2F;verbs handlers without having you to write your own. Meaning things like  <code>get()</code>,  <code>post()</code>,  <code>put()</code>, <code>patch()</code> and <code>delete()</code>  or similar you use to write from Flask&#x2F;Falcon world are almost ‘non-existent’.   Instead, Django gives you <code>list</code>, <code>create</code>, <code>retrieve</code>, <code>update</code>, <code>partial_update</code> and <code>destroy</code>  action handlers which, to me, are nothing more like syntactic sugar  representation to their traditional counterparts.  Viewset is simply an alternate concept to Resources or Controllers domain of API designs.</p><p>Interestingly, with this concept, Django offers a few different Viewset classes implementation out of the box for you to design resource API needs.  They are:</p><p>1 ) <strong>GenericViewSet</strong> - are the views created as a shortcut for common logic code across similar views by injecting Django action mixins into the constructor thus making you not having to write extra code lines between resources.</p><ol start="2"><li><strong>ModelViewSet</strong> - are the views created to give you 6 action handlers for read-write operations into one API resource already just like I did examples above for Question and Choice Resource endpoint - with far fewer lines of code other than plumbing queryset, serializer class, and permission middleware.  That’s incredibly insane!</li></ol><p>One thing that caught my eye is that we have this other concept called Hyperlink APIs where is used to make discoverability of the APIs more meaningful by injecting entities’ URL names instead of its primary keys when describing models relationships. for eg.</p><pre><code class="hljs bash"><span class="hljs-comment"># Instead of this ...</span>http://myapp.tutorial.com/questions/1/choice<span class="hljs-comment"># We get this url generation instead</span>http://myapp.tutorial.com/question/detail-1/choice</code></pre><p>In the above, instead of fetching your primary keys from your model instances (say Question model with primary key of <code>1</code> as eg)  to associate model relationships as part of the URL definition, we can replace it with a different url name as hyperlinks.  We can do this by incorporating HyperlinkedIdentityField’s serializer class that represented as <code>view_name</code> field along with a <code>lookup_field</code> that refers to the particular instance of the model object - which in this case it would the primary key of Question table model ie <code>pk</code> or similar .<br>This is something which Flask&#x2F;Falcon doesn’t do out of the box.</p><p>You can see more example how it is exactly used here - </p><p>In <strong>step 3</strong> - once we’re happy with our API url endpoints setup, we can begin to configure our routes.  In Django,  we imported DefaultRouter which is responsible for creating the hyperlinks for you intelligently based on the HyperLinkedModel Serializer setup you defined in the previous steps.  Then we <code>register</code>  our URL endpoint routes by specifying two arguments; a prefix url name and a Viewset class.  On the surface, this looks pretty straightforward.  It’s not that much different compared to the Flask&#x2F;Falcon counterparts - other than the fact Django provides special Viewset classes that can give developers extra wiring options when determining your action handler definitions when mapping out your URL routing rules.</p><p>Finally in our <strong>step 4</strong> - we look at how the Django handles unit testing as a whole.  As we are reusing Django’s existing test framework, Django REST offers extra utility testing tools for testing API request calls specifically by using their factory classes. To create our test requests, we import <code>APIRequestFactory</code> first and then instantiate it as a <code>post</code> request factory .  Then to test our response assertions without APIRequestFactory, we need our Viewsets to be treated as views so we can inject our HTTP request factory into the <code>view</code> constructor.   That our response can be rendered so that we can perform our assertion on its content straight after. </p><p>With that, that’s pretty much it about Django!</p><h2 id="Thoughts-and-Opinions"><a href="#Thoughts-and-Opinions" class="headerlink" title="Thoughts and Opinions"></a>Thoughts and Opinions</h2><p>Having all said and one with the above, I finally come to the part where I offer my opinions between the two.</p><p>Based on my personal (as well as professional) bias, I prefer working with microframeworks to major frameworks like Django.  To me, a good framework comes with the following:</p><ul><li>Simple code comes with simple design</li><li>Easier and more powerful APIs to grasp</li><li>Simpler and powerful design patterns such IOC.&#x2F;DI you can reason with</li><li>Fewer (and cleaner) abstractions and dependencies for you to learn</li><li>Ease of plugin integration to the web application ecosystem</li><li>Freedom of architectural choice </li><li>Unit&#x2F;Integration tests are arguably more reliable to test for better accuracy and expectations feedback of the system behaviour.</li></ul><p>When reading up design philosophy like <a href="https://flask.palletsprojects.com/en/2.0.x/design/">this</a>, it totally resonates with me. That’s the type of excellent software design mantra I want to stick by.  I feel that I have more ‘’rights’ as a developer to determine how my application can be designed in a certain way. As I mentioned on architectural freedom of choice earlier, microframeworks like Flask&#x2F;Falcon give us the liberty in making design choices upfront when plumbing away all the components and have them all glued together to make a working product.  Involving design choices need to consider all the important factors such as technology choice familiarity, fewer deep-seated dependencies, cleaner abstractions as well as ease of learning barrier for any new developers coming on board when enhancing the product further down the line.  With all this considered, I would argue that you can still get away in writing (and achieving) code simplicity in design as the application grow in greater complexity over time.</p><p>With major frameworks like Django,  RAILS or similar,  I don’t have this “freedom”.  They give you all these toolsets out of the box, from ORMs, security authentication systems, form generation, resource creation, serialization frameworks etc.  <em>They’ve decided the design choices for you already</em>.   The things I learned to build from the microframework world cannot be wholly applied here.  All its own toolsets concepts like classed-based viewsets, serializers are hard to grasp for newcomers thus I already find that it’s already initial barrier for me when learning to build a simple API endpoint.  </p><p>One top of that, even if you got the API endpoint working, their abstractions are not straightforward to follow like for eg what’s the difference ‘’GenericViewSet” and “ModelViewSet”.   Are they same but serve different responsibilities under the hood?  If so, what other extra concepts do I need to come to terms with?  Does it make my job easier to design endpoint resources than I did with Flask&#x2F;Falcon or not? What major benefit do I gain in learning extra concepts vs writing simpler and easier code I do in Flask&#x2F;Falcon? More importantly, will they be easier or harder to change should the major framework developers decide to alter your understanding of these concepts in the future?  In all front, I would argue that I don’t gain any benefit in making code design simpler using these tools.  </p><p>It just made all the things harder.</p><p>While it’s harsh for me to say this, I have to be fair to them.  To their credit, the primary initial benefit of having major frameworks is that they help to save you ahead of time when building your code.  They introduced shortcuts toolchains to assist you in reducing boilerplate code so you never have to find yourself writing lines of code that are repetitive all over the place. They do this by introducing these ‘’black magic” such as class-based viewsets that gets to write DRYer lines of code and you’re well on your way in getting the product shipped out sooner than you realised.  While that’s an attractive selling point in helping you out to achieve deadlines quicker in beginning, but in the long run it will not be sustainable.  </p><p>Sooner rather than later, often, you will find yourself shooting yourself in the foot for paying the price when requirements change over time and the initial assumptions of code design not longer stay constant.  When they changed, those fewer lines of code you will have to change.  How much is the change you may ask?  More important question how big the risk of change will impact the rest of the data flow in the system?.   Not only that what about unit testing&#x2F;integration?  Now the DRY&#x2F;abstraction is no longer serving, will the rest of the application be prone to breaking further? Not so straight-forward.   Those “black magic” will come back to haunt you. 👻</p><p>I would say that using major frameworks that supposed to save you heaps  of time is more of a false economy than anything if you ask me - eventually.</p><p>With major frameworks, it’s infamously said they give you conventions over configuration.  When reading that, it means to me,  as a developer, you are not required to care too much of configuration what makes the bells and whistle of the framework.  All they want to teach you is to embrace their conventions and trust that they all do a good job and do the heavy lifting for you.  You never have to worry or be concerned about their design decisions.  Those same design decisions will come with a set of opinions people made for the community thus you will either have to embrace them truthfully or suffer the consequences should things don’t work out in your favour.</p><p>Either way,  what reveals is that major frameworks like Django (arguably) forces you to be influenced by their design philosophy around these and stick to them to the teeth while Falcon&#x2F;Flask are the complete opposite.  For that reason, they’re  making you feel real ‘’lazy” to write any good code thus you will never appreciate why writing good and simple code truly matters when designing your architecture in the direction you want.</p><p>All in all, I just want to say I’m not giving major frameworks out there a bad name.</p><p>These are just my opinions, and my opinions only.</p><p>I come from a different school of thought when building high-end solutions architecture of the application in various team sizes where I’m fortunate to see good simple and clean code architecture design is laid out. With any frameworks, they come with pros and cons of design.  Django helps to saves you time upfront by eliminating boilerplate designs as a pro; but falls short when if you want to scale design differently compared to its built-in conventions later on.  While Flask&#x2F;Falcon may not have all the fancy ‘black magic’ toolsets Django comes, but the major pro for me is that the learning curve behind these are much simpler to work and encourages you to care deeply of design thinking while still keeping good clean and simple-to-understand code as the main goal.  </p><p>I always firmly believe that every problem domain we aim to solve out there needs the right tool for the right job.   If you think Django&#x2F;RAILS&#x2F;Laravel or similar helps to solve 80-90% of the problems and you don’t mind dealing with the conventions it comes with, then thats the correct tool of choice you should make.  But if the same framework only helps to achieve 50% or under of the problem and their conventions doesn’t serve you any useful purpose at all, then Flask&#x2F;Falcon or similar will be your guy to handle these.</p><p>Just remember that every framework designs comes with certain tradeoffs to build a good scalable application.  </p><p>For me, I would go for simple clean code architecture. It will trump all those ‘black magic’  anytime any day.</p><p>That’s my two cents.</p><p>Till next time, Happy Coding!</p><p><em>Disclaimer:  I’m not framework expert by any chance.  I’m just making a high-level comparison between these and drawing my ow conclusions on how they are good or not so good for anything when building good architectural apps. - with a bit of dash of personal bias here ;)</em></p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_django_flask_falcon.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;It’s been some months since 2021 has begun (and we’re not where near safe coming out of the pandemic woods just yet ). I’ve been itching to come and sit down to write my first blog post for the year.&lt;/p&gt;
&lt;p&gt;What better way to start out writing ins-and-outs on certain Python web frameworks.&lt;/p&gt;
&lt;img src=&quot;/images/django_vs_flask_falcon_cropped.png&quot; class=&quot;&quot; title=&quot;Django vs Flask-Falcon Django vs Flask-Falcon&quot;&gt;

&lt;p&gt;I’ve been working on Python web projects for some time and I’m here to offer my rants&amp;#x2F;thoughts when working between Django and Flask&amp;#x2F;Falcon and outline the comparisons between these two.&lt;/p&gt;
&lt;p&gt;Let’s start with the ones I’m most familiar with.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;NB - Before I begin, this article assumes the reader either has a good understanding or good working knowledge of MVC software patterns as more web frameworks are built around this pattern.  If you don’t know what that is, you may want to visit this &lt;a href=&quot;https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller&quot;&gt;Wikipedia&lt;/a&gt; page as a refresher course, before proceed reading.&lt;/em&gt;&lt;/p&gt;</summary>
    
    
    
    
    <category term="frameworks" scheme="http://awongcm.io/tags/frameworks/"/>
    
    <category term="python" scheme="http://awongcm.io/tags/python/"/>
    
    <category term="django" scheme="http://awongcm.io/tags/django/"/>
    
    <category term="flask" scheme="http://awongcm.io/tags/flask/"/>
    
    <category term="falcon" scheme="http://awongcm.io/tags/falcon/"/>
    
    <category term="architecture" scheme="http://awongcm.io/tags/architecture/"/>
    
  </entry>
  
  <entry>
    <title>Looking back the year of 2020...</title>
    <link href="http://awongcm.io/blog/2020/11/13/looking-back-the-year-of-2020-dot-dot-dot/"/>
    <id>http://awongcm.io/blog/2020/11/13/looking-back-the-year-of-2020-dot-dot-dot/</id>
    <published>2020-11-13T10:55:02.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_looking_back_2020_year_covid.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><pre><code class="hljs javascript"><span class="hljs-comment">// when the giant red button got triggered - 🚨🚨</span><span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">globalWHOAlertSystem</span>(<span class="hljs-params">pandemicAlertLevel, globalToiletPaperSupplyChain</span>) &#123;<span class="hljs-keyword">let</span> message = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">getElementById</span>(<span class="hljs-string">&quot;message&quot;</span>);    message.<span class="hljs-property">innerHTML</span> = <span class="hljs-string">&quot;&quot;</span>;<span class="hljs-keyword">try</span> &#123;<span class="hljs-keyword">do</span> &#123;<span class="hljs-keyword">const</span> areWeDoomed = <span class="hljs-keyword">await</span> globalToiletPaperSupplyChain.<span class="hljs-title function_">verifyStockLevel</span>()<span class="hljs-keyword">if</span> (areWeDoomed) <span class="hljs-keyword">throw</span> <span class="hljs-string">&quot;We are so screwed😱!!&quot;</span>&#125;<span class="hljs-keyword">while</span> (pandemicAlertLevel &gt; <span class="hljs-number">60000</span>); <span class="hljs-comment">// it&#x27;s over 60,000!</span>&#125; <span class="hljs-keyword">catch</span> (err) &#123;message.<span class="hljs-property">innerHTML</span> = err;    &#125;&#125;</code></pre><p>Well, the code snippet above pretty much summarises everything we all knew what the year of 2020 has brought us.</p><p>All small jokes aside, this year has been an incredibly difficult year for all, especially for the tech community in general.</p><p>Countless of tech meetups, tech networking events and tech conferences have either been postponed indefinitely, rescheduled or cancelled altogether around the world.</p><span id="more"></span><p>Thus everything from JS, Python, Docker, AWS, Blockchain, and UI&#x2F;UX meetups were a huge miss for me this year as part of my life-long passions for technology used in business.</p><p>But regardless of how the pandemic turns out, eventually it will be resolved or contained by health community experts trying their very best to combat its deadly virulence from spreading further.  </p><p>Just like the past pandemics before us.</p><p>They come and go.</p><p>And technology realms will always keep moving forth regardless of the nature pandemic comes with..</p><p>Technology trends will always run in constant flow..  </p><p>That we all can bet on.</p><p>However, sadly, not all things can join and follow the same flow.</p><p>Many people on this planet have lost their loved ones during this pandemic; whether they actually contracted the virus or not.</p><p>On a personal level, I lost someone that is very close to me.</p><p>My father recently passed away due to a heavy stroke he fell early this year.</p><p>The hardest part of this pandemic is it’s forced me not to be with my father physically one more time in his presence due to the international border lockdowns rules.  And my family in NZ had little or choice not to wait for the 2 week period for me to arrive at the Auckland aiport airport to attend his funeral.</p><p>With that, I was left with no choice but to live with the rest of my life that I would never get to utter my words of goodbye to my father face-to-face before he was sent for the ashes.</p><p>I had to use virtual experiences via Whatsapp video calls instead to see his lifeless body one last time…</p><p>There are no simple words to make this whole thing better.</p><p>In the beginning, I want to lead my dev life of 2020 to come out with a great bang.</p><p>But I feared this is not the case.</p><p>Can we still be happy coder moving forth in the midst all of this? Will I be able to continue writing blogs post in the near future?</p><p>Naturally, it will.</p><p>When all things passed and the grief subsides..</p><p>All in all, it’s been a trying year for everyone.</p><p>Final message -  look after yourself and your love ones very closely.  </p><p>Appreciate their presence with all your love and care for they do want the best for you in all aspects of life.  You’ll never know once they’re gone from your life..</p><p>Till then, here’s to many years of future Happy Coding days ahead.</p><p>God Bless.</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_looking_back_2020_year_covid.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;hljs javascript&quot;&gt;&lt;span class=&quot;hljs-comment&quot;&gt;// when the giant red button got triggered - 🚨🚨&lt;/span&gt;
&lt;span class=&quot;hljs-keyword&quot;&gt;async&lt;/span&gt; &lt;span class=&quot;hljs-keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;hljs-title function_&quot;&gt;globalWHOAlertSystem&lt;/span&gt;(&lt;span class=&quot;hljs-params&quot;&gt;pandemicAlertLevel, globalToiletPaperSupplyChain&lt;/span&gt;) &amp;#123;
	&lt;span class=&quot;hljs-keyword&quot;&gt;let&lt;/span&gt; message = &lt;span class=&quot;hljs-variable language_&quot;&gt;document&lt;/span&gt;.&lt;span class=&quot;hljs-title function_&quot;&gt;getElementById&lt;/span&gt;(&lt;span class=&quot;hljs-string&quot;&gt;&amp;quot;message&amp;quot;&lt;/span&gt;);
    message.&lt;span class=&quot;hljs-property&quot;&gt;innerHTML&lt;/span&gt; = &lt;span class=&quot;hljs-string&quot;&gt;&amp;quot;&amp;quot;&lt;/span&gt;;
	&lt;span class=&quot;hljs-keyword&quot;&gt;try&lt;/span&gt; &amp;#123;
		&lt;span class=&quot;hljs-keyword&quot;&gt;do&lt;/span&gt; &amp;#123;
			&lt;span class=&quot;hljs-keyword&quot;&gt;const&lt;/span&gt; areWeDoomed = &lt;span class=&quot;hljs-keyword&quot;&gt;await&lt;/span&gt; globalToiletPaperSupplyChain.&lt;span class=&quot;hljs-title function_&quot;&gt;verifyStockLevel&lt;/span&gt;()
			&lt;span class=&quot;hljs-keyword&quot;&gt;if&lt;/span&gt; (areWeDoomed) &lt;span class=&quot;hljs-keyword&quot;&gt;throw&lt;/span&gt; &lt;span class=&quot;hljs-string&quot;&gt;&amp;quot;We are so screwed😱!!&amp;quot;&lt;/span&gt;
		&amp;#125;
		&lt;span class=&quot;hljs-keyword&quot;&gt;while&lt;/span&gt; (pandemicAlertLevel &amp;gt; &lt;span class=&quot;hljs-number&quot;&gt;60000&lt;/span&gt;); &lt;span class=&quot;hljs-comment&quot;&gt;// it&amp;#x27;s over 60,000!&lt;/span&gt;
	&amp;#125; &lt;span class=&quot;hljs-keyword&quot;&gt;catch&lt;/span&gt; (err) &amp;#123;
		message.&lt;span class=&quot;hljs-property&quot;&gt;innerHTML&lt;/span&gt; = err;
    &amp;#125;	
&amp;#125;&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Well, the code snippet above pretty much summarises everything we all knew what the year of 2020 has brought us.&lt;/p&gt;
&lt;p&gt;All small jokes aside, this year has been an incredibly difficult year for all, especially for the tech community in general.&lt;/p&gt;
&lt;p&gt;Countless of tech meetups, tech networking events and tech conferences have either been postponed indefinitely, rescheduled or cancelled altogether around the world.&lt;/p&gt;</summary>
    
    
    
    
    <category term="javascript" scheme="http://awongcm.io/tags/javascript/"/>
    
    <category term="inspiration" scheme="http://awongcm.io/tags/inspiration/"/>
    
    <category term="life" scheme="http://awongcm.io/tags/life/"/>
    
    <category term="happy" scheme="http://awongcm.io/tags/happy/"/>
    
    <category term="coding" scheme="http://awongcm.io/tags/coding/"/>
    
  </entry>
  
  <entry>
    <title>Filter, Map and Reduce functions - the Python way</title>
    <link href="http://awongcm.io/blog/2020/04/29/filter-map-and-reduce-functions-the-python-way/"/>
    <id>http://awongcm.io/blog/2020/04/29/filter-map-and-reduce-functions-the-python-way/</id>
    <published>2020-04-29T03:01:12.000Z</published>
    <updated>2026-03-30T12:13:51.446Z</updated>
    
    <content type="html"><![CDATA[<img src="/images/ai_generated_filter_map_reduce_python_way.jpeg" class="center" width="500" height="500" title="Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator"><p><em><p style="text-align: center;">Generated AI image by Microsoft Bing Image Creator</p></em></p><p>With every new tool, framework or methodology comes along, developers with their insatiable appetite or thirst for knowledge and power, they will find ways to make dedicated time to learn how they work and how they’re planning to use them as part of the day to day job.</p><p>And functional programming (FP) - a newish paradigm has been permeating through the scenes of developer community for some time; everything from Haskell, Elixir, React, AWS Lambdas to Clojure etc.</p><p>Or, at least it’s yet to make establish some <a href="https://www.youtube.com/watch?v=QyJZzq0v7Z4">norms</a> within the community…</p><p>But I must digress.</p><span id="more"></span><img src="https://media.giphy.com/media/X8ynfaP1ZwTvTbxH1b/giphy.gif" class="center" width="500" height="500" title="Let us jump right in. Let us jump right in."><h3 id="Some-Refresher-Points"><a href="#Some-Refresher-Points" class="headerlink" title="Some Refresher Points"></a>Some Refresher Points</h3><p>After dabbling around with Javascript&#x2F;React for a while now, every JS developer would be inclined to tell you using <code>map</code>, <code>filter</code> and <code>reduce</code> are the default go-to tools for expressing their FP-ness in all over their front-end codebase.   </p><p>You’ve probably seen those patterns numerous of times via google searches or countless Medium or FreeCodeCamp tutorial blogpost or what not.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Typical start of your FPer day</span><span class="hljs-keyword">const</span> stayAtHomePersons = [  &#123;    <span class="hljs-attr">id</span>: <span class="hljs-number">1</span>,    <span class="hljs-attr">name</span>: <span class="hljs-string">&#x27;Josh Hamish&#x27;</span>,    <span class="hljs-attr">cookingScore</span>: <span class="hljs-number">90</span>,    <span class="hljs-attr">exerciseScore</span>: <span class="hljs-number">5</span>,    <span class="hljs-attr">isSelfIsolating</span>: <span class="hljs-literal">true</span>  &#125;,  &#123;    <span class="hljs-attr">id</span>: <span class="hljs-number">2</span>,    <span class="hljs-attr">name</span>: <span class="hljs-string">&#x27;Blake Lively&#x27;</span>,    <span class="hljs-attr">cookingScore</span>: <span class="hljs-number">10</span>,    <span class="hljs-attr">exerciseScore</span>: <span class="hljs-number">80</span>,    <span class="hljs-attr">isSelfIsolating</span>: <span class="hljs-literal">true</span>  &#125;,  &#123;    <span class="hljs-attr">id</span>: <span class="hljs-number">3</span>,    <span class="hljs-attr">name</span>: <span class="hljs-string">&#x27;Ken Jeong&#x27;</span>,    <span class="hljs-attr">cookingScore</span>: <span class="hljs-number">0</span>,    <span class="hljs-attr">exerciseScore</span>: <span class="hljs-number">90</span>,    <span class="hljs-attr">isSelfIsolating</span>: <span class="hljs-literal">false</span>  &#125;];<span class="hljs-comment">// No new surprises here.</span><span class="hljs-keyword">const</span> totalSAHPersonScore = stayAtHomePersons  .<span class="hljs-title function_">filter</span>(<span class="hljs-function"><span class="hljs-params">person</span> =&gt;</span> person.<span class="hljs-property">isSelfIsolating</span>)  .<span class="hljs-title function_">map</span>(<span class="hljs-function"><span class="hljs-params">person</span> =&gt;</span> person.<span class="hljs-property">cookingScore</span> + person.<span class="hljs-property">exerciseScore</span>)  .<span class="hljs-title function_">reduce</span>(<span class="hljs-function">(<span class="hljs-params">acc, score</span>) =&gt;</span> acc + score, <span class="hljs-number">0</span>);</code></pre><p>In the JS world, this is how we roll our F.M.Rs mojos.  👩‍💻👨‍💻😎</p><p>Now,  let’s head the Python fence, and see how they’re handle things over there.</p><p><strong>Disclaimer</strong>: <em>Before I proceed further, I just want to say, beforehand, at the time of this writing, as as the entire online community is facing a major unprecedented challenge in our lifetimes and we want all doing the best to treat things with great caution, the examples I’ll be using next may cause upset to the readers and I do not mean to cause any grave amount of discontentment amongst.   The purpose of my blog is to purely demonstrate my learnings over the past few weeks since start of the global pandemic situation, and wish to reiterate my platform will be used for this purpose only.  I wish to apologise in advance.</em></p><h3 id="Beginning-steps"><a href="#Beginning-steps" class="headerlink" title="Beginning steps"></a>Beginning steps</h3><p>Lately, I have been dabbling on how Python does well with its own <code>filter</code>, <code>map</code> and <code>reduce</code> functions approach, and I couldn’t help to look at the following datasets provided from the Dr John Hopkins University’s Github <a href="https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_daily_reports">repo</a> on latest covid19 cases.</p><pre><code class="hljs json"><span class="hljs-comment">/** Daily Covid 19 Cases - NOT ACTUAL DATA **/</span><span class="hljs-punctuation">[</span>    <span class="hljs-punctuation">&#123;</span>        <span class="hljs-attr">&quot;Province/State&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;Hubei&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Country/Region&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;Mainland China&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Last Update&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;2020-03-01T10:13:19&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Confirmed&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;1000&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Deaths&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;10&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Recovered&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;50&quot;</span><span class="hljs-punctuation">,</span>        ...        ...    <span class="hljs-punctuation">&#125;</span><span class="hljs-punctuation">,</span>    <span class="hljs-punctuation">&#123;</span>        <span class="hljs-attr">&quot;Province/State&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Country/Region&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;South Korea&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Last Update&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;2020-03-01T23:43:03&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Confirmed&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;100&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Deaths&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;1&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Recovered&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;2&quot;</span><span class="hljs-punctuation">,</span>        ...        ...    <span class="hljs-punctuation">&#125;</span><span class="hljs-punctuation">,</span>    <span class="hljs-punctuation">&#123;</span>        <span class="hljs-attr">&quot;Province/State&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Country/Region&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;Italy&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Last Update&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;2020-03-01T23:23:02&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Confirmed&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;20&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Deaths&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;2&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Recovered&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;1&quot;</span><span class="hljs-punctuation">,</span>        ...        ...    <span class="hljs-punctuation">&#125;</span><span class="hljs-punctuation">,</span>    <span class="hljs-punctuation">&#123;</span>        <span class="hljs-attr">&quot;Province/State&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;Guangdong&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Country/Region&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;Mainland China&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Last Update&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;2020-03-01T14:13:18&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Confirmed&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;20&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Deaths&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;3&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Recovered&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;10&quot;</span><span class="hljs-punctuation">,</span>        ...        ...    <span class="hljs-punctuation">&#125;</span><span class="hljs-punctuation">,</span>    <span class="hljs-comment">/* and the rest....*/</span><span class="hljs-punctuation">]</span></code></pre><p>Their data were originally in CSV formatter so I wrote my own little Python script that feeds on these CSV files and have them fully JSONified and from here, I can see immediate pattern going on here.</p><p>It reveals that a lot daily cases are collected from every state and provinces in each country and are individually reported for number of cases that are categorised as either confirmed, recovered and death cases.  With this, my immediate thought is this would be a good case to place Python FP’s exercise into good use.</p><p>To start off, I quickly whipped up my own mini API app using Flask&#x2F;Falcon - (which doesn’t really matter to be honest)</p><pre><code class="hljs python"><span class="hljs-keyword">import</span> falcon<span class="hljs-string">&#x27;&#x27;&#x27;</span><span class="hljs-string">Start a Flask/Falcon app starting with a resource endpoint:</span><span class="hljs-string">&#x27;&#x27;&#x27;</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">DailyCovid19Resource</span>(<span class="hljs-title class_ inherited__">object</span>):    <span class="hljs-string">&#x27;&#x27;&#x27;</span><span class="hljs-string">    Pick one of the covid19 daily JSON resources as a date format</span><span class="hljs-string">    &#x27;&#x27;&#x27;</span>    <span class="hljs-keyword">def</span> <span class="hljs-title function_">on_get</span>(<span class="hljs-params">self, req, resp, file_id</span>):        <span class="hljs-keyword">try</span>:            format_matched = re.<span class="hljs-keyword">match</span>(                <span class="hljs-string">&quot;((0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])-[12]\d&#123;3&#125;)&quot;</span>, file_id)  <span class="hljs-comment"># mm-dd-yyyy format</span>            <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> format_matched:                <span class="hljs-keyword">raise</span> ValueError(<span class="hljs-string">&#x27;wrong file_id pattern&#x27;</span>)            data = fetch_json_data(file_id)            country_list = []            <span class="hljs-keyword">for</span> dic <span class="hljs-keyword">in</span> data:                <span class="hljs-keyword">if</span> <span class="hljs-string">&#x27;Country_Region&#x27;</span> <span class="hljs-keyword">in</span> dic:                    country_list.append(dic[<span class="hljs-string">&#x27;Country_Region&#x27;</span>])                <span class="hljs-keyword">elif</span> <span class="hljs-string">&#x27;Country/Region&#x27;</span> <span class="hljs-keyword">in</span> dic:                    country_list.append(dic[<span class="hljs-string">&#x27;Country/Region&#x27;</span>])            unique_countries = <span class="hljs-built_in">list</span>(<span class="hljs-built_in">set</span>(country_list)            new_data = <span class="hljs-built_in">list</span>(<span class="hljs-built_in">map</span>(get_covid19_schema, data)) <span class="hljs-comment"># -&gt; this is where it&#x27;s starting to get interesting</span>      ......      ......api = application = falcon.API()daily_covid19 = DailyCovid19Resource()api.add_route(<span class="hljs-string">&#x27;/covid19/&#123;file_id&#125;&#x27;</span>, daily_covid19)</code></pre><p>In this example, I made my resource endpoint <code>covid19</code> and it expects <code>file_id</code> as  the main parameter_id used to query a specific json file to be fetched from the server.  </p><p>In order to make the right (and exact) search for the specific covid19 json file, I decided to add some a bit regex expression here just to make sure <code>file_id</code> matches the date format my json files are named ie <code>mm-dd-yyyy</code>.  That way I will have Python exception handler to capture the error should the <code>file_id</code> failed to meet the regex pattern matching requirement, raised the ValueError as that will halt entire get resource API operation. </p><p>Once the file_id matching completes, then we can start making the <code>fetch_json_data</code><br> call (as below)  and fetches correct json file off from the server,</p><pre><code class="hljs python"><span class="hljs-comment"># fetch json data file and serialised as json format</span><span class="hljs-keyword">def</span> <span class="hljs-title function_">fetch_json_data</span>(<span class="hljs-params">file_id</span>):    source_file = <span class="hljs-string">&#x27;./some_folder/&#123;&#125;.json&#x27;</span>.<span class="hljs-built_in">format</span>(file_id)    <span class="hljs-keyword">with</span> <span class="hljs-built_in">open</span>(source_file) <span class="hljs-keyword">as</span> covid19_json_file:        data = json.load(covid19_json_file)        <span class="hljs-keyword">return</span> data</code></pre><pre><code class="hljs python"><span class="hljs-comment"># list of unique countries</span>unique_countries = <span class="hljs-built_in">list</span>(<span class="hljs-built_in">set</span>(country_list))</code></pre><p>Once the data is fetched upon returning from the <code>fetch_json_data</code> call, I start building out my list of unique countries by using List and Set combo. With this new list, it will be used later which will be traversing the list of countries data of covid19 cases that matches with the dict’s key <code>Country_Region</code> or <code>Country/Region</code> .  It has two types of keys because John Hopkins University dataset had revised the data set representations in between the early cases of Coronavirus in January 2020 and Mar 2020.</p><p>Beforehand, I need to determine covid19 schema dictionary I want to extract from each item in the array using <code>get_covid19_schema</code> like so.</p><pre><code class="hljs python"><span class="hljs-comment"># Map functions to perform against list to collect row&#x27;s properties we want</span>new_data = <span class="hljs-built_in">list</span>(<span class="hljs-built_in">map</span>(get_covid19_schema, data))............<span class="hljs-keyword">def</span> <span class="hljs-title function_">get_covid19_schema</span>(<span class="hljs-params">item_dict</span>):    schema_keys = [<span class="hljs-string">&#x27;Country_Region&#x27;</span>, <span class="hljs-string">&#x27;Country/Region&#x27;</span>,                   <span class="hljs-string">&#x27;Confirmed&#x27;</span>, <span class="hljs-string">&#x27;Deaths&#x27;</span>, <span class="hljs-string">&#x27;Recovered&#x27;</span>]    mapped_dict = <span class="hljs-built_in">dict</span>((k, item_dict[k])                       <span class="hljs-keyword">for</span> k <span class="hljs-keyword">in</span> schema_keys <span class="hljs-keyword">if</span> k <span class="hljs-keyword">in</span> item_dict)    converted_dict = convert_values_to_int(mapped_dict)    <span class="hljs-keyword">return</span> converted_dict<span class="hljs-comment"># as some of the figures don&#x27;t have numerical values</span><span class="hljs-keyword">def</span> <span class="hljs-title function_">convert_values_to_int</span>(<span class="hljs-params">item_dict</span>):    keys = [<span class="hljs-string">&#x27;Confirmed&#x27;</span>, <span class="hljs-string">&#x27;Deaths&#x27;</span>, <span class="hljs-string">&#x27;Recovered&#x27;</span>]    <span class="hljs-keyword">for</span> k <span class="hljs-keyword">in</span> keys:        <span class="hljs-comment"># fix data that has empty string</span>        item_dict[k] = <span class="hljs-built_in">int</span>(item_dict[k]) <span class="hljs-keyword">if</span> item_dict[k] != <span class="hljs-string">&quot;&quot;</span> <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>    <span class="hljs-keyword">return</span> item_dict</code></pre><p>See how I’m using <code>map</code>  function by passing both a function and an array iterator as parameters?  This will tell the map operator to perform such function on each iteratee of the array iterator and return the consequence of the actions as a result.  According to its API docs,  the <code>map</code> output signature comes as a Map object.  Meaning that it’s only a proxy result and we need intermediary function call that will be responsible for converting it into actual list of each element that has been applied either a list, set or tuple.  Which in this case, it’s going to be a list of my dictionary results.</p><p>This is fascinating as coming from the JS world, we don’t need intermediary function calls when we perform <code>map</code> functions as JS itself does not come with complex data structures (ES6 Sets anyone 🤔?) that Python has.  Arrays are the most basic and the most popular structure that JS Developers used by default so it’s easy to see why we didn’t need to worry getting proxying result set that Python has to come grips with… 😶</p><p>Later on, now that we got the list of <code>new_data</code> of countries that comes with different state&#x2F;provinces that has different covid19 cases data, the next job is to make an aggregate of the total number of covid19 cases for each category from each unique country, which I get from my <code>unique_countries</code> list.  Each of this country’s aggregate numbers take into this form.</p><pre><code class="hljs json"><span class="hljs-punctuation">&#123;</span>        <span class="hljs-attr">&quot;CountryRegion&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;Some Country&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Confirmed&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;total x Numbers&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Deaths&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;total y Numbers&quot;</span><span class="hljs-punctuation">,</span>        <span class="hljs-attr">&quot;Recovered&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-string">&quot;total z Numbers&quot;</span><span class="hljs-punctuation">,</span><span class="hljs-punctuation">&#125;</span></code></pre><p>Then, I put them altogether into one big list as a payload response to the client side.</p><p>To achieve this goal, I thought ahead of using <code>filter</code> and <code>reduce</code> for this…</p><p>And this is what I came up with.</p><pre><code class="hljs python"><span class="hljs-keyword">from</span> functools <span class="hljs-keyword">import</span> reducenew_data_with_sums = []<span class="hljs-keyword">for</span> uc <span class="hljs-keyword">in</span> unique_countries:    country_total_dict = reduce(        (<span class="hljs-keyword">lambda</span> x, y: sum_up_totals_by(uc, x, y)), <span class="hljs-built_in">list</span>(            <span class="hljs-built_in">filter</span>(<span class="hljs-keyword">lambda</span> x: filter_by_country(uc, x), new_data)))    new_data_with_sums.append(country_total_dict)</code></pre><p>There’s a lot of happening here - which I’ll explain.</p><ol><li>I start looping through each country <code>uc</code> from a list of unique countries</li><li>For each country I pick, I want to filter the <code>new_data</code> list of countries cases based on the selected country and their individual covid19 cases. With each returned <code>uc</code> cases, I needed to convert into a list.  This is because in the data set, there are some countries which have several number of provinces&#x2F;states (ie China, Russia or USA) where their total number of cases are defined by the geographical distinction such as this <a href="https://github.com/CSSEGISandData/COVID-19/blob/master/csse_covid_19_data/csse_covid_19_daily_reports/01-31-2020.csv">example</a>.</li><li>Because of this, I need to be aggregating these same covid19 cases from each province&#x2F;state of the same country to get the complete total number of cases for the country.</li><li>Once the total cases for that country is found, I will add it to the <code>new_data_with_sums</code> list.</li><li>Go back step 1 and repeat the process all over again until you reach the end of the <code>unique_countries</code> list.</li></ol><pre><code class="hljs python"><span class="hljs-comment"># This is Step 2 operation</span>filtered_data_by_uc = <span class="hljs-built_in">list</span>(<span class="hljs-built_in">filter</span>(<span class="hljs-keyword">lambda</span> x: filter_by_country(uc, x), new_data)</code></pre><p>The above snippet says with <code>new_data</code>list , I apply <code>filter</code> function by passing <code>filter_by_country</code> callback that is responsible for filtering out countries in the same list,  To filter the data correctly, I rely on the closures upon <code>uc</code>  (which is gathered from <code>unique_countries</code> looping call earlier and its current element (x) that’s being interated upon.</p><p>This is equivalent to JS snippet below.</p><pre><code class="hljs javascript"><span class="hljs-keyword">const</span> filteredList = new_data.<span class="hljs-title function_">filter</span>(<span class="hljs-title function_">filterByCountry</span>(uc));</code></pre><p>Notice they’re conceptually the same, but the syntax between the two clearly differs. <code>lambda</code> keyword is  synonymous  to  anonymous functions for Javascript.  Both of them make use of closures thus both have something in common.  </p><p>And notice we put <code>list</code> as a wrapper around the <code>filter</code> function?  Apparently, <code>filter</code> also returns a Map object, just like <code>map</code> example earlier. You can return in any type of data structure you want as well.</p><p>Here’s the <code>filter_by_country</code> implementation.</p><pre><code class="hljs python"><span class="hljs-keyword">def</span> <span class="hljs-title function_">filter_by_country</span>(<span class="hljs-params">uc, item_dict</span>):    country_keys = [<span class="hljs-string">&#x27;Country_Region&#x27;</span>, <span class="hljs-string">&#x27;Country/Region&#x27;</span>]    <span class="hljs-keyword">for</span> k <span class="hljs-keyword">in</span> country_keys:        <span class="hljs-keyword">if</span> k <span class="hljs-keyword">in</span> item_dict <span class="hljs-keyword">and</span> item_dict[k] == uc:            <span class="hljs-keyword">return</span> item_dict</code></pre><p>With that mind, we take <code>filtered_data_by_uc</code> and pipe that into our next action step</p><pre><code class="hljs python"><span class="hljs-comment"># Step 3</span><span class="hljs-keyword">from</span> functools <span class="hljs-keyword">import</span> reducecountry_total_dict = reduce(        (<span class="hljs-keyword">lambda</span> x, y: sum_up_totals_by(uc, x, y)), filtered_data_by_uc))</code></pre><p>Here, we use <code>reduce</code> method  to take in the <code>filtered_data_by_uc</code> list and take each country’s number of covid19 cases to aggregrate them (using <code>sum_up_totals</code> callback) in get the total sums of confirmed, recovered and death cases, producing into a standalone dict object for that particular country.</p><p>Here’s the implementation for <code>sum_up_totals</code></p><pre><code class="hljs python"><span class="hljs-keyword">def</span> <span class="hljs-title function_">sum_up_totals_by</span>(<span class="hljs-params">uc, x, y</span>):    keys = [<span class="hljs-string">&#x27;Confirmed&#x27;</span>, <span class="hljs-string">&#x27;Deaths&#x27;</span>, <span class="hljs-string">&#x27;Recovered&#x27;</span>]    result = &#123;<span class="hljs-string">&#x27;Country_Region&#x27;</span>: uc&#125;    <span class="hljs-keyword">for</span> k <span class="hljs-keyword">in</span> keys:        result[k] = x[k] + y[k]    <span class="hljs-keyword">return</span> result</code></pre><p>Again, the  <code>reduce</code> function signature is pretty much the same with <code>filter</code> and <code>map</code> counterparts ie using lambdas, closures etc - with the minor difference is that it doesn’t return a list or any similar iterable object, but rather as a single value (or an accumulator if you like) after applying the lambda function as above.</p><p>This comes off as expected as any FP developers from Haskell, Scala, Erlang, Clojure communities etc would tell you that’s how they live and breathe writing this kind of code - like a boss 😎.</p><img src="https://media.giphy.com/media/Ie482rXFNBUbnnGxpg/giphy.gif" class="center" width="500" height="500" title="FP coders - Like A Boss. FP Coders - Like A Boss"><p>But with <code>reduce</code> function, you can also return an iterable object as an accumulator result as well.</p><p>Which let me to think I didn’t need the for-loop to append individual unique countries’ aggregate results into <code>new_data_with_sums</code>.</p><p>I could simply rewrite to this.</p><pre><code class="hljs python"><span class="hljs-keyword">from</span> functools <span class="hljs-keyword">import</span> reducenew_data_with_sums = []<span class="hljs-comment"># notice the third argument is introduced to this reducer function</span>new_data_with_sums = reduce(<span class="hljs-keyword">lambda</span> acc, current: sum_up_totals_each_country(acc, current, new_data),                        unique_countries, new_data_with_sums)</code></pre><p>With the introduction of new <code>sum_up_totals_each_country</code> lambda function, I’ve basically moved <code>filter_by_country</code>  filtering function inside that function where (by using closures and callbacks) it can gain access to the list of <code>unique_countries</code> and the raw <code>new_data</code> to performing the list filtering from there.</p><p>Once the filtering is complete, using the <code>acc</code> which is a proxy to <code>new_data_with_sums</code>,  I start to traverse the same filtered list to accumulate the total cases results for a country that has multiple states and provinces and have it appended to the <code>acc</code></p><p>Here’s the implementation of <code>sum_up_totals_each_country</code>.</p><pre><code class="hljs python"><span class="hljs-keyword">def</span> <span class="hljs-title function_">sum_up_totals_each_country</span>(<span class="hljs-params">acc, current, new_data</span>):    keys = [<span class="hljs-string">&#x27;Confirmed&#x27;</span>, <span class="hljs-string">&#x27;Deaths&#x27;</span>, <span class="hljs-string">&#x27;Recovered&#x27;</span>]    <span class="hljs-comment"># the same list filtering as before</span>    each_country_covid19_list = <span class="hljs-built_in">list</span>(        <span class="hljs-built_in">filter</span>(<span class="hljs-keyword">lambda</span> x: filter_by_country(current, x), new_data))    <span class="hljs-comment"># some countries have multiple states and provinces of data hence for this nested looping</span>    country_case_totals = &#123;&#125;    <span class="hljs-keyword">for</span> each_country_covid19 <span class="hljs-keyword">in</span> each_country_covid19_list:        <span class="hljs-keyword">for</span> k <span class="hljs-keyword">in</span> keys:            temp = each_country_covid19[k]            <span class="hljs-keyword">if</span> k <span class="hljs-keyword">in</span> country_case_totals:                country_case_totals[k] = temp + country_case_totals[k]            <span class="hljs-keyword">else</span>:                country_case_totals[k] = temp    <span class="hljs-comment"># fixed property name as I don&#x27;t need another alias: &#x27;Country/Region&#x27; to be return in the response payload</span>    country_case_totals[<span class="hljs-string">&#x27;Country_Region&#x27;</span>] = current    acc.append(country_case_totals)    <span class="hljs-keyword">return</span> acc</code></pre><p>And the rest is history.</p><h3 id="Concluding-Thoughts"><a href="#Concluding-Thoughts" class="headerlink" title="Concluding Thoughts"></a>Concluding Thoughts</h3><p>Notice how the Python’s very own built-in functional tools (<code>map</code>, <code>filter</code> and <code>reduce</code> ) are slightly different to Javascript counterpart, at least not for the conceptual level but on the syntactical level.  </p><p>Though it is said both Javascript and Python supposedly treat functions as first class citizens (unlike Java where everything is 100% OOD - boo!) yet I found the thing that stands out the most different between the two is that Javascript support <a href="https://en.wikipedia.org/wiki/Method_chaining">method chaining</a> for iterables, right out of the box.  </p><p>Python doesn’t do that by design for its own set of iterables when doing functional programming.  Thus I always wondered why I had to write the code above by wrapping functions one on top of another… which lead me to have the fact that Python’s design philosophy was driven in an imperative-style way for a long time.  It’s never meant to go to the FP route,   so says this <a href="https://python-history.blogspot.com/2009/04/origins-of-pythons-functional-features.html">blog post</a> from the original Python creator himself.  Upon reading these, that may explain the deep culture of imperative style coding Pythonistas have been doing for a very long time using tools like itertools and list comprehensions for more efficient data looping in place.</p><p>In spite of that, it’s still evident we are given the same FP toolset to accomplish here vs I have been doing on the JS&#x2F;React front for some considerable. IMHO, given how multi-paradigm languages evolve and influence each other (just like how Python influenced Javascript to have Pythonic <a href="https://dev.to/massa142/javascript-is-almost-pythonic-3f8">flavours</a>) and now that Python 2 is sunset already at the beginning of 2020, who knows where Python 3 is going to move forward from here.  I strongly believe you can still teach an old dog some new tricks. If Javascript can learn to be Pythonista by day, I don’t see why Python can’t come out of its comfort shell to learn new toolset from other developers community that addresses some of its limitations for modern problems.</p><p>I’m pretty much excited of what lies ahead for Python as a whole and many others.  I can’t wait to start using more of these tools set in future projects. (Scala anyone? 😌😉)</p><p>Till then, Happy Coding, (and remember: stay safe and do your social distance-coding rules)!</p>]]></content>
    
    
    <summary type="html">&lt;img src=&quot;/images/ai_generated_filter_map_reduce_python_way.jpeg&quot; class=&quot;center&quot; width=&quot;500&quot; height=&quot;500&quot; title=&quot;Generated AI image by Microsoft Bing Image Creator Generated AI image by Microsoft Bing Image Creator&quot;&gt;
&lt;p&gt;&lt;em&gt;&lt;p style=&quot;text-align: center;&quot;&gt;Generated AI image by Microsoft Bing Image Creator&lt;/p&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;With every new tool, framework or methodology comes along, developers with their insatiable appetite or thirst for knowledge and power, they will find ways to make dedicated time to learn how they work and how they’re planning to use them as part of the day to day job.&lt;/p&gt;
&lt;p&gt;And functional programming (FP) - a newish paradigm has been permeating through the scenes of developer community for some time; everything from Haskell, Elixir, React, AWS Lambdas to Clojure etc.&lt;/p&gt;
&lt;p&gt;Or, at least it’s yet to make establish some &lt;a href=&quot;https://www.youtube.com/watch?v=QyJZzq0v7Z4&quot;&gt;norms&lt;/a&gt; within the community…&lt;/p&gt;
&lt;p&gt;But I must digress.&lt;/p&gt;</summary>
    
    
    
    
    <category term="javascript" scheme="http://awongcm.io/tags/javascript/"/>
    
    <category term="python" scheme="http://awongcm.io/tags/python/"/>
    
    <category term="functional-programming" scheme="http://awongcm.io/tags/functional-programming/"/>
    
    <category term="filter" scheme="http://awongcm.io/tags/filter/"/>
    
    <category term="map" scheme="http://awongcm.io/tags/map/"/>
    
    <category term="reduce" scheme="http://awongcm.io/tags/reduce/"/>
    
  </entry>
  
  <entry>
    <title>Visualising Javascript dependencies graph tree with Madge</title>
    <link href="http://awongcm.io/blog/2020/02/07/visualising-javascript-dependencies-graph-tree-with-madge/"/>
    <id>http://awongcm.io/blog/2020/02/07/visualising-javascript-dependencies-graph-tree-with-madge/</id>
    <published>2020-02-07T17:07:40.000Z</published>
    <updated>2026-03-30T12:13:51.445Z</updated>
    
    <content type="html"><![CDATA[<p>When you work on a codebase, at any certain point of time, they tend to grow in size.  Thus the larger the codebase becomes, the more Javascript modules you will get to develop and maintain.</p><p>And more often than not, those same Javascript modules are not always clear when you spend time determining the inter or intra-relationships between them.</p><p>For eg,</p><p>A developer (let’s called him Jake) works on modules, A, B, C, and discovers the following dependencies pattern.</p><pre><code class="hljs plaintext">Module A depends on Module B and C,Module B depends on Module CModule C doesn&#x27;t depend on anything.</code></pre><span id="more"></span><p>From this, Jake would assume he only needs to worry about the core functionality shared between A, B and C in linear fashion.. and nothing else.</p><p>But this is a very bad assumption to take because, as he searched through the codebase, he learned&#x2F;realised that there are other modules in the system are <em>also relying</em> on Module B and C and he’s never worked on these other modules before.  </p><p>But yet, he has to touch these modules because any changes he introduced in Module B&#x2F;C not only has to work with Module A, but it must not break in Module XXX etc, that depends on B&#x2F;C as well.  Or.. he may not know anything about Module XXX at all and he would like to communicate better with his teammates how strongly interlinked this module would be across the entired front-end platofrm…</p><p>Sure… you may whip up the VSC and do a global search of the entire codebase to find the relevant modules you see want like so</p><pre><code class="hljs plaintext">project│└───A│   │   A.js│└───B│   │   B.js│└───C    │   C.js</code></pre><p>You still couldn’t figure out clearly on the outset how they supposed to relate to each other visually when navigating the flow of data coming from one module to the next or several modules.  </p><p>You want to see the bigger picture…  The bigger picture where we want to see ALL the dependencies and ALL files at the same time.</p><p>Henceforth, we found the solution here.</p><p>That solution is called <a href="https://github.com/pahen/madge">MadgeJS</a>.</p><p>According to its main Github headline,</p><blockquote><p>Create graphs from your CommonJS, AMD or ES6 module dependencies.</p></blockquote><p>Nuff said.</p><p>It is simply a fantastic tooling that helps you to build visual graphs of your Javascript dependencies tree whenever you wish to discover the interdependent relationships between modules in the whole JS systems - on the fly.</p><p>The short version of this:</p><ol><li>Start and make your project</li><li>Run madge, and feed the source files</li><li>Lastly, you will see a generated graph will pop up.</li></ol><p>For example, if you do the git clone of madge js library, and run npm install when you run madge on its library, you will see below</p><img src="/images/madgejs.png" class="" title="MadgeJS Dependency Tree MadgeJS Dependency Tree"><p>You may need to install <a href="https://github.com/pahen/madge#graphviz-optional">graphviz</a> to view this.</p><p>Which is great for small projects.</p><p>But… what about bigger ones?</p><p>Such as the one from <a href="https://github.com/moment/moment/">Moment JS</a> here.</p><img src="/images/momentjs.png" class="" title="MomentJS Dependency Tree MomentJS Dependency Tree"><p>☝️ That’s pretty insane right?</p><p>So, you can see the visual dependency graph tree could get so many layers of depth of dependency tree branches as far as you can reach.  From this, as developer working in a sizable team, you wamt to know where the modules have been moved around or changes by other developers in the codebase.</p><p>You can, of course, change the depth of the dependency graph as little or more as you want.</p><p>Madge allows you to:</p><ol><li>Query dependency graphs on a single file</li><li>Query dependency graphs on multiple files</li><li>Query dependency from different folder locations</li><li>Query modules that have little or dependency at all</li><li>etc, etc</li></ol><p>With all, you can determine the output of the visualisation graph in different formats such as PNG, SVG or even JSON as well.</p><p>It works on all types of JS frameworks ie React, Angular, VueJS, Svelte, Express etc, etc.</p><p>That’s it, folks!  Have that as another awesome tooling to add as part of your JS toolkit belt.</p><p>Till then, Happy Coding!</p>]]></content>
    
    
    <summary type="html">&lt;p&gt;When you work on a codebase, at any certain point of time, they tend to grow in size.  Thus the larger the codebase becomes, the more Javascript modules you will get to develop and maintain.&lt;/p&gt;
&lt;p&gt;And more often than not, those same Javascript modules are not always clear when you spend time determining the inter or intra-relationships between them.&lt;/p&gt;
&lt;p&gt;For eg,&lt;/p&gt;
&lt;p&gt;A developer (let’s called him Jake) works on modules, A, B, C, and discovers the following dependencies pattern.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;hljs plaintext&quot;&gt;Module A depends on Module B and C,
Module B depends on Module C
Module C doesn&amp;#x27;t depend on anything.&lt;/code&gt;&lt;/pre&gt;</summary>
    
    
    
    
    <category term="javascript" scheme="http://awongcm.io/tags/javascript/"/>
    
    <category term="react" scheme="http://awongcm.io/tags/react/"/>
    
    <category term="tooling" scheme="http://awongcm.io/tags/tooling/"/>
    
    <category term="dependency-graph" scheme="http://awongcm.io/tags/dependency-graph/"/>
    
    <category term="visualization" scheme="http://awongcm.io/tags/visualization/"/>
    
    <category term="angular" scheme="http://awongcm.io/tags/angular/"/>
    
    <category term="vue" scheme="http://awongcm.io/tags/vue/"/>
    
  </entry>
  
  <entry>
    <title>Most common and useful design patterns you should be aware of as a Javascript developer</title>
    <link href="http://awongcm.io/blog/2019/11/16/most-common-and-useful-design-patterns-you-should-be-aware-of-as-a-javascript-developer/"/>
    <id>http://awongcm.io/blog/2019/11/16/most-common-and-useful-design-patterns-you-should-be-aware-of-as-a-javascript-developer/</id>
    <published>2019-11-16T23:37:31.000Z</published>
    <updated>2026-03-30T12:13:51.445Z</updated>
    
    <content type="html"><![CDATA[<p>If you’re anything like me, after you spent considerable time chopping up codes for all types of applications, be it web or mobile app, you’ve already come across with code that shares some similar patterns as the codebase itself has grown over some significant portion of the time.</p><p>From these observations, we programmers developed our conversations on design patterns in making scalable software solutions.</p><p>In particular with JS, with the influx of JS libraries, frameworks, tools etc, we can build our applications to solve some particular problems in so many different ways.  But, no matter how much tooling JS developers are going to be choosing, there’s no better substitute for incorporating useful patterns in your code design where you see fit.</p><span id="more"></span><p>Thus the question remains - what are the common and useful design patterns that modern JS developers will be dealing on a day to day basis?</p><p>At the very core of everything, after working with JS space for several years, you should learn and be aware of these:</p><ol><li>Constructor Pattern</li><li>Prototype Pattern</li><li>Module Design Pattern</li><li>Observer Pattern</li><li>Decorator Pattern</li></ol><h4 id="Constructor-Patterns"><a href="#Constructor-Patterns" class="headerlink" title="Constructor Patterns"></a>Constructor Patterns</h4><p>This pattern is well-known for creating and initializing new objects when memory is allocated.  Just like a traditional object-oriented purist, Javascript also shares the same object constructor blueprint such as Java or C#.  </p><p>Thus, prior to ES6&#x2F;7&#x2F;8&#x2F;9, we had the following traditional way to do things.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Constructor Pattern - old school approach</span><span class="hljs-comment">/* Option 1 - using Object.create function call and object literal presentation */</span><span class="hljs-keyword">var</span> shirt = &#123;<span class="hljs-attr">colour</span>: <span class="hljs-string">&quot;red&quot;</span><span class="hljs-attr">size</span>: <span class="hljs-string">&quot;S&quot;</span><span class="hljs-attr">price</span>: <span class="hljs-number">3.5</span>&#125;<span class="hljs-keyword">var</span> blueshirt = <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">create</span>(shirt)blueshirt.<span class="hljs-property">colour</span> = <span class="hljs-string">&#x27;blue&#x27;</span>blueshirt.<span class="hljs-property">size</span> = <span class="hljs-string">&quot;M&quot;</span>blueshirt.<span class="hljs-property">price</span> = <span class="hljs-number">5.5</span><span class="hljs-comment">/* Option 2 - use the &#x27;new&#x27; keyword */</span><span class="hljs-keyword">function</span> <span class="hljs-title function_">Shirt</span>(<span class="hljs-params">colour, size, price</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = colour;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = size;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = price;<span class="hljs-variable language_">this</span>.<span class="hljs-property">toString</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-keyword">return</span> <span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> + <span class="hljs-string">&quot; shirt has size  &quot;</span> + <span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> + <span class="hljs-string">&quot;, selling for $&quot;</span> + <span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span>; &#125;&#125;<span class="hljs-keyword">var</span> blueshirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;blue&quot;</span>, <span class="hljs-string">&quot;M&quot;</span>, <span class="hljs-number">3.5</span>)<span class="hljs-keyword">var</span> redshirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;red&quot;</span>, <span class="hljs-string">&quot;L&quot;</span>, <span class="hljs-number">5.5</span>)</code></pre><p>Back in my days of BackboneJS web development several years ago, this is the most common pattern approach when making object creations for your Models, Collections and Controllers etc.  You notice this pattern is very apparent all over the <a href="https://addyosmani.com/backbone-fundamentals/#models-1">place</a>.</p><p>But since then, the world moved on, ES6 arrived and we now have the following way that achieves the same thing using new <code>class</code> and <code>constructor</code> approach.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Constructor Pattern - ES6 approach</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Shirt</span> &#123;<span class="hljs-title function_">constructor</span>(<span class="hljs-params">colour, size, price</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = colour;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = size;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = price;&#125;<span class="hljs-title function_">toString</span>(<span class="hljs-params"></span>) &#123;  <span class="hljs-keyword">return</span> <span class="hljs-string">`<span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.colour&#125;</span> shirt has size <span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.size&#125;</span>, selling for $<span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.miles&#125;</span>`</span>    &#125;&#125;<span class="hljs-keyword">let</span> blueShirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;blue&quot;</span>, <span class="hljs-string">&quot;M&quot;</span>, <span class="hljs-number">3.5</span>)<span class="hljs-keyword">let</span> redShirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;red&quot;</span>, <span class="hljs-string">&quot;L&quot;</span>, <span class="hljs-number">5.5</span>)</code></pre><p>You’ll find that most modern JS frameworks that came out during post-ES6 era will embed this constructor patterns all over the place such as React, VueJS, AngularJS.  You can google up “some_js_framework_name constructor example “, and you will see what I mean.</p><h4 id="Prototype-Pattern"><a href="#Prototype-Pattern" class="headerlink" title="Prototype Pattern"></a>Prototype Pattern</h4><p>Next, we have another variation of object creation patterns, but it makes use of JS prototypical inheritance whereby objects are created to prototypes for all object types.  Prototypes act as blueprint for each object constructor created. Prior to ES6,  It is normally done using <code>prototype</code>  bindings.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Prototype Pattern - Old school approach</span><span class="hljs-keyword">var</span> <span class="hljs-title class_">Shirt</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params">colour, size, price</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = colour;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = size;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = price;&#125;<span class="hljs-title class_">Shirt</span>.<span class="hljs-property"><span class="hljs-keyword">prototype</span></span> = &#123;<span class="hljs-attr">changeColour</span>: <span class="hljs-keyword">function</span> (<span class="hljs-params">newColour</span>)&#123; <span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = newColour;&#125;,<span class="hljs-attr">changeSize</span>: <span class="hljs-keyword">function</span> (<span class="hljs-params">newSize</span>)&#123; <span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = newSize;&#125;,<span class="hljs-attr">changePrice</span>: <span class="hljs-keyword">function</span> (<span class="hljs-params">newPrice</span>)&#123; <span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = newPrice;&#125;&#125;<span class="hljs-keyword">var</span> blueShirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;blue&quot;</span>, <span class="hljs-string">&quot;M&quot;</span>, <span class="hljs-number">5.5</span>)blueShirt.<span class="hljs-title function_">changeColour</span>(<span class="hljs-string">&quot;red&quot;</span>);blueShirt.<span class="hljs-title function_">changeSize</span>(<span class="hljs-string">&quot;L&quot;</span>);blueShirt.<span class="hljs-title function_">changePrice</span>(<span class="hljs-number">3.5</span>);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(blueShirt); <span class="hljs-comment">//outputs - Shirt &#123;colour: &quot;red&quot;, size: &quot;L&quot;, price: 3.5&#125;</span></code></pre><p>Its ES6 counterpart is the same as its constructor pattern example above.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Prototype Pattern - ES6 way</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Shirt</span> &#123;<span class="hljs-title function_">constructor</span>(<span class="hljs-params">colour, size, price</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = colour;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = size;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = price;&#125;<span class="hljs-title function_">changeColour</span>(<span class="hljs-params">newColour</span>) &#123;&#125;<span class="hljs-title function_">changeSize</span>(<span class="hljs-params">newSize</span>) &#123;&#125;<span class="hljs-title function_">changePrice</span>(<span class="hljs-params">newPrice</span>) &#123;&#125;<span class="hljs-title function_">toString</span>(<span class="hljs-params"></span>) &#123;  <span class="hljs-keyword">return</span> <span class="hljs-string">`<span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.colour&#125;</span> shirt has size <span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.size&#125;</span>, selling for $<span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.miles&#125;</span>`</span>    &#125;&#125;</code></pre><p>The reason for this is <code>class</code> and <code>constructor</code> are just plain syntactic sugars for prototype bindings as part of the internal mechanics.    This option gives us the freedom to write cleaner code and confidence to write true object-oriented code thus developers coming from Java, C# etc will feel more at ease using these.</p><p>For these same reasons, you can now emulate object inheritance using <code>extends</code> keyword as well</p><pre><code class="hljs javascript"><span class="hljs-comment">// classic OOP inheritance example</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Employee</span> &#123;<span class="hljs-title function_">constructor</span>(<span class="hljs-params">name, salary, tax_rate</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">name</span> = name;<span class="hljs-variable language_">this</span>.<span class="hljs-property">salary</span> = salary;<span class="hljs-variable language_">this</span>.<span class="hljs-property">tax_rate</span> = tax_rate;<span class="hljs-variable language_">this</span>.<span class="hljs-property">total_working_hours</span> = <span class="hljs-number">40</span> &#125;<span class="hljs-title function_">calculateSalary</span>(<span class="hljs-params"></span>)&#123;<span class="hljs-comment">// some calculations to perform here</span>&#125;&#125;<span class="hljs-keyword">class</span> <span class="hljs-title class_">FullTimer</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Employee</span> &#123;<span class="hljs-comment">//properties to describe full-timer&#x27;s role</span>&#125;<span class="hljs-keyword">class</span> <span class="hljs-title class_">Contractor</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Employee</span> &#123;<span class="hljs-comment">//properties to describe contractor&#x27;s role</span>&#125;</code></pre><h4 id="Module-Design-Pattern"><a href="#Module-Design-Pattern" class="headerlink" title="Module Design Pattern"></a>Module Design Pattern</h4><p>This pattern is used as an improvement to the prototype pattern approach. Different types of modifiers (both private and public) are set in the module pattern. You can create similar functions or properties without conflicts.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Module Design Pattern - old school approach</span><span class="hljs-keyword">var</span> <span class="hljs-title class_">Shirt</span> = (<span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;  <span class="hljs-comment">// private variables</span><span class="hljs-keyword">var</span> _colour = <span class="hljs-string">&quot;blue&quot;</span>;<span class="hljs-keyword">var</span> _size = <span class="hljs-string">&quot;M&quot;</span>;<span class="hljs-keyword">var</span> _price = <span class="hljs-number">5.5</span>;<span class="hljs-comment">// public methods and properties</span><span class="hljs-keyword">return</span> &#123;<span class="hljs-attr">colour</span>: _colour,<span class="hljs-attr">size</span>: _size,<span class="hljs-attr">price</span>: _price,<span class="hljs-attr">changeColour</span>: <span class="hljs-keyword">function</span>(<span class="hljs-params">newColour</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">_colour</span> = newColour;&#125;,<span class="hljs-attr">changeSize</span>: <span class="hljs-keyword">function</span>(<span class="hljs-params">newSize</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">_size</span> = newSize;&#125;,<span class="hljs-attr">changePrice</span>: <span class="hljs-keyword">function</span>(<span class="hljs-params">newPrice</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">_price</span> = newPrice;&#125;&#125;&#125;)();</code></pre><p>Notice, in the above, my private variable declarations are denoted with <code>_</code> prefixes.  This is more of a <a href="https://stackoverflow.com/questions/4484424/underscore-prefix-for-property-and-method-names-in-javascript">convention</a> than an actual language feature implementation as JS does not, at the time of writing, have a way to present private variables&#x2F;properties like you do in Java, C# etc.  Just to add that for clarity.</p><p>Again, you can find similar patterns of these used in several jQuery-influenced JS frameworks such as BackboneJS etc.</p><p>After ES6 come along, we now use <code>import/export</code> mechanism instead.  Thus we get:</p><pre><code class="hljs javascript"><span class="hljs-comment">// Module Design Pattern - ES6 way</span><span class="hljs-comment">// Saved in a some-module-design-pattern-es6.js</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Shirt</span> &#123;<span class="hljs-title function_">constructor</span>(<span class="hljs-params">colour, size, price</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = colour;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = size;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = price;&#125;<span class="hljs-title function_">changeColour</span>(<span class="hljs-params">newColour</span>) &#123;&#125;<span class="hljs-title function_">changeSize</span>(<span class="hljs-params">newSize</span>) &#123;&#125;<span class="hljs-title function_">changePrice</span>(<span class="hljs-params">newPrice</span>) &#123;&#125;<span class="hljs-title function_">toString</span>(<span class="hljs-params"></span>) &#123;  <span class="hljs-keyword">return</span> <span class="hljs-string">`<span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.colour&#125;</span> shirt has size <span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.size&#125;</span>, selling for $<span class="hljs-subst">$&#123;<span class="hljs-variable language_">this</span>.miles&#125;</span>`</span>  &#125;&#125;<span class="hljs-keyword">export</span> <span class="hljs-keyword">default</span> <span class="hljs-title class_">Shirt</span>;<span class="hljs-comment">// usage</span><span class="hljs-keyword">import</span> <span class="hljs-title class_">Shirt</span> <span class="hljs-keyword">from</span> <span class="hljs-string">&#x27;./some-module-design-pattern-es6&#x27;</span>;<span class="hljs-keyword">let</span> blueShirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;blue&quot;</span>, <span class="hljs-string">&quot;M&quot;</span>, <span class="hljs-number">5.5</span>)</code></pre><h4 id="Observer-Design-Pattern"><a href="#Observer-Design-Pattern" class="headerlink" title="Observer Design Pattern"></a>Observer Design Pattern</h4><p>This pattern is very useful when objects need to communicate with other sets of objects simultaneously. This is particularly true where there is shared a shared data&#x2F;state that gets changed&#x2F;updated across domain objects that are subscribed to listen&#x2F;respond for changes.</p><p>With this pattern, there is no unnecessary push and pull of events across the states, but rather the modules involved only modify the current state of data.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Observer Pattern - old school approach</span><span class="hljs-keyword">function</span> <span class="hljs-title function_">Observer</span>(<span class="hljs-params"></span>) &#123;  <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span> = [];&#125;<span class="hljs-title class_">Observer</span>.<span class="hljs-property"><span class="hljs-keyword">prototype</span></span> = &#123;  <span class="hljs-attr">subscribe</span>: <span class="hljs-keyword">function</span>(<span class="hljs-params">element</span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">push</span>(element);  &#125;,  <span class="hljs-attr">unsubscribe</span>: <span class="hljs-keyword">function</span>(<span class="hljs-params">element</span>) &#123;    <span class="hljs-keyword">var</span> elementIndex = <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">indexOf</span>(element);    <span class="hljs-keyword">if</span>(elementIndex &gt; -<span class="hljs-number">1</span>) &#123;      <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">splice</span>(elementIndex, <span class="hljs-number">1</span>);    &#125;  &#125;,  <span class="hljs-attr">notifyAll</span>: <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">forEach</span>(<span class="hljs-keyword">function</span>(<span class="hljs-params">observerElement</span>)&#123;      <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">&quot;observerElement: &quot;</span> + observerElement.<span class="hljs-property">name</span> + <span class="hljs-string">&quot; has been notified&quot;</span>;    &#125;)  &#125;,&#125;</code></pre><p>Its ES6 equivalent:</p><pre><code class="hljs javascript"><span class="hljs-comment">// Observer Pattern - ES6 way</span><span class="hljs-comment">// again using class and constructor combo..</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Observer</span> &#123;  <span class="hljs-title function_">constructor</span>(<span class="hljs-params"></span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span> = [];  &#125;,  <span class="hljs-title function_">subscribe</span>(<span class="hljs-params">element</span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">push</span>(element);  &#125;  <span class="hljs-title function_">unsubscribe</span>(<span class="hljs-params">element</span>) &#123;    <span class="hljs-keyword">let</span> elementIndex = <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">indexOf</span>(element);    <span class="hljs-keyword">if</span>(elementIndex &gt; -<span class="hljs-number">1</span>) &#123;      <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">splice</span>(elementIndex, <span class="hljs-number">1</span>);    &#125;  &#125;  <span class="hljs-title function_">notifyAll</span>(<span class="hljs-params">element</span>) &#123;    <span class="hljs-variable language_">this</span>.<span class="hljs-property">observerList</span>.<span class="hljs-title function_">forEach</span>(<span class="hljs-keyword">function</span>(<span class="hljs-params">observerElement</span>)&#123;      <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">&quot;observerElement: &quot;</span> + observerElement.<span class="hljs-property">name</span> + <span class="hljs-string">&quot; has been notified&quot;</span>;    &#125;)  &#125;&#125;</code></pre><p>You will find tons of examples this observer pattern is used very heavily across all JS apps in any old JS frameworks you may find as I explained in my previous examples in this blog post.</p><h4 id="Decorator-Pattern"><a href="#Decorator-Pattern" class="headerlink" title="Decorator Pattern"></a>Decorator Pattern</h4><p>Lastly, this pattern aims to promote code re-use.  They offer the ability to add additional behaviour or features to existing classes in a system dynamically. </p><p>It is common that you’d find applications (especially in the object-oriented world) that contain features requiring large quantity of distinct types of object it has to manage and deal with.  Thus, keeping track for every object definition and creation would be monumental tasks to accomplish at given point of time during the running lifetime of an app.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Decorator Pattern - old school approach</span><span class="hljs-keyword">function</span> <span class="hljs-title function_">Shirt</span>(<span class="hljs-params">brandName</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">brandName</span> = brandName;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;<span class="hljs-keyword">return</span> <span class="hljs-string">&quot;blue&quot;</span>;&#125;;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;<span class="hljs-keyword">return</span> <span class="hljs-string">&quot;M&quot;</span>;&#125;;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;<span class="hljs-keyword">return</span> <span class="hljs-number">3</span>;&#125;;&#125;<span class="hljs-keyword">function</span> <span class="hljs-title function_">swapColour</span>(<span class="hljs-params">shirt</span>) &#123;<span class="hljs-keyword">var</span> c = shirt.<span class="hljs-title function_">colour</span>();shirt.<span class="hljs-property">colour</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;c = <span class="hljs-string">&quot;blue&quot;</span>;<span class="hljs-keyword">return</span> c;&#125;&#125;<span class="hljs-keyword">function</span> <span class="hljs-title function_">changeSize</span>(<span class="hljs-params">shirt</span>) &#123;<span class="hljs-keyword">var</span> s = shirt.<span class="hljs-title function_">size</span>();shirt.<span class="hljs-property">colour</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;s = <span class="hljs-string">&quot;S&quot;</span>;<span class="hljs-keyword">return</span> s;&#125;&#125;<span class="hljs-keyword">function</span> <span class="hljs-title function_">addCostToPrice</span>(<span class="hljs-params">shirt</span>) &#123;<span class="hljs-keyword">var</span> p = shirt.<span class="hljs-title function_">price</span>();shirt.<span class="hljs-property">price</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params"></span>) &#123;<span class="hljs-keyword">return</span> p + <span class="hljs-number">1.50</span>;&#125;&#125;<span class="hljs-keyword">var</span> shirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;Gucci&quot;</span>);<span class="hljs-title function_">swapColour</span>(shirt);<span class="hljs-title function_">changeSize</span>(shirt);<span class="hljs-title function_">addCostToPrice</span>(shirt);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(shirt.<span class="hljs-title function_">colour</span>()); <span class="hljs-comment">// blue</span><span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(shirt.<span class="hljs-title function_">size</span>()); <span class="hljs-comment">// S</span><span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(shirt.<span class="hljs-title function_">price</span>()); <span class="hljs-comment">// 4.50</span></code></pre><p>Its ES6 equivalent would be:</p><pre><code class="hljs javascript"><span class="hljs-comment">// Decorator Pattern - ES6 approach (again using class and constructor combo)</span><span class="hljs-keyword">class</span> <span class="hljs-title class_">Shirt</span> &#123;<span class="hljs-title function_">constructor</span>(<span class="hljs-params">brandName</span>) &#123;<span class="hljs-variable language_">this</span>.<span class="hljs-property">brandName</span> = brandName;<span class="hljs-variable language_">this</span>.<span class="hljs-property">colour</span> = <span class="hljs-string">&quot;blue&quot;</span>;<span class="hljs-variable language_">this</span>.<span class="hljs-property">size</span> = <span class="hljs-string">&quot;M&quot;</span>;<span class="hljs-variable language_">this</span>.<span class="hljs-property">price</span> = <span class="hljs-number">3</span>;&#125;&#125;<span class="hljs-keyword">function</span> <span class="hljs-title function_">swapColour</span>(<span class="hljs-params">shirt</span>) &#123;shirt.<span class="hljs-property">colour</span> = <span class="hljs-string">&quot;white&quot;</span>;<span class="hljs-keyword">return</span> shirt;&#125;<span class="hljs-keyword">function</span> <span class="hljs-title function_">changeSize</span>(<span class="hljs-params">shirt</span>) &#123;shirt.<span class="hljs-property">size</span> = <span class="hljs-string">&quot;L&quot;</span>;<span class="hljs-keyword">return</span> shirt;&#125;<span class="hljs-keyword">function</span> <span class="hljs-title function_">addCostToPrice</span>(<span class="hljs-params">shirt</span>) &#123;shirt.<span class="hljs-property">price</span> += <span class="hljs-number">1.50</span>;<span class="hljs-keyword">return</span> shirt;&#125;<span class="hljs-keyword">const</span> defaultShirt = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Shirt</span>(<span class="hljs-string">&quot;Gucci&quot;</span>)<span class="hljs-keyword">const</span> whiteShirt = <span class="hljs-title function_">swapColour</span>(defaultShirt);<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(whiteShirt); <span class="hljs-comment">// outputs - Shirt &#123;brandName: &quot;Gucci&quot;, colour: &quot;white&quot;, size: &quot;M&quot;, price: 3&#125;</span><span class="hljs-keyword">const</span> whiteLargeShirt = <span class="hljs-title function_">changeSize</span>(<span class="hljs-title function_">swapColour</span>(defaultShirt));<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(whiteLargeShirt); <span class="hljs-comment">// outputs - Shirt &#123;brandName: &quot;Gucci&quot;, colour: &quot;white&quot;, size: &quot;L&quot;, price: 3&#125;</span><span class="hljs-keyword">const</span> priceyWhiteLargeShirt = <span class="hljs-title function_">addCostToPrice</span>(<span class="hljs-title function_">changeSize</span>(<span class="hljs-title function_">swapColour</span>(defaultShirt)));<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(priceyWhiteLargeShirt); <span class="hljs-comment">// outputs - Shirt &#123;brandName: &quot;Gucci&quot;, colour: &quot;white&quot;, size: &quot;L&quot;, price: 4.5&#125;</span></code></pre><p>With all the examples above, what you may realise is that these patterns share one thing in common - they’re come from the basis of object-oriented programming (OOP) design paradigm.</p><p>These are designed to revolve around object-oriented driven software systems.  You can read up examples <a href="https://stackoverflow.com/questions/3631039/design-patterns-used-in-the-jquery-library">here(with jQuery)</a> and <a href="https://addyosmani.com/backbone-fundamentals/">there( with BackboneJS)</a>.</p><p>And recent years, we’ve been told to get educated in building applications using a different programming paradigm.  And that is, functional programming (FP).  </p><p>Without a doubt, we’re slowly starting to hear plenty of noise that functional developers&#x2F;advocates are working on, thus languages like JS are no stranger to this concept.</p><p>Interestingly enough, Javascript treats functions as first-class citizen vs its object-oriented aspects.</p><p>Thus, modern frameworks such as React starting to appear, are embracing these fully.</p><p>The key questions are - what are common design patterns can you employ in the functional programming world, particularly with React?</p><p>Well.</p><p>There’s aplenty of them to describe here.</p><p>Examples of React patterns:</p><ol><li>Functional Components</li><li>Class Components</li><li>Presentational Components or Higher Order Components(HOC)</li><li>Container Components</li><li>Render Props Pattern</li></ol><h4 id="Functional-Components"><a href="#Functional-Components" class="headerlink" title="Functional Components"></a>Functional Components</h4><p>Function components are, as they plainly described, just simple functions that return components</p><p>Here’s an example.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Functional Components</span><span class="hljs-keyword">const</span> <span class="hljs-title function_">Greeting</span> = (<span class="hljs-params"></span>) =&gt; <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">div</span>&gt;</span>Hello small world!<span class="hljs-tag">&lt;/<span class="hljs-name">div</span>&gt;</span></span></code></pre><pre><code class="hljs javascript"><span class="hljs-comment">// Functional Components with props</span><span class="hljs-keyword">const</span> <span class="hljs-title function_">Greeting</span> = (<span class="hljs-params">props</span>) =&gt; <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">div</span>&gt;</span>Hello &#123;props.name&#125;&#125;!<span class="hljs-tag">&lt;/<span class="hljs-name">div</span>&gt;</span></span></code></pre><p>What we’re saying here is that we’re not keeping states in the component so we always pass down the props to the functions so that they are reliably testable.</p><h4 id="Class-Components"><a href="#Class-Components" class="headerlink" title="Class Components"></a>Class Components</h4><p>It is said to be a class-based components because it has data and attributes that associate with the object’s state.</p><p>Stateful components are usually class-based components using thins like constructor properties etc.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Class Based Component</span><span class="hljs-keyword">import</span> &#123;<span class="hljs-title class_">Component</span>&#125; <span class="hljs-keyword">from</span> <span class="hljs-string">&quot;React&quot;</span>;<span class="hljs-keyword">class</span> <span class="hljs-title class_">NumberRandomGenerator</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Component</span> &#123;state = &#123;<span class="hljs-attr">number</span>: <span class="hljs-title class_">Math</span>.<span class="hljs-title function_">random</span>()&#125;<span class="hljs-title function_">render</span>(<span class="hljs-params"></span>) &#123;<span class="hljs-keyword">return</span> (<span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">DataResult</span></span></span><span class="hljs-tag"><span class="language-xml"><span class="hljs-attr">result</span> = <span class="hljs-string">&#123;this.state.number&#125;</span></span></span><span class="hljs-tag"><span class="language-xml"><span class="hljs-attr">title</span>=<span class="hljs-string">&#123;</span>&#x27;<span class="hljs-attr">Your</span> <span class="hljs-attr">name</span> <span class="hljs-attr">is:</span>&#x27; &#125;</span></span><span class="hljs-tag"><span class="language-xml"><span class="hljs-attr">onClick</span>=<span class="hljs-string">&#123;this.setstate(&#123;number:</span> <span class="hljs-attr">Math.random</span>()&#125;)&#125;</span></span><span class="hljs-tag"><span class="language-xml">/&gt;</span></span>)&#125;&#125;</code></pre><h4 id="Presentation-Components-or-Higher-Order-Components-HOC"><a href="#Presentation-Components-or-Higher-Order-Components-HOC" class="headerlink" title="Presentation Components or Higher Order Components (HOC)"></a>Presentation Components or Higher Order Components (HOC)</h4><p>Higher Order Components (HOC) is said to be design pattern, which is also know as a Decorator Pattern.  Commonly, in ReactJS, a HOC is a component that wraps another component by adding extra functionality or extra properties. This allows abstraction from some commonly used logic and keeps your code DRY. It is how you distribute complex component structure between other components in ReactJS and a way to decouple your application logic and UI.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Presentational Components or HOC Components</span><span class="hljs-keyword">import</span> &#123;<span class="hljs-title class_">Component</span>&#125; <span class="hljs-keyword">from</span> <span class="hljs-string">&quot;React&quot;</span>;<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> <span class="hljs-title function_">addSomeData</span> = (<span class="hljs-params">WrappedComponent</span>) =&gt; &#123;  <span class="hljs-keyword">return</span> <span class="hljs-keyword">class</span> <span class="hljs-title class_">extends</span> <span class="hljs-title class_">Component</span> &#123;    <span class="hljs-title function_">render</span>(<span class="hljs-params"></span>) &#123;      <span class="hljs-keyword">return</span> <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">WrappedComponent</span> <span class="hljs-attr">number</span>=<span class="hljs-string">&#123;this.state.number&#125;</span> <span class="hljs-attr">name</span>=<span class="hljs-string">&#123;this.state.name&#125;</span> &#123;<span class="hljs-attr">...this.props</span>&#125; /&gt;</span></span>    &#125;  &#125;&#125;<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> withNumberAndName = <span class="hljs-title function_">addSomeData</span>(<span class="hljs-title class_">DataResult</span>)</code></pre><h4 id="Container-Components"><a href="#Container-Components" class="headerlink" title="Container Components"></a>Container Components</h4><p>Container components’ sole responsibility is to have logic to set state or have functions to emit events up to a parent component. The general rule of the thumb is to keep your component as simple as possible with a Single Responsibility Principle design principle in mind, which essentially means your component must do one thing, but do it well.</p><p>Most often, these types of components are the HOCs that accommodate few presentational components.</p><pre><code class="hljs javascript"><span class="hljs-comment">// Container Components</span><span class="hljs-keyword">import</span> &#123;<span class="hljs-title class_">Component</span>&#125; <span class="hljs-keyword">from</span> <span class="hljs-string">&quot;React&quot;</span>;<span class="hljs-keyword">class</span> <span class="hljs-title class_">CommentListComponent</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Component</span> &#123;state = &#123;<span class="hljs-attr">comments</span>: []&#125;componentDidMount = <span class="hljs-title function_">async</span> () =&gt; &#123;<span class="hljs-keyword">try</span> &#123;<span class="hljs-keyword">const</span> response = <span class="hljs-keyword">await</span> <span class="hljs-title function_">fetch</span>(<span class="hljs-string">&#x27;https://someurl.com&#x27;</span>)<span class="hljs-keyword">const</span> result = <span class="hljs-keyword">await</span> response.<span class="hljs-title function_">json</span>();<span class="hljs-variable language_">this</span>.<span class="hljs-title function_">setState</span>(&#123;<span class="hljs-attr">comments</span>: result&#125;)&#125; <span class="hljs-keyword">catch</span>(error) &#123;<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">&#x27;Error:&#x27;</span>, error);&#125;&#125;<span class="hljs-title function_">render</span>(<span class="hljs-params"></span>) &#123;<span class="hljs-keyword">return</span> <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">CommentList</span> <span class="hljs-attr">comments</span>=<span class="hljs-string">&#123;this.state.comments&#125;</span> /&gt;</span></span>&#125;&#125;</code></pre><h4 id="Render-Props-Pattern"><a href="#Render-Props-Pattern" class="headerlink" title="Render Props Pattern"></a>Render Props Pattern</h4><p>It is a technique or pattern used to share code between components using a prop whose value is a function .</p><p>They’re most helpful in sharing cross-cutting concerns thus allows you to share and re-use patterns and logic across components.</p><p>Here’s one example</p><pre><code class="hljs javascript"><span class="hljs-comment">// Render Props Pattern - based on the official docs - https://reactjs.org/docs/render-props.html</span><span class="hljs-keyword">import</span> &#123;<span class="hljs-title class_">Component</span>, <span class="hljs-title class_">Fragment</span>&#125; <span class="hljs-keyword">from</span> <span class="hljs-string">&quot;React&quot;</span>;<span class="hljs-keyword">import</span> moment <span class="hljs-keyword">from</span> <span class="hljs-string">&quot;moment&quot;</span>;<span class="hljs-keyword">class</span> <span class="hljs-title class_">Watch</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Component</span> &#123;  state = &#123;    <span class="hljs-attr">date</span>: <span class="hljs-title function_">moment</span>();  &#125;  componentDidMount = <span class="hljs-function">() =&gt;</span> (<span class="hljs-variable language_">this</span>.<span class="hljs-property">TICK</span> = <span class="hljs-built_in">setInterval</span>(<span class="hljs-variable language_">this</span>.<span class="hljs-property">update</span>, <span class="hljs-number">1000</span>))  componentWillUnMount = <span class="hljs-function">() =&gt;</span> <span class="hljs-built_in">clearInterval</span>(<span class="hljs-variable language_">this</span>.<span class="hljs-property">TICK</span>)  update = <span class="hljs-function">() =&gt;</span> <span class="hljs-variable language_">this</span>.<span class="hljs-title function_">setState</span>(&#123;<span class="hljs-attr">date</span>: <span class="hljs-title function_">moment</span>()&#125;)  render = <span class="hljs-function">() =&gt;</span> (    <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">div</span>&gt;</span></span><span class="language-xml">      &#123;this.props.render(this.state.date)&#125;</span><span class="language-xml">    <span class="hljs-tag">&lt;/<span class="hljs-name">div</span>&gt;</span></span>  )&#125;<span class="hljs-keyword">const</span> <span class="hljs-title function_">AnalogFace</span> =(<span class="hljs-params">&#123;date&#125;</span>) =&gt; &#123;  <span class="hljs-keyword">const</span> seconds = (<span class="hljs-number">360</span> /<span class="hljs-number">60</span>) * date.<span class="hljs-title function_">seconds</span>();  <span class="hljs-keyword">const</span> minutes = (<span class="hljs-number">360</span> /<span class="hljs-number">60</span>) * date.<span class="hljs-title function_">minutes</span>();  <span class="hljs-keyword">const</span> hours = (<span class="hljs-number">360</span> /<span class="hljs-number">12</span>) * date.<span class="hljs-title function_">format</span>(<span class="hljs-string">&#x27;h&#x27;</span>);  <span class="hljs-keyword">return</span> (    <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">Fragment</span>&gt;</span></span><span class="language-xml">      <span class="hljs-tag">&lt;<span class="hljs-name">span</span>&gt;</span>&#123;seconds&#125;<span class="hljs-tag">&lt;/<span class="hljs-name">span</span>&gt;</span></span><span class="language-xml">      <span class="hljs-tag">&lt;<span class="hljs-name">span</span>&gt;</span>&#123;minutes&#125;<span class="hljs-tag">&lt;/<span class="hljs-name">span</span>&gt;</span></span><span class="language-xml">      <span class="hljs-tag">&lt;<span class="hljs-name">span</span>&gt;</span>&#123;hours&#125;<span class="hljs-tag">&lt;/<span class="hljs-name">span</span>&gt;</span></span><span class="language-xml">    <span class="hljs-tag">&lt;/<span class="hljs-name">Fragment</span>&gt;</span></span>  )&#125;<span class="hljs-keyword">class</span> <span class="hljs-title class_">App</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Component</span> &#123;  render = <span class="hljs-function">() =&gt;</span> (      <span class="language-xml"><span class="hljs-tag">&lt;<span class="hljs-name">Fragment</span>&gt;</span></span><span class="language-xml">        <span class="hljs-tag">&lt;<span class="hljs-name">h1</span>&gt;</span>Checkout this cool watch<span class="hljs-tag">&lt;/<span class="hljs-name">h1</span>&gt;</span></span><span class="language-xml">        <span class="hljs-tag">&lt;<span class="hljs-name">Watch</span> <span class="hljs-attr">render</span>=<span class="hljs-string">&#123;date</span> =&gt;</span> <span class="hljs-tag">&lt;<span class="hljs-name">AnalogFace</span> <span class="hljs-attr">date</span>=<span class="hljs-string">&#123;date&#125;</span> /&gt;</span>&#125; /&gt;</span><span class="language-xml">      <span class="hljs-tag">&lt;/<span class="hljs-name">Fragment</span>&gt;</span></span>  )&#125;</code></pre><h3 id="Concluding-thoughts"><a href="#Concluding-thoughts" class="headerlink" title="Concluding thoughts"></a>Concluding thoughts</h3><p>Now, with the examples I provided in this post, I would like to point out this is not about memorising these patterns by heart and always remembering to apply in <em>all situations</em>.  Some patterns are good for something.  Some may not.  This is also, by far, not the most exhaustive list of design patterns you will ever gonna need.  They are many more patterns out that still need yet to explore and get accustomed such as the ones you get from (in the OOP world) <em>Gang of Four: Design Patterns: Elements of Reusable Object-oriented Software</em> , or (in the FP world) you get several online academia reads or learning resources such as this Github <a href="https://github.com/hemanth/functional-programming-jargon">link</a> as an example.</p><p>I just sharing my years of experience in working within the JS space that I’ve been in both sides of the fence; one being with the OOP world, the other being with the FP world, and how there’s array of battle-tested software design patterns that developers and engineers have used over the years, regardless of the JS frameworks such React&#x2F;Redux&#x2F;VueJS&#x2F;Angular etc you’ll be using.</p><p>The key takeaways from this is always be mindfully aware of the design patterns you will be seeing and using over and over again through your JS software development career.  They come with several shapes and forms such that they will have a major influence on the present and future libraries&#x2F;frameworks&#x2F;tools you will come across in every software projects you’d be doing.  What better way to get your hands dirty is to get very much down to the very basics of them by starting these.  </p><p>From there, once you master them over time, you will evolve to get better and sharper in writing amazingly brilliant software as part of your craft! 🚀🚀🚀🚀🚀</p><p>Till then, Happy Coding!</p>]]></content>
    
    
    <summary type="html">&lt;p&gt;If you’re anything like me, after you spent considerable time chopping up codes for all types of applications, be it web or mobile app, you’ve already come across with code that shares some similar patterns as the codebase itself has grown over some significant portion of the time.&lt;/p&gt;
&lt;p&gt;From these observations, we programmers developed our conversations on design patterns in making scalable software solutions.&lt;/p&gt;
&lt;p&gt;In particular with JS, with the influx of JS libraries, frameworks, tools etc, we can build our applications to solve some particular problems in so many different ways.  But, no matter how much tooling JS developers are going to be choosing, there’s no better substitute for incorporating useful patterns in your code design where you see fit.&lt;/p&gt;</summary>
    
    
    
    
    <category term="javascript" scheme="http://awongcm.io/tags/javascript/"/>
    
    <category term="design-patterns" scheme="http://awongcm.io/tags/design-patterns/"/>
    
    <category term="object-oriented-programming" scheme="http://awongcm.io/tags/object-oriented-programming/"/>
    
    <category term="functional-programming" scheme="http://awongcm.io/tags/functional-programming/"/>
    
  </entry>
  
</feed>
