SoftWhat?

How to Manage AWS CDK Application Configuration

Sun, 26 Mar 2023 00:00:00 +0000

AWS CDK applications need to maintain certain configuration in the version control repository. Configuration can include application name, programming language version, version control branch, build configuration, account and Region per environment, etc. I often see the use of configuration files (e.g. cdk.json) for this purpose, in formats such as JSON and YAML.

In this post, I describe my recommendations for managing AWS CDK application configuration.

Context

From my experience, builders use configuration files mainly to avoid: 1/ changing the application code because they don’t own it 2/ triggering a build 3/ performing a deployment. With the AWS CDK, I think neither of these reasons makes configuration files more efficient than source code. Let’s break them down below.

Recommended AWS CDK Project Structure for Monolithic Python Applications

Sat, 25 Mar 2023 00:00:00 +0000

July 17, 2023: Updated the guidance to align with Application Design Framework recommendations.

My Recommended AWS CDK project structure for Python applications blog post describes an application for a single bounded context - User Management Backend service. The application includes a dedicated repository with source code for the service and the toolchain. This is the option #4 in the figure below.

Source: https://d1.awsstatic.com/events/Summits/reinvent2022/DOP321_Organize-application-components-into-repositories-and-pipelines-with-AWS-CDK.pdf

Some builders initially prefer to start with an application that includes multiple bounded contexts - options #1 and #2 above. In this post, I describe the recommended AWS CDK project structure for these options, with #1 as the example.

Aligning SaaS and Service Planes Definitions

Sun, 05 Feb 2023 00:00:00 +0000

Martin Fowler writes the following about Evocative Names in his Writing Software Patterns article:

One of the valuable features of patterns work is that it develops a vocabulary with which we can talk about how to do things.

Having a clear understanding of what names mean helps communicate intent. When building SaaS applications on AWS, you might stumble into the following terminology:

SaaS control plane and SaaS application plane
Service control plane and Service data plane

In this blog post, I describe the definitions of these terms and explain how they align.

cdk-chalice Adds Support for Accessing Chalice-generated CloudFormation Resources

Mon, 28 Sep 2020 00:00:00 +0000

cdk-chalice 0.8.1 adds support for accessing AWS Chalice-generated CloudFormation resources as native CDK objects. It is useful if you need to explicitly reference these resources in the broader AWS Cloud Development Kit (AWS CDK) application, or customize the resources generated by Chalice. In this blog post, I will explain how this feature works and provide an example.

The new capability is based on the CDK cloudformation-include module, which was released as a developer preview in CDK 1.64.1. That CDK version is now the minimal requirement for cdk-chalice. This module contains a set of classes whose goal is to facilitate working with existing CloudFormation templates in the CDK. It can be thought of as an extension of the capabilities of the CfnInclude class.

Thoughts on Interoperability Between AWS CDK, AWS SAM and AWS Chalice

Sun, 16 Aug 2020 00:00:00 +0000

Today, AWS has three frameworks to build infrastructure logic and runtime logic for applications based on Amazon API Gateway and AWS Lambda. These are the AWS Cloud Development Kit (AWS CDK), AWS Serverless Application Model (AWS SAM), and AWS Chalice. In this blog post, I will describe tradeoffs between some combinations of them.

To begin with, AWS recently announced the general availability of SAM CLI (v1.0.0). SAM CLI allows to locally invoke Lambda functions generated by CDK. CloudFormation and SAM section in CDK FAQ provides a general guideline for choosing between CDK and CloudFormation/SAM. In a gist, it says the following: “If you prefer defining your serverless infrastructure in concise declarative templates, SAM is the better fit. If you want to define your AWS infrastructure in a familiar programming language, we encourage you to try out AWS CDK.”. I would like to further expand on that and add Chalice into the mix.

Adapting Maslow's Pyramid to Personal Development

Sat, 23 May 2020 00:00:00 +0000

The software engineering world is constantly increasing in breadth and depth. There is so much to learn, and it is very challenging to prioritize what and when. Recently, I looked at Maslow’s hierarchy of needs in its pyramid form, and then an idea stroke me - personal development might be very much represented in a similar manner. The missing link for me was adding time sensitivity to the needs (i.e. priorities). One famous example of this approach is a TestPyramid:

Getting Things Done With Outlook and Asana

Tue, 28 Apr 2020 00:00:00 +0000

Nowadays, much of incoming (reactive) work arrives through email. In this post I want to share an approach I found efficient for myself to manage that work, in addition to proactive work. The process below builds on Getting Things Done time management method and the Eisenhower method for setting priorities.

This is the regular state of my Outlook inbox (except some outliers here and there of course 😊). I usually do one of the following actions when processing inbox, and then archive the email immediately:

Automated Path From Hardcoded SQL Queries to SQLAlchemy ORM With Flask

Sat, 11 Apr 2020 00:00:00 +0000

Let’s assume you started by developing the database schema directly on the server (e.g. using MySQL Workbench) and used hardcoded SQL queries in the code (e.g. using one of Transaction Script, Domain Model or Logic in SQL approaches).

Then at some point you feel confident to use SQLAlchemy’s Object-Relational Mapping (ORM) framework. On a side note, I highly recommend to read OrmHate article by Martin Fowler to get some perspective on when to use ORM frameworks. One benefit is versioning the schema as part of business logic code. There are also many tools to perform migrations, and more importantly, downgrades of the database. The reason I use SQLAlchemy here is because it is considered as one of the most popular SQL frameworks for Python.

Organizing Flask-based Web API Directory Structure

Wed, 15 Jan 2020 00:00:00 +0000

Acknowledgements: This post was heavily influenced by Explore Flask book and series of Miguel Grinberg’s posts starting with Designing a RESTful API with Python and Flask.

Still, the above and most of the other examples I found usually refer to directory structure of frontend applications or very simple web APIs. It took me a while to build a big picture of how to build the directory structure of a web API with Flask, hence I thought it might be useful to share. In addition, I wanted to add an example of how to configure a web API for production deployment. One of popular Platform-as-a-Service (PaaS) offerings out there is Cloud Foundry, so I thought it would be useful to provide an example for this platform.

Drilling Down to Amazon EC2 Instances Data Transfer Costs

Thu, 16 May 2019 00:00:00 +0000

The use case I would like to address in this post revolves around the need to understand which specific Amazon EC2 instances incur the highest data transfer charges, including high-level understanding of traffic flow (e.g. to/from Internet, inter-region, intra-region and etc.). For detailed analysis of traffic flow, I would recommend to use Amazon VPC Flow Logs – see Querying Amazon VPC Flow Logs in Amazon Athena (serverless query service) documentation.

First, I’d like to mention that for most cost management needs, you should first take a look at AWS Cost Explorer. It has an easy-to-use interface that lets you visualize, understand, and manage your AWS costs and usage over time. Get started quickly by creating custom reports (including charts and tabular data) that analyze cost and usage data, both at a high level (e.g., total costs and usage across all accounts) and for highly-specific requests (e.g., m2.2xlarge costs within account Y that are tagged “project: secretProject”). Using AWS Cost Explorer, you can dive deeper into your cost and usage data to identify trends, pinpoint cost drivers, and detect anomalies.

Bootstrapping Tcl Unit Testing With Tcltest

Fri, 01 Jul 2016 00:00:00 +0000

If you are working with Tcl, and interested in adding unit tests to your project using tcltest, read on.

Usually there is some bootstrap work required to run your unit tests. For example, every test should load the relevant libraries before using them in a test. There is also a need in a test runner that would execute the test suite and report results. Since there are several ways to implement the above, and there are some gotchas along the way, I thought it would be useful to provide a reference implementation.

Using Couchbase Python SDK on Cloud Foundry

Thu, 09 Jun 2016 00:00:00 +0000

I wanted to use Couchbase as a session store for my app instances, and specifically Couchbase Python SDK. The problem I faced was that Couchbase Python SDK (and some of Couchbase other non-C SDKs) requires Couchbase C SDK to be installed a priori – it is not installed by pip automatically.

Sadly, Conda package manager doesn’t have Couchbase C SDK in its repository, so I could not use Cloud Foundry Python Conda buildpack to install it (it can be very useful to install SciPy for example, which is supported by Conda). I wanted to avoid bundling of Couchbase C SDK in my Git repository (binary files ❌), so set out to look for a way of installing Couchbase C SDK dynamically during application push to Cloud Foundry. Cloud Foundry documentation and Google were not so kind to me this time 😢 – it took a lot of poking around, but eventually I found a nifty feature in Cloud Foundry Python buildpack source code – pre and post compile hooks!