Part 4: Internationalization

In today’s globalized world, reaching a wider audience often means offering your application in multiple languages.

By internationalizing your service, you make it more accessible to users worldwide, enhance the user experience, and potentially grow your user base.

By the end of this tutorial, your APIs will be able to deliver responses based on the user’s language preference.

Introduction to Internationalization (i18n)

Internationalization, or in short i18n, is the process of designing your application to support multiple languages.

It means adapting your app so it can be easily localized. This allows for translation and adjustments for different languages and regions without major code changes.

Internationalization is abbreviated as i18n because there are 18 letters between the first letter “i” and the last letter “n” in “internationalization.”

Internationalization vs. Localization 🌐

Internationalization (i18n) is about making your application capable of being localized by setting up the necessary infrastructure and ensuring flexibility.

Localization (l10n) is the actual process of adapting your application for a specific language or region by translating text and adjusting other content to suit local preferences.

Setting Up i18n in FastAPI

To add i18n capabilities to our FastAPI app, we will use three tools and concepts:

1. PyBabel: We’ll use the PyBabel package to extract, create, and compile translatable strings.
2. Accept-Language Header: We’ll use the standard Accept-Language header to determine the user's preferred language for API responses.
3. Custom Middleware: We’ll develop a custom middleware to handle requests and set the locale based on the request.

Understanding Gettext and PyBabel

Gettext

Gettext is a powerful tool for managing translations in software applications. It allows you to mark strings in your code that need to be translated.

Then, it retrieves the appropriate translations based on the user’s language settings. Gettext is widely used because it simplifies the process of internationalization.

PyBabel

PyBabel is a Python library that works with Gettext. It helps extract translatable strings from your code, manage translation files, and compile them for use.

With PyBabel, you can easily create and update translation files, making the localization (l10n) process straightforward. PyBabel automates many tasks, making it an essential tool for Python developers working on multilingual applications.

Understanding Translation File Formats: .pot, .po, and .mo

Translation files are the backbone of the i18n process. Here’s a brief overview:

• .pot (Portable Object Template): Template files that contain the original text strings from your application.
• .po (Portable Object): Files that contain the translated text strings for a particular language.
• .mo (Machine Object): Compiled binary files generated from .po files, used by the application at runtime.

Let’s Add i18n

1. Create a Simple API

First, let’s look at a minimal FastAPI app. We will add internationalization so the user sees the greeting’s message in their preferred language.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()


class GreetingResponse(BaseModel):
    message: str


@app.get("/greetings", response_model=GreetingResponse)
async def greetings():
    return {"message": "Hello, this is a message in your language!"}

2. Add i18n Module

In this step, we will implement a translation mechanism using the gettext library.

This involves setting up a singleton class to manage translations, creating a middleware to detect the user’s preferred language, and providing a shorthand method for retrieving translated strings.

This will allow our FastAPI app to dynamically respond with messages in the user’s chosen language.

import gettext
from pathlib import Path
from fastapi import Request


class TranslationWrapper:
    """
    Singleton class for managing translations using gettext.

    This class initializes the translation object and provides
    a method for retrieving translated strings.

    Attributes:
        _instance (TranslationWrapper): The singleton instance of 
        the TranslationWrapper class.
        translations (gettext.GNUTranslations): The translation 
        object for managing translations.
    """

    _instance = None

    def __new__(cls):
        """
        Create a new instance of the class if it doesn't 
        exist, otherwise return the existing instance.

        Returns:
            TranslationWrapper: The instance of the class.
        """
        if cls._instance is None:
            cls._instance = super().__new__(cls)
            cls._instance.init_translation()
        return cls._instance

    def init_translation(self):
        """
        Initialize the translation object.

        This method sets up the translation object 
        using the default language and the specified 
        translation directory.
        """
        lang = "en"  # Default language
        # src/translation
        locales_dir = Path(__file__).parent / "translations"
        self.translations = gettext.translation(
            "messages", 
            localedir=locales_dir,
            languages=[lang],
            fallback=True
        )
        self.translations.install()

    def gettext(self, message: str) -> str:
        """
        Get the translated string for the specified message.

        Args:
            message (str): The message to be translated.

        Returns:
            str: The translated string.
        """
        return self.translations.gettext(message)


async def set_locale(request: Request):
    """
    Set the locale based on the request headers.

    This method retrieves the accept-language header 
    from the request and sets the locale accordingly.

    Args:
        request (Request): The incoming request object.
    """
    translation_wrapper = TranslationWrapper()

    lang = request.headers.get("Accept-Language", "en")
    locales_dir = Path(__file__).parent / "translations"

    translation_wrapper.translations = gettext.translation(
        "messages", localedir=locales_dir, languages=[lang], fallback=True
    )

    translation_wrapper.translations.install()


def _(message: str) -> str:
    """
    Get the translated string for the specified message.

    This method is a shorthand for calling gettext() and
    is used to retrieve translated strings.

    Args:
        message (str): The message to be translated.

    Returns:
        str: The translated string.
    """
    translation_wrapper = TranslationWrapper()
    return translation_wrapper.gettext(message)

3. Add Language Middleware

In this step, we add a custom middleware to our FastAPI application to set the language for each request.

The LanguageMiddleware class intercepts incoming requests and sets the language based on the Accept-Language header.

This middleware uses the set_locale function from the i18n module to determine and apply the appropriate language settings before passing the request to the next middleware or route handler.

from fastapi import Request
from starlette.middleware.base import BaseHTTPMiddleware

from i18n import set_locale


class LanguageMiddleware(BaseHTTPMiddleware):
    """
    Middleware for setting the language based on the 
    request headers.

    This middleware sets the language of the application 
    based on the Accept-Language header in the incoming 
    request. It uses the set_locale function from the i18n 
    module to determine the appropriate language for the request.

    Attributes:
        None
    """

    async def dispatch(self, request: Request, call_next):
        """
        Dispatch method to set the language for the request.

        This method intercepts incoming requests, sets the 
        language based on the Accept-Language header, and then
        passes the request to the next middleware or route handler.

        Args:
            request (Request): The incoming request.
            call_next (Callable): The function to call to proceed
            to the next middleware or route handler.

        Returns:
            Response: The response generated by the next middleware
            or route handler.
        """
        await set_locale(request)  # Set the language for the request
        response = await call_next(
            request
        )  # Proceed to the next middleware or route handler
        return response

4. Update API

In this step, we update our FastAPI application to use i18n.

We add the custom LanguageMiddleware to the application to handle setting the language for each request based on the Accept-Language header.

Additionally, we update the /greetings endpoint to return a greeting message translated into the user's preferred language using the _ function from the i18n module.

from fastapi import FastAPI
from pydantic import BaseModel
from i18n import _
from middleware import LanguageMiddleware


app = FastAPI()

# add language middleware to FastAPI
app.add_middleware(LanguageMiddleware)


class GreetingResponse(BaseModel):
    message: str


@app.get("/greetings", response_model=GreetingResponse)
async def greetings():
    # i18n _(str) function to mark this string as translatable
    return {"message": _("Hello, this is a message in your language!")}


# To run the server, save this code to a file (e.g., app.py) and run:
# uvicorn app:app --reload

5. Setup Translation Files

Finally, create the translation folder and extract all the translatable strings using the following pybabel command:

mkdir translations
pybabel extract -o ./translations/messages.pot .

This will generate a translation catalog like this:

# Translations template for PROJECT.
# Copyright (C) 2024 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2024.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2024-06-19 13:50+0330\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.14.0\n"

#: main.py:19
msgid "Hello, this is a message in your language!"
msgstr ""

Next, create the .po files for the languages you want to support. Here are the commands to create translation files for English, German, and French:

pybabel init -i translations/messages.pot -d translations -l en
pybabel init -i translations/messages.pot -d translations -l fr
pybabel init -i translations/messages.pot -d translations -l de

Now you need to translate the strings in each language. Here is an example for the French:

# French translations for PROJECT.
# Copyright (C) 2024 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2024.
#
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2024-06-19 13:50+0330\n"
"PO-Revision-Date: 2024-06-19 14:11+0330\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language: fr\n"
"Language-Team: fr <LL@li.org>\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.14.0\n"

#: main.py:19
msgid "Hello, this is a message in your language!"
msgstr "Bonjour, ceci est un message dans votre langue!"

Then compile the .po files into .mo files. This will compile translation catalogs into binary MO files.

pybabel compile -d translations

And voila! 🎉

See it in Action

Now, you can use the Accept-Language header to choose the language of your API response. Run the FastAPI app and use cURL to send requests with different Accept-Language headers.

English 🏴󠁧󠁢󠁥󠁮󠁧󠁿 🇺🇸

~$ curl -X 'GET'   'http://127.0.0.1:8000/greetings' \
-H 'Accept-Language: en'

{"message":"Hello, this is a message in your language!"}

German 🇩🇪

~$ curl -X 'GET'   'http://127.0.0.1:8000/greetings' \
-H 'Accept-Language: de'

{"message":"Hallo, dies ist eine Nachricht in Ihrer Sprache!"}

French 🇫🇷

~$ curl -X 'GET'   'http://127.0.0.1:8000/greetings' \
-H 'Accept-Language: fr'

{"message":"Bonjour, ceci est un message dans votre langue!"}

Conclusion

Internationalizing your FastAPI application makes it more accessible and user-friendly for a global audience.

Following these steps, you can easily support multiple languages, enhancing the user experience and broadening your application’s reach.

Implementing i18n ensures that your API can respond in the user’s preferred language, making your service more inclusive and adaptable to diverse user needs.

Read More 📜

• How to Structure Your FastAPI Projects
• How to Setup a FastAPI Project using Conda and Poetry
• How to Setup Automatic Versioning for your FastAPI App

Resources

• Command-Line Interface - Babel 2.14.0 documentation
• gettext - Multilingual internationalization services
• Middleware - FastAPI

Part 3: Versioning

In this part, I explain why versioning is crucial for FastAPI applications and how to automate it with semantic versioning, Git commit messages, and the python-semantic-release package.

By the end, you’ll have an automatic versioning system for your FastAPI projects.

How Does Versioning Help Us?

1. Avoid Confusion: Versioning helps keep track of changes and avoids confusion about which version is being used.
2. Maintain Compatibility: Ensures new updates work with older versions, preventing issues for users.
3. Easier Debugging: Knowing the exact version makes it easier to identify and fix bugs.
4. Clear Communication: Clearly communicates what changes have been made to other developers and users.
5. Smooth Updates: Facilitates smooth and predictable updates without breaking the application.

Understanding Semantic Versioning

Semantic Versioning (SemVer) is the most famous versioning standard. it uses a three-part version number: MAJOR.MINOR.PATCH.

1. MAJOR version increments when you make incompatible API changes.
2. MINOR version increments when you add functionality in a backward-compatible manner.
3. PATCH version increments when you make backward-compatible bug fixes.

To learn more about SemVer read the following article or refer to the original docs.

Software Upgrade Versioning: Semantic Versioning

Setting Up Automatic Version Bumping

There are a couple of versioning packages out there. semantic-release is one of the most famous and popular tools for versioning in the Javascript community.

We are going to use thepython-semantic-release which is the Python implementation of semantic-releasepackage.

We Integrate it with out FastAPI project using the three simple steps:

1. Install python-semantic-release

first, install python-semantic-release in your environment.

Use poetry to add python-semantic-release to dev dependencies or the default pip in your activated environment.

(proj-env) user@host:~$ poerty add -G dev python-semantic-release

(proj-env) user@host:~$ pip install python-semantic-release

2. Generate Configuration

Generate the configuration for your project and append it to the pyproject.toml using the following command.

semantic-release generate-config --pyproject >> pyproject.toml

Here is the default config that semantic-release generates:

[tool.semantic_release]
assets = []
build_command_env = []
commit_message = "{version}\n\nAutomatically generated by python-semantic-release"
commit_parser = "angular"
logging_use_named_masks = false
major_on_zero = true
allow_zero_version = true
no_git_verify = false
tag_format = "v{version}"
version_variables = ["src/app/version.py:__version__"] # we added this

[tool.semantic_release.branches.main]
match = "(main|master)"
prerelease_token = "rc"
prerelease = false

[tool.semantic_release.changelog]
template_dir = "templates"
changelog_file = "CHANGELOG.md"
exclude_commit_patterns = []

[tool.semantic_release.changelog.environment]
block_start_string = "{%"
block_end_string = "%}"
variable_start_string = "{{"
variable_end_string = "}}"
comment_start_string = "{#"
comment_end_string = "#}"
trim_blocks = false
lstrip_blocks = false
newline_sequence = "\n"
keep_trailing_newline = false
extensions = []
autoescape = true

[tool.semantic_release.commit_author]
env = "GIT_COMMIT_AUTHOR"
default = "semantic-release <semantic-release>"

[tool.semantic_release.commit_parser_options]
allowed_tags = ["build", "chore", "ci", "docs", "feat", "fix", "perf", "style", "refactor", "test"]
minor_tags = ["feat"]
patch_tags = ["fix", "perf"]
default_bump_level = 0

[tool.semantic_release.remote]
name = "origin"
type = "github"
ignore_token_for_push = false
insecure = false

[tool.semantic_release.publish]
dist_glob_patterns = ["dist/*"]
upload_to_vcs_release = true

3. Add the Version File

Create a Python file to store your version information. Place it in the root folder of your source:

# /src/app/version.py

"""
Version Information

This module provides the code version and is 
automatically updated by the python-semantic-release package.

DO NOT EDIT IT MANUALLY.
"""

__version__ = 0.1.0

Next, update your pyproject.toml to let semantic-release know how to use this file:

[tool.semantic_release]
# ... other configs
version_variables = ["src/app/version.py:__version__"]

And voila! 🎉

Now you can automatically bump your version using the format of your Git Commit Messages.

Automatic Versioning Using Git Commit Messages

python-semantic-release rely on commit messages to detect how to bump to the next version. This ensures versioning is consistent and based on the type of changes made.

By default, it uses the Angular style.

Commit messages should follow a precise format to maintain readability and generate changelogs.

Commit Message Format

• Header: Includes type, scope (optional), and subject.
• Body: Details about the change.
• Footer: Notes on breaking changes or issue references.

<type>(<scope>): <subject>
<BLANK LINE>
<body>
<BLANK LINE>
<footer>

Types

Commit message types determine the next version of your app. For example, the feat type increases the MINOR version number, while the fix type increases the PATCH number. Here are the standard types:

• feat: New feature
• fix: Bug fix
• docs: Documentation changes
• style: Code style changes
• refactor: Code changes without fixing bugs or adding features
• perf: Performance improvements
• test: Testing changes
• chore: Build process or auxiliary tool changes

You can also define custom types in the allowed_tags list in your pyproject.toml file.

Let’s Version

Now, let’s see it in action. Add your changes and commit with the standard format:

(proj-env) user@host:~$ git commit -m "feat: add automatic versioning"

Using semantic-release, you can automatically bump your version based on your commit message. This dry-run command shows the next version without making any changes. Our next version is v0.2.0

(proj-env) user@host:~$ semantic-release --noop version --print
🛡 You are running in no-operation mode, because the '--noop' flag 
was supplied
0.2.0

To create the next version, run:

(proj-env) user@host:~$ semantic-release version
0.2.0
The next version is: 0.2.0! 🚀

semantic-release will update the version file, commit the changes, tag the commit, and push everything to your remote server.

Now you can also add this version to your FastAPI app:

from app.version import __version__

app = FastAPI(
    version=__version__,
)

Automate Using GitHub Actions

You can also set up your CI/CD pipeline to use semantic-release for automatic versioning after each successful merge into the main branch.

Create the Workflow File

Save the following YAML configuration as .github/workflows/version-bump.yml in your repository.

name: Bump Version

on:
  push:
    branches:
      - main

jobs:
  bump-version:
    runs-on: ubuntu-latest
    
    permissions:
        actions: write
        contents: write
  
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
            fetch-depth: 0
      
      - name: Bump version using semantic-release
        uses: python-semantic-release/python-semantic-release@master
        with:
            github_token: ${{ secrets.GITHUB_TOKEN }}

Now every time you push or merge other branches into the main branch, It will bump your code version depending on the commit messages, tag it, and make a release for that version in GitHub.

Conclusion

Versioning is essential for clarity and control in FastAPI projects. Automating it using semantic versioning, Git commit messages, and python-semantic-release improves clarity and maintainability.

By following this guide, you'll have an efficient versioning system for your FastAPI applications, ensuring smooth updates, easier debugging, and clear communication with your team. Automate your versioning to keep your FastAPI projects organized and future-proof.

Read More 📜

• How to Setup a FastAPI Project using Conda and Poetry
• How to Structure Your FastAPI Projects

Resources

• Semantic Versioning 2.0.0
• Introduction
• GitHub - python-semantic-release/python-semantic-release: Automatic semantic versioning for python projects
• Workflow syntax for GitHub Actions - GitHub Docs

Part 2: Environment and Dependencies

In this part, we’ll explore tools and approaches for two crucial parts of a robust project setup:

1. Dependency Management: It’s about handling the libraries and versions needed for your project to function correctly.
2. Environment Configuration: This involves setting up the environment where your project runs, and managing things like Python versions and system dependencies.

Let’s go over each one quickly.

Dependency Management Approaches

Dependency management is the process of ensuring that the required libraries and versions are handled correctly for the project to work.

There are two main approaches.

Using old-fashioned requirements.txt or using the more modern pyproject.toml .

Traditional Approach: requirements.txt

This approach has been tried and tested, offering a straightforward way to list project dependencies.

However, it has limitations like not separating different types of dependencies (such as development and test dependencies) and struggling with complex dependency constraints.

Another shortcoming is that you have to manually check and update the file to include new dependencies and their versions.

Modern Approach: pyproject.toml

The pyproject.toml file serves as a central configuration hub for project-related tools.

It simplifies setup by consolidating all project configurations into one file. This includes building systems, package metadata, dependencies, and tool configurations like testing and linting settings.

Pyproject.toml offers several advantages over requirements.txt. It allows for a comprehensive specification of dependencies, including version constraints and metadata.

Additionally, it supports the separation of different dependency types, such as development and test dependencies, for better organization.

Also, tools like Poetry can automatically add and resolve dependencies and sub-dependencies versions, making management easier and ensuring compatibility with the latest library versions.

For dependency management, we choose to use pyproject.toml approach with Poetry.

Writing your pyproject.toml - Python Packaging User Guide

Environment Configuration

Environment management is crucial to ensure consistency and reproducibility across different systems.

By setting up a virtual environment, we isolate project dependencies from system dependencies, preventing conflicts and ensuring that the project runs smoothly regardless of the underlying system configuration.

Here are three main ways to manage virtual environments in Python.

1. Virtualenv: Widely used for isolated Python environments, Virtualenv allows developers to manage project dependencies separately from the system-wide Python installation.
2. Pyenv: This tool enables users to manage multiple Python versions on one system, offering flexibility for switching between versions based on project needs.
3. Conda: Conda stands out as a powerful environment and package management tool that supports multiple programming languages. It not only creates isolated environments but also simplifies the installation of both Python and non-Python packages, making it an ideal choice for projects with complex dependencies.

We choose Conda for its versatility and ease of use.

Project Setup

We choose Poetry for our dependency management and Conda for our virtual environment.

Note that Poetry can also handle virtual environments using virtualenv. however it might not be its strong suit.

Install Conda and Poetry

First, install Conda and Poetry on your system.

Install Conda on Ubuntu. Refer to the Miniconda documentation for more details.

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh

Install Poetry on Ubuntu. Use the preferred pipx method. Refer to Poetry documentation for more details.

pipx install poetry

As Poetry documents point out multiple time, do not install Poetry in your project enviornment. Instead install it using pipx that handles Poetry’s own dependencies in isolation from your project.

Create Conda Environment

Create your Conda environment and activate it. You can specify the Python version you like for your project.

conda create --name proj-env python=3.11
conda activate proj-env

Setup Poetry

Set up the Poetry project in the activated environment.

Poetry will detect that it is being created in an environment and use the activated environment.

(proj-env) user@host:~$ poetry new poetry-conda-demo

This command will create a project in the poetry-conda-demo folder and set up the minimal following file structure.

.
├── poetry_conda_demo
│   └── __init__.py
├── pyproject.toml
├── README.md
└── tests
    └── __init__.py

And here is the pyproject.toml for our project.

[tool.poetry]
name = "poetry-conda-demo"
version = "0.1.0"
description = ""
authors = ["Amir Lavasani <amirm.lavasani@gmail.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Now we can manage our dependencies using Poetry commands. Adding FastAPI to the project is as simple as this.

(proj-env) user@host:~$ poetry add fastapi

Poetry will install fastapi and all its sub-dependencies in the activated porj-env environment.

It will also automatically update the dependencies section of pyproject.toml .

It also creates a poetry.lock file. The poetry.lock file records the exact versions of dependencies and their sub-dependencies that are installed, ensuring consistent and reproducible environments across different systems.

Finally

In this article, we explored the essential steps for setting up a robust FastAPI backend. We started by reviewing different methods for dependency management, highlighting the pros and cons of each.

We selected pyproject.toml with Poetry for its comprehensive features and ease of use.

Next, we examined popular environment management tools, ultimately choosing Conda for its versatility and simplicity.

Finally, we provided a step-by-step guide to setting up the project, including installing Conda and Poetry, and creating a project using these best practices.

By following these guidelines, you can ensure a well-organized and maintainable FastAPI backend setup.

Read More 📜

How to Structure Your FastAPI Projects

Resources

• Introduction
• Miniconda - Anaconda documentation

Part 1: Blueprint

In this article, we will discuss two main strategies for structuring your FastAPI projects and highlight the scenarios in which each one is more suitable.

Why Proper Structuring Matters?

Here are the most important reasons to structure your code based on best practices.

1. Enhanced Scalability: Well-organized code grows smoothly with your FastAPI project.
2. Improved Maintainability: Easy-to-understand code makes updates and fixes simple.
3. Streamlined Collaboration: Clear structure helps teams work together efficiently.

Key Principles for Structuring Projects

In building FastAPI applications, it’s crucial to follow these best practices:

1. Separation of Concerns: Separate different aspects of your FastAPI project, such as routes, models, and business logic, for clarity and maintainability.
2. Modularization: Break down your FastAPI application into reusable modules to promote code reusability and organization.
3. Dependency Injection: Decouple components by implementing dependency injection, which allows for more flexible and testable code in FastAPI using libraries like fastapi.Dependencies.
4. Testability: Prioritize writing testable code in FastAPI applications by employing strategies like dependency injection and mocking to ensure robust testing and quality assurance.

Structuring Formats

FastAPI applications can be structured in different ways to accommodate various project needs.

There are two main approaches for structuring projects. One is based on file type and the other is based on module functionality.

1. Structuring based on File-Type

In this approach, files are organized by type (e.g., API, CRUD, models, schemas, routers) as represented by FastAPI itself.

This structure is more suitable for microservices or projects with fewer scopes:

.
├── app  # Contains the main application files.
│   ├── __init__.py   # this file makes "app" a "Python package"
│   ├── main.py       # Initializes the FastAPI application.
│   ├── dependencies.py # Defines dependencies used by the routers
│   ├── routers
│   │   ├── __init__.py
│   │   ├── items.py  # Defines routes and endpoints related to items.
│   │   └── users.py  # Defines routes and endpoints related to users.
│   ├── crud
│   │   ├── __init__.py
│   │   ├── item.py  # Defines CRUD operations for items.
│   │   └── user.py  # Defines CRUD operations for users.
│   ├── schemas
│   │   ├── __init__.py
│   │   ├── item.py  # Defines schemas for items.
│   │   └── user.py  # Defines schemas for users.
│   ├── models
│   │   ├── __init__.py
│   │   ├── item.py  # Defines database models for items.
│   │   └── user.py  # Defines database models for users.
│   ├── external_services
│   │   ├── __init__.py
│   │   ├── email.py          # Defines functions for sending emails.
│   │   └── notification.py   # Defines functions for sending notifications
│   └── utils
│       ├── __init__.py
│       ├── authentication.py  # Defines functions for authentication.
│       └── validation.py      # Defines functions for validation.
├── tests
│   ├── __init__.py
│   ├── test_main.py
│   ├── test_items.py  # Tests for the items module.
│   └── test_users.py  # Tests for the users module.
├── requirements.txt
├── .gitignore
└── README.md

In this structure

• app/: Contains the main application files.
• main.py: Initializes the FastAPI application.
• dependencies.py: Defines dependencies used by the routers.
• routers/: Contains router modules.
• crud/: Contains CRUD (Create, Read, Update, Delete) operation modules.
• schemas/: Contains Pydantic schema modules.
• models/: Contains database model modules.
• external_services/: Contains modules for interacting with external services.
• utils/: Contains utility modules.
• tests/: Contains test modules.

2. Structuring based on Module-Functionality

In the second approach, we organize our files based on package functionality for example authentication sub-package, users sub-package, and posts sub-package.

The module-functionality structure is better suited for monolithic projects containing numerous domains and modules. By grouping all file types required by a single sub-package, it improves development efficiency.

This structure is suggested by the fastapi-best-practices GitHub repository.

In this structure, Each package has its own router, schemas, models, etc.

fastapi-project
├── alembic/
├── src
│   ├── auth
│   │   ├── router.py         # auth main router with all the endpoints
│   │   ├── schemas.py        # pydantic models
│   │   ├── models.py         # database models
│   │   ├── dependencies.py   # router dependencies
│   │   ├── config.py         # local configs
│   │   ├── constants.py      # module-specific constants
│   │   ├── exceptions.py     # module-specific errors
│   │   ├── service.py        # module-specific business logic
│   │   └── utils.py          # any other non-business logic functions
│   ├── aws
│   │   ├── client.py  # client model for external service communication
│   │   ├── schemas.py
│   │   ├── config.py
│   │   ├── constants.py
│   │   ├── exceptions.py
│   │   └── utils.py
│   └── posts
│   │   ├── router.py
│   │   ├── schemas.py
│   │   ├── models.py
│   │   ├── dependencies.py
│   │   ├── constants.py
│   │   ├── exceptions.py
│   │   ├── service.py
│   │   └── utils.py
│   ├── config.py      # global configs
│   ├── models.py      # global database models
│   ├── exceptions.py  # global exceptions
│   ├── pagination.py  # global module e.g. pagination
│   ├── database.py    # db connection related stuff
│   └── main.py
├── tests/
│   ├── auth
│   ├── aws
│   └── posts
├── templates/
│   └── index.html
├── requirements
│   ├── base.txt
│   ├── dev.txt
│   └── prod.txt
├── .env
├── .gitignore
├── logging.ini
└── alembic.ini

In this structure

Store all domain directories inside src folder.

1. src/: The highest level of an app, contains common models, configs, and constants, etc.
2. src/main.py: Root of the project, which inits the FastAPI app

Each package has its own router, schemas, models, etc.

1. router.py: is the core of each module with all the endpoints
2. schemas.py: for pydantic models
3. models.py: for database models
4. service.py: module-specific business logic
5. dependencies.py: router dependencies
6. constants.py: module-specific constants and error codes
7. config.py: e.g. env vars
8. utils.py: non-business logic functions, e.g. response normalization, data enrichment, etc.
9. exceptions.py: module-specific exceptions, e.g. PostNotFound, InvalidUserData

When a package requires services or dependencies or constants from other packages, import them with an explicit module name to avoid ambiguity.

from src.auth import constants as auth_constants
from src.notifications import service as notification_service
from src.posts.constants import ErrorCode as PostsErrorCode  # in case we have Standard ErrorCode in constants module of each package

Finally

In conclusion, structuring your FastAPI projects is essential for maintaining scalability, readability, and maintainability as your application grows. By organizing your code effectively, you ensure that it remains manageable and adaptable to evolving requirements.

We discussed two main types of structuring FastAPI projects: the file-type structure and the module-functionality structure.

The file-type structure, represented by FastAPI itself, organizes files based on their type (e.g., routers, schemas, models). This structure is suitable for microservice architectures where individual services have single responsibilities.

On the other hand, the module-functionality structure separates files based on module functionality rather than file type. This approach is more suitable for larger monolithic applications, promoting better organization and maintainability.

In choosing the right structure for your FastAPI project, consider factors such as project size, complexity, and architectural design. Ultimately, whether you opt for the file-type structure or the module functionality structure, prioritizing organization and clarity will contribute to the long-term success of your FastAPI applications.

Resources

• Bigger Applications - Multiple Files - FastAPI
• GitHub - zhanymkanov/fastapi-best-practices: FastAPI Best Practices and Conventions we used at our startup

Linking Abstraction and Implementation

Have you encountered recurring coding challenges? Imagine having a toolbox of tried-and-true solutions readily available. That’s precisely what design patterns provide. In this series, we’ll explore what these patterns are and how they can elevate your coding skills.

Design Patterns in Python: A Series

Understanding the Bridge Pattern

What is the Bridge Design Pattern?

The Bridge Design Pattern is a structural design pattern that decouples an abstraction from its implementation, allowing them to vary independently. It achieves this by providing a bridge between the abstraction and its implementation, enabling changes to one without affecting the other.

When to Use the Bridge Pattern:

When considering the use of the Bridge Design Pattern:

1. Divide and organize a monolithic class with multiple variants: If a class handles various functionalities, such as working with different database servers, and you want to avoid a monolithic structure.
2. Extend a class in orthogonal dimensions: When you need to extend a class in multiple independent dimensions, the Bridge Pattern suggests creating separate class hierarchies for each dimension.
3. Switch implementations at runtime: If you need the flexibility to replace implementation objects within the abstraction dynamically, the Bridge Pattern allows for easy implementation swapping.

Practical Example: File Storage Abstraction

This example demonstrates the Bridge Design Pattern by creating a file storage abstraction with implementations for local, cloud, and network storage.

It shows how the pattern separates storage abstraction from specific implementations for flexibility and maintenance.

Abstraction and Implementation

In the Bridge Design Pattern context, the terms Abstraction and Implementation refer to fundamental concepts that facilitate the separation of concerns and promote code maintainability.

1. Abstraction serves as the high-level control layer, delegating work to the Implementation layer. Abstraction, like a graphical user interface (GUI), interacts with the underlying operating system code (API).
2. Implementation, or the platform, contains platform-specific code and is declared through a common interface. This allows interchangeable implementations linked to the abstraction object.

By splitting classes into Abstraction and Implementation hierarchies, the Bridge pattern simplifies code maintenance and promotes flexibility.

In the above example, GUI changes won’t affect API-related classes, and adding support for new operating systems in API requires minimal modifications.

Terminology and Key Components

Here are the fundamental components involved:

1. Abstraction: Provides high-level control logic and relies on the implementation object for low-level work.
2. Implementation: Declares the interface common to all concrete implementations, allowing communication with the abstraction through declared methods.
3. Concrete Implementations: Contain platform-specific code and provide implementations for the methods declared in the implementation interface.
4. Refined Abstractions: Variants of control logic that work with different implementations via the general implementation interface.
5. Client: Primarily interacts with the abstraction, responsible for linking the abstraction object with one of the implementation objects.

Understanding how these components interact is crucial for effectively utilizing the Bridge Pattern in software design.

Bridge Implementation in Python

Step 1: Define Abstraction

Abstract class representing the high-level control layer.

# Step 1: Define Abstraction (Abstract class)
class Abstraction(ABC):
    """Abstract class representing the high-level control layer."""
    
    def __init__(self, implementation):
        self._implementation = implementation

    @abstractmethod
    def perform_action(self):
        """Perform an action using the implementation."""
        pass

Step 2: Define Implementation

Abstract class representing the platform-specific code.

# Step 2: Define Implementation (Abstract class)
class Implementation(ABC):
    """Abstract class representing the platform-specific code."""
    
    @abstractmethod
    def action_impl(self):
        """Perform the actual action."""
        pass

Step 3: Create Concrete Implementations

Concrete implementation for platforms A and B.

# Step 3: Create Concrete Implementations
class ConcreteImplementationA(Implementation):
    """Concrete implementation for platform A."""
    
    def action_impl(self):
        return "Action performed by Implementation A"

class ConcreteImplementationB(Implementation):
    """Concrete implementation for platform B."""
    
    def action_impl(self):
        return "Action performed by Implementation B"

Step 4: Create Refined Abstractions

Refined abstraction provides variants of control logic.

# Step 4: Create Refined Abstractions
class RefinedAbstraction(Abstraction):
    """Refined abstraction providing variants of control logic."""
    
    def perform_action(self):
        """Perform an action using the implementation."""
        return self._implementation.action_impl()

Main Client Code

The main section showcases usage.

# Main section to showcase usage
if __name__ == "__main__":
    # Create concrete implementations
    implementation_a = ConcreteImplementationA()
    implementation_b = ConcreteImplementationB()

    # Create refined abstractions and link them with concrete implementations
    refined_abstraction_a = RefinedAbstraction(implementation_a)
    refined_abstraction_b = RefinedAbstraction(implementation_b)

    # Use refined abstractions to perform actions
    print(refined_abstraction_a.perform_action())  # Output: Action performed by Implementation A
    print(refined_abstraction_b.perform_action())  # Output: Action performed by Implementation B

GitHub Repo 🎉

Explore all code examples and design pattern implementations on GitHub!

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

Practical Example: File Storage

Step 1: Define Abstraction

Define an abstract class representing the file storage abstraction. This class will declare methods for saving files.

from abc import ABC, abstractmethod

# Step 1: Define Abstraction (Abstract class)
class FileStorage(ABC):
    """Abstract class representing the file storage abstraction."""
    
    @abstractmethod
    def save_file(self, file_name):
        """Abstract method to save a file."""
        pass

Step 2: Define Implementation

Define an abstract class representing the storage implementation. This class will declare methods for saving files specific to each storage location.

# Step 2: Define Implementation (Abstract class)
class StorageImplementation(ABC):
    """Abstract class representing the storage implementation."""
    
    @abstractmethod
    def save(self, file_name):
        """Abstract method to save a file."""
        pass

Step 3: Create Concrete Implementations

Implement concrete classes for local storage, cloud storage, and network storage. Each class will provide a specific implementation for saving files.

# Step 3: Create Concrete Implementations
class LocalStorage(StorageImplementation):
    """Concrete implementation for local file storage."""
    
    def save(self, file_name):
        """Save a file locally."""
        return f"File '{file_name}' saved locally"

class CloudStorage(StorageImplementation):
    """Concrete implementation for cloud file storage."""
    
    def save(self, file_name):
        """Save a file to the cloud."""
        return f"File '{file_name}' saved to the cloud"

class NetworkStorage(StorageImplementation):
    """Concrete implementation for network file storage."""
    
    def save(self, file_name):
        """Save a file to a network location."""
        return f"File '{file_name}' saved to a network location"

Step 4: Create Refined Abstractions

Create a refined abstraction class that extends the file storage abstraction. This class will delegate file storage operations to the appropriate storage implementation based on runtime configurations.

# Step 4: Create Refined Abstraction
class AdvancedFileStorage(FileStorage):
    """Refined abstraction for advanced file storage."""
    
    def __init__(self, storage_impl):
        """Initialize with a specific storage implementation."""
        self._storage_impl = storage_impl
    
    def save_file(self, file_name):
        """Save a file using the specified storage implementation."""
        return self._storage_impl.save(file_name)

Main Client Code

# Main section to showcase usage
if __name__ == "__main__":
    # Create concrete implementations
    local_storage = LocalStorage()
    cloud_storage = CloudStorage()
    network_storage = NetworkStorage()

    # Create refined abstractions and link them with concrete implementations
    advanced_local_storage = AdvancedFileStorage(local_storage)
    advanced_cloud_storage = AdvancedFileStorage(cloud_storage)
    advanced_network_storage = AdvancedFileStorage(network_storage)

    # Use refined abstractions to save files
    print(advanced_local_storage.save_file("example.txt"))    # Output: File 'example.txt' saved locally
    print(advanced_cloud_storage.save_file("example.txt"))    # Output: File 'example.txt' saved to the cloud
    print(advanced_network_storage.save_file("example.txt"))  # Output: File 'example.txt' saved to a network location

Real-World Use Cases for Bridge

1. Python’s Tkinter and PyQT: Similar to Java GUI frameworks, Python’s GUI libraries employ the Bridge pattern to separate the GUI components from platform-specific details.
2. OpenGL and DirectX: Graphics rendering libraries like OpenGL and DirectX utilize the Bridge pattern to abstract away hardware-specific implementations, allowing developers to create cross-platform graphics applications.
3. jQuery: The jQuery JavaScript library employs the Bridge pattern to abstract away browser differences and provide a consistent API for DOM manipulation and event handling across different web browsers.
4. TensorFlow and PyTorch: Deep learning frameworks like TensorFlow and PyTorch use the Bridge pattern to separate the high-level neural network abstraction from specific hardware accelerators and optimization techniques, enabling scalable and efficient deep learning computations.
5. Quartz Scheduler: The Quartz Scheduler library utilizes the Bridge pattern to decouple the scheduling logic from specific job store implementations, allowing developers to store job data in various databases or other storage systems.

Bridge Pors and Cons

When working with the Bridge Design Pattern, it’s essential to understand its advantages and potential drawbacks:

Pros

1. Platform Independence: Bridge allows the creation of platform-independent classes and applications, enhancing portability.
2. Abstraction: Client code interacts with high-level abstractions, shielding it from platform-specific details, thus promoting clarity.
3. Open/Closed Principle: Bridge supports the principle of open/closed, enabling the introduction of new abstractions and implementations independently.
4. Single Responsibility Principle: Bridge encourages separation of concerns, allowing focus on high-level logic in abstractions and platform details in implementations.

Cons

Increased Complexity: Overuse of the pattern in highly cohesive classes can lead to code complexity, making it harder to understand and maintain.

Bridge’s Relations with Other Patterns

When looking at the Bridge Design Pattern, it’s important to know how it connects with other patterns.

Bridge vs. Adapter:

Bridge is typically designed up-front, allowing the development of parts of an application independently.

Conversely, the Adapter is commonly used with existing applications to reconcile otherwise-incompatible classes.

Bridge, State, Strategy, and Adapter:

These patterns share similar structures, based on composition and delegating work to other objects.

However, they address distinct problems and communicate different solutions to developers.

Bridge with Abstract Factory:

Abstract Factory complements Bridge when specific abstractions defined by Bridge require particular implementations.

Abstract Factory encapsulates these relationships, simplifying the complexity of client code.

Bridge with Builder:

Builder can be combined with Bridge, where the director class acts as the abstraction and different builders serve as implementations.

This combination allows for the creation of complex objects while maintaining the separation of concerns.

Conclusion

In conclusion, we’ve delved into the Bridge Pattern in Python, discovering its ability to separate abstraction from implementation, offering flexibility and maintenance advantages.

We’ve explored its key components, pros and cons, and compared it to related patterns. Additionally, we implemented a practical file storage system showcasing the pattern’s power.

Keep coding and exploring new design patterns! 👩‍💻

Next on the Series 🚀

Design Patterns in Python: Flyweight

Read More 📜

• Design Patterns in Python: Template Method
• Design Patterns in Python: Visitor

The Series 🧭

Design Patterns in Python: A Series

Explore the GitHub Repo 🎉

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

References

1. Design Patterns: Elements of Reusable Object-Oriented Software (Book)
2. refactoring.guru Bridge
3. Head First Design Patterns (Book)
4. Wikipedia Bridge Pattern
5. Sourcemaking Bridge Design Pattern
6. Bridge Design Pattern — A different kind of Golden Gate

Optimizing Memory Usage

Have you encountered recurring coding challenges? Imagine having a toolbox of tried-and-true solutions readily available. That’s precisely what design patterns provide. In this series, we’ll explore what these patterns are and how they can elevate your coding skills.

Design Patterns in Python: A Series

Understanding the Flyweight Pattern

What is the Flyweight Design Pattern?

The Flyweight Design Pattern is categorized under structural design patterns and is commonly used to manage objects that share similar or identical states.

The key idea behind the Flyweight pattern is to separate the intrinsic state (shared among multiple objects) from the extrinsic state (unique to each object).

By doing so, we can store the intrinsic state in a centralized location and share it among multiple objects, while the extrinsic state can be managed individually by each object.

Intrinsic vs. Extrinsic State

1. Intrinsic State: This refers to constant data stored within an object, residing solely within the object itself and immutable by external entities. It remains consistent across different contexts and is shared among multiple objects.
2. Extrinsic State: Conversely, the extrinsic state represents the variable data of an object, influenced or altered by external factors or objects. This state is dynamic and can vary between different instances of the same object.

The Flyweight pattern suggests separating extrinsic state from objects, preserving the intrinsic state for reuse across contexts. This minimizes object instances, as they mainly differ in intrinsic state, simplifying the system.

When to Use the Flyweight Pattern:

The Flyweight Pattern is suitable for the following scenarios:

1. Large Number of Objects: When your application needs to create a large number of objects with shared intrinsic state.
2. Memory Optimization: If memory optimization is a concern and you want to reduce the memory footprint by sharing common state among multiple objects.
3. Immutable State: When the objects can be made immutable or the shared state can be safely shared among multiple instances without risk of modification.
4. Performance Improvement: If performance improvement is desired by minimizing object creation and reducing redundant data storage.
5. Extrinsic State Separation: When it’s feasible to separate the intrinsic state (shared among objects) from the extrinsic state (unique to each object) to optimize memory usage.

Practical Example: Game Enemy Character

We use the Flyweight pattern to optimize the management of game enemy characters. We define a Flyweight class for shared intrinsic data, create Concrete Flyweight classes for specific enemy types, and implement a Flyweight Factory for object management.

By separating intrinsic and extrinsic data, we optimize memory usage while allowing for individual customization during gameplay. This approach efficiently handles large numbers of characters, enhancing performance and scalability in game development.

Terminology and Key Components

Understanding the key components of the Flyweight pattern is essential for its effective implementation. Here are the crucial components involved:

1. Flyweight: Contains the intrinsic state shared between multiple objects, with the same flyweight object usable in various contexts. It stores the intrinsic state and receives the extrinsic state from the context.
2. Context: Holds the extrinsic state unique across all original objects. When paired with a flyweight object, it represents the full state of the original object.
3. Client: Calculates or stores the extrinsic state of flyweights. It treats flyweights as template objects and configures them at runtime by passing contextual data into their methods.
4. Flyweight Factory: Manages a pool of existing flyweights, handling their creation and reuse. Clients interact with the factory to obtain flyweight instances, passing intrinsic state for retrieval or creation.

Flyweight Implementation in Python

This implementation demonstrates the abstract version of the Flyweight design pattern, including the Flyweight interface, concrete flyweight classes, flyweight factory, and client class.

Step 1: Define the Flyweight interface

Define the Flyweight interface with an operation method.

from abc import ABC, abstractmethod

# Step 1: Define the Flyweight interface
class Flyweight(ABC):
    """
    The Flyweight interface declares a method for accepting extrinsic state
    and performing operations based on it.
    """

    @abstractmethod
    def operation(self, extrinsic_state):
        """
        Operation method accepting extrinsic state as input.
        """
        pass

Step 2: Create concrete flyweight classes

Create concrete flyweight classes implementing the interface and storing intrinsic state.

# Step 2: Create concrete flyweight classes
class ConcreteFlyweight(Flyweight):
    """
    ConcreteFlyweight implements the Flyweight interface and stores intrinsic state.
    """

    def __init__(self, intrinsic_state):
        self._intrinsic_state = intrinsic_state

    def operation(self, extrinsic_state):
        return f"ConcreteFlyweight: Intrinsic State - {self._intrinsic_state}, Extrinsic State - {extrinsic_state}"

Step 3: Implement the Flyweight Factory

Implement a Flyweight Factory to manage flyweight objects and ensure their uniqueness.

# Step 3: Implement the Flyweight Factory
class FlyweightFactory:
    """
    FlyweightFactory manages flyweight objects and ensures their uniqueness.
    """

    _flyweights = {}

    @staticmethod
    def get_flyweight(key):
        """
        Retrieve or create a flyweight object based on the provided key.
        """
        if key not in FlyweightFactory._flyweights:
            FlyweightFactory._flyweights[key] = ConcreteFlyweight(key)
        return FlyweightFactory._flyweights[key]

Step 4: Define the client class

Define a client class representing objects that use flyweight objects.

# Step 4: Define the client class
class Client:
    """
    Client class represents objects that use flyweight objects.
    """

    def __init__(self, key):
        self._flyweight = FlyweightFactory.get_flyweight(key)

    def operation(self, extrinsic_state):
        """
        Perform an operation using the flyweight object and extrinsic state.
        """
        return self._flyweight.operation(extrinsic_state)

# Example usage
if __name__ == "__main__":
    client1 = Client("shared")
    client2 = Client("shared")
    client3 = Client("unique")

    print(client1.operation("state 1"))  # Output: ConcreteFlyweight: Intrinsic State - shared, Extrinsic State - state 1
    print(client2.operation("state 2"))  # Output: ConcreteFlyweight: Intrinsic State - shared, Extrinsic State - state 2
    print(client3.operation("state 3"))  # Output: ConcreteFlyweight: Intrinsic State - unique, Extrinsic State - state 3

GitHub Repo 🎉

Explore all code examples and design pattern implementations on GitHub!

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

Practical Example: Game Enemy Character

Step 1: Define the Flyweight Interface

Define the abstract EnemyFlyweight interface with a render method.

from abc import ABC, abstractmethod
import random

# Step 1: Define the Flyweight interface
class EnemyFlyweight(ABC):
    """
    Flyweight interface declares a method for accepting extrinsic state
    and performing operations based on it.
    """

    @abstractmethod
    def render(self, position):
        """
        Render method accepting extrinsic state as input.
        """
        pass

Step 2: Create Concrete Flyweight Classes

Create concrete flyweight classes (e.g., EnemyTypeA, EnemyTypeB) implementing the interface and storing intrinsic data like texture.

# Step 2: Create concrete flyweight classes
class EnemyTypeA(EnemyFlyweight):
    """
    ConcreteFlyweight class for enemy type A.
    """

    def __init__(self, texture):
        self._texture = texture

    def render(self, position):
        """
        Render enemy type A with given position.
        """
        print(f"Rendering Enemy Type A at position {position} with texture: {self._texture}")

class EnemyTypeB(EnemyFlyweight):
    """
    ConcreteFlyweight class for enemy type B.
    """

    def __init__(self, texture):
        self._texture = texture

    def render(self, position):
        """
        Render enemy type B with given position.
        """
        print(f"Rendering Enemy Type B at position {position} with texture: {self._texture}")

Step 3: Implement the Flyweight Factory

Implement the EnemyFlyweightFactory to manage flyweight objects and ensure their uniqueness based on texture.

# Step 3: Implement the Flyweight Factory
class EnemyFlyweightFactory:
    """
    FlyweightFactory manages flyweight objects and ensures their uniqueness.
    """

    _flyweights = {}

    @staticmethod
    def get_flyweight(texture):
        """
        Retrieve or create a flyweight object based on the provided texture.
        """
        if texture not in EnemyFlyweightFactory._flyweights:
            if random.randint(0, 1) == 0:
                EnemyFlyweightFactory._flyweights[texture] = EnemyTypeA(texture)
            else:
                EnemyFlyweightFactory._flyweights[texture] = EnemyTypeB(texture)
        return EnemyFlyweightFactory._flyweights[texture]

Step 4: Define the client class

Define the client class (e.g., GameEnvironment) representing objects that use flyweight objects. Instantiate the client class, add enemies with different textures and positions, and Render all enemies in the game environment using their respective flyweight objects.

# Step 4: Define the client class
class GameEnvironment:
    """
    Client class represents objects that use flyweight objects.
    """

    def __init__(self):
        self._enemies = []

    def add_enemy(self, texture, position):
        """
        Add a new enemy to the game environment.
        """
        flyweight = EnemyFlyweightFactory.get_flyweight(texture)
        self._enemies.append((flyweight, position))

    def render_enemies(self):
        """
        Render all enemies in the game environment.
        """
        for flyweight, position in self._enemies:
            flyweight.render(position)

# Example usage
if __name__ == "__main__":
    # Create game environment
    game = GameEnvironment()

    # Add enemies with different textures and positions
    game.add_enemy("texture_a", (10, 20))
    game.add_enemy("texture_b", (30, 40))
    game.add_enemy("texture_a", (50, 60))
    game.add_enemy("texture_b", (70, 80))

    # Render all enemies
    game.render_enemies()

Real-World Use Cases for Flyweight

1. Graphics Processing Units (GPUs): GPU frameworks like OpenGL and DirectX use Flyweight to efficiently manage textures and vertex data.
2. Python Interning: Python caches small integers and certain strings using Flyweight to optimize memory usage.
3. Database Connection Pools: Connection pool implementations in frameworks like Apache DBCP and HikariCP use Flyweight to manage and reuse database connections efficiently.
4. Object Pooling Libraries: Libraries like Apache Commons Pool and ObjectPool use Flyweight to pool and reuse objects, reducing object creation overhead.
5. Caching Libraries: Caching frameworks like Ehcache and Redis use Flyweight to cache objects and reduce memory usage.

Flyweight Best Practices and Potential Drawbacks

Pros:

1. RAM Savings: Significant RAM savings are possible, especially beneficial for programs with numerous similar objects.

Cons:

1. CPU Overhead: Potential trade-off between RAM and CPU cycles, particularly if recalculating context data for flyweight methods.
2. Code Complexity: Increased code complexity, potentially confusing for new team members due to the separation of entity states.

Flyweight’s Relations with Other Patterns

The Flyweight pattern shares similarities and differences with several other design patterns:

Flyweight and Composite

Flyweight can be implemented to represent shared leaf nodes of the Composite tree, saving RAM by reducing the memory footprint of common objects.

Flyweight vs. Facade

While Flyweight focuses on creating many small objects efficiently, Facade concentrates on encapsulating complex subsystems into a single simplified interface.

Flyweight vs. Singleton

Although Flyweight may resemble Singleton if all shared states are reduced to one flyweight object, there are key differences.

Flyweight can have multiple instances with different intrinsic states, while Singleton allows only one instance. Additionally, Singleton objects can be mutable, whereas Flyweight objects are immutable.

Conclusion

In conclusion, we’ve explored the Flyweight pattern, which optimizes memory usage by sharing common data among multiple objects. We discussed its key components, including the Flyweight interface, Concrete Flyweight classes, Flyweight Factory, and Client. Additionally, we highlighted the pros and cons of using the Flyweight pattern.

We compared the Flyweight pattern to other related patterns such as Composite, Facade, and Singleton, illustrating its unique characteristics and applications. Finally, we implemented a practical example of a gaming character using the Flyweight pattern, demonstrating its efficiency in managing large numbers of similar objects.

Keep coding and exploring new design patterns! 👩‍💻

Next on the Series 🚀

Design Patterns in Python: Template Method

Read More 📜

• Design Patterns in Python: Visitor
• Design Patterns in Python: Strategy

The Series 🧭

Design Patterns in Python: A Series

Explore the GitHub Repo 🎉

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

References

1. Design Patterns: Elements of Reusable Object-Oriented Software (Book)
2. refactoring.guru Flyweight
3. Head First Design Patterns (Book)
4. Flyweight Design Pattern in Java
5. The Flyweight Chronicles: How to Make Your Code Weightless!
6. Flyweight Design Pattern
7. Game Programming Patterns Flyweight

Modular Algorithms

Have you encountered recurring coding challenges? Imagine having a toolbox of tried-and-true solutions readily available. That’s precisely what design patterns provide. In this series, we’ll explore what these patterns are and how they can elevate your coding skills.

Design Patterns in Python: A Series

Understanding the Template Method Pattern

What is the Template Method Design Pattern?

The Template Method is a behavioral design pattern that defines the skeleton of an algorithm in the superclass but lets subclasses override specific steps of the algorithm without changing its structure.

Simply put, the Template Method Pattern helps developers create a group of algorithms that share a common structure. It allows for easier management of complexity and encourages a systematic approach when designing related classes.

When to Use the Template Method Pattern:

Use the Template Method pattern when facing the following scenarios:

1. Extend Specific Steps: Use when clients need to extend particular steps of an algorithm without altering its structure.
2. Similar but Not Identical Algorithms: Apply when dealing with multiple classes having almost identical algorithms with slight variations, making modification tedious.
3. Eliminate Code Duplication: Implement when turning a monolithic algorithm into a series of steps, allowing easy extension by subclasses, while consolidating similar implementations to eliminate code duplication.

Practical Example: Web Scraper Template

We will implement a Web Scraper Template to showcase the capabilities of the template method in a practical example.

By encapsulating common steps like sending requests, parsing HTML, and extracting data, this pattern enables a structured and reusable approach to web scraping.

Terminology and Key Components

The template method pattern employs a straightforward class structure. We define an abstract template class with a TemplateMethod and concrete classes that can override specific steps.

1. Abstract Template Class: Declares methods acting as steps in the algorithm and the template method, orchestrating these steps in a specific order. Steps may be either abstract or have default implementations.
2. Concrete Template Classes: Can override all steps but not the template method itself. These classes provide specific implementations for the declared steps, customizing the algorithm.

Template Method Implementation in Python

Step 1: Abstract Class (Template)

The abstract class defines the structure of the algorithm through a template method, orchestrating the sequence of steps while declaring them as abstract methods, and allowing concrete classes to provide specific implementations.

from abc import ABC, abstractmethod

# Step 1: Abstract Class (Template)
class AbstractTemplate(ABC):
    def template_method(self):
        """The template method orchestrating the steps."""
        self.step_one()
        self.step_two()
        self.step_three()

    @abstractmethod
    def step_one(self):
        """Abstract method representing the first step."""
        pass

    @abstractmethod
    def step_two(self):
        """Abstract method representing the second step."""
        pass

    @abstractmethod
    def step_three(self):
        """Abstract method representing the third step."""
        pass

Step 2: Concrete Classes

Concrete classes extend the abstract class, providing concrete implementations for each abstract method, thereby defining the specific steps of the algorithm.

# Step 2: Concrete Class 1
class ConcreteTemplateA(AbstractTemplate):
    def step_one(self):
        """Concrete implementation for the first step in Template A."""
        print("Performing initialization for Template A.")

    def step_two(self):
        """Concrete implementation for the second step in Template A."""
        print("Executing core logic for Template A.")

    def step_three(self):
        """Concrete implementation for the third step in Template A."""
        print("Finalizing and cleaning up for Template A.")

# Step 3: Concrete Class 2
class ConcreteTemplateB(AbstractTemplate):
    def step_one(self):
        """Concrete implementation for the first step in Template B."""
        print("Setting up resources for Template B.")

    def step_two(self):
        """Concrete implementation for the second step in Template B."""
        print("Performing specialized processing for Template B.")

    def step_three(self):
        """Concrete implementation for the third step in Template B."""
        print("Cleaning up connections and resources for Template B.")

Main Section (Client Code)

In the main section (client code), instances of concrete classes are created, and the template method is invoked on these instances to execute the algorithm, demonstrating the flexibility and customization allowed by the Template Method Pattern.

# Main Section
if __name__ == "__main__":
    # Creating instances of concrete classes
    template_a = ConcreteTemplateA()
    template_b = ConcreteTemplateB()

    # Using the template method to perform steps
    print("Executing Template A:")
    template_a.template_method()

    print("\nExecuting Template B:")
    template_b.template_method()

GitHub Repo 🎉

Explore all code examples and design pattern implementations on GitHub!

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

Practical Example: Web Scraper Template

Step 1: Abstract Class (Web Scraper Template)

The abstract class defines a template for web scraping with abstract methods representing common steps like sending HTTP requests, parsing HTML content, and extracting data, allowing concrete classes to provide specific implementations.

from abc import ABC, abstractmethod
import requests
import urllib

from bs4 import BeautifulSoup


# Step 1: Abstract Class (Web Scraper Template)
class WebScraperTemplate(ABC):
    @abstractmethod
    def send_request(self, url):
        """Abstract method for sending HTTP requests."""
        pass

    @abstractmethod
    def parse_html(self, content):
        """Abstract method for parsing HTML content."""
        pass

    @abstractmethod
    def extract_data(self, soup):
        """Abstract method for extracting data from parsed HTML."""
        pass

    # Template method orchestrating the steps
    def scrape_website(self, url):
        """The template method for web scraping."""
        content = self.send_request(url)
        soup = self.parse_html(content)
        data = self.extract_data(soup)
        return data

Step 2: Concrete Classes

This concrete class extends the abstract web scraper template, providing a specific implementation using the requests library for sending HTTP requests and BeautifulSoup for HTML parsing, with a focus on extracting hyperlinks from a webpage.

# Step 2: Concrete Class 1 (RequestsAndBeautifulSoupWebScraper)
class RequestsAndBeautifulSoupWebScraper(WebScraperTemplate):
    def send_request(self, url):
        """Concrete implementation for sending HTTP requests using requests library."""
        response = requests.get(url)
        return response.content

    def parse_html(self, content):
        """Concrete implementation for parsing HTML content using BeautifulSoup."""
        soup = BeautifulSoup(content, 'html.parser')
        return soup

    def extract_data(self, soup):
        """Concrete implementation for extracting data from parsed HTML."""
        # Example: Extracting all hyperlinks from the webpage
        links = [a['href'] for a in soup.find_all('a', href=True)]
        return links

This concrete class also extends the web scraper template but uses urllib for sending HTTP requests and a custom parser for HTML content, showcasing an alternative approach to web scraping, and extracting email addresses from a webpage using regular expressions.

# Step 3: Concrete Class 2 (UrllibAndRegexWebScraper)
class UrllibAndRegexWebScraper(WebScraperTemplate):
    def send_request(self, url):
        """Concrete implementation for sending HTTP requests using urllib."""
        with urllib.request.urlopen(url) as response:
            return response.read()

    def parse_html(self, content):
        """Concrete implementation for parsing HTML content using custom parser."""
        # Custom HTML parsing logic
        return content

    def extract_data(self, content):
        """Concrete implementation for extracting data using regex."""
        import re
        # Example: Extracting all email addresses from the webpage
        emails = re.findall(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', content.decode('utf-8'))
        return emails

Client Code

Creates instances of concrete classes, invoking the template method for flexible web scraping with varied underlying algorithms.

# Main Section (Client Code)
if __name__ == "__main__":
    # Creating instances of the concrete classes
    web_scraper_requests_beautifulsoup = RequestsAndBeautifulSoupWebScraper()
    web_scraper_urllib_regex = UrllibAndRegexWebScraper()

    # Using the template method to scrape websites
    target_url = "https://example.com"

    print("Scraped Data using RequestsAndBeautifulSoupWebScraper:")
    data_requests_beautifulsoup = web_scraper_requests_beautifulsoup.scrape_website(target_url)
    for link in data_requests_beautifulsoup:
        print(link)

    print("\nScraped Data using UrllibAndRegexWebScraper:")
    data_urllib_regex = web_scraper_urllib_regex.scrape_website(target_url)
    for email in data_urllib_regex:
        print(email)

Real-World Use Cases for Template Method

1. Django Web Framework (Python): Employs Template Method in view classes for handling HTTP requests, enabling developers to customize specific steps of request processing.
2. Vue.js Frontend Framework (JavaScript): Implements Template Method in component lifecycle hooks, allowing developers to customize behavior during different phases of a component’s life.
3. TensorFlow Machine Learning Library (Python): Uses Template Method in model training classes, providing a standardized structure while allowing customization of training steps.
4. WordPress Theme Development (PHP): Utilizes Template Method in theme classes, allowing customization of the layout, header, and footer within a consistent theme structure.
5. Angular Framework (TypeScript): Incorporates Template Method in component lifecycle hooks, providing a standardized structure for initialization, change detection, and destruction.

Template Method Best Practices and Potential Drawbacks

When leveraging the Template Method Design Pattern, it’s vital to weigh its advantages and potential challenges:

Pros

1. Selective Overrides: Clients can override specific parts of a large algorithm, reducing the impact of changes on other sections.
2. Code Reusability: Duplicate code can be consolidated into a superclass, promoting a more maintainable codebase.

Cons

1. Client Limitations: Some clients may find their flexibility constrained by the predefined algorithm structure.
2. Liskov Substitution Principle: Suppressing a default step implementation in a subclass could violate the Liskov Substitution Principle.
3. Maintenance Challenges: Template methods become harder to maintain with an increasing number of steps.

Best Practices

1. Clear Abstractions: Establish clear abstractions for components to maintain a cohesive and understandable interface.
2. Use of Interfaces: Implement interfaces to ensure uniformity among components interacting with the template.

Template Method’s Relations with Other Patterns

When considering the Template Method pattern, it’s crucial to recognize its connections with other design patterns, each addressing distinct aspects of software architecture.

Template vs. Factory Method Pattern

The Factory Method is a specialized form of the Template Method, and it can also serve as a step within a larger Template Method.

While both involve object creation, the Template Method focuses on algorithmic structure, while the Factory Method is centered around object instantiation.

Template Method vs. Strategy Pattern

Template Method and Strategy patterns diverge in their approach to modifying object behavior. Template Method, reliant on inheritance, facilitates changes by extending parts of an algorithm in subclasses, operating at the class level.

Conversely, the Strategy pattern, relying on composition, enables runtime behavior alterations at the object level by supplying different strategies.

Conclusion

In concluding our journey through the Template Method Pattern in Python, we’ve delved into a pattern that provides a blueprint for algorithmic structures with customizable steps.

The key components, including the abstract class, concrete subclasses, and the template method itself, showcase a robust design for algorithm definition and refinement. We’ve explored the pros and cons of the Template Method, weighing its benefits in algorithm customization against potential limitations.

Additionally, we implemented a practical Web Scraper Template, demonstrating the pattern’s effectiveness in a real-world scenario.

Happy coding! 👩‍💻

Next on the Series 🚀

Design Patterns in Python: Visitor

Read More 📜

• Design Patterns in Python: Strategy
• Design Patterns in Python: Observer

The Series 🧭

Design Patterns in Python: A Series

Explore the GitHub Repo 🎉

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

References

1. Design Patterns: Elements of Reusable Object-Oriented Software (Book)
2. refactoring.guru Template Method
3. Head First Design Patterns (Book)
4. Template Method Design Pattern in Java
5. Template Method Design Pattern

Structured Processing

Have you encountered recurring coding challenges? Imagine having a toolbox of tried-and-true solutions readily available. That’s precisely what design patterns provide. In this series, we’ll explore what these patterns are and how they can elevate your coding skills.

Design Patterns in Python: A Series

Understanding the Visitor Pattern

What is the Visitor Design Pattern?

The Visitor Design Pattern is a behavioral design paradigm that defines a mechanism for traversing elements in an object structure and performing operations on them without altering their structure. It provides a means to add new operations to existing object structures without modifying those structures.

In simpler terms, the Visitor pattern allows you to define new functionalities (operations) that can be applied to a set of related classes, promoting extensibility without modifying the classes themselves.

When to Use the Visitor Pattern:

1. Multiple Operations on Object Structures: Execute various operations on complex object structures without modifying their classes.
2. Adding New Operations Without Modifying Classes: Easily extend functionalities by adding new operations to existing classes without altering them.
3. Operations Depending on the Type of Elements: Implement behaviors that vary based on the type of elements in the object structure.

The Problem

Consider an application managing a vast graph representing an intricate network of interconnected nodes, each denoting distinct entities such as users, transactions, or products.

The challenge arises when tasked with implementing a logging mechanism to record the activities of these nodes. The initial plan involves adding a logging method to each node class, a seemingly straightforward solution.

However, modifying existing node classes in a production environment poses risks. Additionally, embedding logging behavior within classes primarily designed for core functionalities raises questions about the coherence of such an approach.

The Solution: Visitor

The Visitor pattern resolves this by recommending a separate visitor class for new behavior. This avoids modifying existing classes, reducing risks, and ensuring a clean separation of concerns.

With the Visitor pattern, logging methods for each node type are defined in the visitor class, and dynamically executed via the accept method. This approach allows the addition of new logging behaviors without extensive modifications to existing classes.

Practical Example: Document Exporter

In this practical example, we export document elements like paragraphs, headings, and images to various formats without altering their individual classes.

The Visitor Pattern ensures a modular design by defining a common interface for exporters, allowing dynamic execution of export methods based on element types.

Terminology and Key Components

Understanding the fundamental components of the Visitor pattern is essential for effective implementation. Here are the key components involved:

1. Visitor Interface: Declares visiting methods for concrete elements, allowing methods with the same names in languages supporting overloading, but requiring different parameter types.
2. Concrete Visitor: Implements behavior variations for different element classes, enabling tailored actions during visits.
3. Element Interface: The Element interface includes an accept method with one parameter specifying the type of the visitor interface for visitor acceptance.
4. Concrete Element: Implements the accept method to redirect calls to the appropriate visitor methods, ensuring specific handling for each element.
5. Client: Represents a collection, interacting with elements through an abstract interface and orchestrating visitor acceptance without detailed knowledge of concrete classes.

Visitor Pattern Implementation in Python

here is a step-by-step approach, implementing each of the five key components of the Visitor pattern.

Step 1: Visitor Interface

Visitor is an abstract class defining visiting methods for each concrete element.

from abc import ABC, abstractmethod

# Step 1: Visitor Interface
class Visitor(ABC):
    @abstractmethod
    def visit_concrete_element_a(self, element_a):
        pass

    @abstractmethod
    def visit_concrete_element_b(self, element_b):
        pass

Step 2: Concrete Visitors

ConcreteVisitorA and ConcreteVisitorB implement specific behaviors for visiting each concrete element.

# Concrete Visitor A
class ConcreteVisitorA(Visitor):
    def visit_concrete_element_a(self, element_a):
        print(f"Visitor A visiting {element_a}")

    def visit_concrete_element_b(self, element_b):
        print(f"Visitor A visiting {element_b}")


# Concrete Visitor B
class ConcreteVisitorB(Visitor):
    def visit_concrete_element_a(self, element_a):
        print(f"Visitor B visiting {element_a}")

    def visit_concrete_element_b(self, element_b):
        print(f"Visitor B visiting {element_b}")

Step 3: Element Interface

Element is an abstract class defining the accept method for accepting visitors.

# Step 3: Element Interface
class Element(ABC):
    @abstractmethod
    def accept(self, visitor):
        pass

Step 4: Concrete Elements

ConcreteElementA and ConcreteElementB implement the accept method and define their unique operations.

# Step 4: Concrete Elements
class ConcreteElementA(Element):
    def accept(self, visitor):
        visitor.visit_concrete_element_a(self)

    def operation_a(self):
        return "Operation A"


class ConcreteElementB(Element):
    def accept(self, visitor):
        visitor.visit_concrete_element_b(self)

    def operation_b(self):
        return "Operation B"

Client Code

Demonstrates the usage of concrete elements and visitors.

# Client Code
if __name__ == "__main__":
    # Creating concrete elements
    element_a = ConcreteElementA()
    element_b = ConcreteElementB()

    # Creating concrete visitors
    visitor_a = ConcreteVisitorA()
    visitor_b = ConcreteVisitorB()

    # Accepting visitors
    element_a.accept(visitor_a)
    element_b.accept(visitor_a)

    element_a.accept(visitor_b)
    element_b.accept(visitor_b)

Practical Example: Document Exporter

Step 1: Define the Visitor Interface

Define the Visitor Interface with methods corresponding to each type of element in the document structure.

# Step 1: Define the Visitor Interface
class Exporter:
    def export_paragraph(self, paragraph):
        pass

    def export_heading(self, heading):
        pass

    def export_image(self, image):
        pass

Step 2: Implement Concrete Visitors

Implement Concrete Visitors (HTMLExporter and PDFExporter) providing specific behavior for each type of element.

# Step 2: Implement Concrete Visitors
class HTMLExporter(Exporter):
    def export_paragraph(self, paragraph):
        return f"<p>{paragraph.content}</p>"

    def export_heading(self, heading):
        return f"<h{heading.level}>{heading.text}</h{heading.level}>"

    def export_image(self, image):
        return f'<img src="{image.source}" alt="{image.alt}">'

class PDFExporter(Exporter):
    def export_paragraph(self, paragraph):
        return f"PDF Paragraph: {paragraph.content}"

    def export_heading(self, heading):
        return f"PDF Heading ({heading.level}): {heading.text}"

    def export_image(self, image):
        return f'PDF Image: {image.alt}'

Step 3: Define Element Interface

Define the Element Interface with an accept method to allow visitors to operate on elements.

# Step 3: Define Element Interface
class Element:
    def accept(self, visitor):
        pass

Step 4: Implement Concrete Elements

Implement concrete elements (Paragraph, Heading, Image) with an accept method to let visitors operate on them.

# Step 4: Implement Concrete Elements
class Paragraph(Element):
    def __init__(self, content):
        self.content = content

    def accept(self, visitor):
        return visitor.export_paragraph(self)


class Heading(Element):
    def __init__(self, level, text):
        self.level = level
        self.text = text

    def accept(self, visitor):
        return visitor.export_heading(self)


class Image(Element):
    def __init__(self, source, alt):
        self.source = source
        self.alt = alt

    def accept(self, visitor):
        return visitor.export_image(self)

Step 5: Client Code

Implement the Client Code (Document) with an accept method to iterate through elements and let the visitor operate on them.

Create instances of Concrete Visitors (HTMLExporter, PDFExporter) and use them to export a Document with different types of elements, demonstrating the Visitor Pattern in action.

# Step 5: Client Code
class Document:
    def __init__(self, elements):
        self.elements = elements

    def accept(self, visitor):
        result = []
        for element in self.elements:
            result.append(element.accept(visitor))
        return result

# Usage
html_exporter = HTMLExporter()
pdf_exporter = PDFExporter()

document = Document([
    Paragraph("This is a paragraph."),
    Heading(1, "Main Heading"),
    Image("image.jpg", "A beautiful image")
])

# Export to HTML
html_result = document.accept(html_exporter)
print("HTML Export:")
print('\n'.join(html_result))
print("\n")

# Export to PDF
pdf_result = document.accept(pdf_exporter)
print("PDF Export:")
print('\n'.join(pdf_result))

GitHub Repo 🎉

Explore all code examples and design pattern implementations on GitHub!

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

Real-World Use Cases for Visitor Pattern

1. Document Object Model (DOM) Manipulation in JavaScript: JavaScript libraries like jQuery use the Visitor pattern to traverse and manipulate the DOM efficiently, enabling dynamic updates to web pages.
2. Python Abstract Syntax Tree (AST) Module: Python’s ast module utilizes the Visitor pattern to traverse and analyze Python code at the abstract syntax tree level, providing a foundation for tools like linters and code analyzers.
3. OpenGL Scene Graph Traversal: In graphics libraries like OpenGL, the Visitor pattern is employed to traverse scene graphs efficiently, enabling the rendering of complex 3D scenes.
4. Mozilla SpiderMonkey JavaScript Engine: SpiderMonkey, the JavaScript engine used in Mozilla Firefox, applies the Visitor pattern for traversing abstract syntax trees during JavaScript code execution.
5. Git Version Control System: Git employs the Visitor pattern in its internal object traversal mechanisms, facilitating efficient commit and tree operations.

Visitor Best Practices and Potential Drawbacks

When employing the Visitor pattern, several best practices and considerations contribute to its effective utilization:

Best Practices:

1. Operation on Complex Structures: Use the Visitor when you need to perform an operation on all elements of a complex object structure, like an object tree.
2. Business Logic Cleanup: Employ the Visitor pattern to clean up the business logic of auxiliary behaviors, allowing primary classes to focus on their main responsibilities.
3. Selective Behavior Implementation: Use the pattern when a behavior is relevant only in certain classes of a hierarchy, extracting it into a separate visitor class and implementing only the necessary visiting methods.

Drawbacks:

1. Maintenance Overhead: Updating all visitors when a class is added to or removed from the element hierarchy can introduce maintenance overhead.
2. Limited Access to Private Fields: Visitors might lack access to the private fields and methods of elements, potentially restricting their functionality.

Visitor Pattern’s Relations with Other Patterns

When considering the Visitor pattern, its relations with other patterns become apparent, offering unique advantages in certain scenarios:

Visitor vs. Command Pattern:

• Similarity: Treat Visitor as a potent version of the Command pattern. Both enable objects to execute operations over various objects of different classes.
• Distinction: While Command focuses on encapsulating requests as objects, Visitor extends this concept by executing operations over diverse classes of objects.

Visitor with Composite Pattern:

• Usage: Utilize Visitor to execute an operation over an entire Composite tree. This synergistic combination allows performing operations on complex structures seamlessly.

3. Visitor with Iterator Pattern:

• Collaboration: Combine Visitor with Iterator to traverse a complex data structure and execute operations over its elements, even if they have different classes. This collaboration enhances flexibility in processing diverse elements.

Conclusion

Concluding our exploration of the Visitor Pattern, this design pattern excels in traversing and performing operations on diverse objects within complex structures. The Visitor Interface, Concrete Visitors, and Elements create a modular and focused approach, enabling dynamic behavior changes.

Understanding its pros, cons, and comparisons with related patterns equips developers to make informed choices. The Visitor pattern’s adaptability proves valuable for scenarios requiring dynamic, class-specific behaviors.

Whether traversing abstract syntax trees or performing complex operations, the Visitor pattern stands out as a versatile solution, enhancing flexibility and maintainability in your codebase.

Happy coding! 👩‍💻

Next on the Series 🚀

Design Patterns in Python: Strategy

Explore the GitHub Repo 🎉

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

Read More 📜

• Design Patterns in Python: Observer
• Design Patterns in Python: Memento

The Series 🧭

Design Patterns in Python: A Series

References

1. Design Patterns: Elements of Reusable Object-Oriented Software (Book)
2. refactoring.guru Visitor
3. Head First Design Patterns (Book)
4. Visitor Design Pattern in Java
5. Scaler Visitor Design Pattern

Proximity-Based Predictions

What is KNN?

KNN relies on a straightforward principle: when given a new, unknown data point, it looks at the K nearest labeled data points and assigns the most common label among them to the new point.

This closeness is determined by a distance metric, commonly Euclidean or Manhattan distance.

The Core Principle of Proximity-Based Learning

At the core of proximity-based learning, such as the K-Nearest Neighbors (KNN) algorithm, lies a fundamental concept: closeness dictates similarity.

Grasping Similarity via Proximity

The concept translates into the algorithm’s behavior by seeking the closest ‘neighbors’ — data points that share proximity — to make decisions about new, unseen data. By assuming that nearby points are alike, the algorithm infers patterns and assigns labels based on this proximity.

Decision Boundaries: A Result of Proximity

The algorithm doesn’t just predict outcomes; it also delineates decision boundaries. These boundaries partition the feature space into regions, where each region signifies a particular class or label. Think of it as virtual borders drawn based on the proximity of data points.

Anatomy of the KNN Algorithm

1. Neighbor identification: KNN identifies ‘k’ nearest neighbors to a query point based on their proximity.
2. A decision by the majority: It predicts by a majority vote among the neighbors for classification or averaging for regression.
3. No explicit training phase: KNN’s lazy learning means no explicit training; it stores the entire dataset for inference.
4. Impact of k-value: The k parameter influences model complexity and can affect overfitting or underfitting.

Choosing the Right Distance Metrics in KNN

Each distance metric provides a unique perspective in determining proximity and contributes distinct decision boundaries within the KNN algorithm.

Understanding their characteristics aids in selecting the most appropriate metric for a given dataset.

Let's explore some key distance metrics used in KNN:

Euclidean Distance (p=2)

The most prevalent and straightforward distance measure, exclusively applicable to real-valued vectors.

Manhattan Distance (p=1)

Often known as a taxicab or city block distance, this metric calculates the absolute value between two points.

Minkowski Distance

A generalized form that encompasses both Euclidean and Manhattan distances, allowing the creation of various distance metrics based on the parameter p.

Hamming Distance

Tailored for Boolean or string vectors, identifying discrepancies between vectors. Commonly referred to as the overlap metric.

Coding KNN in Python from Scratch

Implementing the K-Nearest Neighbors (KNN) algorithm from scratch allows a deep dive into its mechanics.

Let’s break down the process into distinct parts and code each step comprehensively.

Part 1: Distance Calculation

The core of KNN involves measuring distances between data points. In this step, we’ll create a function to compute the Euclidean distance between two points.

def euclidean_distance(point1, point2):
    """
    Calculate the Euclidean distance between two points.

    Parameters:
    point1 : list or array-like
        Coordinates of the first point.
    point2 : list or array-like
        Coordinates of the second point.

    Returns:
    distance : float
        The Euclidean distance between the two points.
    """
    # Ensure both points have the same dimensions
    assert len(point1) == len(point2), "Points should have the same dimensions."
    
    # Compute the Euclidean distance
    distance = sum((p1 - p2) ** 2 for p1, p2 in zip(point1, point2)) ** 0.5
    return distance

Part 2: Finding Nearest Neighbors

Next, let’s write a function to find the ‘k’ nearest neighbors of a query point within a dataset.

def find_neighbors(X_train, query_point, k):
    """
    Find the 'k' nearest neighbors of a query point within a dataset.

    Parameters:
    X_train : list or array-like
        Training dataset containing features.
    query_point : list or array-like
        Coordinates of the query point.
    k : int
        Number of neighbors to find.

    Returns:
    neighbors : list
        List of indices of the 'k' nearest neighbors.
    """
    distances = []
    
    # Calculate distance from the query point to each point in the training set
    for i, data_point in enumerate(X_train):
        distance = euclidean_distance(query_point, data_point)
        distances.append((i, distance))
    
    # Sort distances in ascending order
    distances.sort(key=lambda x: x[1])
    
    # Get indices of the 'k' nearest neighbors
    neighbors = [index for index, _ in distances[:k]]
    return neighbors

Part 3: Predicting the Class

Finally, let’s create a function to predict the class of a query point based on the majority class among its nearest neighbors.

def predict(X_train, y_train, query_point, k):
    """
    Predict the class of a query point based on the majority class among its nearest neighbors.

    Parameters:
    X_train : list or array-like
        Training dataset containing features.
    y_train : list or array-like
        Training dataset containing labels.
    query_point : list or array-like
        Coordinates of the query point.
    k : int
        Number of neighbors to consider.

    Returns:
    predicted_class : int or str
        Predicted class label for the query point.
    """
    neighbors = find_neighbors(X_train, y_train, query_point, k)
    neighbor_labels = [y_train[i] for i in neighbors]
    
    # Count occurrences of each label among neighbors
    label_counts = {}
    for label in neighbor_labels:
        if label in label_counts:
            label_counts[label] += 1
        else:
            label_counts[label] = 1
    
    # Get the label with the highest count
    predicted_class = max(label_counts, key=label_counts.get)
    return predicted_class

Exploring Wine Quality Classification with KNN

Introduction to the Dataset

The wine quality dataset comprises 11 features like acidity, residual sugar, pH, and alcohol content, aiming to predict wine quality on a scale from 1 to 10. With 4898 samples, this dataset serves as a playground for exploring classification techniques.

Static K-value: Starting Point

In this phase, we initiate our exploration by reading the dataset, splitting it into training and testing sets, and applying a KNeighborsClassifier from scikit-learn with a static k-value of 15.

import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt
import seaborn as sns


def read_data(file_name):
    # Load the dataset with semicolon delimiter
    data = pd.read_csv(file_name, delimiter=';')
    return data

def split_data(data):
    # Extract features and labels
    X = data.drop('quality', axis=1)
    y = data['quality']

    # Split data into train and test sets
    X_train, X_test, y_train, y_test = train_test_split(
        X,
        y,
        test_size=0.2,
        random_state=42
    )
    return X_train, X_test, y_train, y_test

def fit_model(X_train, y_train, k = 5):
    # Implement KNN classifier
    knn = KNeighborsClassifier(n_neighbors=k)
    knn.fit(X_train, y_train)

    # Predict on the train set
    train_preds = knn.predict(X_train)

    # Calculate and return accuracy on train set
    train_accuracy = accuracy_score(y_train, train_preds)
    return train_accuracy, knn


def test_model(model, X_test, y_test):
    # Predict on the test set
    test_preds = model.predict(X_test)

    # Calculate and return accuracy on test set
    test_accuracy = accuracy_score(y_test, test_preds)
    return test_accuracy

Results:
With a static k-value of 15, the model yields:

• Test Accuracy: 48.44%

The next step is to find the optimum k-value that yields the best accuracy on the training data.

Optimizing KNN: Fine-Tuning Your Model

Enhanced Accuracy with GridSearchCV

Employing GridSearchCV to find the best hyperparameter k-value resulted in a noticeable accuracy surge, hitting 50%. Optimizing the number of neighbors and exploring distance metrics were pivotal in this improvement.

Results:
With an optimized k-value of 19, the model yields:

• Test Accuracy: 50.94%

def find_best_k_with_grid_search(X_train, y_train, X_test, y_test, param_grid):
    # Create a KNN classifier
    knn = KNeighborsClassifier()

    # Perform GridSearchCV
    grid_search = GridSearchCV(knn, param_grid, cv=5, scoring='accuracy')
    grid_search.fit(X_train, y_train)

    # Get the best parameter
    best_k = grid_search.best_params_['n_neighbors']

    # Fit model with best k on entire training set
    best_knn = KNeighborsClassifier(n_neighbors=best_k)
    best_knn.fit(X_train, y_train)

    # Calculate accuracy on train and test set
    train_accuracy = best_knn.score(X_train, y_train)
    test_accuracy = best_knn.score(X_test, y_test)
    
    return grid_search.best_params_, train_accuracy, test_accuracy


# Assuming you have X_train, y_train variables
max_k=50

# Define a grid of hyperparameters
parameters = {'n_neighbors': range(2, max_k + 1)}

best_params, train_accuracy, test_accuracy = find_best_k_with_grid_search(X_train, y_train, X_test, y_test, parameters)
print(f"Best k: {best_params['n_neighbors']}")
print(f"Train Accuracy with Best k: {train_accuracy*100:.2f}%")
print(f"Test Accuracy with Best k: {test_accuracy*100:.2f}%")

Weighing Distance for Precision

By incorporating distance-based weighted averaging within GridSearchCV, the model’s accuracy jumped to nearly 51.56%. Considering the proximity of neighbors in the prediction significantly contributed to this progress.

We added both distance and uniform mode in the parameters to find the best k-value.

parameters = {
    "n_neighbors": range(2, max_k + 1),
    "weights": ["uniform", "distance"]
}

Results:
With an optimized k-value of 43, and using weighted distance, the model yields:

• Test Accuracy: 51.56%

Ensemble Modeling with Bagging

Implementing Bagging with KNN led to a substantial leap in accuracy, reaching around 60.31%. Combining multiple KNN models through Bagging brought more robust predictions.

Results:
With a bagging ensemble model with 100 estimators, each randomly having 0.3 of the data.

• Test Accuracy: 60.31%

def calculate_bagged_knn(X_train, y_train, X_test, y_test, best_k, best_weights):
    # Create a KNeighborsClassifier with best parameters
    knn = KNeighborsClassifier(n_neighbors=best_k, weights=best_weights)
    
    # Create BaggingClassifier with KNeighborsClassifier as base estimator
    bagged_knn = BaggingClassifier(estimator=knn, n_estimators=100, max_samples=0.3)
    
    # Fit the BaggingClassifier on the training data
    bagged_knn.fit(X_train, y_train)
    
    # Calculate training and test accuracies
    train_accuracy = bagged_knn.score(X_train, y_train)
    test_accuracy = bagged_knn.score(X_test, y_test)

    return train_accuracy, test_accuracy

The complete code is accessible on GitHub.

Pros and Cons of the kNN Algorithm

K-Nearest Neighbors (KNN) brings forth advantages and limitations, offering interpretability and adaptability while grappling with computational demands and challenges in high-dimensional spaces.

Advantages of KNN

1. Interpretable and Fast Development: KNN offers interpretability, allowing users to comprehend its functioning. Its simplicity facilitates the rapid development of models without the complexity of more advanced techniques.
2. Adaptable to New Data: It easily accommodates new data points without retraining, adjusting its predictions based on new examples added to the dataset.
3. Few Hyperparameters: Requires minimal parameter tuning — mainly ‘k’ and choice of distance metric — simplifying the training process.
4. Robustness to Noisy Data: Demonstrates resilience to noisy training data, aiding in effective classification, especially with a large dataset.
5. Robustness to Large Training Data: Can be more effective with larger training datasets, leveraging the abundance of information for predictions.

Drawbacks

1. Computationally Intensive and Resource-Heavy: As a lazy algorithm, KNN requires significant computing power and storage due to storing all data points, making it time and resource-consuming.
2. Curse of Dimensionality: Faces challenges with high-dimensional data, struggling to properly classify data points in higher dimensions, potentially leading to less accurate predictions.
3. Needs Optimal ‘k’ Selection: The choice of ‘k’ can significantly impact performance, requiring careful selection and optimization, which might be complex at times.
4. Slower Performance with Increased Data: The algorithm’s efficiency decreases notably as the dataset size or number of predictors/independent variables grows, affecting its speed.

Wrapping Up: Leveraging KNN in Python ML

In this comprehensive exploration of K-Nearest Neighbors (KNN) in Python, we delved into the algorithm’s fundamentals, its pivotal components, and practical implementation aspects.

We’ve reviewed:

Algorithm Insights: Understanding how KNN classifies based on proximity to neighbors.
Significance of K-Value: Recognizing the impact of k-value on model performance and the trade-off between bias and variance.
Distance Metrics Importance: Appreciating the role of distance metrics in shaping decision boundaries and influencing predictions.
KNN Implementation from Scratch: Crafting KNN code from the ground up, enhancing comprehension of its inner workings.
Fine-Tuning on Wine Quality Dataset: Iteratively optimizing KNN on the wine quality dataset to elevate its predictive capacity.
Advantages and Disadvantages: Weighing the pros and cons to comprehend where KNN excels and where it faces limitations.

Hope you enjoyed the KNN pattern exploration.

Happy training! 👩‍💻

Explore the GitHub Repo 🎉

GitHub - AmirLavasani/retro-machine-learning: ⏱ Original implementations from scratch of timeless classic machine learning algorithms such as KNN, SVM, PCA, Decision Trees, K-means, and more.

Resources

1. Pattern Recognition and Machine Learning by Christopher Bishop (Book)
2. realpython The k-Nearest Neighbors (kNN) Algorithm in Python
3. scikit-learn Nearest Neighbors Classification
4. IBM K-Nearest Neighbors Algorithm
5. Machine Learning Basics with the K-Nearest Neighbors Algorithm
6. Medium K-Nearest Neighbor
7. K-Nearest Neighbor (KNN) Explained
8. A Complete Guide to K-Nearest Neighbors

Dynamic Algorithm Switching

Have you encountered recurring coding challenges? Imagine having a toolbox of tried-and-true solutions readily available. That’s precisely what design patterns provide. In this series, we’ll explore what these patterns are and how they can elevate your coding skills.

Design Patterns in Python: A Series

Understanding the Strategy Pattern

What is the Strategy Design Pattern?

The Strategy Design Pattern is a behavioral design paradigm that encapsulates a family of interchangeable algorithms, allowing dynamic selection by a client class.

This decouples algorithmic implementation from the client, promoting flexibility and ease of modification without altering the client’s structure.

When to Use the Strategy Pattern:

Consider employing the Strategy Pattern for the following cases:

1. Dynamic Algorithm Switching: We Need multiple variations of an algorithm within an object with the ability to switch during runtime.
2. Behavioral Differences in Similar Classes: Dealing with numerous classes that share similarities but differ in behavior execution.
3. Business Logic Isolation: Separating business logic from algorithm implementation details.
4. Simplify Massive Conditional Statements: Your class has an extensive conditional statement handling different algorithm variants.

Practical Example: A Trading Strategy System

we’ll implement a trading strategy system in Python using the Strategy Design Pattern. We aim to demonstrate the adaptability and flexibility of the pattern by encapsulating different trading strategies, such as Moving Average and Mean Reversion, as interchangeable components.

Terminology and Key Components

To grasp the essence of the Strategy Design Pattern, let’s delve into its key components:

1. Context: Maintains a reference to one of the concrete strategies and communicates with this object only via the strategy interface.
2. Strategy Interface: Common to all concrete strategies, declares a method the context uses to execute a strategy.
3. Concrete Strategies: Implement different variations of an algorithm the context uses.

Strategy Pattern Implementation in Python

Step 1: Strategy Interface

Define an abstract class Strategy that declares a method execute_strategy common to all concrete strategies.

from abc import ABC, abstractmethod

# Step 1: Create the Strategy Interface
class Strategy(ABC):
    @abstractmethod
    def execute_strategy(self):
        pass

Step 2: Concrete Strategies

Implement concrete strategy classes ConcreteStrategyA and ConcreteStrategyB that provide specific algorithm variations.

# Step 2: Create Concrete Strategies
class ConcreteStrategyA(Strategy):
    def execute_strategy(self):
        return "Executing Strategy A"

class ConcreteStrategyB(Strategy):
    def execute_strategy(self):
        return "Executing Strategy B"

Step 3: Context

Develop a Context class that maintains a reference to one of the concrete strategies and provides a method to execute the strategy.

# Step 3: Create the Context
class Context:
    def __init__(self, strategy):
        # Context maintains a reference to one of the concrete strategies
        self._strategy = strategy

    def set_strategy(self, strategy):
        # Exposes a setter to replace the strategy 
        # associated with the context at runtime
        self._strategy = strategy

    def execute_strategy(self):
        # Context calls the execution method on the linked strategy object
        return self._strategy.execute_strategy()

Client

# Main Section to Showcase Usage
if __name__ == "__main__":
    # Create concrete strategy objects
    strategy_a = ConcreteStrategyA()
    strategy_b = ConcreteStrategyB()

    # Create context with a default strategy
    context = Context(strategy_a)

    # Execute the default strategy
    print(context.execute_strategy())  # Output: Executing Strategy A

    # Switch to a different strategy at runtime
    context.set_strategy(strategy_b)
    print(context.execute_strategy())  # Output: Executing Strategy B

GitHub Repo 🎉

Explore all code examples and design pattern implementations on GitHub!

GitHub - AmirLavasani/python-design-patterns: Explore design patterns using Python with code examples to solve common software challenges.

Practical Example: A Trading Strategy System

In this section, we implement a dynamic trading strategy system in Python, showcasing the capability to switch strategies seamlessly at runtime.

Step 1: Create the Strategy Interface

# Step 1: Create the Strategy Interface
class TradingStrategy(ABC):
    @abstractmethod
    def execute_trade(self, data):
        pass

Step 2: Create Concrete Strategies

# Step 2: Create Concrete Strategies
class MovingAverageStrategy(TradingStrategy):
    def execute_trade(self, data):
        # Calculate Moving Average (Simple example for illustration)
        window_size = 3  # Adjust as needed
        moving_average = sum(data[-window_size:]) / window_size
        return f"Executing Moving Average Trading Strategy. Moving Average: {moving_average:.2f}"

class MeanReversionStrategy(TradingStrategy):
    def execute_trade(self, data):
        # Calculate Mean Reversion (Simple example for illustration)
        mean_value = sum(data) / len(data)
        deviation = data[-1] - mean_value
        return f"Executing Mean Reversion Trading Strategy. Deviation from Mean: {deviation:.2f}"

Step 3: Create the Context

# Step 3: Create the Context
class TradingContext:
    def __init__(self, strategy):
        self._strategy = strategy

    def set_strategy(self, strategy):
        self._strategy = strategy

    def execute_trade(self, data):
        return self._strategy.execute_trade(data)

Client

# Main Section to Showcase Usage
if __name__ == "__main__":
    # Sample data for trading
    trading_data = [50, 55, 45, 60, 50]

    # Create concrete strategy objects
    moving_average_strategy = MovingAverageStrategy()
    mean_reversion_strategy = MeanReversionStrategy()

    # Create context with a default strategy
    trading_context = TradingContext(moving_average_strategy)

    # Execute the default strategy
    result = trading_context.execute_trade(trading_data)
    print(result)  # Output: Executing Moving Average Trading Strategy. Moving Average: 51.67

    # Switch to a different strategy at runtime
    trading_context.set_strategy(mean_reversion_strategy)
    
    # Execute the updated strategy
    result = trading_context.execute_trade(trading_data)
    print(result)  # Output: Executing Mean Reversion Trading Strategy. Deviation from Mean: -1.00

Real-World Use Cases for Strategy Pattern

1. Python’s sort function: The sort function in Python allows users to pass a custom comparison function, showcasing the Strategy Pattern.
2. Redux State Management in React: In React applications using Redux, the middleware pattern employs the Strategy Pattern for handling asynchronous actions.
3. Scikit-learn Machine Learning Library in Python: Scikit-learn’s preprocessing module utilizes the Strategy Pattern for handling missing data.
4. Unity Game Engine: Unity’s input system employs the Strategy Pattern for defining various input handling strategies.
5. TensorFlow Machine Learning Library: In TensorFlow, optimizers like stochastic gradient descent use the Strategy Pattern for defining different optimization strategies.

Best Practices and Considerations

When employing the Strategy Design Pattern, it’s essential to be mindful of both its advantages and potential considerations:

Pros

1. Runtime Algorithm Swap: The ability to switch algorithms dynamically at runtime enhances adaptability and flexibility.
2. Implementation Isolation: The pattern isolates the implementation details of an algorithm, promoting cleaner and more maintainable code.
3. Composition Over Inheritance: It allows replacing inheritance with composition, emphasizing a more flexible design approach.

Cons

1. Programmatic Overhead: For a limited number of rarely changing algorithms, the introduction of extra classes and interfaces may lead to unnecessary complexity.
2. Client Awareness: Clients need awareness of the differences between strategies to make informed selections, introducing potential cognitive load.
3. Functional Alternatives: Some modern programming languages with functional support offer alternatives, implementing different algorithm versions without additional class and interface overhead.

Strategy Pattern’s Relations with Other Patterns

The Strategy Pattern shares structural similarities with other patterns such as Bridge, State, and to some extent, Adapter.

While they all utilize composition for delegating work to other objects, each pattern addresses distinct problems.

Command Pattern

Both Command and Strategy can be used to parameterize an object with an action.

Command: converts any operation into an object, allowing deferred execution, queuing, and remote command handling.

Strategy: describes different ways of doing the same thing, enabling the swapping of algorithms within a single context class.

Decorator Pattern

Both Decorator and Strategy allow altering an object’s behavior.

Decorator: changes the outer layer or “skin” of an object, enhancing its functionalities.

Strategy: focuses on changing the internal workings or “guts” of an object by providing different algorithms.

Template Method Pattern

Both Template Method and Strategy are based on altering parts of an algorithm.

Template Method: relies on inheritance to extend and alter algorithm parts in subclasses (class level).

Strategy: relies on composition to switch behaviors at runtime (object level).

State Pattern

The State pattern can be considered an extension of Strategy, both based on composition.

State: allows dependencies between concrete states, enabling them to alter the context’s state at will.

Strategy: makes objects independent and unaware of each other.

Conclusion

In wrapping up our exploration of the Strategy Pattern in Python, we’ve uncovered a pattern that brings dynamic algorithmic switches to your code. The trio of Strategy Interface, Concrete Strategies, and Context forms a pattern that allows one to change algorithms on the fly.

We covered the pattern's main components, pros and cons, and its relation and comparison to other patterns. We have also implemented a simple trading system that changes algorithms in real time to show the power of the strategy pattern.

Happy coding! 👩‍💻

Next on the Series 🚀

Design Patterns in Python: Observer

Read More 📜

• Design Patterns in Python: Memento
• Design Patterns in Python: Mediator

The Series 🧭

Design Patterns in Python: A Series

References

1. Design Patterns: Elements of Reusable Object-Oriented Software (Book)
2. refactoring.guru Strategy
3. Head First Design Patterns (Book)
4. A Beginner’s Guide to the Strategy Design Pattern
5. Strategy Design Pattern in Java — Example Tutorial
6. sourcemaking Strategy Design Pattern