It is a no-brainer to provide JSON REST endpoints in a Spring Boot application, just like many state-of-the-art frameworks. Just include the Spring Boot Web Starter as a dependency, define a class annotated with @RestController, and implement at least one method annotated with @RequestMapping (or a specific annotation) as a handler for HTTP requests.

However, in a full API lifecycle, there is more to be done. In this article, we introduce an automated tool chain that helps professionalize Spring Boot REST API development and automates a lot of manual efforts.

Code-First But with a specification

In the API community, much like any traditional type of interface, it is often discussed whether to start with an interface specification and implement the rendering API afterwards (API-Design-First) or to generate the specification from the implementation (Code-First). Both approaches are justifiable. In this article, we will focus on the use cases with a code-first approach, as is often done with Spring Boot.

The figure below illustrates that with a code-first approach, first, a Spring Boot application is set up (1), then, REST endpoints and implicitly an API is created (2). Sometimes an API documentation is written, for example, according to the OpenAPI standard (3) and occasionally an API client is created as well (4).

Example: User API

As an case for this article, we provide a User API example project. This services as a reference for the presented tool chain and will likely support you in setting up the tool chain for your own context.

The API allows to create, update, and query users. For this purpose, REST operations are provided in an HTTP compliant manner using GET, POST, PUT or DELETE.

While relevant code snippets are included below, the code for the complete case is publicly available on GitHub: https://github.com/viadee/api-roundtrip-spring-boot

API-Roundtrip — The Full-Service-Package

In this article, we refer to an API roundtrip as the development of a complete API package including the following artifacts:

• REST-API
  The API itself
• API-Documentation
  An API documentation as an OpenAPI specification
• API-Client
  A client for using the API that can be easily integrated in an application
• Integration Tests
  That automatically test the functionality of the API after deployment, for instance, in a test environment

Status Quo

It is fairly established for other interface types such as SOAP or CORBA to provide documentation (WSDL) and pre-built clients. However, such artifacts are far from being common for REST APIs. A significant reason for this is the lack of established automation - or at least the established automation not being commonly known.

Nowadays, tools such as Postman, SOAP UI or REST Assured are often used for integration tests. They all have allowances for sending JSON requests to a REST API.

Because of a necessity of creating and maintaining JSON requests as well as checks for the resulting JSON responses, this often results in high manual effort compared to the original API development.

Often, even a small adjustment in the Spring boot application involves a high manual and often annoying effort considering the overall package.

Automation thanks to SpringDoc and OpenApi Generator

For an effective REST API development improving the status quo, an automated tool chain providing capabilities, at least known from using SOAP and CORBA, is desirable. On one hand, this automates repetitive manual tasks, on the other hand, this resolves the media break in the integration test development.

The latter is achieved when integration tests are implemented in Java, like the Spring Boot application itself.

The following illustration provides an overview of the tool chain we describe in this article.

First, an OpenAPI documentation is generated using SpringDoc based on the Spring Boot REST API implementation. In the second step, appropriate client code and integration test code is generated with OpenAPI Generator facilitating the API documentation.

Of course, implementing the actual test logic still requires manual effort.

In detail, the tool chain consists of the following components:

• Spring Boot
  Implements the REST-API (1)
• SpringDoc
  Generates the API documentation as an OpenAPI specification and a Swagger UI facilitating the Spring Boot application infrastructure. (2)
• OpenAPI
  Generates the code for API client(s) and integration tests from the OpenAPI specification generated by SpringDoc. (3)
• Maven
  Orchestrates and executes the tools respectively with their concurring plugins. Of course, a similar chain can be built with Gradle.
• Integration Tests
  The integration test execution depends on the concrete environment. In this article, we provide an example, where integration tests are triggered with Maven within a deployment pipeline after the deployment itself. (4)

API-Roundtrip — The Tool Chain in a Nutshell

Spring Boot API Implementation

The User API is defined by an interface with corresponding spring annotations. The method names should be chosen with care. By default, they will be used as operationID in the OpenAPI Spec and through this, define the method names of the generated API clients.

It is recommended to explicitly specify a @ResponseStatus as this will automatically result in a cleaner OpenAPI generation later.

The API interface is implemented by a Spring @RestController. For the sake of simplicity, we implemented the storage of the user data with a non-persistent HashMap.

Finally, Maven dependencies must be added in the pom file and a @SpringBootApplication must be created and started. This is necessary to send a first request to the API via http://localhost:8080/api/users .

Up to this point, this is a simple Spring Boot infrastructure. The actual tool chain for our API roundtrip starts from here.

API Documentation Generation

With SOAP Web Services, it is quite common to provide a WSDL and get it automatically generated by the web service infrastructure. However, this is not yet available out-of-the-box for REST APIs.

SpringDoc fills this gap by automatically generating an OpenAPI specification and Swagger-UI online documentation by processing the Spring Boot infrastructure and providing additional configuration capabilities.

To be able to process SpringDoc in your own application, the following Maven-Dependency is sufficient as it indirectly refers the SpringDoc core as well:

This means that a Swagger UI can be reached immediately after a (new) start of the Spring Boot application: http://localhost:8080/swagger-ui.html

An OpenAPI specification can also be retrieved: http://localhost:8080/v3/api-docs

Both types of documentation can be extensively configured and supplemented.

SpringDoc provides various annotations, such as the @OpenAPIDefinition, for enriching the content of the API documentation as shown in the example below:

It is recommendation to add the server specification at this point. Later, during the OpenAPI file generation, the API will be started on a random free port and without explicitly specifying it here, a rather unfamiliar PORT specification will end up in the API specification.

Of course, if the final URL of the API endpoint is already known, it should be entered here.

If the JSON of the OpenAPI specification need to be formatted by a “pretty-printer” with indentations and line breaks, the following configuration should be added in the Spring Boot application.properties or yaml file:

If needed, the URLs of the Swagger UI and the OpenAPI documentation can be set under the actuator or any other context.

Finally, the OpenAPI specification must be stored as a local file to make it accessible for the client code generation later. SpringDoc comes with a Maven plugin for this step and through this, it provides more convenience compared to similar tools such as Spring Fox.

In a typical Spring Boot configuration, entire dependencies are loaded and resolved at runtime only and thus, a running application is needed by the SpringDoc Maven plugin. The plugin downloads the OpenAPI specification from the URL mentioned above.

To get this working, the SpringDoc Maven plugin is configured in the Maven “integration test” phase as shown in the example below:

In addition to the URL for downloading the OpenAPI specification, the target file can also be defined via outputFileName and outputDir parameters. We omitted these parameters in the example, so the file is written by default to target/openapi.json.

To ensure that the Spring Boot application can successfully be launched for the “integration test” phase, we first determine a free port using the codehaus build-helper-maven-plugin. The free port is then configured in the spring-boot-maven-plugin in the “pre-integration-test” phase to start the Spring Boot application (<goal>start</goal>) and stop it during the “post-integration-test” phase (<goal>stop</goal>).

The dynamic port selection ensures that the Maven tool chain works, even if an application is already running on the standard port 8080.

Now, the first step of generating an API documentation is done. Next, we will focus on generating the code for the API client and integration tests.

API Client And Integrationstest Generation

OpenAPI specifications provide a detailed interface description which is sufficient to implement valid requests to the API endpoints. OpenAPI Generator is an quite active and mature open-source tool for generating code and other artifacts based on these specifications. We use it in the roundtrip tool chain for generating code that encapsulates the REST API calls as easy-to-use Java interfaces.

OpenAPI Generator was derived (forked) from the Swagger Code Gen project aiming at a significantly lighter and faster development. The versions released so far, and the active community of the OpenAPI Generator conform to the founder’s goal.

Meanwhile, there are over 130 different code generators, for both client and server implementations, as well as for documentation and other API and data formats such as GraphQL or protobuf.

OpenAPI comes with a Maven plugin, so the code generation can be integrated in the overall workflow straight away. A plugin for Gradle is available, too. The following example shows a concurrent Maven plugin configuration.

This example shows a plugin configuration using the Java code generator with the java8 DateLibrary option, generating date data fields as java.util.LocalDate.

In addition, the various packages for the target code are set and the generator is configured to generate only the client source code and no further documentation or Gradle and Maven artifacts.

Unfortunately, in the current version of OpenAPI Generator, the infrastructure classes to be generated still need to be specified because a feature request for a “Client Code only” setting is still open.

To combine the code generation with the previous generation of the OpenAPI specification, there are two things to consider:

1. In the parent maven module, the api sub-module must be placed in before the client/integration test sub-modules
2. The code generator’s path of the OpenAPI specification (<inputSpec>) must refer to the previously downloaded specification

To compile a Java client code generated with the “native” OpenAPI Generator setting, a few dependencies are required.

We provide a minimal configuration for this as a part of the example application on GitHub: https://github.com/viadee/api-roundtrip-spring-boot/blob/main/client/pom.xml

Integration test Execution

If the integration tests are generated in addition to the pure client code, one valid strategy is to implement them as unit tests, but running them separately with the Maven failsafe and not with the classic surefire plugin. The whole setup becomes even more practical if the integration tests are in a separate maven module, which specifies extra rules such as not bundling integration tests with the overall application.

The failsafe plugin is a part of the well-established surefire plugin but its runs in the Maven integration-tests phase by default.

By default, the failsafe plugin comes with a name filter specifying the unit test classes to be executed. The following code example extends the name filter for the integration tests according to the pattern “*IntegrationTest.java”:

Additionally, the following Maven configuration ensures that the integration tests are not run by the surefire plugin during the normal testing phase and the integration test code is neither deployed nor installed in the Maven repository.

In the sample project provided on GitHub, we have encapsulated the integration test execution in a separate profile to gain even more control when we want to run it within a deployment pipeline.

Conclusion

REST APIs have been adopted widely because of their simplicity. However, this simplicity also comes with many challenges, especially for maintenance and sustainability.

The OpenAPI standard pushed REST API development and management to a new level and the generators and integrations based on it, promise ways out of the maintenance dilemma.

The tool chain presented in the article is entirely based on free and standard tools and takes advantage of the sophisticated Spring Boot infrastructure for REST APIs. Similar automations can also be created with frameworks such as Quarkus and Micronaut, as well as for clients in other languages such as JavaScript or Python.

For those who actively want to extend their professional expertise in API development and API management, we think it’s worth looking at OpenAPI, SpringDoc, and OpenAPI Generator.

Modern APIs are standalone products that significantly enable completely new digital business models and processes. The increasingly conscious handling of high-quality APIs helps to tap this great potential.

Similarly, automation, such as the tool chain presented in the article, helps to develop new quality standards with significantly less manual effort.

Furthermore, OpenAPI specifications provide the base for many other use cases such as public or private API directories or API quality analysis for better, more homogeneous API products.

Are your striving to professionalize your API development? I am happy to discuss your thoughts and questions with you. Just write me an email or get in touch on LinkedIn.

Dr. Benjamin Klatt is an Integration Architect and Agile Coach.

His focus is on the digitization of Products and Processes, Integration Architectures and Agile Methods.

Originally published at https://blog.viadee.de.

Recently, Docker introduced rate-limiting for the Docker Hub image registry. It restricts unauthenticated docker pull requests to 100 requests per 6-hour timeframe and authenticated requests from a free account to 200 requests per 6-hour timeframe (see here).

While there are plenty of ways to mitigate the impact of these limits on your development lifecycle and infrastructure, we still need a way to monitor the available requests for our infrastructure. Gitlab published a lightweight solution in their blogpost.

We took their work one step further and created a Docker image and Helm chart based on their script. Both allow you to easily install the docker-hub-rate-limit-exporter script into your Kubernetes cluster.

By the way, it’s Open Source and you can find all code on GitHub.

The helm chart supports the monitoring of unauthenticated requests and authenticated request to Docker Hub and offers an easy integration with the Prometheus Kubernetes operator. If you follow the guide described here, you will be rewarded with a Grafana-Dashboard that lets you monitor your remaining docker-hub pull requests as can be seen in the image above.

The Problem

If you exceed your Docker Hub rate limit quota, you will be greeted with one of the following errors messages the next time you attempt a docker pull request:

ERROR: toomanyrequests: Too Many Requests.

or

You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.

To avoid getting caught by surprise by those error messages, we suggest using a monitoring solution as described below.

Prerequisites

This guide assumes that you have a running Kubernetes cluster with the Prometheus Operator and Grafana installed. Moreover, you need sufficient privileges to create Helm releases in your Kubernetes cluster.

Installing the Helm chart

To install the Helm chart into your cluster, you simply run the following commands:

$ git clone https://github.com/viadee/docker-hub-rate-limit-exporter.git

$ helm install docker-hub-rate-limit-exporter/chart --namespace=<desired namespace>

This creates a Helm release with the given name in the designated namespace. You can verify that everything works smoothly by port-forwarding the pod that was created:

$ kubectl port-forward 8080:80 -n <namespace>

Once the port-forward is configured, you can open your browser and navigate to http://localhost:8080. Here you will find a set of default metrics along with two metrics that correspond to your Docker Hub rate limit. These are:

• max_requests_total: The maximum number of requests you are allowed to do within the 6-hour timeframe.
• remaining_requests_total: The remaining requests based on your limit and the already sent requests. If this number reaches zero, you are rate-limited in the current timeframe.

Configuring the Prometheus Service Monitor

Next, we need to tell our Prometheus instance to scrap the metrics which are exposed by the deployment in the Helm chart. To do so, we can make use of the ServiceMonitor Custom resource that comes with the Kubernetes Prometheus operator:

Save the above code snipped into a file called serviceMonitor.yaml and apply it to your Kubernetes cluster with the following command:

$ kubectl apply -f serviceMonitor.yaml -n <your namespace>

If you choose to install the ServiceMonitor into the same namespace as your Helm release, you can omit the namespaceSelector blog in the manifest. If not, you additionally have to provide the namespace into which your Helm release was installed. Moreover, you may need to add further labels to your ServiceMonitor depending on your Prometheus configuration.

Configure Docker Hub Credentials

If you have a Docker Hub account and use it to pull images from the registry, you need to configure the Helm chart with those credentials, too. Otherwise, only the unauthenticated rate limit from your IP address will be monitored and not the limit of your Docker Hub account. You can configure the Helm chart to use your Docker credentials by providing additional parameters:

$ helm upgrade docker-hub-rate-limit-exporter/chart --namespace=<desired namespace> --install \
--set config.dockerhubUsername=<you dockerhub username> \
--set config.dockerhubPassword=<Your dockerhub access token>

Alternatively, you can create a values.yaml file:

and install the Helm release into your cluster with the following command:

$ helm upgrade docker-hub-rate-limit-exporter/chart -f values.yaml --namespace=<desired namespace> --install

Creating the Grafana Dashboard

Lastly, we need to configure a Grafana Dashboard to visualize the remaining Docker Hub rate limit. Gitlab created a nice-looking dashboard for that purpose which can be used here. You can find the configuration of that dashboard here.

Simply copy the contents of the linked file and create a new Grafana dashboard. You can do so by following the following steps:

1. Navigate to your Grafana instance.
2. Click on Dashboards and then import.
3. Paste the copied Dashboard json and click on Load.

Congratulations, you are ready to monitor your Docker Hub rate limit! Feel free to visit the GitHub repository and get involved. It’s Open Source and ready to help everyone out there.

Marius Stein is IT consultant at viadee and is part of the competence center Cloud Architectures. He is focused on developing and operating scalable, reliable and secure cloud native solutions.

Originally published at https://blog.viadee.de.

Agile Software Development principles have gained more and more attention in recent years. They help making product teams more flexible and productive. However, to fully leverage the potential of agility in software development projects, you also need an agile operations model and embrace DevOps principles. GitOps follows modern DevOps principles and allows for secure, fast and reliable deployments.

DevOps is not about losing control

Our work with various clients tells us that many IT-Architects wrongly believe that following DevOps principles leads to a loss of control, especially regarding how IT operations are done. More often than not we hear that the operation and development teams need to be strictly separated. Automation along this interface (e.g. Continuous Delivery / Deployment) appears to be difficult because it topples security and neglects auditing and approval requirements.

In this blogpost, we will demonstrate how GitOps can be used to automate deployments while still maintaining approval processes and auditing mechanisms and allowing for a high degree of security.

Continuous Integration and Delivery to implement DevOps paradigms

Before we delve deeper into the concept of GitOps, it is crucial to understand what DevOps actually is. The term DevOps is a combination of the terms Development and Operations, emphasizing the close collaboration between the two tasks — a requirement for a flexible and agile software development lifecycle. The core idea of DevOps is to enable a fast, yet flexible process for the development, integration and deployment of applications. To achieve such a process, heavy automation along each step of the software development lifecycle is crucial. Continuous Integration and Continuous Delivery / Deployment are two principles to achieve this kind of automation.

The principle of Continuous Integration (CI) suggests that code is automatically built and tested as soon as a commit is pushed to the applications repository. Thus, CI enables the software development team to find bugs early by failing tests or builds in the CI pipeline. CI is a well adopted approach and there is a myriad of tools supporting it.

Continuous Delivery / Deployment (CD) relates to automation along the release and deployment process of the software development lifecycle. Although often taken for the same, there is a slight yet important difference between Continuous Delivery and Continuous Deployment.
Continuous Delivery describes the state of being ready to release a certain version of your application to production (or any other environment) at any time by just the single click of a button. On the contrary, Continuous Deployment describes the process of automatically deploying each commit, which results in a successful build, to a certain environment (which can be the development, QA or production environment).

Are you already using CD pipelines?

Although we find that CI mechanisms are widely adopted, we see that CD mechanisms are frowned upon by many IT-Architects for the reasons mentioned above. Large organizations make heavy use of CI yet fall back on a manual process for the deployment to QA and production environments. Not only does this introduce human errors in the deployment process but it also slows down the process. To avoid these impediments, we propose GitOps to stay in control of automatic deployments to any environment while allowing for a custom approval process and seamless auditability.

GitOps to improve transparency

GitOps allows to operate deployments to a Kubernetes cluster only by commits to a git repository. The git repository (also called state repository) acts as the single source of truth for the state of your application landscape. Whenever your application landscape is to be changed, this is done through one or more commits to the state repository. By utilizing Merge / Pull Request features that come with most commonly used source control systems, it is possible to implement any approval process required in a company. Once the approval is given for the changes, GitOps suggests automatic deployments to the desired environment.

By using git as the single source of truth for the application landscape, every change is automatically recorded in the git history. This allows for seamless auditability with regard to who made what changes at what time. Furthermore, having your application landscape historized allows for quick rollbacks to previous versions of your landscape. Going back is as easy as reverting to a previous git commit and pushing that change to the state repository of your application.

CNCF recommends Flux

If you are orchestrating your application landscape through Kubernetes, Flux CD is a daemon that runs inside your Kubernetes cluster and is responsible for synchronizing a GitOps state repository with the cluster. Whenever Flux CD detects a discrepancy between the current state of your Kubernetes cluster and the state that is defined in the state repository, it will update the cluster to eliminate the difference. The figure below illustrates the GitOps operations model with Flux CD.

With GitOps, a developer commits the configuration of a Kubernetes cluster (Kubernetes .yaml files) to the state repository instead of applying it directly to the Kubernetes cluster. This allows for approval processes via Merge / Pull Requests and historizes every state of the Kubernetes cluster to enable seamless auditing. Next, Flux CD is responsible for picking up the changes from the state repository and applying it to the cluster.

One important difference to other operation models is that Flux CD, which runs inside the Kubernetes cluster, actually pulls the changes from the state repository instead of the state repository pushing changes to the cluster. This allows for a higher level of security since we do not need to give administrative privileges to the state repository outside of the cluster. In fact, we could disallow any write operation to the Kubernetes cluster from outside of the cluster. It would even be possible to further maximize security by not exposing the Kubernetes API endpoint at all.

Another benefit of Flux CD is that it allows you to synchronize one state repository with any number of Kubernetes clusters. Thus, the GitOps model allows to manage an IT landscape compromised of many Kubernetes clusters by one or more state repositories. Moreover, as the whole configuration of your application landscape is defined in git, it is easy to redeploy your landscape to a new Kubernetes cluster in case there are issues with the current one.

Take away

All in all, GitOps is a modern CD principle that fulfils the requirements for auditing, approval processes and security. If you want to learn more about GitOps, check out our recorded Websession where we go into more detail with the concept of DevOps, CI / CD, GitOps and Flux CD — including a live demonstration that showcases a typical GitOps workflow.

Are your using GitOps in your organization or planning to do so? I am happy to discuss your thoughts and questions with you. Just write me an email or get in touch on LinkedIn.

Marius Stein is IT consultant at viadee and is part of the competence center Cloud Architectures. He is focused on developing and operating scalable, reliable and secure cloud native solutions.