• Introduction
• TL;DR: Why RFC 9457?
• Let’s write some code!
  • 1. Hand-made ProblemDetail + ExceptionMapper
  • 2. Zalando Problem
  • 3. Quarkus: quarkus-http-problem
  • 4. Spring Boot – a short note
• Conclusion
• What’s Next?

Introduction

If you’ve consumed more than one or two REST APIs, you’ve seen the pattern. One service returns {"error": "..."}, another {"message": "...", "code": 42}, a third returns 200 OK with an error hidden somewhere deep in the response. Your REST client code fills up with special cases for each one. Sounds familiar?

RFC 9457 – Problem Details for HTTP APIs (the successor to RFC 7807) defines a single JSON envelope for errors, served as application/problem+json MIME type. It is a small spec: five well-defined bits of information and an extensions map for anything else you might need.

{
  "type": "urn:problem-type:validation-error",
  "title": "Validation Failed",
  "status": 400,
  "detail": "The request body or parameters failed validation.",
  "extensions": {
    "violations": [
      { "field": "title", "message": "Title is required" }
    ]
  }
}

TL;DR: Why RFC 9457?

Why not keep creating your own?

• Consumers already know the shape. Generated SDKs, gateways, log pipelines, and tracing tools can parse application/problem+json without extra work.
• You can extend it without breaking clients. The extensions map is part of the spec – put what you need in there.
• It separates the category from the instance. type says “this is a validation error” (stable, machine-readable); detail and instance describe what happened this time.

💡 Note: RFC 9457 is just a JSON structure and a content type. No library or framework is required. That’s why there are so many implementations – and why a hand-made one is often a reasonable choice.

Let’s write some code!

I have created a repository called API Guide for Java to showcase the patterns for one of my talks. For this post, have a look at ProblemDetail.java and the mappers next to it under com/mehmandarov/confapi/error/.

1. Hand-made ProblemDetail + ExceptionMapper

What it looks like

Imagine you have a REST interface looking like this:

@GET
@Path("/{id}")
@Operation(summary = "Get room by ID")
@APIResponse(responseCode = "200", description = "Room found")
@APIResponse(responseCode = "404", description = "Room not found")
public Room getById(
        @Parameter(description = "Room ID", required = true)
        @PathParam("id") String id) {
    return repo.findById(id)
            .orElseThrow(() -> new NotFoundException("Room not found: " + id));
}

Now, you can add a single ProblemDetail class – built around the five RFC 9457 elements and an extensions map – and one ExceptionMapper per error category.

public class ProblemDetail {
    private URI type = URI.create("about:blank");
    private String title;
    private int status;
    private String detail;
    private URI instance;
    private final Map<String, Object> extensions = new LinkedHashMap<>();

    public static ProblemDetail of(int status, String title) { /* ... */ }
    public ProblemDetail withType(String typeUri)            { /* ... */ }
    public ProblemDetail withExtension(String key, Object v) { /* ... */ }
    // + getters/setters
}

The interesting part is how it gets used. As you can see from the resource code above, there is no try/catch in resources, ever – every exception is turned into a Problem Details response by an ExceptionMapper:

@Provider
public class ConstraintViolationExceptionMapper
        implements ExceptionMapper<ConstraintViolationException> {

    @Override
    public Response toResponse(ConstraintViolationException ex) {
        List<Map<String, String>> violations = ex.getConstraintViolations()
                .stream().map(this::toMap).toList();

        ProblemDetail problem = ProblemDetail.of(400, "Validation Failed")
            .withType("urn:problem-type:validation-error")
            .withExtension("violations", violations);

        return Response.status(400)
                .type("application/problem+json")
                .entity(problem).build();
    }
}

One mapper per category keeps each file small and obvious: ConstraintViolationExceptionMapper → 400, NotFoundExceptionMapper → 404, NotAuthorizedExceptionMapper → 401, ForbiddenExceptionMapper → 403, and a CatchAllExceptionMapper → 500 that never leaks stack traces to clients.

⚠️A word of caution: The catch-all mapper is the safety net for everything you forgot to handle. Without one, an uncaught exception ends up in the server’s default error page, which often includes stack traces, server versions, and sometimes filesystem paths. However, it might be a good idea to handle most of the common exceptions explicitly, and leave the generic catch-all for something truly unexpected.

✅ Pros:

• Portable across runtimes. The same code runs on Quarkus, Helidon, and Open Liberty. No runtime-specific extension.
• No extra dependencies. RFC 9457 is just a JSON structure; you don’t need a library to emit one.
• Small, readable surface. The error model fits on one slide. When something goes wrong, you can read the source.

❌ Cons:

• You write the boilerplate yourself – one mapper per category.
• Nothing maps validation, WebApplicationException, or uncaught Throwable automatically – you wire each one up. (This can also be one of the pros, depending on the way you look at things.)
• No content negotiation between application/json and application/problem+json unless you add it yourself. (Spring, for example, has a built-in ProblemDetail that does this for you.)

💡 Want to know more? The full code, including all five mappers, lives in com/mehmandarov/confapi/error/.

2. Zalando Problem

What it looks like

The Zalando Problem library (org.zalando:problem + jackson-datatype-problem) gives you Problem and ThrowableProblem types and Jackson serialization. You still write an ExceptionMapper to bridge JAX-RS exceptions to Problem, but you don’t define the envelope yourself.

import org.zalando.problem.Problem;
import org.zalando.problem.Status;

Problem problem = Problem.builder()
        .withType(URI.create("urn:problem-type:validation-error"))
        .withTitle("Validation Failed")
        .withStatus(Status.BAD_REQUEST)
        .with("violations", violations)
        .build();

return Response.status(400)
        .type("application/problem+json")
        .entity(problem).build();

✅ Pros:

• Cross-runtime. Works on Quarkus, Helidon, and Open Liberty – the same artifact deploys on all three.
• Used in production at Zalando (and elsewhere); the model handles cause chains, stack-trace processing, and a few edge cases you probably would not have thought of upfront.
• Jackson integration is done for you via jackson-datatype-problem.

❌ Cons:

• One more dependency to track and upgrade.
• You still write the ExceptionMappers – the library standardises the payload, not the wiring.
• If your stack is JSON-B rather than Jackson, you have a bit of extra work.

3. Quarkus: quarkus-http-problem

If you’re only targeting Quarkus, the quarkus-http-problem Quarkiverse extension is the shortest path. It auto-maps ConstraintViolationException, WebApplicationException, and uncaught Throwable to application/problem+json with no boilerplate from you.

✅ Pros:

• Add the dependency and you get Problem Details for exceptions. No need to write a mapper for each of them.
• Reasonable defaults for validation and security exceptions.

❌ Cons:

• Quarkus only. Doesn’t help on Helidon (Jersey) or Open Liberty (CXF). If “runs on every Jakarta runtime” is a requirement, this is out.
• Less visibility into what gets mapped to what – fine until you need to override a default.

4. Spring Boot – a short note

For completeness, we need to mention Spring Boot 3+ as well, which has Problem Details built in as org.springframework.http.ProblemDetail, with content negotiation and @ExceptionHandler integration already wired up. If you’re on Spring, just use it. The JSON structure is the same RFC 9457; only the wiring differs.

Conclusion

The point of RFC 9457 is not that there’s one correct implementation – there are several reasonable ones – but that there’s one correct envelope. Once your API speaks application/problem+json, clients stop hand-coding error parsers for each new service they consume.

A few rules of thumb:

• On Spring, use the built-in ProblemDetail.
• On Quarkus only, reach for quarkus-http-problem and move on.
• For cross-runtime Jakarta, choose between Zalando Problem (one dependency, more handled for you) and the hand-made approach (no dependencies, about 30 lines you fully understand).

I picked the hand-made approach for the demo project because portability across Quarkus, Helidon, and Open Liberty mattered, and because the ExceptionMapper is the demo – hiding it behind a library would have defeated the point of the talk.

However, “hand-made” doesn’t have to mean “everyone reinvents it from scratch”. Write it once, put it in a small internal library, and reuse it across services. That’s still less code than wiring up a third-party dependency in each runtime.

Summary Comparison

What’s Next?

Error handling is one of the bonus topics in the API Guide for Java. The same repo also covers OpenAPI documentation, security (RBAC, JWT), pagination, async, and versioning strategies – see my earlier post on API versioning in Java using JAX-RS.

Happy shipping of well-formed error messages, folks!

• Introduction
• Why Versioning?
• Show Me The CODE!
• 1. URL Versioning
• 2. Header Versioning
• 3. Media Type Versioning
• 4. Request Parameter Versioning
• 5. Bonus: Combining Strategies
• 6. End-Point Deprecation
• Summary Comparison
• Conclusion
• What’s Next?

Introduction

When working with APIs over time we would often need to make some changes to end-point definitions – like adding or deleting resources or changing the attributes for a resource. To ensure backwards compatibility, we often have to introduce versioning for our APIs. APIs, like all software, evolve. You might be adding optional fields or introducing a breaking change. At some point, you will need versioning to support coexistence of the old and new consumers.

However, versioning the API endpoint introduces a question of how this should be done. In this post, we’ll explore several common API versioning strategies, using Jakarta EE and Java.

💡 Note: There is no silver bullet – instead, we’ll explore pros, cons, and real-world fit.

Why Versioning?

Why not just change the API?
Because breaking contracts is dangerous — clients may not update in sync, and you’ll break production consumers.

Versioning allows you to:

• Support legacy clients
• Introduce new features safely
• Deprecate responsibly

⚠️ Caution: Versioning can cause “version explosion.” Each version increases long-term maintenance cost – aka technical debt.

Best Practice: Prefer backward-compatible changes (e.g., adding fields) whenever possible. To mitigate risks, it’s important to follow best practices for versioning and provide clear documentation and migration paths for users. Also, remember to deprecate old versions to minimize maintenance efforts.

Show me the CODE!

I have created a repository called Random Strings to showcase various concepts. For this blogpost, I would recommend having a look at RandomStringsAPIDemoController.java and request_examples.http. You will find all the info on building and running the code in the repo’s README.md file. Each section below will contain “How to call it” part with an example using curl or HTTP-files, and will be based on this repo.

1. URL Versioning

What it looks like

A version appears directly in the URI path. If your API is at https://example.com/api, and the current version is version 1, the URL for a resource might look like this: https://example.com/api/v1/resource:

@GET
@Path("/v2/")
@Produces(MediaType.APPLICATION_JSON)
public Response getV2() {
    return Response.status(Response.Status.NOT_IMPLEMENTED)
        .entity("This v2 using *path versioning* of the API is not implemented.")
        .build();
}

How to call it

cURL:

curl -X GET http://localhost:8080/api/rnd/v2/ \
  -H "Accept: application/json"

HTTP Request (.http file):

GET http://localhost:8080/api/rnd/v2/
Accept: application/json

✅ Pros:

• Simple and intuitive. Visible.
• Easy to test (e.g., with curl or Postman directly in a browser).
• Plays well with gateways and reverse proxies.
• Clear visual distinction between versions.

❌ Cons:

• Pollutes the URI with versioning logic.
• Breaks REST’s principle of stable resource identifiers.
• Clients have to update URLs when migrating.
• Risk of accumulating too many legacy versions.
• Can result in cluttered and difficult-to-read URLs if there are multiple versions of the API.

🔍 However: Despite its REST purism flaw, URL versioning is extremely practical and widely adopted.

2. Header Versioning

What it looks like

Client specifies version in a custom HTTP header (e.g., Accept-Version, X-API-Version, etc.):

@Path("/hi2")
@GET
@Produces({"application/json"})
public String entryPoint2(@HeaderParam("Accept-Version") String apiVersion) {
    if (apiVersion == null || apiVersion.isEmpty()) {
        return "Default unversioned endpoint hit.";
    }
    String message = "Versioned: Using custom headers. Using version: " + apiVersion +".";
    return message;
}

How to call it

Note: This is for demo purposes only. It has to have a different URL than the regular API; otherwise, it will also intercept calls that do not contain the Accept-Version header.

cURL:

curl -X GET http://localhost:8080/api/rnd/versioned/ \
  -H "Accept: application/json" \
  -H "Accept-Version: 2"

HTTP Request (.http file):

GET http://localhost:8080/api/rnd/versioned/
Accept: application/json
Accept-Version: 2

✅ Pros:

• Keeps URL structure clean and predictable.
• Closer to HTTP semantics (headers = metadata).
• Allows centralized versioning logic in filters/interceptors.

❌ Cons:

• Not self-descriptive — clients must “know the secret handshake”.
• Poor discoverability (not visible in browser without tools).
• Breaks caching in some proxies/CDNs unless explicitly configured.
• Adds complexity to tooling and testing.

⚠️ Challenge: Header versioning can feel “invisible” and cause developer confusion if not well-documented.

3. Media Type Versioning (Content Negotiation)

What it looks like

Client specifies version via a custom media type in the Accept header. This is sometimes called Content Negotiation versioning.

Accept: application/hi.v3+json

In Jakarta EE:

@Path("/hi")
@GET
@Produces({"application/hi.v3+json", "application/hi.v4+json"})
public String entryPoint() throws URISyntaxException {
    String message = "Versioned: Hai there!";
    return message;
}

How to call it

You can request different versions (e.g., v3, v4, v5) by updating the media type:

cURL:

curl -X GET http://localhost:8080/api/rnd/ \
  -H "Accept: application/rnd.v3+json"

HTTP Request (.http file):

GET http://localhost:8080/api/rnd/
Accept: application/rnd.v3+json

✅ Pros:

• Very REST-compliant: changes representation, not resource.
• URI remains stable.
• Supports richer format negotiation (e.g., XML, HAL, etc.).

❌ Cons:

• Requires strict control over media types.
• Not all clients/tooling handle custom media types well.
• Breaks with some reverse proxies and middleware that don’t forward full Accept headers.
• More work to configure content negotiation.

🧪 Observation: Elegant in design, but rarely used consistently in real-world public APIs.

4. Request Parameter Versioning

What it looks like

Technically, it is also possible for the client to specify the version in a URL query parameter (e.g., ?version=2). This, however, might not be a suggested strategy, in my opinion.

https://example.com/api/resource?version=2

How to call it

cURL:

curl -X GET http://localhost:8080/api/rnd?version=2 \
  -H "Accept: application/json"

HTTP Request (.http file):

GET http://localhost:8080/api/rnd?version=2
Accept: application/json

✅ Pros:

• Simplicity & discoverability: Easy to test in a browser without specialized tools.
• Defaulting logic: Straightforward to implement “default to latest” if the parameter is omitted.
• Caching friendly: CDNs treat different query params as unique resources by default.

❌ Cons:

• URI Pollution: Mixes resource identification with technical metadata.
• Routing complexity: Routing based on query parameters usually requires custom middleware or manual logic inside the controller.
• Harder to generate clean, automated documentation (like OpenAPI) when multiple versions share the same path.

5. Bonus: Combining Strategies - Transparent URI Rewriting (Enterprise Pattern)

In large enterprises, you might find that different clients have different needs. Some prefer the explicitness of URL versioning, while others require the clean URIs of Header versioning. You don’t have to choose just one—you can support both without duplicating your backend routing logic.

The common practice is to structure all your resource classes using URL versioning (e.g., @Path("/v1/resource")), but use a @PreMatching Filter to intercept requests and transparently rewrite the URI if a client uses a header instead.

Here is what that looks like in Jakarta EE using a ContainerRequestFilter:

@Provider
@PreMatching
public class HeaderVersionFilter implements ContainerRequestFilter {

      @Override
      public void filter(ContainerRequestContext ctx) {
          String path = ctx.getUriInfo().getPath();

          // If the path is already versioned (e.g., starts with v1, v2), let it pass
          if (path.matches("v\\d+(/.*)?")) return;

          // Otherwise, check if the client provided a version header
          String version = ctx.getHeaderString("X-API-Version");

          if (version != null && !version.isEmpty()) {
              // Transparently rewrite the URI internally to match our URL-based routes
              String newPath = "v" + version + "/" + path;
              URI baseUri = ctx.getUriInfo().getBaseUri();
              URI newUri = UriBuilder.fromUri(baseUri).path(newPath).build();

              ctx.setRequestUri(baseUri, newUri);
          }
      }
}

✅ Pros:

• Ultimate Flexibility: Clients can use http://api.example.com/v2/resource OR http://api.example.com/resource with an X-API-Version: 2 header.
• Single Source of Truth: Your backend controllers only need to use @Path("/v2/"). You don’t have to write duplicate methods to handle both headers and paths.

❌ Cons:

• Magic Routing: It introduces a layer of “magic” where the requested URI differs from the routed URI, which can briefly confuse new developers debugging the application.

💡 Want to know more? Read up on terms Version Normalization and Internal Decoupling.

6. End-Point Deprecation

Eventually, you will need to retire old API versions. Remember: every old version you keep around is technical debt — it increases long-term maintenance cost. When deprecating an endpoint, consider the following best practices:

1. Update the Docs: Use OpenAPI’s @Operation annotation to clearly mark it as deprecated.
2. Add @Deprecated: Use the Java @Deprecated annotation where necessary.
3. HTTP Redirects: Consider returning HTTP codes like 302 Found or 301 Moved Permanently after some time.
4. Add a Link header: Provide a link to the new version in the response headers.
5. Log / Count calls: Track usage (e.g., with MicroProfile @Counted) to know when it is safe to finally remove the endpoint.

Here is a practical example in Jakarta EE showing how to deprecate an endpoint, add a Link header, and track metrics:

@GET
@Path("v0.1/")
@Produces(MediaType.APPLICATION_JSON)
@Operation(summary = "DEPRECATED. Use v2 now. Returns the adjective-noun pair",
           description = "Deprecated function. The pair of one random adjective and one random noun is returned as an array.")
@Counted(name = "totalCountToRandomPairCalls_Versioned_Path_DEPRECATED",
         absolute = true,
         description = "Deprecated function call: Total number of calls to random string pairs.",
         tags = {"calls=pairs"})
@Deprecated
public Response getRndStringPathDeprecated() {
    URI newVersionURI = UriBuilder.fromUri("/api/rnd/v2/").build();
    Link newVersionLink = Link.fromUri(newVersionURI).rel("alternate").build();
    return Response.ok("Deprecated response", MediaType.APPLICATION_JSON)
            .header(jakarta.ws.rs.core.HttpHeaders.LINK, newVersionLink.toString())
            .header("X-API-Version", "0.1")
            .build();
}

How to call it (Deprecated endpoint)

cURL:

curl -X GET http://localhost:8080/api/rnd/v0.1/ \
  -H "Accept: application/json"

HTTP Request (.http file):

GET http://localhost:8080/api/rnd/v0.1/
Accept: application/json

Summary Comparison

The following table summarizes all the different routing strategies implemented in the demo project, illustrating how the HTTP method, path, and headers combine to invoke the correct Java method. The method names refer to the methods in RandomStringsAPIDemoController.java (or RandomStringsController.java):

Conclusion

There is no single correct approach to API versioning. For most teams and public APIs, URL versioning is good enough—it’s visible, easy to test, and plays well with existing tooling. However, you might want to use header versioning if your APIs are primarily consumed by internal services or SDKs that can abstract away the complexity. Reserve media type versioning for hypermedia-rich or REST-purist APIs, and only if your tooling supports it end-to-end.

Consider who your consumers are, whether your API is public or internal, your infrastructure maturity, and your team’s ability to support multiple versions.

What’s Next?

Versioning is just one part of building robust REST APIs. If you want to dive deeper, have a look at the API Guide for Java repository and the slides in the presentation folder. They cover documentation with OpenAPI, security best practices (like RBAC and JWT integration), advanced patterns (pagination, async APIs), and going beyond REST with gRPC and GraphQL.

Happy coding!

• Introduction
• Setup
• Conclusion

Introduction

The built-in development mode for Quarkus is a great functionality that lets you update the application code, resources, and configurations. Setting it up is a great way to develop your applications locally, as you can immediately see the changes reflected in your application.

Furthermore, we have a remote development mode, which lets you make changes to local files immediately available in a containerized environment. Remote development mode works excellently if the container runs in a local Docker or remote containerized environment.

However, running several simultaneous containers with the remote development mode on, mapped to the same domain, may result in warnings and erratic behavior from the client side.

Setup

Imagine a setup where you are running a set of containers, for example, using docker-compose and mapping them all to my.cluster.host.com (or even localhost)through several ports:

First, you will need to update quarkus.live-reload.url in the properties for all the apps (see docs on where and how to do this). Update the settings to the correct domain and port (in our case, it is 8080, 8081, or 8082):

quarkus.live-reload.url=http://localhost:8081

Next, try starting your containers with the remote development mode enabled and connect to the application from a terminal or an IDE. For the second and the consecutive applications, the attempts to establish a connection you will see the following message in the logs:

$> ./mvnw quarkus:remote-dev -Dquarkus.profile=dev

< ... >

[WARNING] Changed debug port to 57409 because of a port conflict
Listening for transport dt_socket at address: 57409

< ... >

Note: Fallback ports will be random and may vary from the one above.

This setup will break the remote reloading from the terminal on the client side (i.e., your IDE). Two or more of your client applications now see that the default port 50005 for a remote debug is in use and start with a new, random port.

The simple fix is to update the debug ports for all other applications to something other than 5005, such as 6006 and 6007. Custom debug ports can be set in the pom.xml files, under quarkus-maven-plugin, for each of the applications that require this update:

<build>
  <plugins>
    <plugin>
      <groupId>io.quarkus.platform</groupId>
      <artifactId>quarkus-maven-plugin</artifactId>
      <version>${quarkus.platform.version}</version>
      <!-- ADD THE CONFIGURATION MENTIONED BELOW THIS LINE -->
      <configuration>
        <debug>6006</debug>
      </configuration>
...

You can choose whether to update the debug ports for all applications in the cluster or for all applications except one, which will get the default port.

Now, you will need to rebuild your apps and re-initiate the remote development mode for each container. And, voilà, everything works!

One last note: Please ensure you do not use the remote development functionality in the production environment.

Conclusion

A tiny config update brings back the development joy of using remote development mode for more than one container simultaneously.

Happy coding!

• Step 1: Define Pipeline Options
• Step 2: Create the Pipeline
• Step 3: Apply Transformations
• Step 4: Run it!
• Conclusion

In this post, I would like to show you how you can get started with Apache Beam and build the first, simple data pipeline in 4 steps.

Step 1: Define Pipeline Options

Let’s start with creating a helper object to configure our pipelines. This is not an absolute necessity, however defining the pipeline options might save you some time later, especially if your pipeline is dependent on a few arguments, that might have pre-defined, default values that you don’t want to provide at every run.

public interface OsloCityBikeOptions extends PipelineOptions {

    /**
     * By default, the code reads from a public dataset containing a subset of
     * bike station metadata for city bikes. Set this option to choose a different input file or glob
     * (i.e. partial names with *, like "*-stations.txt").
     */
    @Description("Path of the file with the availability data")
    @Default.String("src/main/resources/bikedata-stations-example.txt")
    String getStationMetadataInputFile();
    void setStationMetadataInputFile(String value);

    // some other options here...
}

Step 2: Create the Pipeline

Now that you have created the pipeline options object, you will need to create the pipeline object itself and provide the options to it:

OsloCityBikeOptions options = 
        PipelineOptionsFactory.fromArgs(args)
                                .withValidation()
                                .as(OsloCityBikeOptions.class);

Pipeline pipeline = Pipeline.create(options);

(Check out the documentation for the PipelineOptionsFactory class for the description of the methods used above.)

Step 3: Apply Transformations

After defining the pipeline and providing the options class, we can start by applying the transformations using .apply(...). Those can be chained after each other by applying yet another .apply(...), for instance:

PCollection <KV<Integer, LinkedHashMap>> stationMetadata = pipeline
                .apply("ReadLines: StationMetadataInputFiles", TextIO.read().from(options.getStationMetadataInputFile()))
                .apply("Station Metadata", ParDo.of(fnExtractStationMetaDataFromJSON()));
                .apply(MapElements.into(TypeDescriptor.of(String.class)).via(o -> o.toString()))
                .apply("WriteStationMetaData", TextIO.write().to(options.getMetadataOutput()));

Note that a PCollection<T> is an immutable collection of values of type T and that you can provide names for the transformations as the first string argument in the apply(), like in the first two and the last apply methods.

Here we can also specify custom transformations that can be done in parallel. In Beam, they are being referred to as ParDo methods. They are similar to the Mapper or Reducer class of a MapReduce-style algorithm. In this post, we will not be focusing on the contents of such pipeline (i.e. what it is doing), but a simple example of a ParDo can be looking like the second apply in the code above (look for the link in the conclusion for the entire running example).

pipeline.apply("Station Metadata", ParDo.of(fnExtractStationMetaDataFromJSON()));

Step 4: Run it!

After defining the pipeline, its options, and how they are connected, we can finally run the pipeline. The great thing about running the pipelines in Apache Beam is that it is very easy to switch between various runners. Beam provides a portable API layer for building sophisticated pipelines that may be executed across various execution engines or runners. In our example, we can switch from running the pipeline locally (with direct-runner), to running the same pipeline in the Cloud as a managed service (with dataflow-runner) by simply adjusting the values we provide when running the code.

Local runner

Here is an example of running the pipeline with direct-runner:

mvn compile exec:java \
      -Pdirect-runner \
      -Dexec.mainClass=com.mehmandarov.beam.OsloCityBike \
      -Dexec.args="--inputFile=src/data-example.txt \
      --output=bikedatalocal"

Dataflow runner

And here is the example of running the same pipeline in the Cloud as a managed service, using Google Cloud Dataflow. Note that most of the parameters provided are still the same, with a few additional parameters needed for this specific runner.

mvn compile exec:java \
      -Pdataflow-runner \
      -Dexec.mainClass=com.mehmandarov.beam.OsloCityBike \
      -Dexec.args="--project=rm-cx-211107 \
      --inputFile=gs://my_oslo_bike_data/data-2018-*.txt \
      --stagingLocation=gs://my_oslo_bike_data/testing \
      --output=gs://my_oslo_bike_data/testing/output \
      --tempLocation=gs://my_oslo_bike_data/testing/ \
      --runner=DataflowRunner \
      --region=europe-west1"

Other runners

In case you would like to be using various runners or interested in switching between them, it might be a good idea to check the capability matrix in the documentation, as the core concepts of Beam Model can sometimes be implemented to varying degrees in each of the Beam runners.

Conclusion

We have now seen the basic steps needed to create a simple data-parallel processing pipeline and how that can be run and deployed both in the local and managed Cloud environments. We are were also able to run the same pipeline with just a few adjustments to the command line parameters and, in our case, without any changes to the pipeline code.

The entire working example that we have been using here can be found in my GitHub repository, as well as a more advanced example in another repository.

• Intro
• TL;DR: Getting Graph Representation
• A Full Example
• What Now?

Intro

Constructing advanced pipelines, or trying to wrap your head around the existing pipelines, in Apache Beam can sometimes be challenging. We have seen some nice visual representations of the pipelines in the managed Cloud versions of this software, but figuring out how to get a graph representation of the pipeline required a little bit of research. Here is how it is done in a few steps using Beam’s Java SDK.

TL;DR: Getting Graph Representation

If you just want to see a few lines that let you generate the DOT representation of the graph, here it is:

import org.apache.beam.runners.core.construction.renderer.PipelineDotRenderer;

Pipeline p = Pipeline.create(options);
// do stuff with your pipeline
String dotString = PipelineDotRenderer.toDotString(p);

Now, if you want a slightly more comprehensive example, keep on reading.

A Full Example

Here we will be using word count example, particularly the MinimalWordCount class.

Adding Maven Dependency

First, we need to add a dependency to the Maven file under <dependencies> section:

<dependencies>
    <!-- ... all the other dependencies you may have -->
    <dependency>
        <groupId>org.apache.beam</groupId>
        <artifactId>beam-runners-core-construction-java</artifactId>
        <version>${beam.version}</version>
    </dependency>
</dependencies>

The Code

Now, we will need to add a few imports (assuming you already added the Maven dependency mentioned earlier):

import org.apache.beam.runners.core.construction.renderer.PipelineDotRenderer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

To get the DOT representation of the pipeline graph we will be passing the pipeline object to the PipelineDotRenderer class, and in this example, we are only logging the output to the console (hence the log4j imports).

// Create the Pipeline object with the options we defined above
Pipeline p = Pipeline.create(options);

// ... do stuff with your pipeline ...

// Add this piece of code just before running the pipeline:
final Logger log = LoggerFactory.getLogger(MinimalWordCount.class);
String dotString = PipelineDotRenderer.toDotString(p);
log.info("MY GRAPH REPR: " + dotString);

p.run().waitUntilFinish();

That’s it. To see the code in action, run it from the command line:

$ mvn compile exec:java \
        -Dexec.mainClass=org.apache.beam.examples.MinimalWordCount \
        -Pdirect-runner

This code will produce a DOT representation of the pipeline and log it to the console.

A Complete Example

A fully working example can be found in my repository, based on MinimalWordCount code. There, in addition to logging to the console, we will be storing the DOT representation to a file.

In the next section, we will have a brief look at what can be done with the DOT representations.

What Now?

Now that we have a DOT representation of the pipeline graph, we can use it to get a better understanding of the pipeline. For instance, you can generate an SVG or a PNG image from the data. Note that the generated graph might be a bit verbose, but gives a good overview of the pipeline graph.

Here, I have also included examples of the DOT graph and the PNG file generated for that particular pipeline.

Assuming that you have Graphviz tools installed, you can convert a DOT file to a PNG image using this command:

$ dot -Tpng -o pipeline_graph.png pipeline_graph.dot

In addition to Grapgviz (Wikipedia link), there are also online services for converting DOT graphs to graphical representations, like this one.

Update (April 2026): This post has been refreshed to use the modern jakarta.* namespace introduced in Jakarta EE 9+. If you are working with an older codebase that still uses the legacy javax.* namespace, the concepts described here apply identically — only the package prefix differs.

• Intro
• Getting started
• Defining End-Points
  • A (Regular) JSON End-Point
  • An End-Point Returning an Image
  • An End-Point Returning a PDF
• What’s Next?

Intro

In my previous two posts, I have been describing parts of a system for “checking-in” to a location using QR codes in Java. We started with generating QR codes, followed by generating PDF files.

Now, I would like to focus on building microservices around that functionality. We will be creating a few HTTP end-points built with MicroProfile. I will be using the next few posts as an opportunity to show off some of the features that you will be getting out of the box or with minimal effort using MicroProfile.

Getting started

(Assuming that you have Git, Java 9+, and Maven installed.)

Since we will continue using the QR code generator project to showcase various features of MicroProfile, it might be a good idea to familiarize yourself with the code. You might want to start with taking a look at my previous two posts that explain the code for generating QR codes, and generating PDF files in detail.

Now you can clone the project and examine the Maven dependencies in the pom.xml file, as well as any other MicroProfile related dependencies.

After cloning and opening the project in your favorite IDE, build it (again, assuming that you have Java and Maven installed) with the following command in a terminal:

$ mvn clean package

When the project is done building and you have got a Build Success from Maven, you can run the project to make sure everything runs fine:

$ java -jar target/qrcreator.jar

In a few seconds it takes for the app server to start-up, you should be able to access the starter page at http://localhost:8181/qrcreator/index.html.

Defining End-Points

One of the most obvious features any microservice needs is an end-point to receive requests and respond with some kind of data back. Let’s have a closer look into how this is done in MicroProfile. First of all, we will need to define the application path that serves as the base URI for all resource URIs (think of it as a “root” URL) and make sure that the class where it is defined extends jakarta.ws.rs.core.Application, like in ApplicationEntryPoint class here:

package com.mehmandarov.qrcreator;

import jakarta.ws.rs.ApplicationPath;
import jakarta.ws.rs.core.Application;

/**
 * Sets the application path that serves as the base URI for
 * all resource URIs provided by @Path annotation.
 */
@ApplicationPath("/api")

public class ApplicationEntryPoint extends Application {
}

This will set up all the end-point URLs to start with /api, in our case http://localhost:8181/qrcreator/api/.

A (Regular) JSON End-Point

Now, let’s define some endpoints. We will start with a most regular kind – a JSON end-point. This is probably the most common end-point you will encounter.

We will start with creating one that would respond to requests sent to /api/somestring/json. Note that as you can see from the code below, @Path defines somestring in the URL as an id that is passed on as an argument to the createIdKeyTuple method.

We will also define a type of a request (in this case it is a GET request) and specify that it will be returning a JSON document with @Produces annotation.

@GET
@Path("{id}/json")
@Produces("application/json")
public Response createIdKeyTuple(@PathParam("id") String id) {
    ...
}

Now, that we are done with annotations for the method, let’s have a look at the code for this method, that defines a JSON end-point in the QRController class:

@GET
@Path("{id}/json")
@Produces("application/json")
public Response createIdKeyTuple(@PathParam("id") String id) {
    String nameKeyTuple = null;
    try {
        nameKeyTuple = qrCodeContentsSupplier.getQRCodeContents(id);
        return Response.ok(nameKeyTuple).build();
    } catch (InvalidKeySpecException e) {
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    } catch (Exception e) {
        e.printStackTrace();
    }
    return Response.serverError().build();
}

Note that in this code we have only simple exception handling that makes sure we return a correct HTTP status code – OK (200) on a success and server error (500) on an internal error. You can later add other HTTP codes based on your needs.

An End-Point Returning an Image

Returning an image instead of a JSON document is quite similar to the code we have already seen. Here we will have to pay attention to three aspects:

• Different MIME type defined in @Produces annotation: @Produces("image/png")
• Additional elements in the response header that let you control how the created file is displayed in the browser, i.e. shown in the browser “inline”, or made available through a download dialog – "Content-Disposition", "inline;" (see specs for Content-Disposition for more details)
• Additional elements in the response header that let you control the name for the created file: filename=\"" + id + ".png\"

Let’s have a look at the whole method:

@GET
@Path("{id}")
@Produces("image/png")
public Response createQR(@PathParam("id") String id) {
    try {
        byte[] imageData = qrCodeSupplier.qrCodeGenerator(id);
        return Response.ok(imageData)
            .header("Content-Disposition", "inline; filename=\"" + id + ".png\"")
            .build();
    } catch (WriterException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    } catch (InvalidKeySpecException e) {
        e.printStackTrace();
    } catch (Exception e) {
        e.printStackTrace();
    }
    return Response.serverError().build();
}

It is worth noting that the number of catch statements in the try...catch clause will vary and depend on the number and type of exceptions that can be thrown by the underlying methods.

An End-Point Returning a PDF

The last method for today – defining an end-point for returning PDF files – is nearly identical to the one we used for returning images, except for one thing:

• Different MIME type and explicit file encoding in @Produces: @Produces("application/pdf; charset=utf-8").

@GET
@Path("{id}/pdf")
@Produces("application/pdf; charset=utf-8")
public Response createQRPDF(@PathParam("id") String id) {
    try {
        byte[] pdfDocument = pdfDocumentSupplier.pdfDocumentGenerator(id);
        return Response.ok(pdfDocument)
            .header("Content-Disposition", "inline; filename=\"" + id + ".pdf\"")
            .build();
    } catch (WriterException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    } catch (InvalidKeySpecException e) {
        e.printStackTrace();
    } catch (Exception e) {
        e.printStackTrace();
    }
    return Response.serverError().build();
}

Of course, there are also obvious differences in the contents of the byte[] array, but I consider that being outside the scope of this post – you can study those differences on your own.

What’s Next?

Here we have seen how easy it is to define an end-point that can return documents with various MIME types. In the next posts, we will be taking a closer look at things like how you can equip your end-points with metrics, provide auto-generated documentation based on OpenAPI, and add more resilience with fail-over and circuit-breakers.

• Intro
• Choosing a Library
• Generaring PDF Files
  • Adding Maven Dependency
  • Getting Started With the Code
  • Adding Text
  • Adding Images
  • Adding Metadata to the Document
  • Closing, Saving and Returning the Document
• What Now?

Intro

In my previous post, I have started describing a system for “checking-in” to a location using QR codes in Java. Now, I would like to describe another part of that system that will show you how to get started with generating PDF files using Java.

So, let’s have a look at how this can be implemented in your solution – step by step.

Choosing a Library

Generating PDFs is normally something you would like to do using 3rd party libraries, and there are quite a few alternatives available. While choosing a library it might be a good idea to have a closer look at the licensing for that library – some of them might be very permissive and some might force your code to comply to a specific licensing. Some libraries even come with dual licensing, one under a proprietary model and one supporting an open source model.

One should also consider other aspects like maturity and whether it is a high- or low-level library. The latter will tell you how much code you will actually end up writing to implement your features. In the course of this project, I ended up trying two different libraries – iText and, later, Apache PDFBox. Since the point of this code was a tutorial, I decided to stick with PDFBox as it is distributed under more permissive license – Apache License 2.0, as opposed to dual licensed iText that is under AGPL and a commercial license.

Generating PDF Files

As mentioned earlier, this library provides quite extensive functionality for generating PDF files, but it is also quite low-level, so you will have to be prepared to implement a few things you might usually take for granted, e.g. things like calculating coordinates for text that has to be centered on a page and a few other things. However, the library has a great community, so it is quite easy to get help.

Adding Maven Dependency

Ok, let’s get started. First things first, you will need to add the following dependencies to your pom.xml to use PDFBox (assuming you are using Maven to build your project):

    <dependency>
      <groupId>org.apache.pdfbox</groupId>
      <artifactId>pdfbox</artifactId>
      <version>2.0.15</version>
    </dependency>

Getting Started With the Code

For this post, I decided to paste the source code for the whole function doing the PDF generation and separate it with a few sentences, explaining the most interesting parts of the code. You can always piece the code together, or just have a look at the code in my repo. (Bonus: If you are interested in how much work it was to port the code from iText to PDFBox, this commit should give you a rough idea.)

We start with defining a document object, a page object and add a page to a document. Afterwards, we create a content stream object that will be added to the page and document objects. This object will be responsible for holding the text and images we will be generating here.

// Assume that the following variables are declared and set:
//   QRCodeSupplier qrCodeSupplier - to generate QR codes, shown in the previous post
//   String id - a string that will be shown on the top of the PDF file and used in the QR code
//   String timeZone - a string containing current time zone

String headerTitle = id;
PDFont headerFont = PDType1Font.COURIER_BOLD;

int marginTop = 30;
int fontSize = 30;

PDDocument document = new PDDocument();

PDPage page = new PDPage(PDRectangle.A4);
PDRectangle mediaBox = page.getMediaBox();
document.addPage(page);

PDPageContentStream contentStream = new PDPageContentStream(document, 
                                                            page, 
                                                            PDPageContentStream.AppendMode.APPEND, 
                                                            true);

Adding Text

Now we will need to calculate the coordinates for the header text string and make sure it will appear centered independent of the font and size. This is one of the “low-level” parts you will have to deal with when using PDFBox.

// calculate coordinates to center the header text
float titleWidth = headerFont.getStringWidth(headerTitle) / 1000 * fontSize;
float titleHeight = headerFont.getFontDescriptor().getFontBoundingBox().getHeight() / 1000 * fontSize;
float titleStartX = (mediaBox.getWidth() - titleWidth) / 2;
float titleStartY = mediaBox.getHeight() - marginTop - titleHeight;

Next, we will be adding the text itself and setting font and coordinates for it on the page:

// add header text to the document
// Note: This solution will not support fixed-width paragraphs and text flow
contentStream.beginText();
contentStream.setFont(headerFont, fontSize);
contentStream.newLineAtOffset(titleStartX, titleStartY);
contentStream.showText(headerTitle);
contentStream.endText();

Adding Images

Now, let’s examine how to add an image to a PDF document. Here, you can use createFromFile() in case your image is already available, or createFromImage() is you are generating the image on the fly and/or returning it from another function.

Below, you will also find examples of code to scale and to calculate coordinates for centering the image:

// get image as a byte array
ByteArrayInputStream bais = new ByteArrayInputStream(qrCodeSupplier.qrCodeGenerator(id));
BufferedImage bim = ImageIO.read(bais);

// convert image to an object that can be added to the PDF document
PDImageXObject pdImage = LosslessFactory.createFromImage(document, bim);

// calculate coordinates to center the image
float scale = 1f;
int imageOffset = 100;

float imageWidth = pdImage.getWidth() * scale;
float imageHeight = pdImage.getHeight() * scale;
float imageStartX = (mediaBox.getWidth() - imageWidth) / 2;
float imageStartY = titleStartY - imageHeight - imageOffset;

// add image into the document
contentStream.drawImage(pdImage, imageStartX, imageStartY, 
                        pdImage.getWidth() * scale, pdImage.getHeight() * scale);

// closing the stream
contentStream.close();

Adding Metadata to the Document

Here, you can see a few examples of how you can add metadata to your document. This is done with a help of a few methods available thought the API:

// add metadata
document.getDocumentInformation().setTitle("Generated QR code for " + id + ".");
document.getDocumentInformation().setSubject("with a secure string");
document.getDocumentInformation().setAuthor("rm");
document.getDocumentInformation().setCreator("rm");
document.getDocumentInformation().setCreationDate(date);

Closing, Saving and Returning the Document

After your document is built and you have added all the contents, remember to save and close your document. Now you can either save the document to the file with document.save(), or returning the document as a byte array to another function with byteArrayOutputStream.toByteArray():

// save and close document
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
document.save(byteArrayOutputStream);
document.close();

// return document as byte[]
return byteArrayOutputStream.toByteArray();

What Now?

In the last two posts, we have seen how to generate QR codes with a hashed string and PDF files with Java. In the next post, I will be showing how to put it all together into a MicroProfile microservice.

• Intro
• Generaring QR Codes
• Hashing Strings
• What Now?

Intro

I have been testing out new functionality for “checking-in” to a location using QR codes. To make sure the user is at the specified location and is scanning my QR code (and not a “fake” code created by someone else), I needed to add a way of “signing” each code with a value that only I – the provider of the QR code – could know. This would also make it simple enough to be able to implement the same mechanism in the app used to scan the codes to verify the validity on the client side.

I ended up with a solution where I would have a QR code containing a JSON object with a Name and a Key – a hashed and salted name string. The string will be read by the client app used to scan the code and hashed using the same algorithm with the same secret salt value, and compared to the value in the QR code on the client side.

The data structure inside a generated QR code would be like this:

{ 
    "Name", "MyString",
    "Key", "HashedMyStringWithSecretSalt"
}

When it comes to the implementation, I decided to do the generation or codes in Java and, later, implement this as a standalone microservice. Here, I must admit that I was surprised by how simple it was using a specialized library. More about that below.

So, let’s have a look at how this can be implemented in your solution – step by step.

Generating QR Codes

First, I needed a library that can handle QR codes, and I decided to use Zebra Crossing (“ZXing”) library because of its simplicity and popularity (i.e. community around it).

All you need to get started is to add the following dependencies to your pom.xml (assuming you are using Maven to build your project):

<dependency>
  <groupId>com.google.zxing</groupId>
  <artifactId>core</artifactId>
  <version>3.4.0</version>
</dependency>
<dependency>
  <groupId>com.google.zxing</groupId>
  <artifactId>javase</artifactId>
  <version>3.4.0</version>
</dependency>

This library provides quite an extensive functionality both for generating and reading codes. This was more than enough for my use case where I just needed to generate a QR code with a simple JSON object:

public byte[] qrCodeGenerator(String id) throws IOException, 
                                                WriterException, 
                                                InvalidKeySpecException, 
                                                NoSuchAlgorithmException {

    String filePath = "QRCode.png";
    String charset = "UTF-8";
    Map hintMap = new HashMap();
    hintMap.put(EncodeHintType.ERROR_CORRECTION, ErrorCorrectionLevel.L);

    Map<String, String> qrCodeDataMap = Map.of(
            "Name", id,
            "Key", keyProvider.generateVerificationKey(id) 
            // see next section for ´generateVerificationKey´ method
    );

    String jsonString = new JSONObject(qrCodeDataMap).toString();
    createQRCode(jsonString, filePath, charset, hintMap, 500, 500);

    BufferedImage image = ImageIO.read(new File(filePath));
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ImageIO.write(image, "png", baos);
    byte[] imageData = baos.toByteArray();

    return imageData;
}

private void createQRCode(String qrCodeData, 
                          String filePath, 
                          String charset, 
                          Map hintMap, 
                          int qrCodeHeight, 
                          int qrCodeWidth) throws WriterException, 
                                                  IOException {

    BitMatrix matrix = new MultiFormatWriter().encode(
            new String(qrCodeData.getBytes(charset), charset),
            BarcodeFormat.QR_CODE,
            qrCodeWidth,
            qrCodeHeight,
            hintMap
    );

    MatrixToImageWriter.writeToPath(
            matrix,
            filePath.substring(filePath.lastIndexOf('.') + 1),
            FileSystems.getDefault().getPath(filePath)
    );
}

Note also fun little thing – the conversion of Java hashmaps to a JSON object using JSONObject. Sometimes it is much easier to build up data structure the way you want it, and then serialize to JSON:

Map<String, String> qrCodeDataMap = Map.of(
        "Name", "SampleText",
        "Key", "SomeHashedValue"
);
String jsonString = new JSONObject(qrCodeDataMap).toString();

To be able to use JSONObject class, you would need to add the following dependency to your pom.xml:

<dependency>
  <groupId>org.json</groupId>
  <artifactId>json</artifactId>
  <version>20180813</version>
</dependency>

If you are looking for a more simplified interface, you might also check out QRGen that claims to simplify QR code generation API for Java even further and is built on top ZXing. However, ZXing was absolutely fine in my case.

Hashing Strings

Now, I needed to be able to hash a string in a quick and secure manner. For this, I decided to use the method suggested by OWASP for Java. To implement this method you will need to start with updating your pom.xml:

<dependency>
  <groupId>commons-codec</groupId>
  <artifactId>commons-codec</artifactId>
  <version>1.12</version>
</dependency>

And here is the (somewhat simplified) implmentation of the said method in Java:

public String generateVerificationKey(String str) throws NoSuchAlgorithmException,
                                                         InvalidKeySpecException {
    int iterations = 10000;
    int keyLength = 512;

    char[] strChars = str.toCharArray();
    byte[] saltBytes = salt.getBytes();

    SecretKeyFactory skf = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA512");
    PBEKeySpec spec = new PBEKeySpec(strChars, saltBytes, iterations, keyLength);
    SecretKey key = skf.generateSecret( spec );
    byte[] hashedBytes = key.getEncoded( );

    return Hex.encodeHexString(hashedBytes);
}

What Now?

By now you should be able to generate QR codes with a hashed string. In the next post, I will be sharing code on how to embed and generate PDF files with this information with Java, followed by a post where it all will be put together into a MicroProfile microservice. Stay tuned!

• The Existing State of Affairs
• The Moving Parts
• Conclusion

Let’s take a look into what we can do to achieve a better development environment than an average development project – a project that most of us have seen at some point in our professional lives, or maybe even are a part of right now. We will also look into some tools and patterns that will help us convert those projects into a paradise for the developers.

Just a few decades ago, we were working in ways that might look like unproductive, in the best case. Our development models were predominated by waterfalls, our IDEs were basic and we were compiling our projects by hand, using javac, or building up the CLASSPATH depending on the GOTO statements in a huge spaghetti code contained in countless bat files. Our code lived in a very simple versioning systems that were not distributed or supported branching strategies that are praised by the developers today. Our documentation lived in doc files on shared network drives, side by side with the simple issue tracking systems, that don’t even get close to what we have today.

Today, it is all different – we have Git, real issue tracking, IDEs, all that integrated with build servers and collaborative platforms. Yes, everything is much better, more effective and user-friendly, one might think that we are in the paradise already? Well… yes, things are fortunately getting better, however, we are still doing things in a way that might still give you nightmares, several decades from now.

Last years I have been working and invited to evaluate and help with an audit of various projects. Here are some of my observations and thoughts.

The Existing State of Affairs

This tweet describes it pretty well:

YOU ARE IN A LEGACY CODEBASE
> RUN TESTS
YOU HAVE NO TESTS
> READ SPEC
YOU HAVE NO SPEC
> WRITE FIX
YOU ARE EATEN BY AN ELDER CODE HACK.

Some of the issues are, naturally, remnants of the past – the legacy systems; but even those systems and most of the other problems we see today can be avoided if we slightly change our view at some of the main parts of the development process. In most of the cases, we would be aware of those issues, but we might need to explain and motivate the others – often people are responsible for the projects and those who prioritize the development and maintenance backlog.

The Moving Parts

The road to a great nightmare-free future consists of three components: the code quality, the development and build tools, and a good documentation and collaboration systems. When evaluating systems I often start asking some simple questions listed below to get an idea of the system.

1. The Code

The Code Quality

First thing off is the general code quality. I often start by asking about simple things – if the project has a coding standard, and if it is being followed. I also ask to take a quick peek at the code and check minor things like file encodings and MIME types. I also follow up with a question if the team is practicing code reviews, and how they are doing that.

While those things alone don’t have to mean anything, and are minor issues individually, together with other factors they still are initial indicators of possible neglect. This gives me a possibility to map areas where to look further.

In addition to that, there are also some more specific parts that will be listed as sub-sections below.

The third-party libraries

The role of the third party libraries and their use is often forgotten and neglected when considering code and system quality. This is quite unfortunate as this is the part of the code that you might not be able to patch easily, and is harder to maintain compared to your own codebase. Here are some simple questions that might help with getting a better grip on third-party libraries:

• Do you keep track of your third-party libraries?
• Do you regularly check if there are known issues or vulnerabilities in them?
• Do you have a plan for keeping them updated?
• Are the libraries you are using being actively maintained by the authors?
• Are the libraries you are using compatible with each other?
• Do the libraries you are using have appropriate licenses that are compatible with your system? (This also applies to the open source software licenses.)

Issues and vulnerabilities are being found and patched all the time. As an example for this, let me point to Google’s OSS-Fuzz Project that has found numerous security vulnerabilities in several critical open source projects. Unfortunately, even though many people are aware of the security issues in software in general, the library updates still often tend to be forgotten.

It is also worth noting that while most of the issues on my list above are security related, the last one might be of a legal sort, and probably is the most neglected of the issues listed.

The Architecture

I am often being asked to assess a system and tell something about its architecture compared to more modern systems. The different aspects of the system’s architecture will tell a lot about its maintainability both when it comes to further development, bug fixing, and keeping the system running. Some questions that might help with determining the state of the system would be:

• Does your architecture support automated deployment?
• Does your architecture support continuous deploy and delivery?
• Does your architecture support load balancing?
• Does your architecture support microservices?
• How is the architecture implemented in the code?

Tools for Maintaining the Code Quality

Some of this might sound familiar to you when you think of one or several systems you had been working with and you might now be wondering what you can do to improve the code quality? Further steps here would be starting to use proper tools that will be able to tell more about the various aspects of your code. Some of them can be used as plugins to your build system (like Maven), and some could be stand-alone tools.

Stand-alone tools for code analysis:

• SonarQube
• PMD
• FindBugs

Maven plugins to consider (more about plugins in my previous post):

• Assembly
• Versions
• Dependency
• Enforcer
• Surefire
• Failsafe
• Sonar
• Findbugs
• pmd

Bonus: See this post on command line tools for Java projects.

2. Development Tools and Strategies

Now, let’s talk about the development tools. All that fancy code and great architecture will not bring you any closer to a developer’s paradise if there will not be some proper tools to support the development. The code should live in a proper version control system that supports collaboration and things like branching and tagging. There should also be tools that help you with code quality analysis, static code analysis, etc. A good starting point here would be to start with answering the following about the project in question:

• Do you use a proper code versioning tool – Git, or even SVN?
• Do you have a branching (and tagging) strategy?
• Do you have a way of measuring code complexity?
• Do you have a way of measuring test coverage and results?
• Do you run static code analysis?

Some tools that can help you here (again, for Java-based systems):

• IDEs and IDE plug-ins that can do checks at commits, integrate with test and QA tools, etc.
• Build tools: Maven, Gradle, etc.
• Continuous integration tools: Jenkins, TeamCity, Bamboo, etc.
• Frameworks and tools for testing: to run unit tests, integration tests, UI tests, and end-to-end tests.

There are some further strategies and questions to consider:

• Are your environments easy to reproduce with minimal efforts – can you rebuild it by simply running a script?
• Do you have a proper pipeline from packaging, to delivery, to deploy?
• To what environments can you deploy automatically? With the same script, or command?
• Are your environments (like development, testing, staging, pre-prod, production) similar to each other?
• Do you follow the same process to deploy to each environment?
• Are QA and production running on physically separate hardware?
• Are you monitoring all of the environments? (i.e. are you able to see errors before they make it to QA or even production?)

3. Documentation and Collaboration Tools

Last, but not least, we will need to talk about the tools for collaboration and documentation. Without these tools, we will be back to the way things were several decades ago – with documents on shared network drives and other horrors of the 90’s that I mentioned at the beginning of this post. However, good wikis, other collaboration tools, and proper issue tracking will bring your software to another level, encouraging continuous improvement of the system.

• Wiki
• Collaboration – chat, etc.
• Issue tracking tools

No matter how obvious it might seem, it is still important to note that one should avoid multiple documentation and issue tracking systems. Unfortunately, even though it might sound obvious, it is more common than you think – I have seen my share of systems for documentation and issue tracking resulting in fragmented information and confusion.

Conclusion

There are several challenges connected with having and maintaining the good code quality. The first challenge is that a good code quality is not something you can achieve overnight. It takes time and energy to achieve that and it is a continuous process. You will need some tools, techniques, and methodology to prevail, and it will probably be easier to introduce all that from the beginning of a project.

The second challenge would be that it might be hard to convince the stakeholders of the project to invest time and resources into something that does not bring any visible improvements to the table – things like new features and bug fixes are more likely to get prioritized over something that cannot be easily measured.

Actually, while presenting on this topic at JavaOne 2017 in San Francisco, several of the attendees asked me about the ways of getting to a beautiful nightmare-free code and infrastructure, and the ways of convincing the stakeholders that this is the way to go. Unfortunately, there is no one simple solution to this, and the most valuable thing, in this case, would be to show the real value of the good quality code.

The measurements parameters to show the value can be:

• time it takes from the code is written to deploy,
• system stability,
• how often bugs are reported compared to earlier, or
• frequency of errors in logs.

So, what can you do as a developer on a project that might need some help, you might ask? You can just start by continuously suggesting improvements and showing their value to the customer, or the project manager. Now you just need to keep going and gradually improving the system, one small bit at a time.

• Introduction
• Create the Service File
• Activate the Service

Introduction

After your Docker containers are set up and running, you might need to be able to start some of them automatically on a reboot or a crash. There are several ways of getting this done.

One of them is to use restart policies provided by Docker. They can be set to control whether your containers start automatically when they exit, or when Docker restarts.

Alternatively, you can use a process manager such as upstart, systemd, or supervisor instead. In this post, I want to show you how it is done with systemd.

Create the Service File

To create a service file that will be used by systemd (systemctl command), we will first need to get your container name. This can be done by running the following command in your shell:

$ docker ps -a

The output will look something like this. Select the right container from the list, and note its name in the last column. In this example, we will be using mywiki container.

CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                    PORTS                NAMES
573193cf1d5e        hypriot/rpi-busybox-httpd   "/bin/busybox http..."   2 days ago          Exited (0) 5 hours ago                         mytest
e85753d57a67        easypi/dokuwiki-arm         "/bin/sh -c 'php-f..."   1 days ago          Up 23 hours               0.0.0.0:80->80/tcp   mywiki

Now, we will need to create a file (choose an appropriate file name for the service):

$ sudo nano /etc/systemd/system/docker-dokuwiki.service

Paste the following into the file. Set a proper Description, and make sure to update the container name in ExecStart and ExecStop:

[Unit]
Description=DokuWiki Container
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a mywiki
ExecStop=/usr/bin/docker stop -t 2 mywiki

[Install]
WantedBy=local.target

A couple of notes about the script above:

1. This file is called a unit file for systemd.
2. Make sure you don’t have any extra line brakes within the sections, like Unit, or Service.
3. The -a option in the Docker command for ExecStart makes sure it is running in attached mode, i.e., attaching STDOUT/STDERR and forwarding signals.
4. The -t option in the Docker command for ExecStop specifies seconds to wait for it to stop before killing the container.

Activate the Service

Before we can activate the service we have created, we need to reload the unit file. You will also need to run this command anytime you do any modifications to the unit files:

$ sudo systemctl daemon-reload

To activate the service run the following commands (remember to change the service name):

$ sudo systemctl start docker-dokuwiki.service
$ sudo systemctl enable docker-dokuwiki.service

To disable the service run the following commands (remember to change the service name):

$ sudo systemctl stop docker-dokuwiki.service
$ sudo systemctl disable docker-dokuwiki.service

Changes will come to effect on a reboot:

$ sudo reboot

Now you should have a container that will start on a server reboot, Docker restart, or a crash. Congratulations!

As a next step, you might want to look at (external documentation links):

• Adding some more parameters to the unit file.
• Available docker start options.
• Available docker start options.

• Introduction
• Directory Structure
• Code Metrics
• Encoding and MIME types
• Dependencies
• SonarQube

Introduction

This post will give you an overview of some command line tools that will be able to help you to get the feeling on how your project is doing. Most of the tools are widely available in the main Linux distributions and MacOS (some of them might need installation of Homebrew for Mac). On Windows it can be also made available via Cygwin, or Bash on Windows.

The commands below have been run and tested on a MacOS machine, and might sometimes need some slight modifications to be run on a Linux or a Windows machine. However, it should be able to give an idea of what it is possible to do with these tools. I will also be linking to the documentation for each of the programs in the post.

Directory Structure

Let’s start with something simple. Sometimes all you want to see is the contents and structure of the project without leaving the command line. The tree command might be able to help you out here.

It is highly customizable, takes lots of parameters, and is widely available for an installation via most of the package mangers. Command Prompt on Windows also contains a native alternative with the same name.

$ tree sourcecodefolder/ -L 1 -d
sourcecodefolder
├── src
├── tests
├── sql
└── docs

Code Metrics

If you want to see some metrics about the code in your project, like what languages that are used in the project, as well as information about number files, blank lines, comments, and lines of code, you might want to try the cloc command (it stands for Count Lines of Code).

Simply install the program and point it at the project directory, or a zip file:

$ cloc sourcecodefolder/

It will give you a result that might look something like this:

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Java                            33           1226           1026           3017
Python                           4            327            337            888
Markdown                         1             11              0             28
SQL                             10             10             12            212
-------------------------------------------------------------------------------
SUM:                            48           1574           1375           4145
-------------------------------------------------------------------------------

Also, it is possible to show all the information by percent:

$ cloc --by-percent cmb sourcecode.zip

Encoding and MIME types

Files, within a single project, with different, or wrong, encoding might mean trouble with showing non-ASCII characters correctly. They might even give you compilation errors. Therefore, finding and marking those files as quickly as possible might help you manage your project.

While MIME types, on the other hand, are not as bad at causing trouble, the diversity of the MIME types among the same kind of files, like *.java files in this example, might indicate a lack of a standard for developer tools and code standard in the project.

This kind of information can be extracted with file command for each file, or it might be done a bit more automatic for the whole project, and it might look like this:

$ find . -name *.java | \
    xargs file -I $1 | \ 
    gawk 'match($0, /.* (.*); charset=(.*)$/, gr) {print gr[2]}' | \
    sort | uniq -c

A short explanation for the script:

1. find all java files recursively (find command)
2. process them with the file command
3. grab the output and extract data with regex (gawk)
4. sort, extract unique values and count (sort, uniq)

The result will look something like this:

 257 iso-8859-1
 117 unknown-8bit
 678 us-ascii

If you want to drill down a bit further and search for both MIME types and encoding, showing the distribution of encoding per MIME type, you can also extract that from the information supplied by the file command like this:

$ find . -name *.java | \ 
    xargs file -I $1 | \
    gawk 'match($0, /.* (.*); charset=(.*)$/, gr) \
    {print gr[1] " --- " gr[2]}' | \
    sort | uniq -c

A short explanation for the script is almost the same as above. The only difference is that we use information from both of the regex groups (gr[1], and gr[2]). The result will look something like this:

  15 text/html --- iso-8859-1
  34 text/html --- us-ascii
 134 text/plain --- iso-8859-1
  16 text/plain --- unknown-8bit
   1 text/plain --- us-ascii
   1 text/x-c --- iso-8859-1
   8 text/x-c --- us-ascii
   5 text/x-c++ --- iso-8859-1
   6 text/x-c++ --- us-ascii

Dependencies

Now, over to a bit more advanced stuff – analyzing dependencies in your project. Why would you want to do that? Well, in short: dependencies increase complexity, and cyclic dependencies are bad for your project and your health. They also hurt your modularity and complicate the build process.

This kind of analysis can be done with jdepend. It is easy to run and can be run both separately, and as a part of a build tool, like Maven.

For some basic dependency analysis you can also use the command line, and do something like this:

$ find some_module/ -name *.java | \
    xargs cat $1 | \
    gawk 'match($0, /import (no.*);/, gr) {print gr[1]}' | \
    sort | uniq  | \
    grep -v no.ignore.this.namespace # -v option add strings to ignore

A short explanation for the script:

1. find all java files recursively (find command)
2. print them to console with the cat command
3. grab the output and extract import statements with some specific pattern with regex (gawk)
4. sort, extract unique values (sort, uniq)
5. add some namespaces to ignore some strings (grep with a -v option)

SonarQube

Last but not least, you might want to load your code for further analysis to SonarQube. This is an extremely powerful and free tool for doing the static analysis of your code. Normally, you would do that via a plug-in in your continuous integration (CI) software, like Jenkins. However, there might be some cases when you might want to load this data manually, via a command-line.

To load the code to SonarQube you will first need to add a property file in the root directory of each module. It might look like this:

# Contents of sonar-project.properties file:
sonar.projectKey=my:SomeFancyProject
sonar.projectName=My SomeFancyProject
sonar.projectVersion=1.0
sonar.sources=.
sonar.sourceEncoding=ISO-8859-1

Then you will need to install and run SonarQube Scanner from each module directory that contains sonar-project.properties file:

sonar-scanner

As a bonus feature, I might also suggest that if you don’t have time setting up SonarQube on a separate machine, but want to take a quick peek on how your project is doing, you can download and boot it up as a Docker image in no time, and later decide whether you want to create a dedicated machine for running SonarQube, or keep it as it is. Just remember that the database the SonarQube image uses out of the box should not be used for anything other than testing.

• 1. Technical Aspects
• 2. Legal Aspects
• 3. Rapid Development
• 4. Documentation
• 5. Testing and QA
• Bonus: Video

Apache Maven is a build automation tool used primarily for Java projects. It has been around for a while, and it does not seem to be going anywhere anytime soon (whether some of you like it or not). Recently, it has been confirmed yet again, this time in ZeroTurnaround’s Java Tools and Technologies Landscape Report 2016 (see the Build Tools section).

The good thing about Maven is that you can do almost anything with it and its plugins. With such a great ecosystem of plugins, able to do nearly anything, one might wonder where to begin when setting up a new Maven project or improving your old one.

I usually like to think of five main categories – or pillars – that we will be looking into in this post. Each category will have a set of example plugins that are meant to serve merely as a starting point.

In this post, I will assume that you have some experience with build tools in general and Maven in particular.

1. Technical Aspects

First of all, you would like to make sure that all the technical stuff is in place. You might want to automate the packaging, the checks if the project is running the latest version of all artifacts, that all the dependencies are met, and that the unused dependencies are removed. You might even want to define your own rules for all the things that cause troubles, or just simply annoy you down the line, making the build fail if they are not met.

With all that automation in place, you will be getting predictable results and nice packaging from the first day of the project.

Some of the plugins that should be mentioned here:

• Versions plugin does the versions management of artifacts in a project’s POM file
• Dependency plugin helps you analyzing dependencies, building dependency trees, showing unused dependencies, etc
• Assembly plugin helps you packaging all the dependencies, modules, site documentation, and other files into a single distributable archive
• Enforcer plugin lets you make your own rules! You can set up your build to break if some of your requirements are not met

2. Legal Aspects

With all the technical stuff out of the way, we might want to make sure that the legal side is taken care of as well.

Yes, I know, it might be less fun thinking about licenses than writing code, but it is still something that has to be done. You still have to release your code under some kind of license, and you will have to make sure that the third-party licenses do not violate your own licensing. So, why not leave that job to a plugin?

The following plugin will manage the license of a maven project and its dependencies; it will also update file headers, download dependency licenses, check third-party licenses, etc.

• License plugin

3. Rapid Development

Now, back to coding. Or even better – to seeing the results of your hard work. The chances are that you will need some kind of web or application server for running your code, and you want to able to deploy to that server in no time. Another kind of plugins that will be helping you from the very first day of the project.

Some of the plugins that can be mentioned here:

• Apache Tomcat plugin
• Jetty plugin
• WildFly plugin
• …or any other deploy plugin, depending on your application

4. Documentation

Every decent project must also be properly documented. However, this is something that developers might postpone until the end. Well, no more! This kind of plugins will help you to get started early and will help you to produce some beautiful (maybe?) and maintainable docs.

You might even consider coupling these with some rules in the Enforcer plugin, but tread carefully as too many, and too strict rules can, and usually do, backfire.

• Site plugin
• Asciidoctor plugin

5. Testing and QA

By now, your code should be looking great, with all the right licenses, and up and running in no time. So, how about squashing some [virtual] bugs? The list below might help you setting up a proper QA environment and fixing bugs before the come crawling to your production servers.

• Surefire plugin
• Failsafe plugin
• SonarQube plugin
• FindBugs plugin
• PMD plugin

Bonus: Video

A video of a talk I gave about this topic at JavaZone in Oslo (in Norwegian).

On June 16th, I will be doing a workshop at JBCNConf in Barcelona, Spain. I will be talking about semantic technologies, reasoning, and Java. I have been asked to post some information about the workshop, so here it comes.

• What is it all about?
• Contents
• Workshop requirements

What is it all about?

Ever heard of Zebra Puzzles? Those logical puzzles that are claimed to be invented by Albert Einstein as a little boy? Those that are based on simple logical facts and go something like this:

“The Brit lives in the Red house. The Swede keeps dogs as pets. The Dane drinks tea… Who owns the zebra?”

Some claim that only 2% of the population can solve it without any help.

However, with the help of semantic technologies, we can solve it in (almost) no time. Of course, you have to know RDF, OWL, Jena, reasoning, inference. The good news is that I will be showing you how it works in this session.

Later, I will also post a link to the code for the rest of you to try all that at home.

Contents

We will start with introducing the puzzle. Then we will continue by looking at the toolbox for solving it. We will build a semantic representation of the puzzle (basically, a graph!), programmatically add some inferred facts and reason and solve the puzzle.

Suddenly solving puzzles is a piece of cake for the other 98%.

Workshop requirements

Now, over to what you will need:

• Your super computer
• Java 7, or later
• Git
• IDE of your choice
• Maven 3.x

Looking forward to running this workshop next week! See you there!

p.s. Want to read more about Zebra Puzzles? Go ahead, but beware of some potential spoilers!