Book's Upgrade: Migrating from Spring Boot 2.6 to 2.7

A new upgrade chapter! This one covers the required changes on the book’s source code to update it to Spring Boot 2.7.1 and some extras.

• An H2 major upgrade surprise and some extras
• Migrate to Spring Boot 2.7
• Major H2 upgrade
  • Feature not supported: “AUTO_SERVER=TRUE && DB_CLOSE_ON_EXIT=FALSE”
  • Error executing DDL “create table user …”
• Bonus: Migrating bootstrap to application’s properties
  • Remove the legacy bootstrap starter
  • Migrate the properties and remove bootstrap files
  • Remove the workarounds added previously in our tests
• Bonus: a proper H2 server
  • Creating an H2 server docker image
  • Using the H2 server in our containerized system
• Bonus: getting rid of an ugly error on macOS
• Summary

An H2 major upgrade surprise and some extras

Smooth upgrades couldn’t last for too long. This upgrade breaks the book’s code sources again, but nothing that can’t be fixed with some minor changes.

This release of the book sources also prepares the road for the upcoming Spring Boot 3.x major upgrade.

In summary, these are the changes covered in this chapter:

• (Mandatory) Fix the error caused by using incompatible H2 features
• (Mandatory) Fix the error caused by using a reserved H2 keyword
• (Optional) Migrate to the new configuration system introduced in Spring Boot 2.4
• (Optional) Use an H2 database server to avoid data isolation while scaling up the system
• (Optional) Use the macOS’ Netty DNS resolver to avoid ugly warnings in logs

We’ll take as a baseline the last changes to upgrade the projects to Spring Boot 2.6. The new source code is available on GitHub (while you’re there, give it a star!).

Migrate to Spring Boot 2.7

First, we update the pom.xml files in all the Spring Boot projects. See Listing 1 as an example (gamification service).

• The version of the spring-boot-starter-parent is now 2.7.1, to set the new Spring Boot version and get all the newest dependencies.
• Our project is changed to 2.7.1-SNAPSHOT to follow Spring Boot’s versioning.
• We upgraded Spring Cloud to the latest version, 2021.0.3. A minor update in this case.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.7.1</version>
    <relativePath/> <!-- lookup parent from repository -->
  </parent>
  <groupId>microservices.book</groupId>
  <artifactId>gamification</artifactId>
  <version>2.7.1-SNAPSHOT</version>
  <name>gamification</name>
  <description>Gamification Microservice - Learn Microservices with Spring Boot - Book - Upgrade 2.7.1</description>

  <properties>
    <java.version>17</java.version>
    <spring-cloud.version>2021.0.3</spring-cloud.version>
  </properties>

  <dependencies>
      <!-- for the upgrade itself, we don't need to modify anything here -->        
  </dependencies>    
</project>

Listing 1. Upgrading Spring Boot and Spring Cloud

Major H2 upgrade

Spring Boot 2.7 brings a major upgrade of the database we use in the book’s code from 1.x to 2.x. Many changes in this major version are backward-incompatible and two of them impact our code (breaking it). See this section in the Spring Boot’s 2.7 release notes for more details.

Feature not supported: “AUTO_SERVER=TRUE && DB_CLOSE_ON_EXIT=FALSE”

In the book, we use the “Auto Server” H2 feature to automatically create a database server when starting a microservice instance. The main goal was to keep the database simple using just files that can be shared across microservice instances. With this feature, multiple microservice replicas can connect to the same database file instead of having independent data stores, which wouldn’t fit into a microservice architecture.

Combined with that feature, we set the flag DB_CLOSE_ON_EXIT=FALSE. I intended to avoid that stopping a microservice would cause the database server to go down. But I misunderstood the docs because that’s not how it works. This flag is only useful when the application needs to do something with the database in its own shutdown hook.

The new major release disallows the usage of these two flags together because that may cause incompatibility issues. See this comment for more details.

We can solve this very quickly: just removing the DB_CLOSE_ON_EXIT parameter from the connection URL. See Listing 2 for an example of the URL of the gamification microservice. We must do this also for the multiplication microservice.

spring.datasource.url=jdbc:h2:file:~/gamification;AUTO_SERVER=TRUE;

Listing 2. Removing the DB_CLOSE_ON_EXIT parameter in application.properties

Error executing DDL “create table user …”

If you try to start the multiplication microservice after the upgrade in a new workspace, you’ll get an error like the one in Listing 3:

Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "create table [*]user (id bigint not null, alias varchar(255), primary key (id))"; expected "identifier"; SQL statement: [...]

Listing 3. Syntax error in H2 while creating the user table

The error description might not help too much, but the root cause is that our table name is user and this is now a syntax keyword. We must escape it, change the table name or configure H2 to avoid considering this a keyword. To keep it simple, we’ll go for this last option which requires only one change in the database URL in application.properties. We use the NON_KEYWORDS parameter in H2 to achieve this. See Listing 4.

spring.datasource.url=jdbc:h2:file:~/multiplication;AUTO_SERVER=TRUE;NON_KEYWORDS=USER;

Listing 4. Fixing the H2 error with the user table

Bonus: Migrating bootstrap to application’s properties

In addition to the mandatory upgrades to make the code work with 2.7, I also switched to the new configuration approach introduced by Spring Boot 2.4.x. It’s good to do this in preparation for the Spring Boot 3.x release, which will stop working with the old approach.

The required changes to move to the new configuration approach are in the two subsections below.

Remove the legacy bootstrap starter

We added the legacy starter as a way to move forward quickly without making any changes. This starter is available from the Spring Boot release 2.4 onwards as a convenient way to provide backwards compatibility without needing to upgrade to the new configuration system.

Let’s remove it because we’re going to apply the new configuration changes later. See Listing 5 for the changes. We need to do this in all our microservice projects.

<!-- Remove this dependency -->
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>

Listing 5. Removing the spring cloud starter bootstrap to remove support for legacy configuration

Migrate the properties and remove bootstrap files

The second step is to copy and paste all properties in each corresponding project’s bootstrap configuration file to the corresponding application properties file. See Listing 6 for the complete contents of the multiplication microservice’s configuration after the change.

# Gives us access to the H2 database web console
spring.h2.console.enabled=true
# Creates the database in a file
spring.datasource.url=jdbc:h2:file:~/multiplication;AUTO_SERVER=TRUE;NON_KEYWORDS=USER;
spring.datasource.username=sa
# Creates or updates the schema if needed
spring.jpa.hibernate.ddl-auto=update
# For educational purposes we will show the SQL in console
#spring.jpa.show-sql=true

# Gamification service URL
service.gamification.host=http://localhost:8081

amqp.exchange.attempts=attempts.topic

# Shows declaration of exchanges, queues, bindings, etc.
logging.level.org.springframework.amqp.rabbit.core.RabbitAdmin = DEBUG

# Using the new configuration approach
spring.config.import=optional:consul:

# From the legacy bootstrap.properties
spring.application.name=multiplication

spring.cloud.consul.config.prefix=config
spring.cloud.consul.config.format=yaml
spring.cloud.consul.config.default-context=defaults
spring.cloud.consul.config.data-key=application.yml

logging.level.org.springframework.amqp.rabbit.connection.CachingConnectionFactory = WARN

Listing 6. Migrating from bootstrap to application properties

Apart from the copy-pasted contents, you’ll see a new property spring.config.import=optional:consul:. The new configuration approach in Spring Cloud introduced this property to instruct Spring Cloud Config (centralized configuration) to try to sync the latest configuration from Consul. Since we prefix it with optional, the microservice will start anyways if the connection to Consul is not available.

Remove the workarounds added previously in our tests

With the upgrade of the book code repositories to Spring Boot 2.4, we needed to introduce an additional workaround to make it work with the legacy configuration system: a test.properties file in the test package, and a @TestPropertySource annotation in the tests using those configuration properties. See Listing 7 for the contents of that test.properties file.

# Disable Consul Config for tests. Search for the `TestPropertySource`
# annotation to find out which tests require this.
spring.cloud.consul.config.enabled=false

Listing 7. The legacy test.properties file we need to remove now

We don’t need these files nor the annotations anymore. Actually, if we leave them there, the tests won’t work with the new configuration approach. We can simply remove these files and the corresponding annotations.

Bonus: a proper H2 server

One of the challenges with the book’s code sources appears when we want to start multiple microservice replicas using docker containers. Since we use database files, containers can’t simply share the database. Workarounds like creating shared volumes are possible, but that’s not a production-ready strategy.

Let’s see how we can use a proper H2 database standalone server, which we can deploy as a container as well.

Creating an H2 server docker image

First, we create a Dockerfile that we’ll use to build a custom docker image that starts H2 as a standalone server. See Listing 8.

FROM amazoncorretto:17

# Based on https://github.com/oscarfonts/docker-h2/blob/master/alpine/Dockerfile
ENV DOWNLOAD https://github.com/h2database/h2database/releases/download/version-2.1.214/h2-2022-06-13.zip
ENV DATA_DIR /home/h2-data

RUN yum -y install curl unzip && yum -y clean all  && rm -rf /var/cache

RUN mkdir -p ${DATA_DIR} \
    && curl -L ${DOWNLOAD} -o h2.zip \
    && unzip h2.zip -d /home/ \
    && rm h2.zip

EXPOSE 81 1521

WORKDIR /home/h2-data

# Databases shouldn't be created on first access on production (remove -ifNotExists and create them e.g. via console)
CMD java -cp /home/h2/bin/h2*.jar org.h2.tools.Server \
  -ifNotExists \
 	-web -webAllowOthers -webPort 81 \
 	-tcp -tcpAllowOthers -tcpPort 1521 \
 	-baseDir ${DATA_DIR} ${H2_OPTIONS}

Listing 8. Dockerfile for an H2 standalone server

We base this image on Amazon’s JDK 17 (Corretto) and download the latest H2 version available at the moment of writing this post. Then, we install some required tools on top of the base image, unzip H2 contents, set the H2 data directory, and expose the required ports for connections. Last, we use the java command to start an H2 server with some basic parameters and allow database creation on first access.

Now we can build the image from the folder where this file is stored and save it to the local docker repository. See Listing 9. I’m using a version that matches the H2 version I downloaded in the Dockerfile.

$ docker build -t h2-server:2.1.214 .

Listing 9. Building an H2 server docker image

Using the H2 server in our containerized system

We can now include the container specification in our existing docker-compose.yml file. See the highlighted lines in Listing 10. Don’t forget we have to point the database URL in both microservices to the new container. This is what we do with the new SPRING_DATASOURCE_URL environment variable value.

version: "3"

services:
  frontend:
    image: challenges-frontend:1.0
    ports:
      - '3000:80'
  multiplication:
    image: multiplication:2.7.1-SNAPSHOT
    environment:
      - SPRING_PROFILES_ACTIVE=docker
      - SPRING_CLOUD_CONSUL_HOST=consul
      - SPRING_DATASOURCE_URL=jdbc:h2:tcp://h2-server:1521/home/h2-data/multiplication;NON_KEYWORDS=USER;
    depends_on:
      - rabbitmq-dev
      - consul-importer
    networks:
      - microservices
  gamification:
    image: gamification:2.7.1-SNAPSHOT
    environment:
      - SPRING_PROFILES_ACTIVE=docker
      - SPRING_CLOUD_CONSUL_HOST=consul
      - SPRING_DATASOURCE_URL=jdbc:h2:tcp://h2-server:1521/home/h2-data/gamification
    depends_on:
      - rabbitmq-dev
      - consul-importer
    networks:
      - microservices
  gateway:
    image: gateway:2.7.1-SNAPSHOT
    ports:
      - '8000:8000'
    environment:
      - SPRING_PROFILES_ACTIVE=docker
      - SPRING_CLOUD_CONSUL_HOST=consul
    depends_on:
      - rabbitmq-dev
      - consul-importer
    networks:
      - microservices
  logs:
    image: logs:2.7.1-SNAPSHOT
    environment:
      - SPRING_PROFILES_ACTIVE=docker
      - SPRING_CLOUD_CONSUL_HOST=consul
    depends_on:
      - rabbitmq-dev
      - consul-importer
    networks:
      - microservices
  consul-importer:
    image: consul-importer:1.9
    depends_on:
      - consul-dev
    networks:
      - microservices
  consul-dev:
    image: consul:1.9
    container_name: consul
    ports:
      - '8500:8500'
      - '8600:8600/udp'
    command: 'agent -dev -node=learnmicro -client=0.0.0.0 -log-level=INFO'
    networks:
      - microservices
  rabbitmq-dev:
    image: rabbitmq:3-management
    container_name: rabbitmq
    ports:
      - '5672:5672'
      - '15672:15672'
    networks:
      - microservices
  h2-server-dev:
    image: h2-server:2.1.214
    container_name: h2-server
    ports:
      - '81:81'
      - '1521:1521'
    networks:
      - microservices

networks:
  microservices:
    driver: bridge

Listing 10. Using an H2 server in our containerized microservices system

You can try to run this file and then spin up new replicas as detailed in the book. You’ll see that all these replicas will then share the same database, without glitches e.g. on the leaderboard results.

Bonus: getting rid of an ugly error on macOS

I use macOS with an M1 chip and this setup causes some annoying errors from Netty, as I documented on the previous upgrade. In summary, I get some exception traces from Netty warnings, due to missing DNS native libraries.

If you want to avoid these ugly warnings you can just fix that issue by adding the corresponding DNS native libraries. See Listing 11 for the dependency I need to add in my specific case.

<!-- Avoids Netty's DNS-related errors logged in MacOS -->
<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-resolver-dns-native-macos</artifactId>
    <!-- Change the classifier if your Mac is not M1, see e.g. https://stackoverflow.com/questions/65954571/spring-boot-2-4-2-dns-resolution-problem-at-start-on-apple-m1 -->
    <classifier>osx-aarch_64</classifier>
    <version>4.1.78.Final</version>
</dependency>

Listing 11. Fixing the Netty macOS ugly warning

Summary

That’s all! A couple of mandatory fixes with this upgrade, both caused by the H2 major upgrade, and some bonus content that you can use to keep up-to-date with the Spring Boot changes. On top of that, we now have the option to use an H2 database server, which makes the whole setup even more similar to a real production-ready microservice architecture.

If you still don’t have the book, I strongly recommend you get your copy now. I keep updating the book’s source code and will be sharing more content around the same use case.