Full Reactive Stack Backend: WebFlux, Reactive MongoDB and Spring Boot
Table of Contents
Through this post (part of a series), you’ll learn how to develop the backend Reactive Web Application. We’ll use Spring WebFlux, available from Spring Boot 2.0 version, and connect to a Mongo database using its reactive driver with Spring Data.
This post is part of the Full Reactive Stack Guide:
- Full Reactive Stack – Introduction
- Backend side with WebFlux, Spring Boot and MongoDB (this post)
- Frontend side with Angular and EventSource
- Running the application, Comparing WebFlux and MVC, and Conclusions
In this post
You’ll build (or analyze the existing code if you prefer) a Spring Boot 2.0 application that uses two different backend approaches:
- A reactive style, using WebFlux and a MongoDB
- The classical stack, using a blocking REST controller and a blocking query to MongoDB.
Additionally, we’ll need:
- Docker, to simplify MongoDB deployment.
- A class to pre-load the data into MongoDB. For that we’ll use a
Before that, we’ll introduce some topics for a better understanding for those not familiar with reactive programming. If you already have experience, feel free to jump directly into the code explanation.
Project Reactor – Main Features
Project Reactor is a framework built by Pivotal and powered by Spring. It implements Reactive Programming patterns and, more specifically, the Reactive Streams specification.
If you’re familiar with Java 8 Streams you’ll quickly find many similarities between a
Stream and a
Flux (or its single-element version,
Mono). The main characteristics that differ between these Fluxes and Monos and the Stream API are that the first two follow a Publisher-Subscriber pattern and implement backpressure.
For instance, if you declare a Flux that takes elements from a database, say numbers, maps them applying an operation, and filters them according to a threshold… nothing happens. It will start doing all the operations only when a subscriber starts pulling data. Furthermore, the command processing in the Flux is always triggered by the subscriber when it asks for more elements (applying the backpressure concept).
As a direct conclusion of this very brief summary, the main advantage of using Reactor is that you’re in total control of the data flow: you can rely naturally on the subscriber’s ability to ask for more information when it’s ready to process it, or buffer some results on the publisher’s side, or even revert back to a full push-approach like in the Java Streams API.
Fluxes and Monos
In our scenario, we use the Flux class. It implements the Publisher interface and, as we briefly introduced, it is just a reactive stream that pushes items whenever the subscriber instructs it to do so.
There is a handy version of a Flux for the special case in which the reactive stream will either emit only one item, or none: Mono. It contains a different subset of methods so, for instance, concatenating a Mono with another Mono, or a Flux, will return a Flux.
Spring is including Reactor in some of their libraries, thus applying reactive programming patterns to them. Following a smart approach, they’re not getting rid of the previous programming style so don’t worry, updating to Spring 5 doesn’t mean being forced to adopt reactive programming.
The first big framework into which Spring is leveraging Reactor is, as you may guess, the Web framework. Starting with the version 5 you can use Spring WebFlux, which comes with major updates like a new way of declaring the routes, and transparent support for Server-Sent Events using the Reactor API.
Spring Data has also embraced Reactive Patterns through its Reactive module, with the inclusion of the
ReactiveCrudRepository. That means you can perform reactive queries to databases like MongoDB, which already have a reactive version of its driver, and pull data in a flow controlled by the subscribers instead of pushing the query results.
WebFlux – Main Features
Standard Controllers and Router Functions
Spring WebFlux comes with an extra new feature called Router Functions. The idea is to apply Functional Programming to the Web layer and to get rid of the declarative way of implementing Controllers and RequestMapping’s – see the full docs here. However, as you can see depicted in the picture below, Spring 5 still allows you to use the
@RequestMapping annotations, so you can actually decide.
In this guide, I’ll use the classic, simple way of declaring routes and controllers: declarative annotations. The main reason is that I don’t want to distract you with multiples changes at once but also, to be honest, I’m not yet convinced of the convenience of that new style of declaring routes (feel free to judge for yourself).
WebFlux comes with a reactive web client too:
WebClient. You can use it to perform requests to controllers that use reactive streams, and benefit from backpressure as well. It’s a kind of reactive and fluent version of a
As described in the first part of this guide, we won’t use here this WebClient to avoid endogamic Java-Spring examples. We’ll create an Angular frontend to consume the endpoints, and detail what we get and what we don’t get when compared to WebClient.
Creating the application
Spring Boot Reactive Skeleton
To start with, the easiest option is to navigate to http://start.spring.io/ and create a Spring Boot application skeleton project. Make sure to select a Spring Boot version 2.0 or newer, the Reactive Web and the Reactive MongoDB dependencies.
Spring Boot 2.0 can integrate the WebFlux capabilities so let’s use its power to set up our backend. The first change we apply is already in our dependency management: we don’t include the MVC starter but the WebFlux one (
spring-boot-starter-webflux). This starter contains the Reactor Netty server library dependencies, so Spring Boot will know that’s the one to start at runtime, and we’ll have available the Reactor API during development.
On the other hand, the Reactive MongoDB dependencies are included with the starter
spring-boot-starter-data-mongodb-reactive. This one pulls the Spring Data MongoDB dependency, containing Spring Data Commons 2.0 with reactive support, and the asynchronous and reactive-streams versions of the MongoDB driver.
Planning our way
You’ll create a typical layered backend application. Since there is no business logic involved, the Controllers will use directly the repositories. Besides, you’ll create two parallel ways of retrieving data from the Mongo database: the reactive way and the classic way. In any case, we’ll focus on the reactive part of it while explaining the ideas.
See the image for the final structure of the backend application (as you find it on GitHub).
We have twice the Repository and the Controller elements as our main logic, and a
Quote class representing our domain. Our intention is to expose a list of Quotes using a Reactive Web API –
QuoteMongoReactiveRepository -, and compare it with the classic blocking REST API –
QuoteMongoBlockingRepository. We need to implement a
CorsFilter to enable cross-origin requests to simplify connection from the frontend, and we create the data loader (
QuijoteDataLoader) too inside the same project (so it’ll create the quotes at startup into MongoDB if they’re not there yet).
We’ll implement two endpoints per controller: one that retrieves all the quotes available in the repository and another that supports pagination. This is useful for us to compare both approaches in a more realistic scenario, in which pagination is usually required.
Reactive Repository with Spring 5 and Spring Data
To implement a truly Full Reactive Stack we’ll use a MongoDB datastore using its reactive driver. The
QuoteMongoRepository implementation consumes quotes from a Spanish masterpiece: Don Quixote. Therefore, this repository data is the source of our
Creating a basic Reactive repository is as simple as creating a classic one in Spring Data JPA: you just need to create an interface that extends
ReactiveCrudRepository, which is the reactive version of
CrudRepository. You’ll have access to default methods to create, delete, update and read Quotes.
Since we want the paged version of
findAll, we define a new interface method that uses a
Pageable and has a defined query to return all the quotes with an id – thus all of them. See the MongoDB documentation if you’re curious about this json query specification. Note that, to avoid side effects, you shouldn’t name your method starting with
query, otherwise you’ll activate the Query Methods feature.
findAll and the
retrieveAllQuotesPaged that we’ll use from the controller return a Flux. That means that our subscriber (the controller) can control how fast the data should be pulled from the database.
Saving entities in a reactive manner
One thing that draws the attention of people not familiar with reactive patterns is how the
save method (and its variants) work. Let’s compare the
CrudRepository and the
Now imagine that we want to save a Quote and we’re so confident about the result that we just ignore it, so we use:
You can do that with an interface extending
CrudRepository and the entity will be persisted. However, if you do that with a
ReactiveCrudRepository, the entity is not stored. The reason is simple: the latter is returning a Mono, which is a Publisher, and it won’t start working until you subscribe to it. If you want to keep the blocking behavior that
CrudRepository offers, you need to call instead
quoteRepository.save(quote).block(). Anyway, we’ll see an example of this while explaining the
The Quote class
Below you can find the
Quote class implementation. It contains just an identifier, the book title and the quote contents.
The Reactive Controller
Let’s focus now on the most important part of our backend application for the purpose we have in this post: the Reactive Controller.
If you’re familiar with Spring Controllers, you’ll find out quickly that the only part of the code that seems new is the
Flux object we’re returning as a result of the methods. In Spring MVC, we would return a
Iterable) instead (which we can still use, as we’ll see later).
Note that, if we would have chosen the Router Functions alternative instead of the annotated controllers (
@GetMapping annotations), the implementation would be quite different. Anyhow, the resulting functionality would be exactly the same.
The controller is calling the
QuoteMongoRepository to retrieve all quotes. The printed versions of Don Quixote can easily get to more than 500 pages so you can imagine that there are a lot of quotes (more than 5000). Thanks to the reactive approach, we don’t need the full list of results to be available in the backend before getting them: we can consume the quotes one by one as soon as the MongoDB driver is publishing results.
Simulating poor performance
To evaluate the Reactive Web properly, we should simulate things like an irregular network latency or an overloaded server. To keep it simple, we’ll go for the latter and play the situation in which every quote has a processing time of 100 milliseconds. It will take almost ten minutes (>5000 quotes * 100 ms each) to retrieve the whole data; if you want to try a smaller set of quotes you can tell your publisher to take only the first N elements (as described in the code’s comment).
This will also help us visualize the differences between Reactive and MVC strategies. Since we’ll run client and server on the same machine, if we don’t introduce the delay the response times are so good that it would be hard to see the differences.
When we created the Repository we introduced an additional method to retrieve results in pages. We’re exposing that behavior in the Controller, mapped to the URL
Creating pages for results is always a good practice, no matter if you’re calling the backend in a blocking or non-blocking style. On one hand, the client might be not interested in getting all the results at once; on the other hand, you want to make the best use of resources by keeping the request’s processing time as short as possible.
Note that in Spring MVC you don’t need to explicitly create the
size parameters as I’m doing in this case. However, I didn’t get the automatic conversion of parameters into a
Pageable working when I specified that as the endpoint’s method argument so I chose the explicit way. In any case, see how easy is to get a
Pageable just by calling
PageRequest.of(page, size). Let me know via comments if you find how to do that with automatic conversion.
Enabling CORS in Spring WebFlux
We want to connect to the backend from a client application deployed in a different port (at least during development). In Web terms, that means that the frontend part of our application has an origin which is different from the backend. Unless we explicitly state that as valid in our configuration, that connection is going to be refused.
To enable CORS with WebFlux we need to implement a new reactive-like
WebFilter (returns a
Mono) to set some headers related with access control. The most important one is
Access-Control-Allow-Origin, which we set to allow any (which is good only for development purposes).
Apart from that header, you can see in this snippet that we’re setting some others. They are related to the process used by the browser to open a channel with the server (SSE). What we want for this experiment is to allow any HTTP method, and all related headers. Important note: don’t do this in production environments. Be more restrictive and adapt this configuration to your case, e.g. don’t use
* as allowed origins but the specific hosts instead.
Returning a Flux from a Controller: behind the scenes
Let’s dive a bit more into details. The new Spring WebFlux documentation lists the reactive types (Flux included) as possible return types. But, what happens when we request content to the server? How does WebFlux convert it into a valid response? It actually depends on how we request it:
- If we request content without using an
Acceptheader, or setting it to
application/json, we’ll get a synchronous, JSON-formatted response.
- If we want to use the Server-Sent Events support in Spring to implement our full reactive stack, we set in our request (explicitly or behind the scenes) the
text/event-stream, therefore activating the reactive functionality in Spring.
We’ll test this within this post, once we have our backend set up.
The blocking Controller and Repository
To better show the comparison between the blocking and reactive approaches, let’s create a separate Controller with specific paths, and connect them to a standard
CrudRepository. This code is pretty straightforward and well-known to all developers familiar with MVC.
Here is the Controller. Note that here we also apply the same delay as in the reactive approach, this time all at once since the query is returning results also as a whole pack.
The Repository is very similar to reactive, but this time we extend
CrudRepository and return a
Loading data into MongoDB with a CommandLineRunner
We have all the code we need to run our Spring Boot application. But we don’t have yet any way of storing quotes so we’ll solve this by reading them from a text version of the book and storing them into MongoDB the first time the application runs.
In the project repository, you’ll see a file containing the ebook in text mode:
pg2000.txt. The first time we run the application every paragraph will be stored as a Quote in MongoDB. To achieve this, we inject a
CommandLineRunner implementation in the application’s context: the
There are several aspects that might be interesting to point out:
- Since the repository is reactive, we need to
block()to wait for the result of the one-element publisher (
Mono) containing the number of quotes in the repository.
- We apply a reactive pattern to subscribe to the result of
ReactiveCrudRepository. Remember that, if we don’t consume the result, the quote is not stored.
Running the backend
Running MongoDB with Docker
This step is optional, you can also install MongoDB on your machine and it’ll work the same way. However, in a next post I’ll show you how to run the complete reactive system using Docker so it’s good to follow this approach to get a first contact with it.
You need to install Docker if you haven’t done it yet. Then, create a file named
docker-compose.yml with this content:
I’ll explain this in more details in the dedicated post/section of this guide. For now, just keep in mind that this will create a docker container with a MongoDB instance in your Docker host (which can be
localhost or a virtual machine IP), and expose the default connection port, 27017. It’ll create a volume too, and the quotes data will remain after stopping the container. To run this container, you need to execute:
Running the Spring Boot Reactive application
We now run our backend application to see if everything works as expected. To do so, we can either use our IDE to run the
ReactiveWebApplication class, or we can use the included maven wrapper from the
If everything goes well you should see messages like these on the console:
Playing with Reactive and Classic endpoints
Let’s test our API endpoints using cURL, a command line tool that supports Server-Sent Events, so we can have a look at the blocking and non-blocking alternatives in our application. On Windows, you’ll need either a Linux shell simulator (like Git Bash, included with the official Git distribution package, or Cygwin) or to download cURL from this site.
Then we try the paged version of the reactive endpoint:
What’s going wrong? It will take at least 5 seconds to complete (due to the delay of 100ms per element). After that time, we’ll get a big JSON output in the console. It’s still a blocking call.
We didn’t specify what are the contents we accept in our first request so Spring WebFlux, by default, is returning us a JSON and treating the request as a blocking one. To activate the Reactive patterns in the shape of a Server-Sent Events channel, we need to ask for a
To test a stream approach, we just need to add a header:
The output format is different now and so it is the time that it takes to get each quote on the console. This time, WebFlux is creating a Server-Sent Events connection and publishing a quote every 100 milliseconds. We’re not employing any kind of backpressure here so our client is pulling data continuously and printing it on screen.
Let’s keep that header in our request and evaluate what happens in the rest of the cases:
curl -H "Accept: text/event-stream" "http://localhost:8080/quotes-blocking-paged?page=0&size=50". This call will block for five seconds since we’re requesting data to the classic controller (non-reactive). The same will happen if we call
http://localhost:8080/quotes-blockingand, in that case, you’ll need some patience since it will take almost ten minutes to complete.
curl -H "Accept: text/event-stream" "http://localhost:8080/quotes-reactive"will also take almost ten minutes to finish but, in this reactive scenario, we’re getting quotes as soon as they’re dispatched.
At this point we got a very important conclusion: it doesn’t matter if you use a Reactive Web approach in the backend, it won’t be really reactive and non-blocking unless your client is able to handle it as well.
We’ll implement the real Reactive Web Client in the next section of this guide. And, as promised, we’ll get out of Spring Boot and Reactor to implement the client code so we make it more realistic.