Microservice Communication in ASP.NET 5 with RabbitMQ

Microservice Communication in ASP.NET 5 with RabbitMQ

Follow Me Here

In my previous posts on Ocelot Gateway for Microservices , Microservices with API Gateway and Creating Customer Microservice using Dapper and Swagger , we discussed a lot about the Microservices in ASP.NET 5. We also started from scratch and built the Visual Studio solution hands-on.

In this post, we shall talk about the all-important task of communication between the microservices using RabbitMQ which is a message broker.

What is a message broker?

A message broker is software that enables applications, systems, and services to communicate with each other and exchange information. The message broker does this by translating messages between formal messaging protocols. This allows interdependent services to “talk” with one another directly, even if they were written in different languages or implemented on different platforms.

Message brokers are software modules within messaging middleware or message-oriented middleware (MOM) solutions. This type of middleware provides developers with a standardized means of handling the flow of data between an application’s components so that they can focus on its core logic. It can serve as a distributed communications layer that allows applications spanning multiple platforms to communicate internally.

The functions of message brokers are:

  • Validation, storage, routing and delivery of messages to appropriate destination
  • They serve as intermediaries between other applications (hence the name broker, I think)
  • Allow senders to send messages without the knowledge of the receivers, including if they are active or even their number
  • This way the message brokers help decouplong of processes and services within the system
Fig. 1

One of the very simplistic models of a message broker is a software module that provides queues (enqueuing) for the incoming messages and allows the messages to be taken off (dequeued) by the microservices (Fig. 1). This model can be viewed as Point-to-Point messaging, where there is a one-to-one relationship between the message’s sender and receiver. Each message in the queue is sent to only one receipient and consumed only once. This type of messaging pattern is called when a message should acted upon only once. The payroll and financial transactions processing are suitable use cases of this type of messaging. The sender and receiver need a guarantee that each payment will be sent once only.

An Introduction to RabbitMQ

As per the official website of RabbitMQ , it is one of the most popular open-source message brokers.

The RabbitMQ has no idea of the message producer or the subscriber of the message. A message can include any kind of information. It could, for example, have information about a process or task that should start on another application (which could even be on another server), or it could be just a simple text message. The queue-manager software stores the messages until a receiving application connects and takes a message off the queue. The receiving application then processes the message.

RabbitMQ is a Publish/subscribe type of messaging style. Here the publisher of each message publishes it to a topic, and multiple consumers (subscribers) subscribe to topics from which they wish to receive messages. All messages published to a topic are distributed to all consumers that subscribed to it. This exhibits a one-to-many relationship between the message’s publisher and its consumers.

The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn’t even know if a message will be delivered to any queue at all.

The producer can only send messages to an exchange. An exchange is a very simple thing. On one side it receives messages from producers and the other side it pushes them to queues (Fig. 2). The exchange must know exactly what to do with a message it receives. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded? The rules for that are defined by the exchange type.

Fig. 2 Courtesy RabbitMQ.com

An example of publish/subscribe (a.k.a as pub-sub) can be seen in the airline industry, where the flight status updates by an airline can be used by multiple parties on the ground: baggage handlers, crews performing the maintenance and refueling of the flights, flight attendants preparing for the plane’s next trip, operators responsible to update displays notifying the public

Installing RabbitMQ as a docker container

Even though you have the option of downloading and installing directly from the RabbitMQ website and install it on your OS, it will require Erlang language to be installed first. The reason for this requirement of installing Erlang programming language first is because the RabbitMQ server is built on it.

It is easier and desirable to install a docker container for RabbitMQ that creates the environment with all its dependencies within it, so you are ready to go without the tedious and memory consuming process involved in the direct installation in the first approach.

The only pre-requisite is to have Docker desktop for windows installed on your machine and have it running. The docker image name is :

rabbitmq:3-management

Once the Docker desktop for windows is running, Open Visual Studio 2019 and open Windows Developer Command Prompt.

When the Command Prompt is opened, issue the following command

docker run -d -t -it --hostname my-rabbitmq --name rabbitmq3-server -p 15672:15672 -p 5672:5672  rabbitmq:3-management

Here default RabbitMQ user is kept as “user” and default password as “password”. You can choose your username and password in the above command.

When the docker container is created, just run the Product.Microservices project and browse to

http://localhost:15672/                                         

Here is where you’ll find the management dashboard as below

RabbitMQ Dashboard Login Screen

The default username (user) and password (password) as mentioned in the above docker command was used to login to the dashboard which shows as

I shall take the discussion of various features later for the management dashboard.

Add a shared Model Library to the exiting Solution

I shall be continuing with the existing solution named Microservices.API we created earlier in the post Create Data-Driven Microservices with ASP.NET 5 with API Gateway and add a .NET Core library project to this solution. Accordingly right-click the solution and click Add a new project

On the Add a new project dialog, Choose Class Library .NET Core and click Next

On the Configure your new project window, give it the name Shared.Models and click Create

In the library project, add a Models folder and create a Class named CustomerProduct and copy the following code to replace the existing code

using System;
using System.Collections.Generic;
using System.Text;

namespace Shared.Models.Models
{
    public class CustomerProduct
    {
        public string ProductName { get; set; }
        public DateTime AddedOnDate { get; set; }
        public string ProductDescription { get; set; }
        public decimal Price { get; set; }
    }
}

Put the reference to the Shared Models library in both the Product and Customer microservices

Product as the Publisher Microservice

Note that in our requirement, the product microservice is essentially the publisher that sends a notification to the queue of RabbitMQ whenever a new product is added to the database. The customer microservice will be the consumer who will subscribe to the notification and consume the message when the customer microservice is online.

Mass Transit as RabbitMQ abstraction layer

Using the notions of queues and exchanges, listening for messages with loops and all the like, a RabbitMQ library permits us to communicate in a raw fashion. Though we can perfectly use this when communicating between our micro-services, it will require much work, and what if in the future we want to change the messaging system from Rabbitmq to Azure Service bus or something else? If we wish to do things in a clean and maintainable way we can do so by using what is called Mass Transit.

Mass Transit is a nice layer of .NET friendly abstraction over the message broker services (whether it is RabbitMQ, Azure Service Bus or such). To meet this goal, MassTransit brings a lot of the application-specific logic closer to the developer in an easy to configure and understand manner.

MassTransit is a free, open-source distributed application framework for .NET. MassTransit makes it easy to create applications and services that leverage message-based, loosely-coupled asynchronous communication for higher availability, reliability, and scalability.

MassTransit works with several well-supported message transports and provides an extensive set of developer-friendly features to build durable asynchronous services. It supports multiple transports including:

  • RabbitMQ
  • Azure Service Bus
  • ActiveMQ
  • Amazon SQS
  • In Memory

MassTransit adds the following prominent features to the transport(here RabbitMQ), among others

  • Concurrency
  • Connection Management
  • Exception, retries
  • Serialization
  • Message header and correlation
  • Routing
  • Unit Testing

Installing the Required MassTransit Packages

Open up Package Manager Console and install the following NuGet packages for the product microservice. Note that we will be installing the same packages in our second microservice (customer microservice) too, but with a different configuration.

Install-Package MassTransit
Install-Package MassTransit.RabbitMQ
Install-Package MassTransit.AspNetCore

Configuring MassTransit in the publisher (Product Microservice)

In the product Microservice, replace the existing code in the Startup class (Startup.cs file) with the following code:

using MassTransit;
using MediatR;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.OpenApi.Models;
using Product.Microservice.Context;
using System;
using System.Reflection;

namespace Product.Microservice
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            
                services.AddMassTransit(x =>
                {
                    x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(config =>
                    {
                        config.UseHealthCheck(provider);
                        config.Host(new Uri("rabbitmq://localhost"), h =>
                        {
                            h.Username("guest");
                            h.Password("guest");
                        });
                    }));
                });
                services.AddMassTransitHostedService();
                
            

            services.AddControllers();
            services.AddSwaggerGen(c =>
            {
                c.SwaggerDoc("v1", new OpenApiInfo { Title = "Product.Microservice", Version = "v1" });
            });
            services.AddDbContext<ApplicationContext>(options =>
                options.UseSqlServer(
                    Configuration.GetConnectionString("DefaultConnection"),
                    b => b.MigrationsAssembly(typeof(ApplicationContext).Assembly.FullName)));
            services.AddMediatR(Assembly.GetExecutingAssembly());
            services.AddScoped<IApplicationContext, ApplicationContext>();

        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                app.UseSwagger();
                app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "Product.Microservice v1"));
            }

            app.UseHttpsRedirection();

            app.UseRouting();

            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
            });
        }
    }
}

In the Startup class, we have added MassTransit to the services collection. In the Configure Services method we have selected RabbitMQ as the transport bus and then initialized the RabbitMQ host URI with the guest credentials.

Now right-click the Controllers folder in the product microservice and add a new API Controller with the name of CustomerProduct and replace the code with the following code

using MassTransit;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Shared.Models.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace Product.Microservice.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class CustomerProductController : ControllerBase
    {
        private readonly IBus _busService;
        public CustomerProductController(IBus busService)
        {
            _busService = busService;
        }
        [HttpPost]
        public async Task<string> CreateProduct(CustomerProduct product)
        {
            if (product != null)
            {
                product.AddedOnDate = DateTime.Now;
                Uri uri = new Uri("rabbitmq://localhost/productQueue");
                var endPoint = await _busService.GetSendEndpoint(uri);
                await endPoint.Send(product);
                return "true";
            }
            return "false";
        }

    }
}

In the above code, we are injecting the IBus service in the CustomerProduct Controller. We are then setting the URI and named the queue as productQueue. Then we set the send endpoint for our URI (address) and finally push the message to the queue.

Configure the MassTransit in the Customer Microservice (Consumer)

Open up the Package Manager Console and install the same MassTransit NuGet packages for the customer microservice (that we earlier installed for the product microservice). This microservice will act as the consumer for the publisher (product) microservice and shall consume the message published from the RabbitMQ message broker queue as we shall see next.

After the NuGet packages are installed in the consumer (Customer Microservice), go to the customer Microservice project folder structure and create a new folder with the name “Consumers”. Now add a new class within the Consumers folder and name it “ProductConsumer”. Add the following code to replace the existing default code in the class

using MassTransit;
using Shared.Models.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace Customer.Microservice.Consumer
{
    public class ProductConsumer : IConsumer<CustomerProduct>
    {
        public async Task Consume(ConsumeContext<CustomerProduct> context)
        {

            await Task.Run(() => { var obj = context.Message; });
        }
    }
}

Now open up the Startup class and paste the following code to replace the existing code in the class

using GreenPipes;
using MassTransit;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.OpenApi.Models;
using Customer.Microservice.Consumer;
using System;

namespace Customer.Microservice
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMassTransit(x =>
            {
                x.AddConsumer<ProductConsumer>();
                x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(cur =>
                {
                    cur.UseHealthCheck(provider);
                    cur.Host(new Uri("rabbitmq://localhost"), h =>
                    {
                        h.Username("guest");
                        h.Password("guest");
                    });
                    cur.ReceiveEndpoint("productQueue", oq =>
                    {
                        oq.PrefetchCount = 20;
                        oq.UseMessageRetry(r => r.Interval(2, 100));
                        oq.ConfigureConsumer<ProductConsumer>(provider);
                    });
                }));
            });
            services.AddMassTransitHostedService();

            services.AddControllers();
            services.AddSwaggerGen(c =>
            {
                c.SwaggerDoc("v1", new OpenApiInfo { Title = "Customer.Microservice", Version = "v1" });
            });
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                app.UseSwagger();
                app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "Customer.Microservice v1"));
            }

            app.UseHttpsRedirection();

            app.UseRouting();

            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
            });
        }
    }
}

The main features of the above code are that it adds a new consumer “ProductConsumer” and sets the host URI path with the guest credentials. It then sets the receiving end points for a queue named “productQueue”

Testing Communication between microservices with RabbitMQ using the integrated Swagger tool

In this section, we shall test the communication between the publisher and consumer microservices using the built-in swagger tool. We shall test it with two different cases

First Case: Pulisher and Consumer are both online (meaning both Product and Customer Microservices are running)

Second Case: Consumer is offline with the publisher on line (Product Microservice is running while the Customer Microservice is not running) (Offline scenarios in practical world could be the non-availability of a service due to network faults or many other technical factors)

First Case Test: Let’s go to the  project folder structure and then right-click on the solution, and then click on the properties. Now, select the Multiple startup projects radio button then start both services and then click on the apply button as you do see below in the screenshot.

Now, run the project, and then you will see both services are up and running. 

In the Product.Microservice, form the request body for CustomerProduct (the request URI is automatically populated) as below screenshot and click Execute

The server response code is 200 ok as below

On checking the RabbitMQ admin dashboard with guest credentials we see the message payload as below:

So it’s working fine and the message is actually in the message broker queue which is productQueue whose details are as follows:

We can see below that there is one Consumer

Now let’s test the case 2: (Publisher Online, Consumer Offline)

For this case we again configure the Solution properties as follows:

Now run the solution and since customer microservice is not running (offline) we see the following screen

On clicking Execute and putting a breakpoint in the relevant code we find the breakpoint is hit :

The server shows a 200 OK response

On viewing the RabbitMQ we find there is one queued message and 0 concumers

Now we stop the solution and reconfigure the Solution to start both the microservices but do not issue any fresh message from the Product Microservice and check our customer microservice code, as below. (Note: We shall get the same result if we keep the product microservice offline and customer microservice online, once the message from the product (publisher) microservice is queued by the RabbitMQ.

We find the breakpoint is hit that would show us the message at the consumer end. We then inspect the RabbitMQ and find there is now one consumer

Conclusion

In this post, we have seen the concept of message broker and used RabbitMQ as our message broker. We have also integrated MassTransit with our ASP.NET Core publisher and consumer microservices and have seen the benefit of using MassTransit in our projects. We have demonstrated how the message broker supports communication between microservices in an asynchronous manner.

Source Code

https://github.com/krchome/Microservices-With-Web-API-in-ASP.NET-5

Follow Me Here

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.