Demoing microservices architecture in spring ecosystem with simple e-commerce application
This repo contains demo about how to build simple ecommerce microservices app from scratch step by step
based on spring boot (spring cloud, spring cloud gateway, spring data JPA, spring web, ...), deployed on Kubernetes and AWS using EKS.
The content part explains some concepts in microservices architecture like : service discovery, load balancing, distributed tracing, massaging & Advanced Message Queuing Protocol, ...
The app composed of 5 microservices [customer - product - order - payment - notification] communicating with each other using REST API (+ Open Feign) and messaging system (RabbitMQ).
Over the demonstration and documentation, we will cover a bunch of concepts in the microservice architecture :
mvn archetype:generate -DgroupId=dev.nano -DartifactId=ecomicroservices -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
In order to create microservices instances in top of the parent module, we need to generate submodule for each microservice then we add the necessary dependencies.
application.yml
file.modelMapper
to mapStruct
In the beginning, I have used modelMapper to map entities to DTOs and vice versa, but I have found that mapStruct is more efficient and faster than modelMapper. That's why I decided to move to mapStruct.
mapStruct
is a library that helps to map objects between two classes, same as modelMapper
.mapStruct
is more flexible and faster than modelMapper
and it is easier to use.restTemplate
After creating all microservices, we're supposed to send requests (Http requests) between them via restTemplate
to get data from other microservices.
#Example in customer microservice
OrdersResponse ordersResponse = restTemplate.postForObject(
"http://localhost:8003/api/v1/orders",
customerOrdersRequest,
OrdersResponse.class
);
Service discovery
is a tool that helps for detecting the devices & services that are running on the network, in order to connect to them.Medium Photo By Isuru Jayakantha
pom.xml
for all microservices.<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
@EnableEurekaClient
annotation in Application class.@LoadBalanced
restTemplate
to OpenFeign
OpenFeign
is a client-side library that simplifies the use of RESTful web services.OpenFeign
gives more flexibility to use APIs from different microservices just creating interfaces (CustomerOrdersFeignClient & CustomerProductFeignClient) that target the method body in the controller.<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
@EnableFeignClients
annotation in Application class._Load balancing_
is the process of distributing network traffic across multiple servers, to ensures no single server bears too much demand.
There is a variety of load balancing methods (that decide in which microservice instance to send the request), which use different algorithms best suited for a particular situation:
In the LB we manage a lot of things like : Authentication, High availability, Logging, caching, TLS, etc.
But in production we don't want to manage all these things, we can use some providers (aws, azure, google, etc.)
There's another concept in LB called Health Check
, which is used to check the availability of the service to respond to the requests.
Some resources explain LB :
Spring Cloud Gateway
is a service that allows you to route requests to different microservices. pom.xml
:<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
application.yml
file.8001
/api/v1/customers", 8004
`/api/v1/customers".Sleuth Zipkin
is a distributed tracing library that allows you to trace the flow of requests and responses between microservices.pom.xml
:<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-sleuth-zipkin</artifactId>
</dependency>
#Spring
spring:
application:
name: product
zipkin:
base-url: http://localhost:9411
**_scaling issues_**
.Solution :
- The solution in this case, is about adding a message broker to the application, so in case of an outage of the notification microservice, we can send requests asynchronously to the message broker,
until the notification microservice is up again.
Stand-for : Advanced Message Queuing Protocol
Is a messaging protocol that allows clients apps to communicate with each other using a messaging middleware brokers (RabbitMQ, Kafka, etc.).
Product / Customer & Orders microservices are producers of messages in our app.
Notification microservice is a consumer of messages in our app.
Our queue is the RabbitMQ broker.
Illustration :
Note : We can create multiple queues in RabbitMQ.
Add new module called "ampq", which contains the RabbitMQ configuration and RabbitMQ producer.
RabbitMQConfig
:
RabbitMQProducer
:
Add maven dependency for RabbitMQ in pom.xml
:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
public class NotificationConsumer {
@RabbitListener(queues = "${rabbitmq.queue.notification}")
public void consumer(CustomerNotificationRequest customerNotificationRequest) {
log.info("Consumed {} from queue", customerNotificationRequest);
notificationService.sendNotification(customerNotificationRequest);
}
}
JIB
is a maven plugin that allows you to create a Docker image for your app.pom.xml
of the whole microservice using profiles
tag :<profiles>
<profile>
<id>jib-build-push-docker-image</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</profile>
</profiles>
**docker-jib**
in the parent module by specifying java image and its version, as well as the goal of the build (check pom.xml of the parent module).docker login # (if you are not logged in)
!! Note !!
: Concerning the packaging of microservices to JARs, first we need to install from the root using maven to be able to use modules as dependencies in other project.mvn clean package -P build-docker-image
docker-compose up -d
The master node
is the node that is responsible for managing the cluster (brain of the cluster), where all the decisions are made.The worker nodes
are the nodes that are responsible for running the applications.
Note: We should never deploy our DB on K8S, because in case that node can die then you will lose the hole data of the application. Instead of that, we can use some services like RDS via AWS.
Deploy Postgres to K8S :
config-map
that contains the credentials of the database.service
manifest in order to define how we can access the database (ClusterIP).volume
using "PersistentVolume" and "PersistentVolumeClaim".statefulset
instead of deployment because the stateful-set support long-lived app such as databases (Save data in case that pod restart).Run the following commands :
minikube start
cd k8s/miniKube
kubectl apply -f bootstrap/postgres
get pods (retrieve "postgres" pod name)
kubectl exec -it postgres-0 -- psql -U miliariadnane
CREATE DATABASE customer; CREATE DATABASE order; CREATE DATABASE product; CREATE DATABASE notification;
kubectl apply -f bootstrap/rabbitmq
kubectl apply -f bootstrap/zipkin
To read more about ingress, you can check k8s documentation : Ingress k8s doc
Create a new cluster (master node) :
IAM role
", it means create AWS user.Security Group
" means a list of permissions that the user can access.Create cluster for node groups (worker nodes) :
Create worker nodes & connect to cluster
EC2 instances
".eksctl
check this linkMonitoring
is a concept that allows us to monitor and supervise the performance of our application using metrics like Http traces, health checks, CPU, memory, etc.micrometer
.<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.9.2</version>
</dependency>
management:
endpoints:
web:
exposure:
include: "*"
endpoint:
health:
show-details: always
Add docker images to docker compose.
Monitoring diagram :
env:
- name: SPRING_PROFILES_ACTIVE
value: eks
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: dbsecretd
key: db_username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: dbsecret
key: db_password
Install kubectl : link
Install aws cli : link
Configure and credential file settings in local machine : link
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Install eksctl :
Add IAM permissions : The IAM security principal that you're using must have permissions to work with Amazon EKS IAM roles and service linked roles, and a VPC and related resources.
Verify kubectl configuration
kubectl cluster-info
Update kube config (exist in ~/.kube/config)
Run the following command :
exe : aws eks update-kubeconfig --name [name-of-eks-cluster] --region [aws-region]
aws eks update-kubeconfig --name demo-microservices --region ca-central-1
Check nodes status :
kubectl get nodes
Add secrets :
kubectl create secret generic dbsecret --from-literal=db_username=your-db-username --from-literal=db_password=your-db-password