1 Introduction

In this tutorial, we will introduce the basics of the service grid architecture and understand how it complements the distributed system architecture.

We will mainly focus on Istio, which is the implementation of the service mesh. In this process, we will introduce the core architecture of Istio and learn how to benefit from it on Kubernetes.

2.What is a service grid?

In the past few decades, we have seen how monolithic applications start to break down into smaller applications. It has found unprecedented popularity in cloud native computing and microservice architecture . In addition, containerization technologies such as Docker and orchestration systems such as Kubernetes are only helpful in this regard.

Although there are many advantages to adopting a microservice architecture on a distributed system like Kubernetes, it has considerable complexity . Since distributed services must communicate with each other, we must consider discovery, routing, retry, and failover.

There are other issues, such as security and observability, we must also pay attention to the following issues:

Now, it can be very cumbersome to establish these communication functions in each service, especially when the service range expands and the communication becomes complicated. This is where the service grid can help us. Basically, the service grid eliminates the responsibility of managing all service-to-service communication in a distributed software system .

The service mesh can do this through a set of network agents. Essentially, requests between services are routed through a proxy that runs with the service but is located outside the infrastructure layer:

These proxies basically create a mesh network for the service-hence the name service mesh! Through these agents, the service grid can control all aspects of service-to-service communication. In this way, we can use it to solve the eight fallacies of distributed computing .This is a set of assertions that describe the wrong assumptions we often make about distributed applications.

3.Characteristics of service grid

Now, let us understand some of the features that the service grid can provide us. Please note that the actual function list depends on the implementation of the service mesh. However, in general, we should expect most of these features in all implementations.

We can roughly divide these functions into three categories: traffic management, security and observability.

3.1 Traffic management

One of the basic characteristics of the service grid is traffic management. This includes dynamic service discovery and routing. It also enables some interesting use cases, such as traffic shadowing and traffic splitting . These are very useful for performing releases and A/B testing.

Since the communication between all services is handled by the service grid, it also enables some reliability features . For example, the service mesh can provide retries, timeouts, rate limiting, and circuit breakers. These off-the-shelf failure recovery functions make communication more reliable.

3.2 Security

The service grid usually also handles the security aspects of service-to-service communication . This includes enforcing traffic encryption through mutual TLS (MTLS), providing identity verification through certificate verification, and ensuring authorization through access policies.

There may also be some interesting security use cases in the service mesh. For example, we can implement network segmentation to allow certain services to communicate while prohibiting other services. In addition, the service grid can provide accurate historical information for audit requirements.

3.3 Observability

Strong observability is a basic requirement for dealing with the complexity of distributed systems . Since the service grid can handle all communications, it is properly placed to provide observability functions. For example, it can provide information about distributed tracing.

The service grid can generate many indicators, such as latency, traffic, errors, and saturation . In addition, the service grid can also generate access logs to provide a complete record of each request. These are very useful for understanding the behavior of individual services and the entire system.

4.Introduction to Istio

Istio is an open source implementation of the service grid originally developed by IBM, Google and Lyft . It can be layered transparently onto distributed applications and provides all the advantages of a service grid, such as traffic management, security, and observability.

It is designed to be used in conjunction with various deployments, such as local deployment, cloud hosting, Kubernetes containers, and service programs running on virtual machines. Although Istio has nothing to do with the platform , it is often used with microservices deployed on the Kubernetes platform.

Fundamentally speaking, Istio works by deploying the extended version of Envoy as a proxy to each microservice as a secondary vehicle:

This agent network constitutes the data plane of the Istio architecture. The configuration and management of these agents is done from the control plane:

The control plane is basically the brain of the service grid . It provides discovery, configuration and certificate management for the Envoy proxy in the data plane at runtime.

Of course, we can only realize the advantages of Istio when we have a large number of microservices that communicate with each other. Here, the sidecar proxy forms a complex service mesh in a dedicated infrastructure layer:

Istio is very flexible in terms of integration with external libraries and platforms. For example, we can integrate Istio with external logging platforms, telemetry or strategy systems.

5.Understand Istio components

We have seen that the Istio architecture consists of a data plane and a control plane. In addition, there are several core components that make Istio work.

In this section, we will introduce these core components in detail.

5.1 Data plane

Istio's data plane mainly includes an extended version of the Envoy proxy . Envoy is an open source edge and service proxy that can help isolate network issues from underlying applications. The application only localhostsends and receives messages without knowing the network topology.

The core of Envoy is a network proxy running at the L3 and L4 layers of the OSI model . It performs connection processing by using a pluggable network filter chain . In addition, Envoy supports additional L7 layer filters for HTTP-based traffic. Moreover, Envoy has first-class support for HTTP/2 and gRPC transmissions.

Many of the functions provided by Istio as a service mesh are actually enabled by the basic built-in functions of the Envoy proxy:

• Flow control: Envoy enables fine-grained flow control applications through rich routing rules for HTTP, gRPC, WebSocket and TCP traffic

• Network resilience: Envoy includes out-of-the-box support for automatic retry, circuit breaking and fault injection

• Security: Envoy can also implement security policies and apply access control and rate limiting to communications between basic services

One of the other reasons why Envoy excels on Istio is its scalability. Envoy provides a pluggable extension model based on WebAssembly . This is very useful in custom strategy execution and telemetry generation. In addition, we can also use the Istio extension based on the Proxy-Wasm sandbox API to extend the Envoy proxy in Istio.

5.2 Control plane

As mentioned earlier, the control plane is responsible for managing and configuring the Envoy proxy in the data plane. istiodThe component responsible for this in the control plane. Here, it istiodis responsible for translating advanced routing rules and flow control behaviors into Envoy-specific configurations and propagating them to the sidecars at runtime.

If we look back at the Istio control plane architecture, we will notice that it used to be a set of independent components that cooperated with each other. It includes components such as Pilot for service discovery, Galley for configuration, Citadel for certificate generation, and Mixer for extensibility. Due to complexity, these ** separate components are combined into a istiodsingle component called **.

Since istiod, istiod still uses the same code and API as the previous components. For example, Pilot is responsible for abstracting platform-specific service discovery mechanisms and synthesizing them into a standard format that Sidecar can use. Therefore, Istio can support discovery for multiple environments (such as Kubernetes or virtual machines).

In addition, istiodsecurity is provided, and powerful service-to-service and end-user authentication is achieved through built-in identity and credential management. In addition, using istiod, we can implement security policies based on service identities. The process istiodalso acts as a certificate authority (CA) and generates certificates for mutual TLS (MTLS) communication in the data plane.

6.How Istio works

We have already understood what are the typical characteristics of a service grid. In addition, we introduced the foundation of the Istio architecture and its core components. Now, it's time to understand how Istio provides these functions through the core components of its architecture.

We will focus on the same category of features that we have experienced before.

6.1 Traffic Management API

We can use the Istio traffic management API to fine-tune the traffic in the service mesh. We can use these APIs to add our own traffic configuration to Istio. In addition, we can use Kubernetes custom resource definitions (CRD) to define API resources. The key API resources that help us control traffic routing are virtual services and target rules:

Basically, virtual services allow us to configure how to route requests to services in the Istio service grid. Therefore, a virtual service consists of one or more routing rules that are evaluated in sequence. After evaluating the routing rules of the virtual service, the target rules are applied. Target rules help us control the traffic to the target, for example, grouping service instances by version.

6.2 Security

Security in Istio starts with providing a strong identity for each service. Run the Istio agent istiodautomatic key and certificate rotation along with each envoy agent job :

Istio provides two types of authentication: peer-to-peer authentication and request authentication . Peer-to-peer authentication is used for service-to-service authentication, in which Istio provides mutual TLS as a full-stack solution. Request authentication is used for end user authentication, where Istio uses a custom authentication provider or OpenID Connect (OIDC) provider to provide JSON Web Token (JWT) authentication.

Istio also allows us to implement access control to services by simply applying authorization policies to services. The authorization policy implements access control to the inbound traffic in the Envoy proxy. In this way, we can apply access control at various levels: grid, namespace, and service scope.

6.3 Observability

Istio generates detailed telemetry for all service communications within the mesh network, such as metrics, distributed tracking, and access logs. Istio generates a rich set of agent-level indicators, service-oriented indicators and control plane indicators .

Previously, the Istio telemetry architecture used Mixer as a core component. But starting from Telemetry v2, the functions provided by the mixer have been replaced with Envoy proxy plugins:

In addition, Istio generates distributed traces through the Envoy proxy. Istio supports many tracking backends, such as Zipkin , Jaeger , Lightstep and Datadog . We can also control the sampling rate of the tracking rate. In addition, Istio also generates access logs for service traffic in a set of configurable formats.

7.Practice with Istio

Now that we have learned enough background knowledge, we can see the practical application of Istio. First, we will install Istio in the Kubernetes cluster. In addition, we will use a simple microservice-based application to demonstrate the capabilities of Istio on Kubernetes.

7.1 Installation

There are multiple ways to install Istio, but the easiest way is to download and unzip the latest version of a specific operating system (such as Windows). The client binary file binin the extracted package directory istioctl. We can use istioctl** to install Istio on the target Kubernetes cluster :**

istioctl install --set profile=demo -y

This will install the Istio component on the default Kubernetes cluster using the demo configuration file. We can also use any other vendor-specific configuration files instead of demos.

Finally, when we deploy any application on this Kubernetes cluster, we need to instruct Istio to automatically inject the Envoy sidecar proxy:

kubectl label namespace default istio-injection=enabled

Our kubectlpremise is that we already have a Kubernetes cluster like Minikube and Kubernetes CLI on our machinekubectl

7.2 Online ordering application

For demonstration purposes, we will imagine a very simple online ordering application. The application contains three microservices that interact with each other to satisfy the end user's order request:

We did not discuss the details of these microservices, but they can be created very simply using Spring Boot and REST API. Most importantly, we created a Docker image for these microservices so that we can deploy them on Kubernetes.

7.3 Deployment method

Deploying containerized workloads on a Kubernetes cluster like Minikube is very simple. We will use Deploymentand Serviceresource types to declare and access workloads. Usually, we define them in YAML files:

apiVersion: apps/v1beta1
 kind: Deployment
 metadata:
 name: order-service
 namespace: default
 spec:
 replicas: 1
 template:
 metadata:
 labels:
 app: order-service
 version: v1
 spec:
 containers:
 - name: order-service
 image: kchandrakant/order-service:v1
 resources:
 requests:
 cpu: 0.1
 memory: 200
 ---
 apiVersion: v1
 kind: Service
 metadata:
 name: order-service
 spec:
 ports:
 - port: 80
 targetPort: 80
 protocol: TCP
 name: http
 selector:
 app: order-service

order-service Deploymentand Servicea very simple definition. Similarly, we can define YAML files for inventory-serviceand shipping-service.

kubectlDeploying these resources is also very simple:

kubectl apply -f booking-service.yaml -f inventory-service.yaml -f shipping-service.yaml

Since we have enabled automatic injection of the Envoy sidecar proxy for the default namespace, everything will be handled by us. Alternatively, we can use kube-injectthe command to istioctlmanually inject the envoy cross-fighting agent.

7.4 Access to the application

Now, Istio is mainly responsible for handling all mesh network traffic. Therefore, by default, no traffic in and out of the grid is not allowed . Istio uses gateways to manage inbound and outbound traffic from the grid. In this way, we can precisely control the flow into or out of the grid. Istio provides some pre-configured gateway proxy deployments: istio-ingressgatewayand istio-egressgateway.

We will create a gateway and a virtual service for our application to achieve this:

apiVersion: networking.istio.io/v1alpha3
 kind: Gateway
 metadata:
 name: booking-gateway
 spec:
 selector:
 istio: ingressgateway
 servers:
 - port:
 number: 80
 name: http
 protocol: HTTP
 hosts:
 - "*"
 ---
 apiVersion: networking.istio.io/v1alpha3
 kind: VirtualService
 metadata:
 name: booking
 spec:
 hosts:
 - "*"
 gateways:
 - booking-gateway
 http:
 - match:
 - uri:
 prefix: /api/v1/booking
 route:
 - destination:
 host: booking-service
 port:
 number: 8080

Here, we use the default entry controller provided by Istio. In addition, we have defined a virtual service to route our requests to booking-service.

Similarly, we can also define an egress gateway for outbound traffic from the grid .

8.Common use cases of Istio

Now, we have seen how to deploy a simple application on Kubernetes using Istio. However, we still haven't taken advantage of any interesting features that Istio has enabled for us. In this section, we will introduce some common use cases of service mesh and learn how to implement them for our simple application using Istio.

8.1 Request routing

There are several reasons why we may have to handle request routing in a specific way. For example, we **may deploy micro shipping-service** and want to route only a small portion of requests to the new version.

We can use virtual service routing rules to achieve this:

apiVersion: networking.istio.io/v1alpha3
 kind: VirtualService
 metadata:
 name: shipping-service
 spec:
 hosts:
 - shipping-service
 http:
 - route:
 - destination:
 host: shipping-service
 subset: v1
 weight: 90
 - destination:
 host: shipping-service
 subset: v2
 weight: 10
 ---
 apiVersion: networking.istio.io/v1alpha3
 kind: DestinationRule
 metadata:
 name: shipping-service
 spec:
 host: shipping-service
 subsets:
 - name: v1
 labels:
 version: v1
 - name: v2
 labels:
 version: v2

Routing rules also allow us to define matching conditions based on attributes such as header parameters. In addition, the destination field specifies the actual destination of the traffic that matches the condition.

8.2 Open circuit

A circuit breaker is essentially a software design pattern used to detect faults and encapsulate logic that prevents further cascading of faults . This helps create resilient microservice applications to limit the impact of failures and delay spikes.

In Istio, we can use DestinationRule trafficPolicyconfiguration inventory-serviceto apply circuit breaks when calling services such as inventory :

apiVersion: networking.istio.io/v1alpha3
 kind: DestinationRule
 metadata:
 name: inventory-service
 spec:
 host: inventory-service
 trafficPolicy:
 connectionPool:
 tcp:
 maxConnections: 1
 http:
 http1MaxPendingRequests: 1
 maxRequestsPerConnection: 1
 outlierDetection:
 consecutive5xxErrors: 1
 interval: 1s
 baseEjectionTime: 3m
 maxEjectionPercent: 100

Here, we will DestinationRule maxConnections1, httpMaxPendingRequests1, maxRequestsPerConnection1. This actually means that if we increase the number of concurrent requests to more than 1, the circuit breaker will start catching some requests.

8.3 Enable mutual TLS

Mutual authentication refers to the situation where both parties perform identity verification at the same time in an authentication protocol such as TLS . By default, all traffic between services with proxies uses mutual TLS in Istio. However, services without a proxy continue to receive traffic in plain text format.

Although Istio automatically upgrades all traffic between services with proxies to mutual TLS, these services can still receive plain text traffic. We can choose to use PeerAuthenticationstrategies to implement mutual TLS across the entire grid:

apiVersion: "security.istio.io/v1beta1"
 kind: "PeerAuthentication"
 metadata:
 name: "default"
 namespace: "istio-system"
 spec:
 mtls:
 mode: STRICT

We also provide the option to enforce mutual TLS for each namespace or service instead of mesh-wide. However, PeerAuthenticationthe PeerAuthentication strategy of the service takes precedence over the namespace-scoped strategy .

8.4 Use JWT for access control

JSON Web Token (JWT) is a standard for creating data whose payload contains JSON that declares many claims . In order to pass the identity of the authenticated user and the standard or custom statement between the identity provider and the service provider, this has been widely accepted.

We can enable authorization policies in Istio to allow access to JWT-basedbooking-service

apiVersion: security.istio.io/v1beta1
 kind: AuthorizationPolicy
 metadata:
 name: require-jwt
 namespace: default
 spec:
 selector:
 matchLabels:
 app: booking-service
 action: ALLOW
 rules:
 - from:
 - source:
 requestPrincipals: ["[email protected]/[email protected]"]

Here, AuthorizationPolicyall requests are forced to have a valid JWT and requestPrincipalset to a specific value. Istio declares the isssum of JWT through combinationsub requestPrincipal

9.Thinking afterwards

Therefore, so far, we have seen how service grids like Istio make our lives easier to deal with many common problems in distributed architectures such as microservices. But despite this, Istio is still a complex system that will increase the complexity of the final deployment . Like all other technologies, Istio is not a panacea and must be used with caution.

9.1 Should we always use service mesh?

Although we have seen enough reasons to use a service mesh, let us list some reasons that might prompt us not to use it:

• The service grid handles all service-to-service communication, and the deployment and operation of the service grid requires additional costs. For simpler applications, this may be unreasonable

• Since we are accustomed to dealing with some of these issues, such as circuit interruptions in the application code, it may lead to repeated processing in the service mesh

• Increasing reliance on external systems such as service grids may harm application portability, especially since there is no industry standard for service grids

• Since the service mesh usually works by intercepting the mesh traffic passing through the proxy, it may add undesirable delays to the request

• The service mesh adds many other components and configurations, which need to be handled precisely. This requires expertise and adds to the learning curve

• Finally, we may end up mixing operational logic (should exist in the service grid) with business logic (should not exist in the service grid)

Therefore, as we have seen, the story of service grids does not only involve benefits, but it does not mean that they are not real. For us, it is important to carefully evaluate our needs and application complexity, and then weigh the benefits of service grids and the added complexity they add.

9.2 What are the alternatives to Istio?

Although Istio is very popular and has the support of some leaders in the industry, it is certainly not the only option. Although we cannot make a comprehensive comparison here, let's take a look at the two options Linkerd and Consul.

Linkerd is an open source service mesh that has been created for the Kubernetes platform . It is also very popular and currentlyhas the status of an incubation projectin CNCF . It works like any other service mesh like Istio. It also uses TCP proxy to handle grid traffic. Linkerd uses micro-agents written in Rust, called Linkerd agents.

Overall, Linkerd is not more complicated than Istio because it only supports Kubernetes. But, other than that, the list of features available in Linkerd is very similar to those available in Istio. Linkerd's core architecture is also very similar to Istio . Basically, Linkerd consists of three main components: user interface, data plane and control plane.

Consul is an open source implementation of HashiCorp 's service grid . Its advantage is that it can be well integrated with HashiCorp's other infrastructure management product suites to provide a wider range of functions. The data plane in Consul can flexibly support agents and native integration models. It comes with a built-in proxy, but it can also be used with Envoy.

In addition to Kubernetes, Consul can also be used with other platforms such as Nomad . Consul works by running Consul agents on each node to perform health checks. These agents communicate with one or more Consul servers that store and replicate data. Although it provides all the standard functions of a service grid such as Istio, it is a more complex system to deploy and manage .

10.Conclusion

All in all, in this tutorial, we introduced the basic concepts of the service grid pattern and the functions it provides us. In particular, we introduced Istio in detail. This covers the core architecture of Istio and its basic components. In addition, we detailed the details of installing and using Istio for some common use cases.