API Management and the Service Mesh

API management excels on socialization and self-subscription of APIs while ensuring security across application boundaries. A Service Mesh is more about the internal routing and resilience patterns required within an application or domain boundary.

Context

There is both noise and religious positions around API Management and Service Mesh existence, co-existence, usage and interactions. Confusion around the topic is abundant and there is no definitive white or black position for them as vendors are also confusing the functionalities for the rest of us mortals.

One generally accepted and quite full of common-sense position is that the Service Mesh serves for East-West traffic. But then, Istio, one of my go-to solutions out there, includes an Ingress Gateway that’s clearly intended for north-south traffic.

Istio Ingress Gateway docs:

Gateway describes a load balancer operating at the edge of the mesh receiving incoming or outgoing HTTP/TCP connections.

What’s the confusion? Well, here’s a list of features that the API Management (API Gateway) and the Service Mesh share:

  • Telemetry collection
  • Distributed tracing
  • Service discovery
  • Load balancing
  • TLS termination/origination
  • JWT validation
  • Request routing
  • Traffic splitting
  • Canary releasing
  • Traffic shadowing
  • Rate limiting

What’s nice is that, overlapping as it is, these features make sense both in the API Management and the Service Mesh thus ultimately, they should not add to the confusion but complete the solution in a consistent way.

Circuit breaker is an indispensable pattern in a distributed application. I like Martin Fowler’s summary for the pattern:

The basic idea behind the circuit breaker is very simple. You wrap a protected function call in a circuit breaker object, which monitors for failures. Once the failures reach a certain threshold, the circuit breaker trips, and all further calls to the circuit breaker return with an error, without the protected call being made at all. Usually you’ll also want some kind of monitor alert if the circuit breaker trips.

From Wikipedia: East-West traffic indicates flow of data among devices within a specific data center. The other direction of traffic flow is North-south traffic, which typically indicates data flow that either enters/leaves the data center from/to a system physically residing outside the data center.[1]

Xsigo East-West traffic

API Gateway Pattern

The API gateway pattern describes an additional hop in the network that every request will have to go through in order to consume the underlying APIs. In this context, some people call the API gateway a centralized deployment.

  • Insulates the clients from how the application is serving the APIs
  • Insulates the clients from the problem of determining the locations, types, authentication requirements of service instances
  • Provides the optimal API for each client
  • Reduces the number of requests/roundtrips. API gateway enables clients to retrieve data from multiple services with a single round-trip. Fewer requests also means less overhead and improves the user experience. An API gateway is essential for mobile applications.
  • Translates from a “standard” public web-friendly API protocol to whatever protocols are used internally

Drawbacks

  • complexity – the API gateway is yet another moving part that must be developed, deployed and managed
  • response time – for most applications the cost of an extra network hop is insignificant.

Related:

Service Mesh Pattern

William Morgan made the observation that the the interconnection between proxies form a mesh network. In early 2017, William wrote a definition for this platform, and called it a Service Mesh:

A service mesh is a dedicated infrastructure layer for handling service-to-service communication. It’s responsible for the reliable delivery of requests through the complex topology of services that comprise a modern, cloud native application. In practice, the service mesh is typically implemented as an array of lightweight network proxies that are deployed alongside application code, without the application needing to be aware.

Probably the most powerful aspect of his definition is that it moves away from thinking of proxies as isolated components and acknowledges the network they form as something valuable in itself.

A service mesh can be defined as an infrastructure layer which handles the inter-service communication in a microservice architecture. Service mesh reduces the complexity associated with a microservice architecture and provides lot of the functionalities like:

  • Load balancing
  • Service discovery
  • Health checks
  • Authentication
  • Traffic management and routing
  • Circuit breaking and failover policy
  • Security
  • Metrics and telemetry
  • Fault injection

Service Mesh decouples complexity from the application and puts it in a service proxy to handle. These service proxies allow the application to focus on the implementation of business functionalities.

In a microservice architecture, if you have several services talking with each other, rather than baking common necessary functionalities like configuration, routing, telemetry, logging, circuit breaking, etc. inside every microservice, it makes more sense to abstract it into a separate component — called a ‘service proxy.’

The Service Mesh is generally deployed as a proxy alongside services also known as the sidecar pattern.

The sidecars abstract the complexity away from the application and handle the functionalities like service discovery, traffic management, load balancing, circuit breaking, etc.

As I mentioned, Istio is one of my go-to solutions out there and provides additional capabilities in your microservices architecture like intelligent routing, load balancing, service discovery, policy enforcement, in-depth telemetry and all mentioned above.

 

As you can see above, I am not referring to products but patterns (sure, I mention all time favourites but viable and higly performant alternatives exist and abide to the same rules). If you look at traditional applications, serverless or microservices, these patterns apply (not Istio, the pattern)