In getting started with .net core microservices with Dapr I introduced my favourite framework for developing event driven microservices , Dapr - Distributed Application Runtime and I have also previously discussed how services work in Dapr. In this post, I'm going to dive a little deeper into using Dapr and discuss State Management in microservices.
What is state
State has various different meanings amongst developers, depending on where they are working within the application stack, answers may include:
- Data
- Variables within code
- Variables across the entire application
- A data store like Mongo, Redis, SQL Server
- The behaviour of an application at any given time
In computer science the state of a program is defined as its condition regarding stored inputs. The stored inputs can be anything from variables, constants or data of any sort. As an application is executed the values , or state , of the data is changed.
What is State Management
Statement management can be a key aspect of any application design. It tends to cover a broad range data storage capabilities in your application, which may include accessing files on the file system or accessing a database of some description, with the purpose of manipulating the state of an object.
State management is often an amalgamation of numerous independent behaviours:
- Data persistence
- Information flow
- Programming paradigms
- I/O ( networking & caching)
- Application architecture
- Presentation behaviour
- Templating
Stateless versus Stateful
Learning Dapr has a great chapter on State Management and it well worth reading if you want to learn more, but I will try to paraphrase some of important concepts here.
Web services are typically either of two categories: Stateless or Stateful.
A stateless service doesn't maintain contextual information between requests. Whereas a stateful service maintains contextual information during a transaction.
Stateless services are the preferred option for cloud computing services, because stateless services are easier to scale and manage.
The way state is managed defines whether a microservice is stateful (when it takes the responsibility of persisting the state upon itself) or stateless (when the state is not in its scope of responsibility).
Learning Dapr
Building Distributed Cloud Native Applications
Authoritative guide to Dapr, the distributed application runtime that works with new and existing programming languages alike.
State in distributed systems
Distributed services should be stateless, however there will still be occasions when you need to track state of object data, but you don't necessarily need to commit to a database . An example of this may be a typical Shopping Cart on an e-commerce store. You want to enable a customer to add products to a shopping cart and enable them to continue shopping enabling them to edit items in the cart as they go.
There are a few things developers will typically need to achieve in this scenario and why storing this data in a volatile session storage may not be ideal. First, you will want to ensure that when it comes to check out time, all the products the customer has chosen is available and can be processed.
A typical e-commerce browsing session may take several hours, as the customer flits between tasks and comparing various products etc. Also the customer may want to edit their basket, by removing, adding or editing quantities of products.
Thirdly, you may want to query the data in the carts to determine which products customers are interested in, check your stock compared to customer interest or even initiate abandoned cart strategies.
All these scenarios require state to managed.
Dapr State Management
Dapr simplifies the interaction with a state store by providing a State Management building block. Dapr brings state management to services through a reliable state endpoint, enabling your to transform stateful services into stateless services.
Dapr-compatible state stores are required to support optimistic concurrency and ETags. An entity tag (ETag) is an HTTP header used for Web cache validation and conditional request from browsers to resources. The value of an ETag is an identifier that represents a specific version of the resource.
Dapr also requires state stores to support eventual consistency by default. A state store can also opt to support strongly consistent writes.
Dapr requires Data stores to ensure transactional isolation for a single insert, update or delete operations. The dapr state API also defines bulk read, delete and update operations, but it does not mandate that operations be handled as a single transaction.
Dapr provides State Management via one of it Building Block, the State Management Building Block , enabling your application to use Dapr’s state management API to save and read key/value pairs using a state store component.
Dapr State Stores
Dapr has a number of Community state store adapters you can use:
- Hashicorp consul
- Etcd
- Cassandra
- Memcached
- MongoDB
- ZooKeeper
- Google Cloud Firestore
- AWS DynamoDB
- Couchbase
- PostgreSQL
- Redis
- Azure Cosmos DB
What problems do Dapr State stores solve?
Tracking state in a distributed application can be challenging. For example:
- The application may require a number of different types of data stores.
- Different consistency levels may be required for accessing and updating data.
- Multiple users may update data at the same time, requiring conflict resolution.
- Services must retry any short-lived transient errors that occur while interacting with the data store.
The Dapr state management building block addresses these challenges, by streamlining the tracking of state without dependencies or a learning curve on third-party storage SDKs.
Building a Service with Dapr State Management
Lets build a simple .net API service and make use of the Dapr State Management Building Block. In this example we'll simply make use of my API Project template, which I discussed in How to create project templates in .net core and I have installed on my machine using nuget.
dotnet new --install Threenine.ApiProject
We'll generate a solution and project using the commands below, if you want more of a discussion of the commands used below check out Manage Project and Package References with .net CLI
mkdir daprstate && cd daprstate dotnet new sln -n daprstate dotnet new apiproject -n ShoppingCart -o src/ShoppingCart dotnet sln "daprstate.sln" add "src/ShoppingCart/ShoppingCart.csproj"
What we have done with the steps above is create a basic Web API projects making use of the API endpoints pattern implementing the Mediator Pattern using Mediatr.
If we open the project in Rider: Fast & powerful cross-platform .NET IDE we see the layout
We now only need to add the Dapr.AspnetCore package to our application.
dotnet add package Dapr.AspNetCore --version 1.4.0
Then we just need to update our StartUp.cs to wire up the dependency injection for our Dapr Client
public void ConfigureServices(IServiceCollection services)
{
services.AddDaprClient();
/// .... rest of implementation
}
For the purposes of this Tutorial, I am going to use Docker-Compose to spin up a MongoDB server, just to illustrate how easy it is to get your Dapr services to communicate with externally created state stores. In production or even on your own machine you may want to use Kubernetes (K8s) to manage your containers.
I'll create a a simple MongoDb docker-compose file and will run that using the technique I outline in How to run docker compose files in Rider. The database we create here is simply for local development purposes, only to illustrate how this works.
version: '3.7'
services:
mongodb_container:
image: mongo:latest
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_USER}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PWD}
ports:
- 27017:27017
volumes:
- mongodb_data_container:/data/db
volumes:
mongodb_data_container:
Create Dapr Components
In order to make use of this in our application we need to create a Components folder in our project and create another yaml file which we'll name mongo.yaml and we'll add the following code
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: cart
spec:
type: state.mongodb
version: v1
metadata:
- name: host
value: localhost:27017
- name: username
value: dapr
- name: password
value: daprPassword
- name: databaseName
value: admin
- name: collectionName
value: cart
scopes:
- cart-service
For the purpose of this article I will keep it simple, and I have just simple kept the username, password and collectionName as plain text within the file. In a later tutorial we'll build on this to make use of Secrets, but for now I just want to illustrate configuring a state store.
A couple of important aspects of this file to point out. The first is the metadata: name: this is the name which we will refer to our state store later in our code.
spec: type: we stipulate that this is a state component and we will be making use of Mongo DB.
Dapr users a modular architecture that allows new state stores to be plugged in as Dapr components. Dapr ships with Redis as the default state store, but there are a growing list of data stores available, in our case we have elected to make use of Mongo DB.
The scopes element in the configuration constrains application access to the state store component. Only the cart-service can access the state store.
Create Post Endpoint
We need to create our Post end point which will be responsible for creating a our Cart. I won't go into too much detail about the basic Post endpoint as all the code is available for you to view in the State Management Dapr Tutorials . We'll focus on the Dapr specific items and our endpoint will call a class that we'll create CartStateService which for intents and purposes is nothing more than an Adapter Pattern over the Dapr Client to make use of the StateStateAsync and GetStateAsync methods which we will need to make use of.
public class CartStateService : IService<Item>
{
private const string DAPR_STORE_NAME = "cart";
private readonly DaprClient _client;
public CartStateService(DaprClient client)
{
_client = client;
}
public async Task<List<Item>> Update(string session, JsonPatchDocument<List<Item>> items)
{
var currentItems = await Get(session);
items.ApplyTo(currentItems);
await Save(session, currentItems);
return currentItems;
}
public async Task Save(string session, List<Item> entity)
{
await _client.SaveStateAsync(DAPR_STORE_NAME, session , entity);
}
public async Task<List<Item>>Get(string session)
{
return await _client.GetStateAsync<List<Item>>(DAPR_STORE_NAME, session);
}
}
I have intentionally left the class as simple as possible to illustrate the basic functionality we can use to carry out rudimentary state management in our applications.
The DAPR_STORE_NAME is a constant which will have the same value that we provided our Dapr component we defined earlier. We also inject our Dapr Client into our class, which will have a number of methods we can use to manipulate state.
Dapr offers a simple key/value-based state API for applications to store state. The API hides the underlying data store details so that the operation concerns are not abstracted away from application develoerps.
Running a Dapr application.
We can carry out a test of application by running our application making use of the Dapr CLI. We start our application with the following terminal command.
Before running the command below, ensure you have installed and initialised Dapr on your machine, check out Getting started with .net core microservices with dapr , as I walk you through the process and highlight some key aspects of working with Dapr
dapr run --app-id "cart-service" --app-port "5001" --dapr-grpc-port "50010" --dapr-http-port "5010" --components-path "./components" -- dotnet run --project ./ShoppingCart.csproj --urls="http://+:5001"
Once we execute this command our app will be running and you should be able to navigate to http://localhost:5001/swagger to view the documentation page and interact with the API. However , I prefer to make use of HTTP Request in Rider to test my API. These are included in the source code for the project.
## Post Cart Item
POST http://{{host-url}}/cart
accept: application/json
x-session-id: {{create-session-id}}
Content-Type: application/json-patch+json
[
{
"sku": "abc-100",
"quantity": 2,
"amount": 9.99
}
]
> {%
client.test("Request executed successfully", function() {
client.assert(response.status === 201, "Response status is not 201");
});
%}
###
## Get
GET http://{{host-url}}/cart
accept: application/json
x-session-id: {{create-session-id}}
> {%
client.test("Request executed successfully", function() {
client.assert(response.status === 200, "Response status is not 200");
});
%}
###
Once we execute the Requests, we should be able to see the data in our Mongo DB. I'll make use of the MongoDB plugin for Rider and browse to the database. We can browse to our collection and see our data.
We have implemented a vary rudimentary state store using Dapr and dapr is handling all the communication with our Mondo Container via the side car. The State API is provided by the Dapr sidecar container that stays next to each service.
All this communication is basically enabled by making use of the Dapr Client
Conclusion
In this example, we have developed a relatively simple shopping cart functionality for a website using Dapr state management block.
Dapr offers key/value storage APIs for state management. If a microservice uses state management, it can use these APIs to leverage any of the supported state stores, without adding or learning a third party SDK.
When using state management your application can leverage several features that would otherwise be complicated and error-prone to build yourself such as:
- Distributed concurrency and data consistency
- Retry policies
- Bulk CRUD operations
- What is this Directory.Packages.props file all about? - January 25, 2024
- How to add Tailwind CSS to Blazor website - November 20, 2023
- How to deploy a Blazor site to Netlify - November 17, 2023