Making API calls with HttpClientFactory in Console applications

.Net Core console applications are by far the most popular applications most developers will routinely develop in order to implement some kind of automation-based task.

I have previously discussed how by using IHost in .net core console applications developers can create asynchronous Linux Daemons and how to use Configuration API in .net core console application .

In this post, I will expand on both of these examples and provide an example of how to use .net core console applications and the HttpClient to develop a Linux daemon to interact with a Web-Based API to extract and process data.

What is the HttpClient

In .net core the HttpClient Class provides a base class for sending HTTP requests and receiving HTTP responses from a resource identified by a URI.

In previous versions on.net framework developers misused the HttpClient by typically wrapping it in a using block, primarily because the HttpClient implements a Dispose method.

The using statement is a C# language feature for dealing with disposable objects. Once the using block is complete then the disposable object. In fact, many refactoring tools like Jetbrains Resharper including my personal favourite cross-platform .net IDE Jetbrains Rider will explicitly warn if you don’t. However, In this case disposing of HttpClient was the wrong thing to do.

It is unfortunate that HttpClientimplements IDisposable which encourages the wrong behaviour.

The dispose method is called and whatever resources are in use are cleaned up. This is a typical pattern in .NET and is used for everything from database connections to stream writers. Any object which has external resources that must be cleaned up using the IDisposable interface.

You will undoubtedly find code similar to this in almost all code bases, where the HttpClient is used to download resources or connect to API’s.

C#

The underlying issue with this code is that while this HttpClient class is disposable, using it with the using statement is not the best choice because even when you dispose HttpClient object, the underlying socket is not immediately released and can cause a serious issue named ‘sockets exhaustion’.

HttpClient is intended to be instantiated once and reused throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads.

Possible approaches to solve the problem are based on the creation of the HttpClient object as singleton or static. However, the HttpClient was not designed to be used as a SIngleton or static object as it doesn’t respect DNS changes.

In .net core 2.1 , a new HttpClientFactory was introduced to be used to implement resilient HTTP calls by integrating Polly, which is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner.

What is the HttpClientFactory

The HttpClientFactory is a new mechanism to create HttpClient instances that are properly managed behind the scenes. The HttpClientFactory manages the lifetime of the handlers so that we have a pool of them which can be reused, while also rotating them so that DNS changes are respected.

The expensive part of using HttpClient is actually creating the HttpClientHandler and the connection. By Pooling these within the HttpClientFactory enables more efficient use of the connections. Using the HttpClientFactory to request a HttpClient, creates a new instance each time, meaning no need to worry about mutating its state. The HttpClient may or may not use an existing HttpClientHandler from the pool and therefore use an existing open connection.

How to use the HttpClientFactory

Like most objects in the .net core framework, the HttpClientFactory has been optimised so it can be made use of by Dependency Injection. I will illustrate a typical approach take when making use of the HttpClientFactory, in this approach I’ll be developing a Linux Daemon which will be used to contact a Rest API end-point too extract and process data.

Add the following reference to your project

Shell

It is worth checking out using IHost in .net core console applications and expand further on typical usage scenarios. We will also see an example how to use Configuration API in .net core console application .

In this scenario, we generate a simple Console Application then modify the code to make use of the IHost to enable dependency injection, logging and configuration. You will need to add a number of packages, details of which can be found in the articles referenced.

I have also added the requisite json files:

  • appsettings.json
  • appsettings.Development.json
  • appsettings.Production.json
  • hostsettings.json
C#

Information

In .net core 3.0 you will be able to generate projects which will have all this boiler plated code available as the Worker Service Template

Now that the basic project template has been configured we’re all good to go. As you might of deduced from the code, the service we’re going to develop is going to connect to the Adzuna API service to extract data. However, I will attempt to make the concepts as generic as possible so you can follow along and connect to any Web API service of choice.

The first thing we need to do before we wire up our HttpClientFactory is to edit our appsettings.[Environment].json files so the contain the configuration data we’ll need

JS

You will notice we are going store our configuration details that we will need these are credentials you receive when you register to use the Adzuna API.

The Adzuna API enables developers to get the very latest ads, job, property and car ads, and data with Adzuna’s API. Adzuna’s up-to-the-minute employment data to power your own website, reporting and data visualisations.

In particular scenario I also wanted to create a Configuration class to contain the Base Urls that I will be using in my application. So I created another POCO class as follows.

C#

We add the details to our environment specific App Settings file as follows:

JS

Information

Always ensure you include the trailing / when including a base address because this is the only permutation that seems to work.

With the details we can now edit our Program.cs to enable the dependency injection and to retrieve the configuration details from our App Settings file

C#

You’ll notice on line 42 we configure the HttpClient making use of the AddHttpClient method. If you are unable to access this method you may need to add the dependency Microsoft.Extensions.Http to your project.

Shell

We’re now ready to start writing code for our service,add a new class we’ll call JobProcessingService which will implement the IHostedService interface which defines two methods for objects that are managed by the host.

The daemon we’re going to be be developing here, will potentially be running several times an hour and will be running several jobs at least, so it is important for us to build in some kind of cancellation process using tokens. For this purpose of this post, I will implement a “Good Enough”approach but this is by no means a complete approach and it simply has the basics, but provides at least some understanding of what required.

C#

Now I background task is complete we’ll move onto the actual implementation of our Service which implements the HttpClient, again the implementation here is just enough to provide a flavour of the implementation for demo purpose, but you will see enough of how to implement the HttpClient and make use of it, in a real-world scenario.

The major drawback with the code in this implementation is that we make a call retrieve data and deserialize it but we don’t actually do anything with it! But it this just provides and example of how to use the HttpClient and the HttpClientFactory

C#

You may have noticed I am using an extension method on the Stream class to DeserializeFromJsonI thought I would include the extension method for clarity. The code for this method is in a new class I called StreamExtensions.cs

C#

The code for these sections is almost complete, we now just need to wire up our application startup.

We’ll now edit the Program.cs to look like the following

C#

Summary

We have seen an example of how to make use HttpClientFactory within a .net console application to create a HttpClient instance to be used within a HostedService application.

Gary Woodfine

Technical Director at Denizon
Gary is Technical Director at Denizon, an independent software vendor specialising in IoT, Field Service and associated managed services,enabling customers to be efficient, productive, secure and scalable in a way which helps them address and reduce their ecological impact.

Denizon's product line successfully integrate IoT, Artificial Intelligence and Blockchain technology to enable efficient, productive, secure and scalable solutions to help organisations address increasing energy demands, ecological impact and Health & Safety concerns of their staff.

Latest posts by Gary Woodfine (see all)

Tags: