- 14 minutes to read

Blob Storage Policy

Info
This guide explains how to apply a Nodinite-specific policy that enables comprehensive logging from the Azure API Management platform to Nodinite.

✅ Track all API requests and responses, including payloads and headers, with ease
✅ Achieve asynchronous, scalable logging for any payload size
✅ Monitor and receive real-time alerts with Nodinite agents
✅ Maintain compliance and traceability through end-to-end logging

The diagram below shows the end-to-end flow for logging events from Azure API Management to Blob Storage and then to Nodinite.

graph LR subgraph "Payload (Any size)" AA[Azure API Management
with Policy 2] --> |1. Log Event,
Any size | roBS[Blob Storage] end subgraph "Nodinite" roBS --> C{Pickup Service} C --> F[Log API] end

Diagram: End-to-end flow for logging API events from Azure API Management to Blob Storage and Nodinite.

When you activate the Blob Storage Policy, the code running in the Azure API Management platform generates a Nodinite-specific JSON Log Event and stores it in the Blob Storage Container. This approach captures every request and response, regardless of payload size, and ensures all relevant data remains available for monitoring and analysis.

Tip
Use the Nodinite Azure Monitoring Agent to receive alerts if Log Events accumulate in Blob Storage. The Nodinite Pickup Log Events Service Logging Agent consumes these Log Events. You can also configure a backout container for malformed messages, ensuring you do not lose data and that all issues are monitored.

The Nodinite Pickup Log Events Service Logging Agent processes Log Events from Blob Storage asynchronously, providing scalable and reliable logging.

Tip
The Nodinite Non-Events Monitoring Agent alerts you if the number of events deviates from expectations, helping you maintain operational integrity.

Configuration Steps

Follow these steps to enable logging to Nodinite in your Azure API Management Service APIs:

Next, configure the Nodinite Pickup Log Events Service Logging Agent to consume Log Events from Blob Storage. Route malformed Log Events to a backout container for further analysis.

Configure the Nodinite Azure Monitoring Agent to ensure you do not stockpile Log Events in your destination or backout containers.

Create and use Nodinite Log Views with Search Fields to provide business users and stakeholders with actionable insights into your logged data.

Optionally, set up the Nodinite Non-Events Monitoring Agent to validate that you receive the expected number of Log Events during a given period.

Managed Identity

In the Blob Storage Policy, you use a user-assigned Managed Identity to securely access Azure resources.

Create (or reuse) a Managed Identity in the Azure Portal.
Note the Client ID; you will need it in the Named Values section.

Image: Managed Identity Client ID in Azure Portal.
Assign the Storage Blob Contributor role to the Managed Identity on the Storage Account referenced by the logging-blob-url named value.

Image: Assigning Storage Blob Contributor role to Managed Identity.
Add the Managed Identity to the API Management Service where you will apply the Blob Storage Policy.

Image: Adding Managed Identity to API Management Service.

Named Values

The Blob Storage Policy uses variables to avoid hard-coding values. You can expand the sample with additional properties as your business and policy require.

In the Azure Portal, navigate to the API Management Service and click Named Values in the sidebar.

Image: Named Values configuration in Azure Portal.

Create the following named values:

logging-blob-mid-client-id – Enter the Client ID of the Managed Identity from the previous step.
logging-blob-url – Enter the URL for the Blob Storage Container. Ensure the value ends with a trailing slash (/). You can omit the trailing slash if you also change the code in the sample.
Image: Blob Storage Container URL example.

Policy Fragments

The Blob Storage Policy uses three Policy Fragments:

Inbound Policy Fragment – Creates a Nodinite JSON Log Event for Inbound Processing.
Outbound Policy Fragment – Creates a Nodinite JSON Log Event for Outbound Processing.
On Error Policy Fragment – Creates a Nodinite JSON Log Event when an error occurs in Inbound or Outbound Processing.

Tip
All events in the sample use the x-ms-client-tracking-id header value. This ID enables end-to-end logging and is vital for all Nodinite Log Events. You can group all Log Events for an interchange in a Nodinite Log View by the built-in Search Field Application interchange id. If you call an Azure Function using the Nodinite Serilog sinks, add this header to the context to retain the value through the whole chain of events. If you invoke a Logic App, the system handles this automatically.
Image: Grouping by Application interchange id in Nodinite Log View.

Inbound Policy Fragment

In the Azure Portal, navigate to the API Management Service, then, in the sidebar, click Policy Fragments.
Create a new Policy Fragment and name it, for example, InboundLoggingPolicyFragment.

Property	Value	Comment
LogAgentValueId	3	Once set, you should NOT change this value. Ensure it is not already in use before you start to log.
EndPointDirection	10	Two-way Receive
EndPointTypeId	71	Microsoft Azure API Management
EventDirection	21	ExternalIncomingRequest

The fields are all part of a Nodinite Log Event.

<fragment>
	<!-- Start Logging -->
	<set-header name="x-ms-client-tracking-id" exists-action="skip">
		<value>@(Guid.NewGuid().ToString())</value>
	</set-header>
	<set-variable name="correlationId" value="@{
            return context.Request.Headers.GetValueOrDefault("x-ms-client-tracking-id");
        }" />
	<send-request mode="new" timeout="20" response-variable-name="blobdata" ignore-error="true">
		<set-url>@($"{{logging-blob-url}}{Guid.NewGuid().ToString()}.json")</set-url>
		<set-method>PUT</set-method>
		<set-header name="x-ms-version" exists-action="override">
			<value>2019-07-07</value>
		</set-header>
		<set-header name="x-ms-blob-type" exists-action="override">
			<value>BlockBlob</value>
		</set-header>
		<set-body>@{
            var body = context.Request?.Body?.As<string>(preserveContent: true) ?? "";
            var bodyToLog = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(body));

            var headers = context.Request.Headers.Where(h => !h.Key.Equals("authorization", StringComparison.InvariantCultureIgnoreCase) && !h.Key.Equals("ocp-apim-subscription-key", StringComparison.InvariantCultureIgnoreCase));
            var correlationId = context.Variables.GetValueOrDefault<string>("correlationId");

            Dictionary<string, string> contextProperties = new Dictionary<string, string>();
            contextProperties.Add("CorrelationId", correlationId);
        
            foreach (var h in headers) {
                contextProperties.Add(string.Format("Header#{0}", h.Key), String.Join(", ", h.Value));
            }
            
            var requestLogMessage = new {
                LogAgentValueId = 3,
                EndPointName = context.Api.Name + "/" + context.Operation.Name,
                EndPointUri = context.Deployment.ServiceName + "/" + context.Api.Name + "/" + context.Operation.Name,
                EndPointDirection = 10,
                EndPointTypeId = 71,
                EventDirection = 21,
                OriginalMessageTypeName = context.Variables.GetValueOrDefault<string>("loggingSchemaRoot", "UndeclaredSchemaRoot") + "#" + context.Variables.GetValueOrDefault<string>("loggingSchemaName", "UndeclaredSchemaName"),
                LogDateTime = DateTime.UtcNow,
                ApplicationInterchangeId = correlationId,
                Context = contextProperties,
                LogText = context.Variables.GetValueOrDefault<string>("logtext", ""),
                Body = bodyToLog
            };

            return JsonConvert.SerializeObject(requestLogMessage);
            }</set-body>
		<authentication-managed-identity client-id="{{logging-blob-mid-client-id}}" resource="https://storage.azure.com" />
	</send-request>
	<!-- End Logging -->
</fragment>

Outbound Policy Fragment

Property	Value	Comment
LogAgentValueId	3	Once set, you should NOT change this value. Ensure it is not already in use before you start to log.
EndPointDirection	10	Two-way Receive
EndPointTypeId	71	Microsoft Azure API Management
EventDirection	25	ExternalIncomingResponse

<fragment>
	<!-- Start Logging -->
	<set-header name="x-ms-client-tracking-id" exists-action="skip">
		<value>@(Guid.NewGuid().ToString())</value>
	</set-header>
	<set-variable name="correlationId" value="@{
            return context.Response.Headers.GetValueOrDefault("x-ms-client-tracking-id");
        }" />
	<send-request mode="new" timeout="20" response-variable-name="blobdata" ignore-error="true">
		<set-url>@($"{{logging-blob-url}}{Guid.NewGuid().ToString()}.json")</set-url>
		<set-method>PUT</set-method>
		<set-header name="x-ms-version" exists-action="override">
			<value>2019-07-07</value>
		</set-header>
		<set-header name="x-ms-blob-type" exists-action="override">
			<value>BlockBlob</value>
		</set-header>
		<set-body>@{
            var body = context.Response?.Body?.As<string>(preserveContent: true) ?? "";
            var bodyToLog = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(body));

            var headers = context.Response.Headers.Where(h => !h.Key.Equals("authorization", StringComparison.InvariantCultureIgnoreCase) && !h.Key.Equals("ocp-apim-subscription-key", StringComparison.InvariantCultureIgnoreCase));
            var correlationId = context.Variables.GetValueOrDefault<string>("correlationId");

            Dictionary<string, string> contextProperties = new Dictionary<string, string>();
            contextProperties.Add("CorrelationId", correlationId);
        
            foreach (var h in headers) {
                contextProperties.Add(string.Format("Header#{0}", h.Key), String.Join(", ", h.Value));
            }
            
            var requestLogMessage = new {
                LogAgentValueId = 3,
                EndPointName = context.Api.Name + "/" + context.Operation.Name,
                EndPointUri = context.Deployment.ServiceName + "/" + context.Api.Name + "/" + context.Operation.Name,
                EndPointDirection = 10,
                EndPointTypeId = 71,
                EventDirection = 25,
                OriginalMessageTypeName = context.Variables.GetValueOrDefault<string>("loggingSchemaRoot", "UndeclaredSchemaRoot") + "#" + context.Variables.GetValueOrDefault<string>("loggingSchemaName", "UndeclaredSchemaName"),
                LogDateTime = DateTime.UtcNow,
                ApplicationInterchangeId = correlationId,
                Context = contextProperties,
                LogText = context.Variables.GetValueOrDefault<string>("logtext", ""),
                Body = bodyToLog
            };

            return JsonConvert.SerializeObject(requestLogMessage);
            }</set-body>
		<authentication-managed-identity client-id="{{logging-blob-mid-client-id}}" resource="https://storage.azure.com" />
	</send-request>
	<!-- End Logging -->
</fragment>

On-Error Policy Fragment

Note
The LogStatus is set to -1 to indicate an Error condition.

<fragment>
	<!-- Start Logging -->
	<set-header name="x-ms-client-tracking-id" exists-action="skip">
		<value>@(Guid.NewGuid().ToString())</value>
	</set-header>
	<set-variable name="correlationId" value="@{
            return context.Request.Headers.GetValueOrDefault("x-ms-client-tracking-id");
        }" />
	<send-request mode="new" timeout="20" response-variable-name="blobdata" ignore-error="true">
		<set-url>@($"{{logging-blob-url}}{Guid.NewGuid().ToString()}.json")</set-url>
		<set-method>PUT</set-method>
		<set-header name="x-ms-version" exists-action="override">
			<value>2019-07-07</value>
		</set-header>
		<set-header name="x-ms-blob-type" exists-action="override">
			<value>BlockBlob</value>
		</set-header>
		<set-body>@{
            var body = context.Request?.Body?.As<string>(preserveContent: true) ?? "";
            var bodyToLog = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(body));

            var headers = context.Request.Headers.Where(h => !h.Key.Equals("authorization", StringComparison.InvariantCultureIgnoreCase) && !h.Key.Equals("ocp-apim-subscription-key", StringComparison.InvariantCultureIgnoreCase));
            var correlationId = context.Variables.GetValueOrDefault<string>("correlationId");

            Dictionary<string, string> contextProperties = new Dictionary<string, string>();
            contextProperties.Add("CorrelationId", correlationId);
        
            foreach (var h in headers) {
                contextProperties.Add(string.Format("Header#{0}", h.Key), String.Join(", ", h.Value));
            }
            
            var requestLogMessage = new {
                LogAgentValueId = 3,
                EndPointName = context.Api.Name + "/" + context.Operation.Name,
                EndPointUri = context.Deployment.ServiceName + "/" + context.Api.Name + "/" + context.Operation.Name,
                EndPointDirection = 10,
                EndPointTypeId = 71,
                EventDirection = 25,
                OriginalMessageTypeName = context.Variables.GetValueOrDefault<string>("loggingSchemaRoot", "UndeclaredSchemaRoot") + "#" + context.Variables.GetValueOrDefault<string>("loggingSchemaName", "UndeclaredSchemaName"),
                LogDateTime = DateTime.UtcNow,
                ApplicationInterchangeId = correlationId,
                Context = contextProperties,
                LogText = context.Variables.GetValueOrDefault<string>("logtext", ""),
                Body = bodyToLog,
                LogStatus = -1
            };

            return JsonConvert.SerializeObject(requestLogMessage);
            }</set-body>
		<authentication-managed-identity client-id="{{logging-blob-mid-client-id}}" resource="https://storage.azure.com" />
	</send-request>
	<!-- End Logging -->
</fragment>

Add a reference to Policy Fragments in All APIs

In the Azure Portal, navigate to the API Management Service, then, in the sidebar, click APIs, then All APIs.

Inbound Processing

Use the Inbound Policy Fragment to create Nodinite Log Events for the Request.

Inbound Policy Fragment (Inbound Processing)

Click on the Add policy button.
Add a reference to the Inbound Policy Fragment.

 <inbound>
    <include-fragment fragment-id="InboundLoggingPolicyFragment" />
</inbound>

Outbound Processing

Use the Outbound Policy Fragment to create Nodinite Log Events for the Response.

Outbound Policy Fragment (Outbound Processing)

Click on the Add policy button.
Add a reference to the Outbound Policy Fragment.

<outbound>
    <include-fragment fragment-id="outboundLoggingPolicyFragment" />
</outbound>

On Error

Use the On-Error Policy Fragment to create Nodinite Log Events when there is an error.

<on-error>
    <include-fragment fragment-id="onerrorLoggingPolicyFragment" />
</on-error>

Complete example with all Policy Fragments

<policies>
    <inbound>
        <include-fragment fragment-id="InboundLoggingPolicyFragment" />
    </inbound>
    <backend>
        <forward-request />
    </backend>
    <outbound>
        <include-fragment fragment-id="outboundLoggingPolicyFragment" />
    </outbound>
    <on-error>
        <include-fragment fragment-id="onerrorLoggingPolicyFragment" />
    </on-error>
</policies>

Variables

In the final steps with the Blob Storage Policy, you need to set a few values. These are commonly used to attribute the Nodinite JSON Log Event. You can develop additional dynamics based on the provided examples.

The following variables are part of the default Nodinite Blob Storage Policy:

Variable	Default Value	Example	Comment
`loggingSchemaRoot`	`UndeclaredSchemaRoot`	`Nodinite.Logging.Schemas`	The prefix part of the Message Type name
`loggingSchemaName`	`UndeclaredSchemaName`	`Orders`	The suffix part of the Message Type name. Should be set on individual operation level
`logtext`	``	`Hello World`	Empty by default; Can be set on individual operation level

Info
Using the example values, the name of the Message Type in the resulting Nodinite JSON Log Event is Nodinite.Logging.Schemas#Orders. If possible, please use a versioning scheme in real-world scenarios, i.e. Nodinite.Logging.Schemas/1.0#Orders

In the On-Error, the LogStatus is set to -1 (indicating an Error), this is a candidate you may be interested in setting as a variable. Feel free to modify the default Policy Fragments to fit your use case.

Log Status Codes

Set variable on All operations

In the Azure Portal, navigate to the API Management Service. In the sidebar, click APIs, and then select one of the APIs, then All operations.

Complete example

Below is an example of setting the value for loggingSchemaRoot for all operations (inbound, outbound, and on-error). You can override the value if you want to on the operation level.

<policies>
    <!-- Throttle, authorize, validate, cache, or transform the requests -->
    <inbound>
        <set-variable name="loggingSchemaRoot" value="Nodinite.Logging.Schemas" />
        <base />
    </inbound>
    <!-- Control if and how the requests are forwarded to services  -->
    <backend>
        <base />
    </backend>
    <!-- Customize the responses -->
    <outbound>
        <set-variable name="loggingSchemaRoot" value="Nodinite.Logging.Schemas" />
        <base />
    </outbound>
    <!-- Handle exceptions and customize error responses  -->
    <on-error>
        <set-variable name="loggingSchemaRoot" value="Nodinite.Logging.Schemas" />
        <base />
    </on-error>
</policies>

Set variables on the API level (inbound)

For a specific operation, for example, a POST, in the inbound processing step, you can set the suffix part of the Message Type name as described in the Variables heading. Also, you can set a Log Text (a description field for the Nodinite Log Event)

 <inbound>
    <set-variable name="loggingSchemaName" value="Orders" />
    <set-variable name="logtext" value="Post OK" />
    <base />
    <set-backend-service id="apim-generated-policy" backend-id="nodinitefunctionapps" />
</inbound>

Set variables on the API level (outbound)

In the outbound processing step, we now make changes to the variables. You should set these values according to your business case and match the data in orbit.

 <outbound>
    <set-variable name="loggingSchemaName" value="OrderResponse" />
    <set-variable name="logtext" value="Response OK" />
    <base />
</outbound>

Set variables on the API level (on-error)

In the on-error processing step, we again make changes to the variables. You should set these values according to your business case and match the data in orbit.

 <on-error>
    <set-variable name="loggingSchemaName" value="Error" />
    <set-variable name="logtext" value="Error in processing" />
    <base />
</on-error>

Next Step

Search Fields
Log Views

Interested in logging from other Azure Related Services?

Explore Nodinite for 30 days – completely free

See how Nodinite resolves your technical issues, in a

LIVE Demo!