- 7 minutes to read

About Logic Apps Logging options

Nodinite provides different options to give you end-to-end Logging for your system integrations solutions like Azure Logic Apps. Each option has its pros and cons; Find out about these in this user guide.

Using Nodinite, you have many options to enable Logging from your solutions built with Azure Logic Apps. In this user guide, we present different approaches to help you understand these options, and also to use the best option to use, to solve your business case.

  1. Diagnostics Logging (debug level Logging)
  2. Custom Logging (log only specific events, )

Info

You can use both options in the same Workflow, one does not exclude the other.

All the options have the potential to provide any of the following:

  • Events (e.g. operational data, time, shape name, ...)
  • Payload (e.g. the data body, json, xml, ...)
  • Tracked properties (e.g. Key-value based properties)
graph LR subgraph "Azure Cloud / Subscriptions" roLA(fal:fa-clouds Your Logic Apps) ---roIF{Logging Option} roIF -.- | 1. Diagnostics enabled | B(fa:fa-list Azure Event Hub) roIF -.- | 2. Custom Logging| C(fal:fa-bars Intermediate Storage) end subgraph "Nodinite Instance" B---|Diagnostics data|roMA(fal:fa-monitor-waveform Azure Logic Apps
Logging and Monitoring agent) C --- |Nodinite JSON LogEvent|roPS(fal:fa-truck-pickup Pickup Service) roMA --- |Filter| F[fal:fa-cloud-download Nodinite's Log API] roPS --- F end

Your need for the data of interest, and your business case should drive what option(s) you implement.

1. Diagnostics Logging

Great for the IT-pros!

Using the Azure Logic Apps built-in diagnostics Logging has the following pros and cons (the list is not comprehensive):

Pros

The Azure built-in Logging feature is straightforward to get started with and works very well for Logic Apps that do not contain hefty loops with nested sub-loops. Remember, this diagnostic feature creates 2 events for each shape, 1 for the activating event (in) and the other one for the result.

There is a cost associated with each trigger, and you should design your solutions with this in mind. Logic Apps might not always be the proper solution for your logic. Instead, execute the heavy lifting (such as computing and looping) using Azure functions. If you need Logging for Azure functions, please review the Nodinite Serilog feature.

# Comment
Built-in Logic Apps has support for verbose Logging with a simple configuration
No coding required You do not need to change any of your existing, or future solutions
ARM Template support You can re-use, and enable the diagnostics settings by specifying the required properties part of your build and deploy routines
Correlation support Group events for individual runs using the Nodinite 'System Interchange Id' search field

Cons

The simplicity and the pros with the Diagnostics Logging is not a silver bullet. Please review the notes in the table below:

# Comment
Excessive Logging With the diagnostic setting enabled, you get all the events (in and out) from every single shape. From real-world scenarios, we have seen ~300.000 event for a single run(!). Thanks to the Nodinite Logging, the customer got aware of this situation and could redesign the solution.
Complex naming Nodinite names the Endpoints and Message Types. Make sure to override when possible, using the Context Options
No control Needless to say,
Limited filters In Nodinite you can filter on different levels to limit what actually gets logged into Nodinite. However, the diagnostic setting is an all or nothing decision. This means a lot of data is managed both in Azure, and by Nodinite
Performance If you have extensive Logging, then you may need to use multiple Event Hub entities, with higher partition counts. There is an increase in complexity and potentially the costs
Delay The Azure system process writing to the event hub is not immediate; Data can be delayed 10-15 minutes(!)

Enable diagnostics based Logging

From within the Prerequisites user guide, you will find a detailed section about how to enable the diagnostics Logging for Azure Logic Apps for use with the Nodinite Azure Logic Apps Agent.

Further reading: Set up Azure Monitor logs and collect diagnostics data for Azure Logic Apps

Tips and tricks

You can remedy some of the cons by adding content to your Tracked Properties; please review the Context Options user guide. Specifically, you should review the following properties that may get an easier administration within Nodinite

  • Message Type - Control the name of the message type by adding the ExtendedProperties/1.0#MessageTypeName as the name, and a versioned name as value (i.e. Orders/1.0#Order, EDIFACT.DESADVD96A, ...)

  • Repair and Resubmit operation - By adding, for example, Filename and a full path as value, from within a Log View, it is possible to Repair and/or Resubmit messages. This property controls the target for such operations.

DO NOT CHECK THESE OPTIONS - These will just increase the cost in Azure and waste system resources.
Invalid options
Nodinite does not need the metrics or other categories.

2. Custom Logging

Great for the business!

The other option is to perform custom Logging as part of your design and Logging Strategy.

With this coded approach, only the events that you need gets logged. This option requires a coding effort, and you need to add code and logic to your Logic Apps. With the Nodinite Logging strategy, at least, the changes you perform rarely needs to be changed once you put them into production. With our recommendation, all the data you have is logged (what more than everything is there to log anyway?).

Using a custom coded approach to enable Logging has the following pros and cons (the list is not comprehensive):

Pros

Total control yields high-quality logging with some initial costs

If you need Logging for Azure functions, please check out the Nodinite Serilog feature.

# Comment
Not noisy Only log the events you require, keep your footprint and processing to a minimum
Total control Since you do some of the codings, you also control the logic, the naming and the content
Low maintenance With the Nodinite Logging strategy everything that is known in that log-poing should be logged; hence, there should be few reasons to adjust (such as revisit, re-code, re-test, re-deploy)
Correlation support You still have the power of grouping, just with fewer events
No vendor lock-in With the Nodinite Logging strategy, the actual target for your logs should be unknown to the system creating the logs. This makes it easy for you to use one or more products to get the insights from your Logging. Our competitors tend to provide you with stuff on the marketplace or otherwise, that if used in many places, it is hard and expensive to replace
Faster visibility A custom coded solution may produce a faster

Cons

The total control comes with a price. Please review the notes in the table below:

# Comment
Coding is required Even if you re-use connectors or other shared components; and you still need to add them to the Logic App designer, there is some engineering effort required to do so
Adds complexity Depending on your solution

Enable custom coded Logging

To begin with, there are many possible solutions to enable custom coded Logging for your Azure Logic Apps. The example here gets you started but you may need to refine and polish it for use in a production environment.

Your mission, regardless of the path you take (for example by providing your own set of custom connectors, Azure functions or other services that you opt to use), is to create a Nodinite JSON Log Event.

graph LR subgraph "Custom Solution" roBroker(fal:fa-brackets-curly Custom Code)--> |Nodinite JSON Log Event| roSink(fal:fa-file-export Intermediate storage ) end subgraph "Nodinite instance" roPS(fal:fa-truck-pickup Pickup Service) -->roNI(fal:fa-cloud-download Log API) roSink --> roPS end

This Nodinite JSON Log Event is persisted to an intermediate storage and is eventually consumed by the Nodinite Pickup Service making your logs available within the Nodinite Log Views.

For Azure, the intermediate storage use any of the following (other storages exist, please review the Nodinite Pickup Service for additional details):

  • Service Bus
  • Event Hub

Your first decision is to decide on the intermediate storage. Your motivation to use any of these may come from pricing, scaling, geo-redundancy, experience to mention a few.

From within your Azure Logic App, you need to add some code. This guide does not provide you with all the details on a generic Logging strategy. You need to invent and decide on this first. Please consult our professional Partners who has great experience working with Azure and Nodinite.

In this example, we will add a Compose shape and create a small JSON formatted message to get you started. In addition to the minimum set of required properties, the Body is attached with Nodinite Logging example. For all the available properties and the details for each, please review the Nodinite JSON Log Event user guide.

Replace the properties as required by your specific use case. Additional fields and context is attached, either in the message or as Tracked Properties.

{
  "Body": @{base64(body('INT001B.OrderIncomingSB'))},
  "EndPointDirection": 0,
  "EndPointName": "INT101: Receive Hello World Log Events",
  "EndPointTypeId": 60,
  "EndPointUri": "C:\\temp\\in",
  "LogAgentValueId": 5,
  "LogDateTime": @{utcNow()},
  "OriginalMessageTypeName": "OVERRIDDENBYTRACKEDPROPERTY"
}