Prerequisites for the Nodinite Azure Logic Apps Logging and Monitoring Agent

Info

This page describes the prerequisites to successfully install and run the Nodinite Azure Logic Apps Log and Monitoring Agent and you will enjoy a plug-and-play experience for Logic Apps Monitoring and Logic Apps Logging.

Before you dig in and enable the built-in Azure Logic Apps tracking feature, please read the About Logic Apps Logging options user guide first.

Monitoring

The Nodinite Azure Logic Apps Log and Monitoring Agent features Azure Monitoring without the need for many costly (consultancy hours) time-consuming hands-on tweaks featured in the Azure portal.

The Nodinite Azure Logic Apps Log and Monitoring Agent often replace Azure Monitoring!

Logging and/or failed runs Monitoring

To enable Logging and the Monitoring of failed runs from Workflows, the Agent makes use of the following Azure services:

• Enable Logging in Workflows
• An Azure Event Hub (target for diagnostics Logging).
  • You can use one or more Event Hub entities as the target.
• A storage container to keep track of the current checkpoint.
  • Each Event Hub Entity must have its unique container to store the bookmark (checkpoint).

Important

You must have one unique storage container per Event Hub entity! Otherwise, you will overwrite the checkpoint, thus preventing the operational Logging feature.

• The Agent makes use of a mandatory SQL Server Agent Database. This database boosts performance and provides long-term statistics. Every instance of the Agent requires its unique database.

You can install instances of this Agent on-premise using TCP/IP for local network access and/or in the cloud/off-site using Service Bus Relaying (see also the external link for additional information MicrosoftServiceBusRelayingLink) as long as the Agent can access the Nodinite Log API.

We recommend that you keep this Agent close to the Nodinite Core Services. This documentation covers a local network setup (usually on the Nodinite application server).

Software Requirements

The Nodinite Azure Logic Apps Log and Monitoring Agent is a Windows Service you usually install on the Nodinite application server.

Important

This agent is updated timely to comply with the Microsoft support policy. You must regularly patch your .NET and .NET Core environments, please review the NET and .NET Core Support Policy

Supported Versions

Cloud technologies are evolving fast, and Microsoft deprecates older versions of their APIs now and then. Nodinite will always support the APIs supported by Microsoft. This means you need to update Nodinite and our Azure Logic Apps Log and Monitoring Agent from time to time.

Make sure to subscribe to our Release Notes.

What Azure User rights do the Azure Logic Apps Logging and Monitoring Agent require?

• The Agent uses the Azure REST API to read logged events with tracked properties and can modify the state of your logic apps. Therefore the Agent needs access rights. Carefully read and follow the instructions detailed in the Azure Applications Access user guide.

• Azure REST API access (contributor)

  • Apply to the Resource groups to be included in Logging and Monitoring where you have your Logic Apps

• Event Hub

  • Event Hub Connection string

• Storage

  • Storage connection string

Enable Logic Apps Logging in Azure

To enable Logging to Nodinite from your Logic Apps, you must enable and configure some features in the Azure portal. You can always script, and use ARM Templates to create the necessary services and apply the proper settings. The Nodinite Azure Logic Apps Log and Monitoring Agent requires information about these configurations.

The following Azure services and settings are required to enable Logging to Nodinite from Azure Logic Apps:

• Configure each Logic Apps Diagnostic setting with the 'Workflowruntime' Logging option (Standard and Consumption)

  Please review the ARM Template user guide for an example to use with your build and deployment process.

• Storage account - Stores the checkpoint (offset position in the Event Hub stream). You must use one unique configuration per event hub entity.

  The Event Hub "only" supports client-side checkpointing.

• Event Hub entity - Match according to the diagnostics settings in Azure. The Agent can tap many event hubs simultaneously. This means that you can consider the geo-redundancy, the number of partitions to use, and the retention.

Event Hub Entity

The Azure Event Hub entity is used as the target for Logging when the diagnostics logging for a Logic App is enabled.

Note

The different diagnostic settings can target a single or multiple Event Hubs. You must match these different settings with the configuration within the Nodinite Azure Logic Apps Log and Monitoring Agent.

• Use one or more dedicated event hubs entities for the Logic Apps Logging. This is set in the Azure diagnostic settings.

Important

If you use diagnostic Logging also for other Azure resources like API Management Services, then you must provide a different set of Event Hub Entities (names) for the different type of Azure resources

• You can use multiple Event Hubs for different diagnostic settings
• Access Control (Shared Access Policy)
• Use at least 2 partitions, see note below:

  Important

  For high volume solutions (>2.000.000 events per day), you might need to [re]-create the Event Hub with 4 (or even more partitions).

• Try to keep the number of Event Hub entities as small as possible

  You can re-use the same Event Hub entity from many diagnostic settings set on the Azure Logic Apps.

• You must use an Event Hub in the same region as your Logic Apps.

Here's an example of an Azure Event Hub entity in use to enable the Azure Logic Apps Logging feature.

Details about the Azure Event Hub entity.

Data is updated regularly on the Event Hub.

Additional comments about the Event Hub

The target for Logging from the Azure Logic App is an Event Hub entity. These named Event Hubs entities MUST be unique for Logging from Azure Logic Apps.

1. Logging from Azure functions can share the same Eventy Hub if the content logged is a Nodinite JSON Log Event.
2. Since the content is in a different format; the Logging from Azure Logic Apps requires one or more OTHER event hubs as the target.

Using a shared Event Hub for a Nodinite JSON Log Event has the advantage of less administration (i.e. you must configure the Nodinite Pickup Log Events Service Logging Agent to fetch these events)
The disadvantage with a shared event hub entity can be performance-related, or other factors to consider like pricing (Logging from multiple regions), scalability, retention settings, number of partitions, ...

Tip

Make sure to use separate event hub entities for different purposes and needs, i.e. multiple Logic App Logging, multiple API Management Services, multiple APIs, and multiple Azure functions, all according to your architecture.

Storage Account

Nodinite uses the storage to persist a bookmark to the position in the event stream. This feature provides reliable Logging. If you stop or restart the Agent, it will resume the event hub stream operation from the last known checkpoint with the offset stored in the storage container.

You need to create (or reuse) an Azure Storage Account.

A Storage Account is required to enable the Nodinite Azure Logic Apps Logging feature.

• One container, per unique combination of:
  • per Log Agent configuration
    • Subscription
    • Event Hub Entity
• Access Control (Contributor)

Important

If you have multiple instances and/or multiple Event Hubs entities; Make sure to map the storage container 1:1 with the Event Hub entity. You must not reuse the same storage container name in multiple configurations. If you do so, the checkpoint is read and overwritten yielding lost data and/or duplicates.

• Region/Location - Place in the same region as your Logic Apps and Resource groups
• Kind - General-purpose v2 account

Here's an example of a storage container to keep track of the Event Hub checkpoint.

Next, you must get the Connection String the Agent uses to access the Azure Storage Account:

Example with details about the Azure Storage container connection.

Over time, this object should be tiny, and it gets modified regularly.

Logic Apps

Info

Logically, the same type of settings apply for both Standard and Consumption based Logic Apps. The settings apply on either the Workflow (Consumptiom), or the Logic Apps (Standard). The latter thenb applies to all Workflows in that Logic App.

Important

Ensure that only the Workflow runtime checkbox is checked. Otherwise, the Agent is not operational.

• Diagnostics Setting (Consumption) - Repeat for each Logic App where Logging should be enabled.
  
  Here's an example of a Consumption-based Azure Logic App with the essential diagnostic settings to enable the Logging feature for use with Nodinite.

• Diagnostics Setting (Standard)

  
  Here's an example of a Standard-based Azure Logic App with the essential diagnostic settings to enable the Logging feature for use with Nodinite.

Tip

Use Powershell scripts to automate this configuration or make it a default setting part of your deployment template.

Next, you must Add the SAS Policy with the connection string Nodinite uses to connect with the Event Hub:

Here's an example of a SAS Policy with Event Hub connection details.

The connection string is available in the Azure portal, copy and paste it to the Nodinite Configuration.

What Windows User Rights does the Azure Logic Apps Logging and Monitoring Agent require?

The Agent is installed as a Windows Service usually on the Nodinite application server. Virtual machines are supported.

• Local named account or domain account (preferred).
• Access and run-time rights.
  • Follow the 'How to set logon as a Windows service right' user guide for detailed instructions.

What Firewall settings are required for the Azure Logic Apps Logging and Monitoring Agent?

The Azure Logic Apps Log and Monitoring Agent has both inbound and outbound communication:

1. Between the Monitoring Service and the Azure Logic Apps Log and Monitoring Agent.
2. Between the Azure Logic Apps Log and Monitoring Agent and Azure Management API.
3. Between the Azure Logic Apps Log and Monitoring Agent and Azure Event Hub.
4. Between the Azure Logic Apps Log and Monitoring Agent and Azure Storage.
5. Between the Azure Logic Apps Log and Monitoring Agent and the Monitoring Database.
6. Between the Azure Logic Apps Log and Monitoring Agent and the Log API.

1. Between the Monitoring Service and the Azure Logic Apps Logging and Monitoring Agent

The following ports must be allowed on the Windows server where the Agent is installed and running:

And further with 'Option 1' or 'Option 2' as documented next:

Option 1 (Local network)

Option 2 (Cloud/Hybrid)

Use Service Bus Relayed connections when Nodinite and Agent are on totally different networks.

Nodinite uses the same principle technique as the On-Premise data gateway, see 'Adjust communication settings for the on-premises data gateway' user guide.

The following ports must be open for outbound communication with '*.servicebus.windows.net' from both on-premise and off-site (Nodinite Windows Server(s) and where the Agent is installed):

2. Between the Azure Logic Apps Logging and Monitoring Agent and Azure Management API

3. Between the Azure Logics Apps Log and Monitoring Agent and Azure Event Hub

The following ports must be open for outbound communication with '*.servicebus.windows.net' from both on-premise and off-site Windows Servers where Agent is installed:

Troubleshooting guide: https://learn.microsoft.com/en-us/azure/event-hubs/troubleshooting-guide

4. Between the Azure Logics Apps Log and Monitoring Agent and Azure Storage

You must allow communication between the Nodinite Azure Logic Apps Log and Monitoring Agent and your Storage Account. The system Administrator may have one or more rules in place in any of the following locations:

• Virtual Machine
• Storage Account
• Virtual Network

If you have enabled rules, you may need to tune these to allow communication.

Troubleshooting guide: Configure Azure Storage firewalls and virtual networks

5. Between the Azure Logics Apps Log and Monitoring Agent and the Monitoring Database

This topic is well covered in the user guide for Monitoring Databases.

6. Between the Azure Logic Apps Logging and Monitoring Agent and the Log API

When Logging is enabled the Nodinite Azure Logic Apps Log and Monitoring Agent requires one of the following outbound TCP ports to be open for access to the Nodinite Log Api (configurable)

Tip

If the Azure Logic Apps Log and Monitoring Agent and the Log API are on the same server, you should stick with HTTP for the best performance. The information is managed locally on the server (e.g. Not on the network.).

Frequently asked questions

Additional solutions to common problems and the FAQ for the Nodinite Azure Logic Apps Logging and Monitoring Agent exist in the Troubleshooting user guide.

Next Step

Install the Azure Logic Apps Logging and Monitoring Agent

Related

Add or manage a Monitoring Agent Configuration
Monitoring Agents
Administration
Monitoring Agents