Azure Event Hub Configuration Example

Unlock seamless integration by configuring the Nodinite Pickup Logging Service to fetch JSON Log Events directly from your Azure Event Hub. This guide empowers you to set up secure, scalable, and reliable log event collection for your enterprise systems, with robust checkpointing and storage for compliance and analytics.

✅ Effortless integration with Azure Event Hub
✅ Secure and reliable log event collection
✅ Scalable setup for enterprise environments
✅ Robust checkpointing and long-term storage

The diagram above illustrates how the Nodinite Pickup Logging Service interacts with Azure Event Hub entities, processes messages, and stores them in the Nodinite Log Database. The Storage Container is used for checkpointing to ensure reliable message processing.

Event Hubs

The Event Hubs section lets you manage all Azure Event Hub sources for your log events.

In the Remote Configuration GUI, navigate to the Event Hubs tab. Here, you can add, edit, or remove Event Hub configurations. Each configuration defines how the Nodinite Pickup Log Events Logging Service connects to and processes messages from a specific Azure Event Hub entity.

Example of the Event Hub configuration tab in the Remote Configuration GUI.

Click the Add button to create a new Event Hub configuration. Fill in the required fields, such as Event Hub entity name, storage container for checkpointing, and authentication credentials. You can also configure Azure AD authentication for enhanced security.

Example of an Event Hub configuration accordion in the Remote Configuration GUI.

Expand the accordion for each Event Hub configuration to access advanced settings, including Replace Rules for message processing. These rules allow you to modify message content before it's stored in Nodinite, ensuring consistency and compliance with your logging standards.

Important

Assign a unique storage container to each Event Hub entity. Sharing containers will overwrite checkpoints and disrupt logging operations.

General tab

The General tab provides the fundamental settings for your Event Hub configuration within the Remote Configuration GUI.

The General tab contains basic settings for the Event Hub configuration, including enabling/disabling the configuration and providing descriptive information.

Key Fields:

Guidelines:

• Use descriptive names that clearly indicate the Event Hub entity and purpose (e.g., "Production Event Hub - Order Events")
• In the Description field, note important details such as the entity's business purpose, owner contact, or any special processing notes
• The Display Name appears throughout the Nodinite interface, so keep it concise yet meaningful
• Leave the configuration disabled during initial setup if you're not ready to start consuming messages

Source tab

The Source tab contains the Event Hub entity configuration and storage checkpoint settings.

The Source tab contains settings for connecting to the Azure Event Hub entity and configuring checkpoint storage.

Key Fields:

Guidelines:

• Ensure the specified Event Hub entity exists in your Event Hub namespace before enabling this configuration
• Use a descriptive Event Hub entity name that reflects the type of events it receives (e.g., order-events, integration-logs)
• Always use a unique, dedicated storage container for each Event Hub entity to prevent checkpoint conflicts
• The storage container must exist in the same Azure Storage Account specified in the Authentication tab
• For optimal performance, use standard storage account tiers for checkpoint storage
• Test the configuration with a single Event Hub entity before scaling to multiple sources

Destination tab

The Destination tab configures where processed log events are stored. By default, events are sent to the Nodinite Log API where they are written to the Log Database for long-term storage and analysis.

The Destination tab contains settings for connecting to the Log API and authentication options.

Key Fields:

Guidelines:

• Always ensure Use Log API is checked to maintain proper log event storage
• For local/development environments, use unprotected Log API URLs (e.g., http://localhost:40002/LogAPI/)
• For production environments with authentication, check the Protected checkbox and provide valid OAuth 2.0 credentials
• When using Protected authentication, ensure your Client ID, Client Secret, and Scope are correctly configured with your identity provider (IDP)
• The IDP Token Endpoint must be accessible from the Pickup Service instance
• Keep Client Secret values secure and never commit them to version control

Example Configurations:

• Unprotected: http://localhost:40002/LogAPI/ (Protected checkbox unchecked)
• Protected: https://mynodinite.acme.com:40002/LogAPI/ Requires Client ID, Client Secret, Scope, and IDP Token Endpoint configured with your OAuth provider

Example of the Destination tab with Protected authentication enabled.

Authentication tab

The Authentication tab contains the credentials required to connect to and authenticate with your Azure Event Hub and Storage Account. This tab has two sub-sections: Event Hub Authentication and Storage Authentication.
![Authentication Tab][7]
The Authentication tab contains settings for connecting to Azure Event Hub and Storage Account, including authentication options.

Event Hub Authentication

Configure the credentials for connecting to your Azure Event Hub entity.

Connection String Method (unchecked "Use App Registration"):

Azure AD App Registration Method (checked "Use App Registration"):

Guidelines for Connection String Authentication:

• Use connection string authentication for straightforward setups without Azure AD requirements
• Keep connection strings secure and never commit them to version control
• Ensure the connection string includes the EntityPath parameter with your Event Hub entity name
• Enable Enable AMQP Redirect if you encounter firewall or proxy issues with standard AMQP connections

Guidelines for Azure AD App Registration Authentication:

• Use Azure AD App Registration for enhanced security and to avoid storing shared keys directly
• Ensure your Azure AD tenant has the App Registration created with appropriate permissions
• Grant the App Registration the "Azure Event Hubs Data Receiver" role on the Event Hub namespace
• Keep Client Secret values secure and regenerate them periodically
• The Tenant ID is required for Azure AD authentication; ensure it matches your organization's Azure AD tenant

Storage Authentication

Configure the credentials for connecting to your Azure Storage Account (used for checkpointing).

Storage Account Key Method (unchecked "Use App Registration"):

Azure AD App Registration Method (checked "Use App Registration"):

Guidelines for Storage Account Key Authentication:

• Use storage account key authentication for straightforward setups without Azure AD requirements
• Keep storage account keys secure and never commit them to version control
• Rotate storage account keys periodically for enhanced security
• The Storage Account Name must match the account where your checkpoint container is located

Guidelines for Azure AD App Registration Authentication:

• Use Azure AD App Registration for enhanced security and to avoid storing storage keys directly
• Ensure your Azure AD tenant has the App Registration created with appropriate permissions
• Grant the App Registration the "Storage Blob Data Contributor" role on the Storage Account
• Keep Client Secret values secure and regenerate them periodically
• Tenant ID and Client credentials are required for Azure AD authentication

Security Best Practices:

• Use separate credentials for development, staging, and production environments
• Implement Azure RBAC (Role-Based Access Control) to limit access to Event Hub and Storage resources
• Regularly review access permissions and remove unused accounts or apps
• Monitor audit logs in Azure to detect unusual access patterns
• For production environments, prefer Azure AD App Registration authentication over connection strings and storage keys when possible
• When using connection strings or storage keys, consider rotating them according to your security policies
• Keep all credentials secure and use Azure Key Vault to manage sensitive values centrally

Replace Rules

Read about Replace Rules in the shared configuration section. Replace Rules allow you to modify message content before it's stored in Nodinite, ensuring consistency and compliance with your logging standards.
The Replace Rules tab allows you to configure rules for modifying message content.

Configuration file

C:\Program Files\Nodinite\Logging Agent - Pickup Service\Settings.json

{
  ...
  "EventHubs": [
    {
      "Enabled": true,
      "LogAPI": "http://localhost/Nodinite/Dev/LogAPI/",
      "UseConfigurationDatabase": false,
      "ConfigurationDatabaseConnectionString": null,
      "EventHubConnectionString": "Endpoint=sb://replaceme.servicebus.windows.net/;SharedAccessKeyName=somename;SharedAccessKey=replaceme-1337=;EntityPath=replaceme",
      "EventHubName": null,
      "StorageContainerName": "event-hub-replaceme",
      "StorageAccountName": "replaceme",
      "StorageAccountKey": "replaceme-42=",
       "ReplaceRules": [
        {
          "Name": "Fix Endpoint real customer id to {customerId}",
          "ReplaceName": false,
          "ReplaceUri": true,
          "Pattern": "/([0-9]{4,})$",
          "Group1Value": "{customerId}"
        }]
    }]
  ...                      
}

EventHubs is an array of EventHub configurations. Each entry defines how the Log Agent connects to and processes messages from a specific Event Hub entity. This setup ensures your Nodinite JSON Log Events are reliably collected and managed.

Important

Restart the Nodinite Pickup Logging Service after making configuration changes to apply your updates.

Storage Account Name*

You can find the StorageAccountName in the Azure Portal as shown below:

Example of how to find the Storage Account Name in the Azure Portal.

Storage Account Keys

You can find the StorageAccountKey in the Azure Portal as shown below:

Example of how to find the Storage Account Keys in the Azure Portal.

Important

Use the value from a Key, not the connection string.

Next Step

Configure

Related Topics

• JSON Log Event
• Replace Rules
• API Management Logging with Event Hub