How to understand Microsoft Entra application required for log ingestion API

Question

My use case is to ingest data from Amazon S3 into Microsoft Sentinel using log ingestion api. I have a Python script that reads data from s3, post-processes events and send to sentinel via log ingestion api. The part where it should send to sentinel does not work because of the issue described below.

I am reading https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-ingestion-api-overview and I managed to set up a DCR, a DCE, tables in log analytics workspace. However, I do not understand why an entra application should be set up.

From the documentation, "Once your DCR is created, you must grant access to it for the application that you created in the first step." Here, application refers to entra application. I came across a sample code which explains uploading data to log analytics workspace with log ingestion api.

import os

from azure.core.exceptions import HttpResponseError
from azure.identity import DefaultAzureCredential
from azure.monitor.ingestion import LogsIngestionClient

endpoint = os.environ['DATA_COLLECTION_ENDPOINT']
credential = DefaultAzureCredential()

client = LogsIngestionClient(endpoint=endpoint, credential=credential, logging_enable=True)
stream_name = os.environ['LOGS_DCR_STREAM_NAME']

rule_id = os.environ['LOGS_DCR_RULE_ID']
body = [
      {
        "Time": "2021-12-08T23:51:14.1104269Z",
        "Computer": "Computer1",
        "AdditionalContext": "context-2"
      },
      {
        "Time": "2021-12-08T23:51:14.1104269Z",
        "Computer": "Computer2",
        "AdditionalContext": "context"
      }
    ]

try:
    client.upload(rule_id=rule_id, stream_name=stream_name, logs=body)
except HttpResponseError as e:
    print(f"Upload failed: {e}")

After doing an az login and running the above script, I see the message:

Message: The authentication token provided does not have access to ingest data for the data collection rule with immutable Id 'dcr-***************************'

Why is Microsoft Entra application needed?

I tried creating an application on entra admin control but on azure portal I do not have permissions to assign new role to the app created. Hence, I want to know is creating a new application required or is there a workaround?

Answer 1

I have referred to the MS Docs to configure the prerequisites which is needed to send the data to Azure monitor logs using log ingestion API.

• After creating all the components, you need to provide RBAC role to the Data Collection Rule instance as explained in this documentation.

Navigate to Data Collection Rule instance -> click on Access Control (IAM) -> select Add role assignment -> select Monitoring Metrics Publisher role.

• Assign this role to the registered app and also to our own account if in local, it is validating using your object id.

• After role assignment, wait for 15-20 minutes and then execute your code.

dce_endpoint = "https://********.eastus-1.ingest.monitor.azure.com" 
dcr_immutableid = "dcr-a163*******bb3" 
stream_name = "Custom-DCR_Data_CL" 

# Import required modules
import os
from azure.identity import DefaultAzureCredential
from azure.monitor.ingestion import LogsIngestionClient
from azure.core.exceptions import HttpResponseError
from datetime import datetime

credential = DefaultAzureCredential()
client = LogsIngestionClient(endpoint=dce_endpoint, credential=credential, logging_enable=True)

time_generated = datetime.now().strftime('%Y-%m-%d %H:%M:%S') 
body = [
        {
        "TimeGenerated": time_generated,
        "Name": "Ikhtesam",
        "Computer": "Computer1",
        "AdditionalContext": "context-1"
      },
      {
        "TimeGenerated": time_generated,
        "Name": "Afreen",
        "Computer": "Computer2",
        "AdditionalContext": "context-2"
      }
    ]

try:
    client.upload(rule_id=dcr_immutableid, stream_name=stream_name, logs=body)
except HttpResponseError as e:
    print(f"Upload failed: {e}")

I am able to get the logs in the custom table.