Search code examples
azureazure-aksazure-log-analytics

AKS configured Container Insights does capture excluded namespaces

I have an AKS cluster running on which I enabled Container Insights. The Log Analytics workspace has a decent amount of logs in there. Now I do have my applications running on a separate namespace, and one namespace which has some Grafana containers running (which I also don't want in my captured logs).

So, I searched on how I could reduce the amount of captured logs and came across this Microsoft docs article.

I deployed the template ConfigMap to my cluster and for [log_collection_settings.stdout] and [log_collection_settings.stderr] I excluded the namespaces which I don't want to capture.

When calling kubectl edit configmap container-azm-ms-agentconfig -n kube-system I get the following:

kubectl edit configmap screenshot

Which means that my config is actually in there.

Now when I open a query window in Log Analytics workspace and execute the following query:

KubePodInventory
| where Namespace == "kube-system"

I get plenty of results with a TimeGenerated column that contains values that are like 5 minutes ago, while I setup the ConfigMap a week ago.

In the logs of one of the pods omsagent-... I see logs like the following:

Both stdout & stderr log collection are turned off for namespaces: '*.csv2,*_kube-system_*.log,*_grafana-namespace_*.log'
****************End Config Processing********************
****************Start Config Processing********************
config::configmap container-azm-ms-agentconfig for agent settings mounted, parsing values
config::Successfully parsed mounted config map

While looking here at StackOverflow, I found the following answers which make me believe that this is the right thing that I did:

So, not sure what I am doing wrong here. Anyone an idea?

Solution

  • Since I hate it myself that some people don't post an answer even if they already have one, here it is (although not the answer you want, at least for now).

    I posted the issue on GitHub where the repository is maintained for Container Insights. The issue can be seen here on GitHub.

    If you don't want to click the link, here is the answer from Microsoft:

    We are working on adding support for namespace filtering for inventory and perf metrics tables and will update you as soon this feature available.

    So, currently we are not able to exclude more data than the ContainerLog table with this ConfigMap.