I'm trying to set up an Azure infrastructure with Pulumi. So far I have used pulumi preview command and it seems that most things are in place.
I've been able to sort out most things but not the parts associated with the Azure AD.
For example, the code below works:
var group = Output.Create(
GetGroup.InvokeAsync(
new GetGroupArgs
{
Name = "Administrators 2"
}));
var tenantId = config.Apply(c => c.TenantId);
var keyVault = new KeyVault(
name,
new KeyVaultArgs
{
ResourceGroupName = resourceGroup.Name,
EnabledForDeployment = true,
EnabledForTemplateDeployment = true,
PurgeProtectionEnabled = true,
SkuName = "standard",
TenantId = tenantId,
AccessPolicies = new KeyVaultAccessPolicyArgs[]
{
// <------------------------ Nothing here and it works.
}
});
But the code below doesn't work.
var group = Output.Create(
GetGroup.InvokeAsync(
new GetGroupArgs
{
Name = "Administrators 2"
}));
var tenantId = config.Apply(c => c.TenantId);
var keyVault = new KeyVault(
name,
new KeyVaultArgs
{
ResourceGroupName = resourceGroup.Name,
EnabledForDeployment = true,
EnabledForTemplateDeployment = true,
PurgeProtectionEnabled = true,
SkuName = "standard",
TenantId = tenantId,
AccessPolicies = new KeyVaultAccessPolicyArgs[]
{
new KeyVaultAccessPolicyArgs // <--- When I add this, it stops working.
{
SecretPermissions = new[] { "get", "list" },
ObjectId = group.Apply(g => g.ObjectId),
TenantId = tenantId
}
}
});
error: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 2
What json?... I suspected that the service principal doesn't have good enough permissions to set these things.
debug: Serialize property[resource:fe-modules-kv-[azure:keyvault/keyVault:KeyVault].accessPolicies.id[0].objectId]: Recursing into Output
error: Running program 'C:\dev\...........\bin\Debug\netcoreapp3.1\Frontend.dll' failed with an unhandled exception:
Grpc.Core.RpcException: Status(StatusCode=Unknown, Detail="invocation of azuread:index/getGroup:getGroup returned an error: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 2")
at Pulumi.GrpcMonitor.InvokeAsync(InvokeRequest request)
at Pulumi.Deployment.InvokeAsync[T](String token, InvokeArgs args, InvokeOptions options, Boolean convertResult)
at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
at Pulumi.Output`1.Pulumi.IOutput.GetDataAsync()
at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
at Pulumi.Serialization.Serializer.SerializeDictionaryAsync(String ctx, IDictionary dictionary)
at Pulumi.Serialization.Serializer.SerializeInputArgsAsync(String ctx, InputArgs args)
at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
at Pulumi.Serialization.Serializer.SerializeListAsync(String ctx, IList list)
at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
at Pulumi.Deployment.SerializeFilteredPropertiesAsync(String label, IDictionary`2 args, Predicate`1 acceptKey)
at Pulumi.Deployment.PrepareResourceAsync(String label, Resource res, Boolean custom, ResourceArgs args, ResourceOptions options)
at Pulumi.Deployment.RegisterResourceAsync(Resource resource, ResourceArgs args, ResourceOptions options)
at Pulumi.Deployment.ReadOrRegisterResourceAsync(Resource resource, ResourceArgs args, ResourceOptions options)
at Pulumi.Deployment.CompleteResourceAsync(Resource resource, ResourceArgs args, ResourceOptions options, ImmutableDictionary`2 completionSources)
at Pulumi.Output`1.GetValueAsync()
at Pulumi.Deployment.Logger.TryGetResourceUrnAsync(Resource resource)
at Pulumi.Deployment.Runner.WhileRunningAsync()
The part of debug: Serialize property[resource:fe-modules-kv-[azure:keyvault/keyVault:KeyVault].accessPolicies.id[0].objectId]: Recursing into Output tells us the problem is not with the tenant but I haven't yet been able to grasp enough.
I searched a lot about this and made sure that my service principal has the necessary roles to be able to manage user permissions by assigning it the roles User Account Administrator and Company Administrator. The below script does that and I confirmed that in the Azure portal.
Connect-AzureAD -TenantId "0000000000000000000"
$userAccountAdministratorRoleName = "User Account Administrator"
$companyAdministratorRoleName = "Company Administrator"
$userAccountAdministratorRole = Get-AzureADDirectoryRole | Where-Object { $_.displayName -eq $userAccountAdministratorRoleName }
if ($userAccountAdministratorRole -eq $null) {
# Instantiate an instance of the role template
$roleTemplate = Get-AzureADDirectoryRoleTemplate | Where-Object { $_.displayName -eq $userAccountAdministratorRoleName }
Enable-AzureADDirectoryRole -RoleTemplateId $roleTemplate.ObjectId
# Fetch User Account Administrator role instance again
$userAccountAdministratorRole = Get-AzureADDirectoryRole | Where-Object { $_.displayName -eq $userAccountAdministratorRoleName }
}
$companyAdministratorRole = Get-AzureADDirectoryRole | Where-Object { $_.displayName -eq $companyAdministratorRoleName }
if ($companyAdministratorRole -eq $null) {
# Instantiate an instance of the role template
$roleTemplate = Get-AzureADDirectoryRoleTemplate | Where-Object { $_.displayName -eq $companyAdministratorRoleName }
Enable-AzureADDirectoryRole -RoleTemplateId $roleTemplate.ObjectId
# Fetch User Account Administrator role instance again
$companyAdministratorRole = Get-AzureADDirectoryRole | Where-Object { $_.displayName -eq $companyAdministratorRoleName }
}
Write-Host "User account administrator role: $userAccountAdministratorRole"
Write-Host "Company administrator role: $companyAdministratorRole"
$sp = Get-AzureADServicePrincipal -All $true | Where-Object { $_.displayName -eq 'Pulumi' }
Add-AzureADDirectoryRoleMember -ObjectId $userAccountAdministratorRole.ObjectId -RefObjectId $sp.ObjectId
Add-AzureADDirectoryRoleMember -ObjectId $companyAdministratorRole.ObjectId -RefObjectId $sp.ObjectId
What else am I missing? Or where can I get more information?
It's the service principal. As per the Pulumi.yml file
config:
azure:clientId: 0000000000
azure:clientSecret:
secure: 000000000000
azure:location: WestEurope
azure:subscriptionId: 0000000000
azure:tenantId: 000000000000000000000000
The below debug code show demonstrates that.
debug: Invoke RPC prepared: token=azuread:index/getServicePrincipal:getServicePrincipal, obj={ "objectId": "000000-0000-0000-0000-0000000000" }
debug: 2020/06/06 17:24:31 Testing if Service Principal / Client Certificate is applicable for Authentication..
debug: 2020/06/06 17:24:31 Testing if Multi Tenant Service Principal / Client Secret is applicable for Authentication..
debug: 2020/06/06 17:24:31 Testing if Service Principal / Client Secret is applicable for Authentication..
debug: 2020/06/06 17:24:31 Using Service Principal / Client Secret for Authentication
debug: 2020/06/06 17:24:31 Getting OAuth config for endpoint https://login.microsoftonline.com/ with tenant 000000-877e-440f-b0ba-0000000
Actually, a few hundred lines after, I find the following.
terraform-provider-azurerm/dev pid-222c6c49-1b0a-5959-a213-6608f9eb8820
debug: AzureRM Request:
debug: POST /335f20ab-877e-440f-b0ba-d7eabfa258e4/oauth2/token?api-version=1.0 HTTP/1.1
debug: Host: login.microsoftonline.com
debug: User-Agent: Go/go1.14.1 (amd64-windows) go-autorest/adal/v1.0.0
debug: Content-Length: 174
debug: Content-Type: application/x-www-form-urlencoded
debug: Accept-Encoding: gzip
debug:
debug: client_id=346ead82-d584-4427-b2fa-e54b94ed10cf&client_secret=[secret]&grant_type=client_credentials&resource=https%3A%2F%2Fmanagement.azure.com%2F
debug: 2020/06/08 08:39:56 Testing if Service Principal / Client Certificate is applicable for Authentication..
debug: 2020/06/08 08:39:56 Testing if Multi Tenant Service Principal / Client Secret is applicable for Authentication..
# *** Testing if Service Principal / Client secret
debug: 2020/06/08 08:39:56 Testing if Service Principal / Client Secret is applicable for Authentication..
# *** And then it tests Managed Service Identity is applicable? Why?
debug: 2020/06/08 08:39:56 Testing if Managed Service Identity is applicable for Authentication..
debug: 2020/06/08 08:39:56 Testing if Obtaining a token from the Azure CLI is applicable for Authentication..
debug: 2020/06/08 08:39:56 Using Obtaining a token from the Azure CLI for Authentication
debug: AzureRM Response for https://login.microsoftonline.com/335f20ab-877e-440f-b0ba-d7eabfa258e4/oauth2/token?api-version=1.0:
It would be nice for us to have better quality feedback instead of:
Error getting authenticated object ID: Error parsing json result from the Azure CLI
However, I found the problem. The Pulumi.AzureAD library expects the service principal credentials to be environment variables.
I had them set up in the config using pulumi config set commands but that was not enough.
This is what you need to do. Pulumi does not or cannot set this automatically and Terraform expects those environment variables.
$env:ARM_CLIENT_ID="000000000000000000"
$env:ARM_CLIENT_SECRET="00000000000000000"
$env:ARM_TENANT_ID="00000000000000000"
$env:ARM_SUBSCRIPTION_ID="000000000000000000000"