When I call "Request Sync" on the Google HomeGraph API I receive a "403 Forbidden" response.
I'm writing a Smart Home Action, and have successfully implemented SYNC, QUERY and EXECUTE. Testing on my mobile I can see and interact with devices okay. I'm now trying to implement Request Sync, but can't appear to interact with the API. I am making what seems to be successful requests for an Access Token. The token always begins with "ya29.c." which in my naïve understanding suggests an empty header and payload (trying it on https://jwt.io). However, when testing it at https://accounts.google.com/o/oauth2/tokeninfo?access_token= it appears valid, showing both my service account unique ID and the scope I intended. When I make a call to the API, either manually posting the data, or via Google's own code, it gives me a blunt 403 error. I do not know where I can get any more information on this error other than the exception objects. I'm new to GCP and couldn't find any sort of log. Given I've tried different methods and all return a 403 I'm inclined to suspect the issue is more a account or credential-related than the code, but can't be certain.
(I'm no longer able to reproduce any errors relating to API keys being missing or invalid).
Although the documentation doesn't show it, I've seen some people use an API key. When I don't include the API key with a p12 certificate, or include an incorrect one it errors (either with API key missing, or API key invalid accordingly). I have created an unrestricted API key in IAM, and am using that. I can't appear to explicitly relate this to HomeGraph API, but it says that it can call any API.
This example fetches an access token, then tries to call the API via POST with and without the API key. It then tries to authenticate and call the API via the Google library code. Each fails with a 403.
using Google;
using Google.Apis.Auth.OAuth2;
using Google.Apis.HomeGraphService.v1;
using Google.Apis.HomeGraphService.v1.Data;
using Google.Apis.Services;
using Lambda.Core.Constants;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using static Google.Apis.HomeGraphService.v1.DevicesResource;
public class Example
{
public void RequestSync()
{
const string UrlWithoutKey = @"https://homegraph.googleapis.com/v1/devices:requestSync";
const string UrlWithKey = @"https://homegraph.googleapis.com/v1/devices:requestSync?key=" + OAuthConstants.GoogleApiKey;
string accessToken = this.GetAccessToken();
// Manual Attempt 1
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithoutKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// Manual Attempt 2
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// SDK Attempt
try
{
this.CallRequestSyncApiWithSdk();
}
catch (GoogleApiException ex)
{
// Google.Apis.Requests.RequestError
// The caller does not have permission[403]
// Errors[Message[The caller does not have permission] Location[- ] Reason[forbidden] Domain[global]]
// at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response) in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 243
// at Google.Apis.Requests.ClientServiceRequest`1.Execute() in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 167
string msg = ex.Message;
}
}
private string GetAccessToken()
{
string defaultScope = "https://www.googleapis.com/auth/homegraph";
string serviceAccount = OAuthConstants.GoogleServiceAccountEmail; // "??????@??????.iam.gserviceaccount.com"
string certificateFile = OAuthConstants.CertificateFileName; // "??????.p12"
var oAuth2 = new GoogleOAuth2(defaultScope, serviceAccount, certificateFile); // As per https://stackoverflow.com/questions/26478694/how-to-produce-jwt-with-google-oauth2-compatible-algorithm-rsa-sha-256-using-sys
bool status = oAuth2.RequestAccessTokenAsync().Result;
// This access token at a glance appears invalid due to an empty header and payload,
// But verifies ok when tested here: https://accounts.google.com/o/oauth2/tokeninfo?access_token=
return oAuth2.AccessToken;
}
private string CallRequestSyncApiManually(string accessToken, string url)
{
string apiRequestBody = @"{""agentUserId"": """ + OAuthConstants.TestAgentUserId + @"""}";
var client = new HttpClient();
var request = (HttpWebRequest)WebRequest.Create(url);
var data = Encoding.ASCII.GetBytes(apiRequestBody);
request.Method = "POST";
request.Accept = "application/json";
request.ContentType = "application/json";
request.ContentLength = data.Length;
request.Headers.Add("Authorization", $"Bearer {accessToken}");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
return responseString;
}
private void CallRequestSyncApiWithSdk()
{
var certificate = new X509Certificate2(OAuthConstants.CertificateFileName, OAuthConstants.CertSecret, X509KeyStorageFlags.Exportable);
var credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(OAuthConstants.GoogleServiceAccountEmail)
{
Scopes = new[] { "https://www.googleapis.com/auth/homegraph" },
}.FromCertificate(certificate));
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
// Complains if API key is not provided, even though we're using a certificate from a Service Account
ApiKey = OAuthConstants.GoogleApiKey,
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
}
Account screenshots. (I'm not allowed to post images yet, so they're links)
My Service Account has Owner & Service Account Token Creator enabled
I have tried skipping manually obtaining the access token, as per Devunwired's suggestion. Whilst this does eliminate the error I was getting from not providing the API key, I still end up with the 403. My reasoning for doing the access token part manually was part of debugging a 403 I was getting with the API call. That way I could at least see part of the process working. I'm happy to use the library version for the solution as the access token doesn't appear to be the issue.
public void GoogleLibraryJsonCredentialExample()
{
try
{
GoogleCredential credential;
using (var stream = new FileStream(OAuthConstants.JsonCredentialsFileName, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(new[] { OAuthConstants.GoogleScope });
}
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
catch (Exception ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
}
Is it possible that I need to be making the API call from a verified or white-listed domain? At the moment I'm running it from a console app running on my development machine. My understanding of domain verification is that it does not apply to incoming calls, and therefore shouldn't be the problem.
The problem was nothing to do with my permission to talk to the HomeGraph API or that user. Instead it was where HomeGraph wanted to call my Smart Home Action, but the access token had expired. When attempting to refresh the token, an erroneous implementation on my part led to a blunt 403, which Google was then relaying back to me.
For those interested, the issue was that rather than omitting the expiry date for a token that should never expire, I was setting it to DateTime.MaxValue (subsequently sent through some further processing). Unfortunately when this is finally cast to an int, it is a value that exceeds int.Max. The subsequent time on the expiry was set to epoch (i.e. in the past), and therefore the token validation failed due to expiry.
For anyone else still having the same issue, double check your agentUserId matches exactly the value shown in your SYNC output payload. In my case I'd checked this.
Many thanks to anyone who's looked at this.