Search code examples
phpgoogle-cloud-platformgoogle-cloud-dataflowgoogle-php-sdk

PHP Google_Service_Dataflow not finding file in bucket


I've come across an issue using the googleapis/google-api-php-client library, specifically the Dataflow Service that I cannot solve.

When I try to use the library I set up the request like so:

$this->client = new \Google_Client();
$this->client->setAuthConfig(config_path('google-service-account.json'));
$this->client->setIncludeGrantedScopes(true);
$this->client->addScope(\Google_Service_Dataflow::CLOUD_PLATFORM);

$body = [
    "gcsPath" => "gs://{$this->bucket}/{$this->template}",
    "location" => "us-central1",
];

$parameters = new \Google_Service_Dataflow_LaunchTemplateParameters;
$parameters->setJobName($this->jobname);
$parameters->setParameters($body);

$service = new \Google_Service_Dataflow($this->client);
$request = $service->projects_templates->launch($this->project, $parameters);

And I get the following error:

{
  "error": {
    "code": 400,
    "message": "(11f8b78933fc59c3): Bad file name: , expected 
    'gs://\u003cbucket\u003e/\u003cpath\u003e'",
    "errors": [
      {
        "message": "(11f8b78933fc59c3): Bad file name: , expected 
        'gs://\u003cbucket\u003e/\u003cpath\u003e'",
        "domain": "global",
        "reason": "badRequest"
      }
    ],
    "status": "INVALID_ARGUMENT"
  }
}

It seems that the path is getting corrupted along the way, I've checked and it gets fine until the Guzzle object is instantiated to send the request inside the library.

I'm pretty lost at this point so any suggestion or clue is welcome.

Thank you in advance.


Solution

  • No gcsPath is given in the query params for the request constructed by the SDK.

    This is because gcsPath is set to be an option for Google_Service_Dataflow_LaunchTemplateParameters.

    It is documented that request query parameters be given as optional params (See https://github.com/googleapis/google-api-php-client-services/blob/v0.81/src/Google/Service/Dataflow/Resource/ProjectsTemplates.php#L73.)

    $opt_params = [
        "gcsPath" => "gs://{$this->bucket}/{$this->template}",
        "location" => "us-central1",
    ];
    
    $template_params = [
       // Keep template params here.
    ];
    
    $launch_params = new \Google_Service_Dataflow_LaunchTemplateParameters;
    $launch_params->setJobName($this->jobname);
    $parameters->setParameters($template_params);
    
    $service = new \Google_Service_Dataflow($this->client);
    $request = $service->projects_templates->launch($this->project, $parameters, $opt_params);