Search code examples
javaapigoogle-bigqueryruntime-errortalend

java.lang.NoSuchMethodError: com.google.api.services.bigquery.model.JobConfigurationQuery.getConnectionProperties()Ljava/util/List;

I have some problems with com.google.api.services. I'am using this version "BigQuery API v2 (revision 459)":

https://www.javadoc.io/doc/com.google.apis/google-api-services-bigquery/v2-rev459-1.25.0/com/google/api/services/bigquery/model/JobConfigurationQuery.html

and indeed the method "getConnectionProperties()" doesn't not exist... This is the complete error:

Exception in thread "main" java.lang.NoSuchMethodError: com.google.api.services.bigquery.model.JobConfigurationQuery.getConnectionProperties()Ljava/util/List; at com.google.cloud.bigquery.QueryJobConfiguration$Builder.(QueryJobConfiguration.java:250) at com.google.cloud.bigquery.QueryJobConfiguration$Builder.(QueryJobConfiguration.java:95) at com.google.cloud.bigquery.QueryJobConfiguration.fromPb(QueryJobConfiguration.java:1060) at com.google.cloud.bigquery.JobConfiguration.fromPb(JobConfiguration.java:128) at com.google.cloud.bigquery.JobInfo$BuilderImpl.(JobInfo.java:158) at com.google.cloud.bigquery.Job.fromPb(Job.java:497) at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:363) at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:340) at bce_datahub.bigquery_template_0_1.bigquery_template.tJava_1Process(bigquery_template.java:383) at bce_datahub.bigquery_template_0_1.bigquery_template.runJobInTOS(bigquery_template.java:757) at bce_datahub.bigquery_template_0_1.bigquery_template.main(bigquery_template.java:583)

I saw that this version:

https://www.javadoc.io/doc/com.google.apis/google-api-services-bigquery/latest/com/google/api/services/bigquery/model/JobConfigurationQuery.html

contain this method but if I replace it with this version I get another error:

NoClassDefFoundError: com/google/api/services/bigquery/Bigquery$Builder

So this version seem doesn't have this class.

I'm developing a Java program to:

  1. Execute a query on Bigquery
  2. Export a BigQuery table on Google cloud storage.

For the step 2 I don't have problems. These are the jara that I'm using:

enter image description here

I'm using Talend Studio so I have to install jars: one by one. This is a part of the script that generate the problem:

import java.io.File;
import java.io.FileInputStream;

import com.google.cloud.RetryOption;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.QueryJobConfiguration;
import com.google.cloud.bigquery.Job;
import com.google.cloud.bigquery.JobInfo;
import com.google.cloud.bigquery.Table;
import com.google.cloud.bigquery.TableId;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.auth.oauth2.AccessToken;

import org.threeten.bp.Duration;

//INPUT
String credentialsPath = "...";
String projectId = "...";
String datasetName = "...";
String tableName = "...";                                                           
String bucketName = "...";                                                  
String objectName = "...";                                          
String destinationUri = "...";
String dataFormat = "...";
String destFilePath = "...";
String ddl =

    "DROP TABLE IF EXISTS `" + datasetName + "." + tableName + "`;"
    + "CREATE TABLE `" + datasetName + "." + tableName + "` AS "
    + "SELECT * FROM ...";




//Authentification
File credentialsFilePath = new File(credentialsPath);
FileInputStream serviceAccountStream = new FileInputStream(credentialsFilePath);
GoogleCredentials credentials;
credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);

// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests
BigQuery bigquery = BigQueryOptions.newBuilder()
                .setProjectId(projectId)
                .setCredentials(credentials).build()
                .getService();



// Execute query on BigQuery
QueryJobConfiguration config = QueryJobConfiguration.newBuilder(ddl).build();
Job job = bigquery.create(JobInfo.of(config));

Job completedJob =
    job.waitFor(
        RetryOption.initialRetryDelay(Duration.ofSeconds(1)),   //Checking period time [Optional]
        RetryOption.totalTimeout(Duration.ofMinutes(3))         //Timeout [Optional]
    );
        
if (completedJob == null)
{
    System.out.println("Job not executed since it no longer exists.");
    return;
} else if (completedJob.getStatus().getError() != null)
{
    System.out.println("BigQuery was unable to execute the query due to an error: \n" + job.getStatus().getError());
    return;
}
    
System.out.println("Table create successful. Check in "  + datasetName + " for the " + tableName + " table.");

...
Solution

  • Resolved.

    I install the jar that I found at https://mvnrepository.com/artifact/com.google.apis/google-api-services-bigquery/v2-rev20201030-1.31.0.

    I don't understand why the jar that I downloaded from here https://www.javadoc.io/doc/com.google.apis/google-api-services-bigquery/latest/com/google/api/services/bigquery/model/JobConfigurationQuery.html

    doesn't work.

    enter image description here

    Seems that they are different. However I resolved my problem.