I am trying to upload data from certain fields in a CSV file to an already existing table.
From my understanding, the way to do this is to create a new table and then append the relevant columns of the newly created table to the corresponding columns of the main table.
How exactly do I append certain columns of data from one table to another? As in, what specific commands?
I am using the bigquery api and the python-client-library.
You can use pandas library for that.
import pandas as pd
data = pd.read_csv('input_data.csv')
useful_columns = [col1, col2, ... ] # List the columns you need data[useful_columns].to_csv('result_data.csv', index=False) # index=False is to prevent creating extra column