I'm coding a scraper that uses gspread to read and write In Google Sheets.
In the "writting" part of the code I had to add a try-except
because of an APIError
caused by the quota limit of writting, so when the except Is executed It have wait 100 seconds and then continue. The problem is that It ignores the item that caused the except, but It was supposed to repeat that Item
def process_cpf_list(self):
# SKIP OVER COLUMN HEADING IN THE SPREADSHEET
cpfs = self.sheet.col_values(self.cpf_col)[1:]
bot_url = BOT()
for row, cpf in enumerate(cpfs):
nome, idade, beneficio, concessao, salario, bancos, bancocard, consig, card = bot_url.search_cpfs(cpf)
# UPDATE THE SHEET
print("Atualizando...")
try:
row = row + 2
self.sheet.update_cell(row, self.nome_col, nome)
self.sheet.update_cell(row, self.age_col, idade)
self.sheet.update_cell(row, self.beneficio_col, beneficio)
self.sheet.update_cell(row, self.concessao_col, concessao)
self.sheet.update_cell(row, self.salario_col, salario)
self.sheet.update_cell(row, self.bancos_col, bancos)
self.sheet.update_cell(row, self.bancocard_col, bancocard)
self.sheet.update_cell(row, self.consig_col, consig)
self.sheet.update_cell(row, self.card_col, card)
print('Cliente atualizado!')
except APIError:
print('Esperando para atualizar...')
time.sleep(100)
continue
cpf_updater = CpfSearch('TESTE')
cpf_updater.process_cpf_list()
It shouldn't repeat the item in this case. When you iterate with for loop, once the item is picked, it won't be picked again.
You can add a while loop that will try infinitely to update unless the update succeeded:
for row, cpf in enumerate(cpfs):
nome, idade, beneficio, concessao, salario, bancos, bancocard, consig, card = bot_url.search_cpfs(cpf)
# UPDATE THE SHEET
print("Atualizando...")
max_retries = 3
row = row + 2
while max_retries:
try:
self.sheet.update_cell(row, self.nome_col, nome)
self.sheet.update_cell(row, self.age_col, idade)
self.sheet.update_cell(row, self.beneficio_col, beneficio)
self.sheet.update_cell(row, self.concessao_col, concessao)
self.sheet.update_cell(row, self.salario_col, salario)
self.sheet.update_cell(row, self.bancos_col, bancos)
self.sheet.update_cell(row, self.bancocard_col, bancocard)
self.sheet.update_cell(row, self.consig_col, consig)
self.sheet.update_cell(row, self.card_col, card)
print('Cliente atualizado!')
break
except APIError:
max_retries -= 1
print('Esperando para atualizar...')
time.sleep(100)
I have added a simple retries mechanism, which will try 3 times before it skips the row.