Search code examples
postgresqlmariadbamazon-rdstypeorm

TypeOrm is not doing bulk save when using MariaDb as database

I am trying to insert more than 1000 entries into a mariadb database.

let repo = await this.getRepository(RawOhlcv)
return await repo.save<RawOhlcv>(entityObj, { chunk: 1 })

This is the typeorm logs

query: INSERT INTO `raw_ohlcv`(`raw_ohlcv_id`, `stock_id`, `symbol`, `date`, `open`, `high`, `low`, `close`, `volume`, `is_active`, `created_at`, `updated_at`) VALUES (DEFAULT, DEFAULT, ?, ?, ?, ?, ?, ?, ?, ?, DEFAULT, DEFAULT) -- PARAMETERS: ["20MICRONS","2019-09-04",33,34.2,33,33.05,8787,true]
query: SELECT `RawOhlcv`.`raw_ohlcv_id` AS `RawOhlcv_raw_ohlcv_id`, `RawOhlcv`.`is_active` AS `RawOhlcv_is_active`, `RawOhlcv`.`created_at` AS `RawOhlcv_created_at`, `RawOhlcv`.`updated_at` AS `RawOhlcv_updated_at` FROM `raw_ohlcv` `RawOhlcv` WHERE `RawOhlcv`.`raw_ohlcv_id` = ? -- PARAMETERS: [617]
query: INSERT INTO `raw_ohlcv`(`raw_ohlcv_id`, `stock_id`, `symbol`, `date`, `open`, `high`, `low`, `close`, `volume`, `is_active`, `created_at`, `updated_at`) VALUES (DEFAULT, DEFAULT, ?, ?, ?, ?, ?, ?, ?, ?, DEFAULT, DEFAULT) -- PARAMETERS: ["21STCENMGM","2019-09-04",12.3,12.3,12.15,12.3,244,true]
query: SELECT `RawOhlcv`.`raw_ohlcv_id` AS `RawOhlcv_raw_ohlcv_id`, `RawOhlcv`.`is_active` AS `RawOhlcv_is_active`, `RawOhlcv`.`created_at` AS `RawOhlcv_created_at`, `RawOhlcv`.`updated_at` AS `RawOhlcv_updated_at` FROM `raw_ohlcv` `RawOhlcv` WHERE `RawOhlcv`.`raw_ohlcv_id` = ? -- PARAMETERS: [618]
query: INSERT INTO `raw_ohlcv`(`raw_ohlcv_id`, `stock_id`, `symbol`, `date`, `open`, `high`, `low`, `close`, `volume`, `is_active`, `created_at`, `updated_at`) VALUES (DEFAULT, DEFAULT, ?, ?, ?, ?, ?, ?, ?, ?, DEFAULT, DEFAULT) -- PARAMETERS: ["3IINFOTECH","2019-09-04",1.7,1.75,1.7,1.7,405693,true]
query: SELECT `RawOhlcv`.`raw_ohlcv_id` AS `RawOhlcv_raw_ohlcv_id`, `RawOhlcv`.`is_active` AS `RawOhlcv_is_active`, `RawOhlcv`.`created_at` AS `RawOhlcv_created_at`, `RawOhlcv`.`updated_at` AS `RawOhlcv_updated_at` FROM `raw_ohlcv` `RawOhlcv` WHERE `RawOhlcv`.`raw_ohlcv_id` = ? -- PARAMETERS: [619]

So each entry is being individually inserted and there is even a select query running before the next entry.

Whereas when I try to do the same with a postgres database, it inserts all the entries at once.

How can I achieve bulk insert with mariadb?

PS: I am using AWS RDS

Solution

  • The save query supplied by Typeorm is interpreted like this when using AWS RDS. But if you use a pure insert query using query builder, then it does bulk insert

    let repo = await this.getRepository(RawOhlcv)
await repo
  .createQueryBuilder()
  .insert()
  .values(entityArray)
  .execute()