Search code examples
node.jspostgresqlsequelize.js

Sequelize is not returning all rows for given query


I've got the following sequelize models:

import { DataTypes, Model } from 'sequelize';

class A extends Model{}

A.init {
  name: DataTypes.STRING,
  start_date: DataTypes.DATE,
  end_date: DataTypes.DATE,
  deleted: DataTypes.BOOLEAN
  
}
class B extends Model {}
B.init {
   a_id: DataTypes.Integer,
   text: DataTypes.String,
   created_at: DataTypes.STRING,
   updated_at: DataTypes.DATE,
}
B.belongsTo(A);
A.hasMany(B);

I'm trying to run a query to get the first 50 records of A with a left outer join of B. For some reason I'm only getting 49 records. What's weird is that if I take the query that is generated in the logging property and use a tool like DBeaver, I get all 50 rows.

Here is how the query is created:

const params = {
  limit: 50,
  offset: 0,
  order: [ [ 'id', 'desc']],
  attributes: [
    'id',
    'name',
    'start_date',
    'end_date',
  ],
  subQuery: false,
  where: { deleted: false },
  include: [{model: B}],
  logging: console.log
}

const results = await A.findAll(params);
//length of results is 49

And here is the query that is generated by the logging property:

(Note I did not include all of the columns in the select for readability)

SELECT * 
FROM "A" LEFT OUTER JOIN B ON "A"."id" = "B"."a_id" 
WHERE "A"."deleted" = false 
ORDER BY "A"."id" DESC 
LIMIT 50 OFFSET 0;

Sequelize query is returning 49 rows. The query that's generated from the Sequelize query is returning 50.

Some other data points

  • If I take out the include from the parameters, I get all 50 results in the sequelize findAll.
  • If I change the limit to 25 the sequelize findAll returns 25 results.
  • If I change the limit to 100 the sequelize findAll returns 99 rows but the generated query returns 100.
  • I have about 1000 rows for table A. There is a one to many relationship between table A and table B. All of the rows in table A have at least one row of table B. The only exception is there is exactly one row in table A that has no rows in table B.
  • I've tried purging table B to see if there was a problem with the join but got the same results as when there was data in that table.

Update:

  • Here's the code that I use to print out the results:
const results = await this.A.findAll(findParams);
console.log('Final results: ', results.length);
  • When I purged table B all rows for table A showed up.

Solution

  • By default, Sequelize will compose the associated model into an inner array. Because of the composition, when you count the length of result, it is actually counting the number of distinct A model records.

    Ex:

    SQL output

    | id | name | a_id | text |
    |  1 |   a1 |    1 |   b1 |
    |  1 |   a1 |    1 |   b2 |
    |  2 |   a2 |    2 |   b3 |
    

    This will be composed into

    [
      {
        "id": 1,
        "name": "a1",
        "b" : [
          {
            "text": "b1"
          },
          {
            "text": "b2"
          }
        ]
      },
      {
        "id": 2,
        "name": "a2",
        "b": [
          {
            "text": "b3"
          }
        ]
      }
    ]
    

    If you count the length of the parent array, you see 2 and SQL output is 3. What you actually need to count is the number of elements in inner arrays to match with the SQL's count.

    If you do not want Sequelize to compose in this way, you can turn off this composition by

    params = {
      ...
      raw: true,
      nest: true
    }