QSqlRelationalTableModel only populates first 256 records

Question

I have the following simple model using PyQt5, sqlite3 and python3.5.2.

class choicesModel(QDialog):
    def __init__(self, parent=None):
        super(choicesModel, self).__init__()

        query = QSqlQuery()
        query.prepare("SELECT COUNT(*) FROM resident_choices")
        query.exec_()
        query.next()
        dbrows = query.value(0)
        print("rows in db: ", dbrows)

        self.choicesModel = QSqlRelationalTableModel(self)
        self.choicesModel.setEditStrategy(QSqlTableModel.OnManualSubmit)
        self.choicesModel.setTable("resident_choices")
        self.choicesModel.select()
        rows = self.choicesModel.rowCount()
        print("rows returned by model.select(): ", rows)

Executing the model produces the following output:

rows in db: 831

rows after model.select(): 256

It appears that the initial select only populates the first 256 records. Is this an sqlite3 issue and if so how do I populate all records (the db only has about 850 records total). If not how can I force all records to be loaded?

The upshot is that using a proxymodel and proxy view does not initially show all records until I change the proxy filter condition - that is, I have to trigger the loading of the rest of the records.

Answer 1

The number of records is hard-coded as #define QSQL_PREFETCH 255 in the QSqlQueryModel class. To override it, you will have to explicitly fetch all the rows yourself:

while self.choicesModel.canFetchMore():
    self.choicesModel.fetchMore()

You will probably need to run code like this every time the model is updated, so it might be better to create your own sub-class and reimplement fetchMore.