Is it more efficient to highlight cells in QAbstractTableModel's data method or QItemDelegate's paint method?

Question

I'm building a Qt table in Python to display a large pandas DataFrame. The table uses a custom PandasTableModel (subclassing QAbstractTableModel) to connect to the DataFrame, and I want to highlight cells conditionally—e.g., red for False values and green for True.

I have found two ways of doing this:

1. Using the data method in the model: Returning a specific background color for certain cells based on their value.

   class PandasTableModel(QtCore.QAbstractTableModel):
   ...
       @override
       def data(self, index, role):
           if not index.isValid():
               return None
           value = self._dataframe.iloc[index.row(), index.column()]
           if role == QtCore.Qt.ItemDataRole.DisplayRole:
               return str(value)
           elif role == QtCore.Qt.ItemDataRole.BackgroundRole:
               if value == "True":
                   return QtGui.QColor(0, 255, 0, 100)
               elif value == "False":
                   return QtGui.QColor(255, 0, 0, 100)
           return None
   

2. Using a custom delegate's paint method: Setting the cell's background color in the paint method of a delegate (subclassed from QItemDelegate):

   class PandasItemDelegate(QtWidgets.QItemDelegate):
   ...
       @override
       def paint(self, painter, option, index):
           super().paint(painter, option, index)
           value = index.data(QtCore.Qt.ItemDataRole.DisplayRole)
           if value in ["True", "False"]:
               color = QtGui.QColor(0, 255, 0, 100) if value == "True" else QtGui.QColor(255, 0, 0, 100)
               painter.fillRect(option.rect, color)
   

Which approach would be more efficient, especially for a large DataFrame, in terms of memory and processing speed?

Answer 1

Premise: the proper comparison

First of all, there is no such thing as more/better/optimal/faster/etc. in their absolute meaning. What's "best" is up to your requirements and context: it may be good for you and terrible for others. And that doesn't even consider user expectations in most cases.

Then, before considering "efficiency", you should also think about consistency, effectiveness, and the implications of each approach you may choose.

Finally, while the two concepts you're suggesting are relatively similar, their results certainly are not, especially because your implementations are potentially quite different.

Distinction between model data and view display

An important thing to remember about item/views is that the model is always expected to reliably return consistent data (no matter "what" shows that data), while the view may choose to display that data in its own way.

In the model data() case, you let the view (and therefore, its delegate) draw the item background based on the BackgroundRole, using a context to define what color to return. The delegate will theoretically fill the cell with that background, and then draw the remaining "display" contents (possibly some text, an icon and/or a checkbox, if they exist) above that. The "theoretically" has an important meaning, because it may be completely ignored in some cases.

In the delegate case, you're specifically drawing the color depending on the given field, but above the "display" contents: how that is drawn may change because of composition (see below).

The two common Qt delegates (QItemDelegate and QStyledItemDelegate) override the background color using the Highlight palette role when an item is selected, meaning that in the "item data" case the user will probably never be able to see the background of a selected item.
Furthermore, if the view uses a QStyledItemDelegate and a proper Qt Style Sheet is set (for instance, using the ::item { background: <color>; } selector/property combination), the background is possibly completely ignored to begin with.

This means that if you follow the model approach, the background will probably never be shown for any selected item at all.

If showing that color is of utmost importance, you cannot rely on the model approach only.
Which brings us to an appropriate delegate approach, but that's not the only issue.

Color composition

As explained above, there is an important difference in how you applied those methods: the first draws the background first (before anything else), the other after that (above what has been drawn before).

You probably did not notice or consider the difference enough, but it is there, because drawing above something is achieved through composition, and the order on which things are drawn depends on the opacity of the colors involved. Drawing with an opaque color above something, always clears what was "behind" it with the new color, but if you use a partially transparent color, then it's blended with what was previously drawn.

This is fundamentally identical to physical painting: if the paint you're using is too diluted (as in a not fully opaque color), the first pass will just blend what's behind with the new color; draw a line using a yellow marker and then draw another line with a blue one intersecting the first one: what color do you get?

See the difference using the background role, in the model data approach (the text is in opaque color and is drawn on a semi transparent background):

And then drawing the background above, in your delegate attempt (the opposite):

The difference in the text is quite visible.
It's also important to notice that the "background" is not identical: in the first case, the delegate uses the given color as a reference, eventually basing the painted rectangle on that color (the style decides what to do with it); in the second case, the color is the same, because you explicitly filled the rectangle with that color.

What's "best"

Performance is important, but should always be balanced with actual efficiency: not only "how fast" an approach can be, but also how effective it may be.

The fact that the model data may be large must be carefully considered: it's not always relevant.

Qt item views are relatively smart, and they query the model only about what they need; some roles are only requested upon drawing, others for related aspects (eg: to know how wide a column should be): background/foreground roles are only relevant when displaying items, therefore if the view has a size that can only show 10 rows and 10 columns, it really doesn't matter if the model has one hundred or one million items, at least for what concerns those roles: you can rest assured that data() will be called with those roles only for the items that are currently visible.

Besides the paint composition issues noted above (regarding your approaches), the only difference is that you're using a custom delegate that draws things on its own, and it does it from Python, which introduces a relatively important bottle neck: using the default delegate (that would "do things" on the "C++ side" of things, therefore much faster) against relying it on the more efficient C++ and precompiled approach.

If your view normally shows a reasonable amount of items ("how much reasonable" is up to you to decide, but if you're under the count of 1000 on a decent computer, that should be fine) that shouldn't be an issue, especially if you're using the simpler QItemDelegate.

QItemDelegate and QStyledItemDelegate

While they have the same purpose and achieve similar results, the two standard delegates Qt provides are different.

QItemDelegate is the simplest one, it's generally faster and less "fancy", as it doesn't rely too much on the QStyle. The background and selection colors, if any, are shown as plain colors, without applying 3D/glow/hover gradients. Even though those are done on the "C++ side", filling dozens of rectangles with a given color is obviously simpler than computing those gradients for each one of them. Private subclasses of that delegate are used for QComboBox popups, QColumnView and the internal QTableView of QCalendarWidget.

QStyledItemDelegate is the default for all standard Qt item views. It's normally more compliant with the OS, but its complexity may add some significant overhead if you're going to show lots of items at the same time (in the order of many thousands) and the model is also large: the problem will be mostly visible during scrolling.

Considerations about possible approaches and optimizations

As said above, any Python implementation may add a significant bottleneck, so you need to carefully choose which delegate to use (considering how much display matters) and how to implement it.
Knowing which, when and how often each function is called, is mandatory for proper comparison in each consideration.

For instance, QItemDelegate provides drawBackground(), which would be nice to implement; unfortunately, it is not a "virtual" function, meaning that trying to override it is pointless unless it's called explicitly.

Using QStyledItemDelegate and overriding initStyleOption() to set the option.backgroundBrush may be a possibility, but consider that that function is also called a lot, even when nothing is being painted yet (sizeHint() calls it).

It also is important to know if the background color should be applied to any item in the view, or only for specific rows or columns. For obvious reasons, if you have a lot of columns and you only need this behavior for just one or a few of them, then you shall create a single delegate for the view and call setItemDelegateForColumn() for those columns.

If you don't care much about the selection overriding the background color and text/decoration display, using the model data approach with QItemDelegate might be one good performant choice.

Yet, there's still room for improvement. For instance, there's little point in creating new QColors every time in the model depending on the cell value. A possible solution could be to use a basic dictionary:

class PandasTableModel(QtCore.QAbstractTableModel):
    boolColors = {
        "False": QColor(0, 255, 0, 100), 
        "True": QColor(255, 0, 0, 100), 
    }
    ...
    @override
    def data(self, index, role):
        if not index.isValid():
            return
        if role == QtCore.Qt.ItemDataRole.DisplayRole:
            return str(self._dataframe.iloc[index.row(), index.column()])
        elif role == QtCore.Qt.ItemDataRole.BackgroundRole:
            return self.boolColors.get(
                self._dataframe.iloc[index.row(), index.column()])

^{My changes/removal in the return None cases are irrelevant in efficiency and just up to personal style (I don't particularly agree with the PEP-8 on this).}

Note that the above syntax is not that elegant, but if the model is quite large and the view may show thousands of items at any given time, its efficiency may be preferred: since data() is called very often, and lots of those calls may end up in unused roles, it's better to get the value only when actually required; local variable allocation (value = ...), even though relatively fast, should be done if the variable is actually needed more than once within the same block.

Then, let's consider the composition issue: as said above, you shall not draw the background above everything. If you want to display both the background and the selection, you need to do composition appropriately, meaning that you shall consider both colors.

There are various options for this. As said above, which one to choose is up to your specific needs.

For instance, you may still choose the simpler QItemDelegate, but rely on its drawBackground() behavior, which is either based on the palette (when selected) or BackgroundRole, considering the option state.

The following is a possible example, without the model data() implementation of the background role:

class BackgroundDelegate(QItemDelegate):
    boolColors = {
        # Each state has two color values, depending on the selection
        "False": (QColor(0, 255, 0, 100), QColor(0, 255, 0)), 
        "True": (QColor(255, 0, 0, 100), QColor(255, 0, 0)), 
    }
    transparent = QColor(Qt.transparent)
    def paint(self, qp, opt, index):
        bgColors = self.boolColors.get(index.data())
        if bgColors:
            # always draw the background
            if opt.state & QStyle.State.State_Selected:
                qp.fillRect(opt.rect, bgColors[1]

                # override the background color for the "Highlight" palette 
                # role by making it transparent, so that nothing will be 
                # drawn in drawBackground(); yet, keep the "option" state, 
                # which may be relevant for other aspects decided by the 
                # style, such as the font weight, italic, etc.
                opt.palette.setColor(QPalette.ColorRole.Highlight, 
                    self.transparent)
            else:
                qp.fillRect(opt.rect, bgColors[0]
        super().paint(qp, opt, index)

Final considerations

The above should clearly summarize how no best/more/better/etc. option could ever exist without a proper context.

What has been shown above is just a fragment of possible options, and should also explain how every possible implementation should be carefully considered: in your case, while appropriate in principle, your two options were actually resulting in two quite different and inconsistent results, meaning that none of them were appropriate for reasonable comparisons.

Finally, ensure that code consistency (including function signature) is also appropriate: some of your overrides are not. For instance, data() expects that the role is keyworded (as in optional) defaulting to Qt.DisplayRole.
That may not be so relevant for performance or efficiency, but it's still important for consistency, especially in subclassing. The @override decorator isn't mandatory for functionality, yet you've considered adding it as more relevant than the expected signature (which does not require the role argument).
Check your priorities: while typing may be relevant, it can not be more important than the expected signature syntax.