How to plot my own logistic regression decision boundaries and SKlearn's ones on the same figure

Question

I have an assignment in which I need to compare my own multi-class logistic regression and the built-in SKlearn one.

As part of it, I need to plot the decision boundaries of each, on the same figure (for 2,3, and 4 classes separately).

This is my model's decision boundaries for 3 classes:

Made with this code:

x1_min, x1_max = X[:,0].min()-.5, X[:,0].max()+.5
x2_min, x2_max = X[:,1].min()-.5, X[:,1].max()+.5

xx1, xx2 = np.meshgrid(np.linspace(x1_min, x1_max), np.linspace(x2_min, x2_max))
grid = np.c_[xx1.ravel(), xx2.ravel()]
for i in range(len(ws)):
    probs = ol.predict_prob(grid, ws[i]).reshape(xx1.shape)
    plt.contour(xx1, xx2, probs, [0.5], linewidths=1, colors='green')

where

• ol - is my Own Linear regression
• ws - the current weights

That's how I tried to plot the Sklearn boundaries:

for i in range(len(clf.coef_)):
    w = clf.coef_[i]
    a = -w[0] / w[1]
    xx = np.linspace(x1_min, x1_max)
    yy = a * xx - (clf.intercept_[0]) / w[1]
    plt.plot(xx, yy, 'k-')

Resulting

I understand that it's due to the 1dim vs 2dim grids, but I can't understand how to solve it.

I also tried to use the built-in DecisionBoundaryDisplay but I couldn't figure out how to plot it with my boundaries + it doesn't plot only the lines but also the whole background is painted in the corresponding color.

Answer 1

A couple fixes:

1. Change clf.intercept_[1] to clf.intercept_[i]

2. If the xlimits and ylimits in the plot look strange, you can constrain them.

ax.set_xlim([x1_min, x1_max])
ax.set_ylim([x2_min, x2_max])

MRE:

import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import make_blobs
from sklearn.linear_model import LogisticRegression

X, y = make_blobs(n_features=2, centers=3, random_state=42)
fig, ax = plt.subplots(1, 2)

x1_min, x1_max = X[:,0].min()-.5, X[:,0].max()+.5
x2_min, x2_max = X[:,1].min()-.5, X[:,1].max()+.5

def draw_coef_lines(clf, X, y, ax, title):
    for i in range(len(clf.coef_)):
        w = clf.coef_[i]
        a = -w[0] / w[1]
        xx = np.linspace(x1_min, x1_max)
        yy = a * xx - (clf.intercept_[i]) / w[1]
        ax.plot(xx, yy, 'k-')
    ax.scatter(X[:, 0], X[:, 1], c=y)
    ax.set_xlim([x1_min, x1_max])
    ax.set_ylim([x2_min, x2_max])
    ax.set_title(title)

clf1 = LogisticRegression().fit(X, y)
clf2 = LogisticRegression(multi_class="ovr").fit(X, y)

draw_coef_lines(clf1, X, y, ax[0], "Multinomial")
draw_coef_lines(clf2, X, y, ax[1], "OneVsRest")
plt.show()