Can't reproduce Sum of Squares Calculation in torcheval.metrics.r2_score

Question

I calculate R^2 manually and compare the results to torcheval.metrics.regression.r2_score without multioutput but I do not tie out the total sum squared calculation, and hence manual R^2 is different than torcheval:

Manual approach:

#Manual approach code
ss_total = torch.sum((var1 - torch.mean(var1)) ** 2)
ss_residual = torch.sum((var1 - var2) ** 2)
r2 = 1 - (ss_residual / ss_total)
print("R^2 manual",r2, "my ss_total", ss_total, "ss_residual", ss_residual)
#R^2 manual tensor(-1.4128, device='cuda:0') my ss_total tensor(3.7081, device='cuda:0') ss_residual tensor(8.9471, device='cuda:0')

Torcheval.metrics approach tss formulae documentation without multioutput:

sum_squared_obs = torch.sum((actual - torch.mean(actual)) ** 2)
tss sum squared calculation = sum_squared_obs - torch.square(sum_obs) / num_obs
r_squared = 1 - (rss / tss)

#torcheval.metrics.regression.r2_score tested in script
metric = R2Score(device=device)
update = metric.update(var1, var2)
print("sum_squared_residual",update.sum_squared_residual)
print("sum_obs",update.sum_obs)
print("torch.square(sum_obs)",torch.square(update.sum_obs))
print("num_obs",len(var1))
print("sum_squared_obs",update.sum_squared_obs)
r2_py = metric.compute()
print("R^2 pytorch",r2_py)
#sum_squared_residual tensor(8.9471, device='cuda:0')
#sum_obs tensor(-29.9617, device='cuda:0')
#torch.square(sum_obs) tensor(897.7044, device='cuda:0')
#num_obs 64
#sum_squared_obs tensor(22.2245, device='cuda:0')
#R^2 pytorch tensor(-0.0914, device='cuda:0')
#R^2 var_weight pytorch tensor(-0.0914, device='cuda:0')

I cannot tie out tss.

Can someone explain what is the difference between the two approaches?

I have saved the actual and predicted values in excel and calculate R^2 with both approaches.

Answer 1

R2 is not symmetric. You get different results because you change which variable is the ground truth.

In your first section of code, you treat var1 as the ground truth values and var2 as the predicted values.

In your second section of code, you treat var2 as the ground truth and var1 as the predicted value.

import torch
from torcheval.metrics.regression import R2Score

var1 = torch.randn(64)
var2 = torch.randn(64)

def compute_r2(inputs, targets):
    ss_total = torch.sum((targets - torch.mean(targets)) ** 2)
    ss_residual = torch.sum((targets - inputs) ** 2)
    r2 = 1 - (ss_residual / ss_total)
    return r2

def compute_r2_metric(inputs, targets):
    metric = R2Score()
    metric.update(inputs, targets)
    r2 = metric.compute()
    return r2

# treat var1 as input, var2 as target
print(compute_r2(var1, var2))
> tensor(-0.8324)

print(compute_r2_metric(var1, var2))
> tensor(-0.8324)

# treat var2 as input, var1 as target
print(compute_r2(var2, var1))
> tensor(-1.9208)

print(compute_r2_metric(var2, var1))
> tensor(-1.9208)