Search code examples
c#json.netxunit

Why Assert.Equal two different JObject(Newtonsoft.Json.Linq.JObject) got pass

I am quite new to c#, xUnit as well as Newtonsoft.Json. While I am trying compare two different JOject using Assert.Equal() method in unit test, it pass, refer to the below example code

using Newtonsoft.Json.Linq;

namespace TestProject1
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {
            JObject jobj1 = JObject.FromObject(new { foo = "bar" });
            JObject jobj2 = JObject.FromObject(new { foo = 1 });
            JObject jobj3 = JObject.FromObject(new { foo = "b" });
            Assert.Equal(jobj1, jobj2); // output: pass
            Assert.Equal(jobj1, jobj3); // output: failure
            Assert.True(jobj1.Equals(jobj2)); // output: failure
            Assert.True(jobj1.Equals(jobj3)); // output: failure
        }
    }
}

I don't quite understand how this happens, should I look deep into xUnit or Newtonsoft.Json?

I find there is a DeepEquals method from Newtonsoft.Json, but I am not sure which comparator does xUnit call.

Solution

  • This is kind of a bug of xUnit asserts + Newtonsoft Json.NET. The reason is two-fold:

    • From Newtonsoft side:
      • JToken (base type for "everything" in the library) implements IEnumerable
      • JValue implements IComparable (and IEnumerable because it inherits from JToken) which throws when two JValues are incompatible
    • From xUnit side - if type implements IComparable and comparer throws error, the error is ignored, then if type implements IEnumerable it is treated as a collection and JValue is an empty collection, so as collections they are equal

    Minimal repro looks like:

    [Fact]
public void Test2()
{
    var jvLeft = JToken.FromObject(1);
    var jvRight = JToken.FromObject("bar");

    // some "debug" checks
    Assert.True(jvLeft is JValue);
    Assert.Empty(jvLeft);
    Assert.Throws<FormatException>(() => (jvRight as IComparable).CompareTo(jvLeft));

    Assert.Equal(jvRight, jvLeft); // output: pass
}

    One of the fixes is to use Assert.StrictEqual (works correctly for JValues only):

    [Fact]
public void Test2()
{
    var jvLeft = JToken.FromObject(1);
    var jvRight = JToken.FromObject("bar");
    Assert.StrictEqual(jvRight, jvLeft); // output: fail
}

    Or use/follow up with the JToken.DeepEquals (as you have discovered):

    [Fact]
public void Test12()
{
    JObject jobj1 = JObject.FromObject(new { foo = "bar" });
    JObject jobj2 = JObject.FromObject(new { foo = 1 });
    Assert.Equal(jobj1, jobj2); // output: pass
    Assert.True(JToken.DeepEquals(jobj1, jobj2)); // output: fail
}

    Notes: