Working in .NET 8, I am creating a method on my repository class that takes an object called Criteria and queries EF based on the conditions in that object. Criteria is capable of generating an Expression which is then passed to the Where() LINQ extension method for the query. The problem is that, for a particular Expression tree, I get an exception that does not happen if I provide the same expression as code.
The expression is: x => x.UserRoles.Any(y => y.RoleNo == 100101) where x is of type UserEntity, y is of type UserRoleEntity and 100101 could be any integer role ID.
The exception this produces is: "System.InvalidOperationException: 'The LINQ expression 'y' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'."
I also find this exception unclear, because it's complaining about "y", which is the parameter for the nested lambda. I can't imagine what could be wrong with that part of the expression.
public async Task<List<User>> GetByCriteria(ICriteria<User> criteria)
{
// Using an expression in code
var test = _dbContext.Set<UserEntity>()
.Where(x => x.UserRoles.Any(y => y.RoleNo == 100101)).AsNoTracking();
List<UserEntity> codeEntities = await test.ToListAsync();
The above code works. Here I'm using a coded expression within the Where() call.
public async Task<List<User>> GetByCriteria(ICriteria<User> criteria)
{
Expression<Func<UserEntity, bool>> expression = criteria.AsExpression<UserEntity>();
// Using a generated expression
var queryable = _dbContext.Set<UserEntity>()
.Where(expression).AsNoTracking();
List<UserEntity> exprEntities = await queryable.ToListAsync();
This code produces the exception when ToListAsync() is called. The variable expression is an Expression that was generated by the Criteria object. In the debug view, I can see that this expression is the same as the one in code from before.
I have tried running the coded expression vs the generated Expression object, which shows that the lambda I'm using is correct.
I have also tried running with and without the AsNoTracking() call. This makes no difference.
I have compared the Queryable objects from the two examples in the debugger and found no differences between them.
I have tried running outside of the debugger and still get the exception.
If I place both sets of sample code together, so that it first runs the coded lambda and then runs the generated Expression, the exception does not happen. I'm guessing this is because EF has cached the first (successful) query and doesn't try to query again for the second one.
UPDATE: I have taken a look at LinqKit's PredicateBuilder as a possible way to make my code simpler and make the error easier to find. Unfortunately, it can't do what I was hoping for. It eases the building of the top-level expression, which is a series of logical ANDs, but does nothing for building the actual conditional expressions, which is the hard part here. I need to be able to take dynamic property names, values and operators and compose them together, so it looks like the Expression API is still the only way to go.
I will continue analyzing my code to hopefully find a difference between the coded and generated Expressions.
UPDATE: I have found the issue and posted the details below in a new response. Thanks to everyone who gave me ideas.
I've discovered what the problem was that caused the exception in my original post. It turns out that Gert Arnold had the correct idea when he commented:
A common error is to have different instances of parameters for range variables in one lambda expression. It's not enough if they have the same name, they should be the same instance.
My code generating the individual Expressions for each predicate had a programming error that escaped my notice many times. See in the method below the line marked "THIS IS THE PROBLEM" - it should have been placed before the foreach loop, not within it.
public Expression AsExpression(ParameterExpression lambdaParameterEx)
{
Expression? conditionalEx = default;
// Generate an Expression for testing the logical condition represented by the Criterion.
bool isLast = true;
foreach (var propertyName in NameParts.Reverse())
{
// This is the thing we are comparing to.
ParameterExpression parameterEx =
Expression.Parameter(_entityType, "y"); /*** THIS IS THE PROBLEM ***/
if (isLast)
{
conditionalEx = GetComparisonExpression(parameterEx, propertyName);
isLast = false;
}
else
{
// Create a lambda Expression from the conditional so that it can be embedded within the outer lambda Expression.
// We have to call Expression.Lambda dynamically since it's a generic method and we don't know the type at compile time.
Type lambdaGenericType = typeof(Func<,>).MakeGenericType(new Type[] { _entityType, typeof(bool) });
MethodInfo? lambdaMethodInfo = typeof(Expression)
.GetMethods()
.Where(m => m.Name == "Lambda"
&& m.IsGenericMethodDefinition
&& m.GetParameters()
.Where(p => (p.Position == 0
&& p.ParameterType == typeof(Expression)) || (p.Position == 1
&& p.ParameterType == typeof(ParameterExpression[])))
.Count() == 2)
.Single()
.MakeGenericMethod(lambdaGenericType);
if (lambdaMethodInfo != null)
{
Expression? lambdaEx = lambdaMethodInfo.Invoke(
null,
new object?[] {
conditionalEx,
new ParameterExpression[] { parameterEx } }
) as Expression;
if (lambdaEx != null)
{
MemberExpression propertyEx = GetPropertyExpression(lambdaParameterEx, propertyName);
// Wrap the lambda inside a call to Where(). Where() is called against the collection property.
conditionalEx = GetWhereExpression(lambdaParameterEx, propertyEx, lambdaEx, lambdaGenericType);
}
}
}
}
return conditionalEx!;
}
This caused the ParameterExpression for "y" to be created twice, once as the property in the comparison [y.RoleNo == 100101], and once as the parameter for the inner lambda [(y => ...)]. Since these ParameterExpressions failed the .Equals() test, they were seen as different by EF.
Thanks to everyone who provided ideas for me. I also have to give credit to this post here. The update post from the OP contains a nice class that can compare almost any two Expression trees. By stepping through it in the debugger, I found where the comparison failed and that allowed me to narrow down where the root of the problem was.