I have the following Linq queries that differs only on the where clause:
Query 1:
var initialList = await
context.AlertDetailsDbSet
// Pouring the TrackedVehicule related graph
.Include(ad => ad.TrackedVehicule.Registration)
.Include(ad => ad.TrackedVehicule.Device)
.Include(ad => ad.TrackedVehicule.GpsBox.Icon)
.Include(ad => ad.TrackedVehicule.GpsBoxTypeNavigation.Icon)
.Include(ad => ad.TrackedVehicule.VehiculeGpsBoxInfo.VehiculeConfigurationNavigation)
// Loading the associated alert
.Include(ad => ad.Alert)
#if NETSTANDARD2_1
// This filter is not supported with EF Core 2.X
// It should be done client side
.Where(ad => ad.Alert.CompanyId == companyId && ad.Alert.AlertRule.Contains("SerializableContextAlert") && ad.AlertDateTime >= startDate && ad.AlertDateTime <= endDate && ad.TrackedVehicule != null && ad.TrackedVehicule.CompanyId == companyId && ad.TrackedVehicule.RowEnabled == true && (ad.TrackedVehicule.GpsBoxType == 29 || ad.TrackedVehicule.GpsBoxType == 12))
#endif
.Skip(0)
// A limit is applied on returned elements
.Take(1000)
.OrderByDescending(ad => ad.AlertDateTime)
// .Skip(skip)
// .Take(take)
// We disable the tracking mechanism because the context will be thrown away as soon as we have the data
.AsNoTracking()
// We want an asynchrounous execution
.ToListAsync(cancellationToken)
Query 2:
var initialList = await
context.AlertDetailsDbSet
// Pouring the TrackedVehicule related graph
.Include(ad => ad.TrackedVehicule.Registration)
.Include(ad => ad.TrackedVehicule.Device)
.Include(ad => ad.TrackedVehicule.GpsBox.Icon)
.Include(ad => ad.TrackedVehicule.GpsBoxTypeNavigation.Icon)
.Include(ad => ad.TrackedVehicule.VehiculeGpsBoxInfo.VehiculeConfigurationNavigation)
// Loading the associated alert
.Include(ad => ad.Alert)
#if NETSTANDARD2_1
// This filter is not supported with EF Core 2.X
// It should be done client side
.Where(ad => ad.Alert.CompanyId == companyId && ad.Alert.AlertRule.Contains("SerializableContextAlert") && ad.TrackedVehicule != null && ad.TrackedVehicule.RowEnabled == true && ad.TrackedVehicule.CompanyId == companyId && (ad.TrackedVehicule.GpsBoxType == 29 || ad.TrackedVehicule.GpsBoxType == 12) && ad.AckTime.HasValue == acknwoledged)
#endif
.OrderByDescending(ad => ad.AlertDateTime)
.Skip(skip)
.Take(take)
// We disable the tracking mechanism because the context will be thrown away as soon as we have the data
.AsNoTracking()
// We want an asynchrounous execution
.ToListAsync(cancellationToken)
Basically, the first query is (among aother things) filtering on a boolean property, while the second one is filtering on a range of dates.
My problem is that those queries are not translated to the same T-TSQL leading to different results.
TSQL 1 :
exec sp_executesql N'SELECT [t].[AlertDetailID], [t].[AckMachineName], [t].[AckPhoneNumber], [t].[AckTime], [t].[AckUserName], [t].[AlertDateTime], [t].[AlertID], [t].[AlertedCorridorId], [t].[AlertedItemId],
[t].[AssociatedVehiculeID], [t].[AssociatedVehiculeUserID], [t].[Context], [t].[CustomID], [t].[CustomInfo], [t].[DbInsertTime], [t].[IsFleetAlert], [t].[MessageStatus], [t].[ReceivedTime], [t].[RowVersion],
[t].[SafeProtectCustomInfo], [t].[TrackedVehiculeID], [t].[TrackedVehiculeUserID], [v0].[VehiculeID], [v0].[Address], [v0].[AddressProtocol], [v0].[BoardID], [v0].[Category], [v0].[CompanyID], [v0].[CustomId], [v0].[DbInsertTime], [v0].[Description], [v0].[GpsBoxSubType], [v0].[GpsBoxTrackingDelay], [v0].[GpsBoxType], [v0].[HardwareID], [v0].[HasGeoWorker], [v0].[IconID], [v0].[Name], [v0].[PhoneNumber], [v0].[RowEnabled], [v0].[RowVersion], [v0].[VehiculeUserID], [r].[VehiculeID], [r].[CompanyId], [r].[LatestRegistrationStatusChangeDate], [r].[RegistrationMailSent], [r].[RegistrationStatusId], [d].[VehiculeID], [d].[AppVersion], [d].[Brand], [d].[Details], [d].[Imei], [d].[Model], [d].[Name], [d].[OsVersion], [d].[RowVersion], [g].[GpsBoxTypeID], [g].[GpsBoxSubTypeID], [g].[IconId], [g].[Name], [i].[IconId], [i].[Category], [i].[FileName], [i].[Icon], [g0].[GpsBoxTypeId], [g0].[Category], [g0].[IconId], [g0].[Name], [i0].[IconId], [i0].[Category], [i0].[FileName], [i0].[Icon], [v1].[VehiculeID], [v1].[BoardConfiguration], [v1].[BoardConnected], [v1].[BoardCurrentAddress], [v1].[BoardLastCommunicationTime], [v1].[Connected], [v1].[CurrentAddress], [v1].[CurrentDelay], [v1].[FuelConsumptionEstimationTime], [v1].[HeartBeatPeriod], [v1].[InsureCoherence], [v1].[KillHedgehog], [v1].[LastCommunicationTime], [v1].[LastConnexionTime], [v1].[LastParkMileage], [v1].[LastParkMileageTime], [v1].[NormalTrackingMode], [v1].[SmartModeDelay], [v1].[SmartModeDistance], [v1].[TimeModeDelay], [v1].[UpdateStartTime], [v1].[UpdateStatus], [v1].[VehiculeConfiguration], [g1].[ConfigurationID], [g1].[ConfigName], [g1].[Firmware], [g1].[GpsBoxType], [g1].[Master], [g1].[MasterName], [g1].[SimContract], [g1].[System], [a1].[AlertID], [a1].[AlertEnabled], [a1].[AlertRule], [a1].[Always], [a1].[CalendarID], [a1].[Category], [a1].[CompanyID], [a1].[Description], [a1].[DisplayName], [a1].[RowEnabled], [a1].[RowVersion], [a1].[Shared], [a1].[UserID]
FROM (
SELECT [a].[AlertDetailID], [a].[AckMachineName], [a].[AckPhoneNumber], [a].[AckTime], [a].[AckUserName], [a].[AlertDateTime], [a].[AlertID], [a].[AlertedCorridorId], [a].[AlertedItemId], [a].[AssociatedVehiculeID], [a].[AssociatedVehiculeUserID], [a].[Context], [a].[CustomID], [a].[CustomInfo], [a].[DbInsertTime], [a].[IsFleetAlert], [a].[MessageStatus], [a].[ReceivedTime], [a].[RowVersion], [a].[SafeProtectCustomInfo], [a].[TrackedVehiculeID], [a].[TrackedVehiculeUserID], [a0].[AlertID] AS [AlertID0]
FROM [AlertDetail] AS [a]
INNER JOIN [Alert] AS [a0] ON [a].[AlertID] = [a0].[AlertID]
LEFT JOIN [Vehicule] AS [v] ON [a].[TrackedVehiculeID] = [v].[VehiculeID]
WHERE (((((([a0].[CompanyID] = @__companyId_0) AND (CHARINDEX(N''SerializableContextAlert'', [a0].[AlertRule]) > 0)) AND [v].[VehiculeID] IS NOT NULL) AND ([v].[RowEnabled] = CAST(1 AS bit))) AND ([v].[CompanyID] = @__companyId_0)) AND (([v].[GpsBoxType] = 29) OR ([v].[GpsBoxType] = 12))) AND (CASE
WHEN [a].[AckTime] IS NOT NULL THEN CAST(1 AS bit)
ELSE CAST(0 AS bit)
END = @__acknwoledged_1)
ORDER BY [a].[AlertDateTime] DESC
OFFSET @__p_2 ROWS FETCH NEXT @__p_3 ROWS ONLY
) AS [t]
LEFT JOIN [Vehicule] AS [v0] ON [t].[TrackedVehiculeID] = [v0].[VehiculeID]
LEFT JOIN [SafeProtect].[Registration] AS [r] ON [v0].[VehiculeID] = [r].[VehiculeID]
LEFT JOIN [SafeProtect].[Device] AS [d] ON [v0].[VehiculeID] = [d].[VehiculeID]
LEFT JOIN [GpsBoxSubType] AS [g] ON ([v0].[GpsBoxType] = [g].[GpsBoxTypeID]) AND ([v0].[GpsBoxSubType] = [g].[GpsBoxSubTypeID])
LEFT JOIN [cst].[Icon] AS [i] ON [g].[IconId] = [i].[IconId]
LEFT JOIN [GpsBoxType] AS [g0] ON [v0].[GpsBoxType] = [g0].[GpsBoxTypeId]
LEFT JOIN [cst].[Icon] AS [i0] ON [g0].[IconId] = [i0].[IconId]
LEFT JOIN [VehiculeGpsBoxInfo] AS [v1] ON [v0].[VehiculeID] = [v1].[VehiculeID]
LEFT JOIN [GpsBoxConfiguration] AS [g1] ON [v1].[VehiculeConfiguration] = [g1].[ConfigurationID]
INNER JOIN [Alert] AS [a1] ON [t].[AlertID] = [a1].[AlertID]
ORDER BY [t].[AlertDateTime] DESC',N'@__companyId_0 int,@__acknwoledged_1 bit,@__p_2 int,@__p_3 int',@__companyId_0=1013,@__acknwoledged_1=0,@__p_2=0,@__p_3=999
TSQL 2:
exec sp_executesql N'SELECT [t].[AlertDetailID], [t].[AckMachineName], [t].[AckPhoneNumber], [t].[AckTime], [t].[AckUserName], [t].[AlertDateTime], [t].[AlertID], [t].[AlertedCorridorId], [t].[AlertedItemId],
[t].[AssociatedVehiculeID], [t].[AssociatedVehiculeUserID], [t].[Context], [t].[CustomID], [t].[CustomInfo], [t].[DbInsertTime], [t].[IsFleetAlert], [t].[MessageStatus], [t].[ReceivedTime], [t].[RowVersion],
[t].[SafeProtectCustomInfo], [t].[TrackedVehiculeID], [t].[TrackedVehiculeUserID], [v0].[VehiculeID], [v0].[Address], [v0].[AddressProtocol], [v0].[BoardID], [v0].[Category], [v0].[CompanyID], [v0].[CustomId], [v0].[DbInsertTime], [v0].[Description], [v0].[GpsBoxSubType], [v0].[GpsBoxTrackingDelay], [v0].[GpsBoxType], [v0].[HardwareID], [v0].[HasGeoWorker], [v0].[IconID], [v0].[Name], [v0].[PhoneNumber], [v0].[RowEnabled], [v0].[RowVersion], [v0].[VehiculeUserID], [r].[VehiculeID], [r].[CompanyId], [r].[LatestRegistrationStatusChangeDate], [r].[RegistrationMailSent], [r].[RegistrationStatusId], [d].[VehiculeID], [d].[AppVersion], [d].[Brand], [d].[Details], [d].[Imei], [d].[Model], [d].[Name], [d].[OsVersion], [d].[RowVersion], [g].[GpsBoxTypeID], [g].[GpsBoxSubTypeID], [g].[IconId], [g].[Name], [i].[IconId], [i].[Category], [i].[FileName], [i].[Icon], [g0].[GpsBoxTypeId], [g0].[Category], [g0].[IconId], [g0].[Name], [i0].[IconId], [i0].[Category], [i0].[FileName], [i0].[Icon], [v1].[VehiculeID], [v1].[BoardConfiguration], [v1].[BoardConnected], [v1].[BoardCurrentAddress], [v1].[BoardLastCommunicationTime], [v1].[Connected], [v1].[CurrentAddress], [v1].[CurrentDelay], [v1].[FuelConsumptionEstimationTime], [v1].[HeartBeatPeriod], [v1].[InsureCoherence], [v1].[KillHedgehog], [v1].[LastCommunicationTime], [v1].[LastConnexionTime], [v1].[LastParkMileage], [v1].[LastParkMileageTime], [v1].[NormalTrackingMode], [v1].[SmartModeDelay], [v1].[SmartModeDistance], [v1].[TimeModeDelay], [v1].[UpdateStartTime], [v1].[UpdateStatus], [v1].[VehiculeConfiguration], [g1].[ConfigurationID], [g1].[ConfigName], [g1].[Firmware], [g1].[GpsBoxType], [g1].[Master], [g1].[MasterName], [g1].[SimContract], [g1].[System], [a1].[AlertID], [a1].[AlertEnabled], [a1].[AlertRule], [a1].[Always], [a1].[CalendarID], [a1].[Category], [a1].[CompanyID], [a1].[Description], [a1].[DisplayName], [a1].[RowEnabled], [a1].[RowVersion], [a1].[Shared], [a1].[UserID]
FROM (
SELECT [a].[AlertDetailID], [a].[AckMachineName], [a].[AckPhoneNumber], [a].[AckTime], [a].[AckUserName], [a].[AlertDateTime], [a].[AlertID], [a].[AlertedCorridorId], [a].[AlertedItemId], [a].[AssociatedVehiculeID], [a].[AssociatedVehiculeUserID], [a].[Context], [a].[CustomID], [a].[CustomInfo], [a].[DbInsertTime], [a].[IsFleetAlert], [a].[MessageStatus], [a].[ReceivedTime], [a].[RowVersion], [a].[SafeProtectCustomInfo], [a].[TrackedVehiculeID], [a].[TrackedVehiculeUserID], [a0].[AlertID] AS [AlertID0]
FROM [AlertDetail] AS [a]
INNER JOIN [Alert] AS [a0] ON [a].[AlertID] = [a0].[AlertID]
LEFT JOIN [Vehicule] AS [v] ON [a].[TrackedVehiculeID] = [v].[VehiculeID]
WHERE ((((((([a0].[CompanyID] = @__companyId_0) AND (CHARINDEX(N''SerializableContextAlert'', [a0].[AlertRule]) > 0)) AND ([a].[AlertDateTime] >= @__startDate_1)) AND ([a].[AlertDateTime] <= @__endDate_2)) AND [v].[VehiculeID] IS NOT NULL) AND ([v].[CompanyID] = @__companyId_0)) AND ([v].[RowEnabled] = CAST(1 AS bit))) AND (([v].[GpsBoxType] = 29) OR ([v].[GpsBoxType] = 12))
ORDER BY (SELECT 1)
OFFSET @__p_3 ROWS FETCH NEXT @__p_4 ROWS ONLY
) AS [t]
LEFT JOIN [Vehicule] AS [v0] ON [t].[TrackedVehiculeID] = [v0].[VehiculeID]
LEFT JOIN [SafeProtect].[Registration] AS [r] ON [v0].[VehiculeID] = [r].[VehiculeID]
LEFT JOIN [SafeProtect].[Device] AS [d] ON [v0].[VehiculeID] = [d].[VehiculeID]
LEFT JOIN [GpsBoxSubType] AS [g] ON ([v0].[GpsBoxType] = [g].[GpsBoxTypeID]) AND ([v0].[GpsBoxSubType] = [g].[GpsBoxSubTypeID])
LEFT JOIN [cst].[Icon] AS [i] ON [g].[IconId] = [i].[IconId]
LEFT JOIN [GpsBoxType] AS [g0] ON [v0].[GpsBoxType] = [g0].[GpsBoxTypeId]
LEFT JOIN [cst].[Icon] AS [i0] ON [g0].[IconId] = [i0].[IconId]
LEFT JOIN [VehiculeGpsBoxInfo] AS [v1] ON [v0].[VehiculeID] = [v1].[VehiculeID]
LEFT JOIN [GpsBoxConfiguration] AS [g1] ON [v1].[VehiculeConfiguration] = [g1].[ConfigurationID]
INNER JOIN [Alert] AS [a1] ON [t].[AlertID] = [a1].[AlertID]
ORDER BY [t].[AlertDateTime] DESC',N'@__companyId_0 int,@__startDate_1 datetime,@__endDate_2 datetime,@__p_3 int,@__p_4 int',@__companyId_0=1013,@__startDate_1='2020-09-07 21:59:59',@__endDate_2='2020-09-08 21:59:59',@__p_3=0,@__p_4=999
It took me some time to spot a tiny by impacting difference between the two queries.
If you look carefully, the first subquery contains an ORDER BY which is what I want:
ORDER BY [a].[AlertDateTime] DESC
The second linq query, despite using the same OderBy is translated to a TSQL subquery that contains
ORDER BY (SELECT 1)
That sounds crasy to me.
I hope that someone could explain what is going on.
It has nothing to do with filtering (Where), but the order of ordering (OrderBy{Descending}) and row limiting (Skip / Take) operators in LINQ query.
The fake ORDER BY (SELECT 1) definitely is generated for the LINQ query having the following fragment
.Skip(0)
.Take(1000)
.OrderByDescending(ad => ad.AlertDateTime)
// .Skip(skip)
// .Take(take)
As you can see, here the row limiting operators are applied before the ordering. Since SqlServer OFFSET FETCH SQL construct requires ORDER BY, EF Core translator inserts fake one (it would have been better to throw exception or let the database do that, but that's another story. Actually before EF Core 3.0 there was CoreEventId.RowLimitingOperationWithoutOrderByWarning which could be configured to throw, but for some unknown reason it has been obsoleted and has no effect in EF Core 3.x).
The correct place of row limiting of course is after ordering, as it is in the commented out code. So simply remove the (temporary I guess) Skip(0).Take(1000) and uncomment the .Skip(skip).Take(take). Problem solved.