Is “N+1 Queries” Still A Performance Issue In Entity Framework Core 3?

In a previous post we saw that EF 2.1 is highly susceptible to the N+1 queries problem. After the release of a new version of Entity Framework Core (EF) the first question coming to mind is: "Is it still a big issue in EF 3.1?" And if the answer is no, is there anything else we have to be aware of?

In this article:

pg
Pawel Gerr is architect consultant at Thinktecture. He focuses on backends with .NET Core and knows Entity Framework inside out.

Examples on Github:

What is the "N+1 Queries Problem"?

The N+1 Queries Problem is an O/R mapper specific issue. It may occur when trying to load data from a table (like ProductGroups) along with its related data (like Products) having a one-to-many relationship whith the related data is on the many-side.

An example of a LINQ query that may lead to N+1 Queries with some O/R mappers:

				
					Context.ProductGroups
          .Select(g => new
          {
            ProductGroup = g,
            Products = g.Products
          })
          .ToList();
				
			

In this example an O/R mapper may execute 1 SQL statement to load N ProductGroups and after that N SQL statements to load Products for each ProductGroup separately. Hence the name: N+1 Queries.

Let’s revisit the previous blog post’s queries to see whether EF 3.1 still suffers from the N+1 Queries Problem as earlier versions of EF did.

Fetching a collection navigational property

Fetching a collection navigational property in EF 2.1 leads to 2 queries, one for ProductGroups and the other for the Products.

				
					var groups = Context.ProductGroups
          .Where(g => g.Name.Contains("Group"))
          .Select(g => new
          {
            ProductGroup = g,
            Products = g.Products.Where(p => p.Name.Contains("1")).ToList()
          })
          .ToList();
				
			

With EF 3.1 we get just 1 SQL statement that fetches both, the ProductGroups and Products.

				
					SELECT
    [p].[Id], [p].[Name], [p].[RowVersion],
    [t].[Id], [t].[GroupId], [t].[Name], [t].[RowVersion]
FROM [ProductGroups] AS [p]
LEFT JOIN 
(
    SELECT [p0].[Id], [p0].[GroupId], [p0].[Name], [p0].[RowVersion]
    FROM [Products] AS [p0]
    WHERE CHARINDEX(N'1', [p0].[Name]) > 0
) AS [t] ON [p].[Id] = [t].[GroupId]
WHERE CHARINDEX(N'Group', [p].[Name]) > 0
ORDER BY [p].[Id], [t].[Id]
				
			

This example proves that neither EF 2.1 nor 3.1 causes the N+1 Queries Problem, yet there are some differences:

  • EF 2.1 requires ToList() when fetching Products, EF 3.1 does not
  • EF 2.1 executes 2 SQL statements, EF 3.1 just 1

Using "FirstOrDefault" inside "Select"

When selecting the first Product for each ProductGroup using FirstOrDefault() EF 2.1 executes 1 SQL statement for loading N ProductGroups and N SQL statements for loading the Products.

				
					var groups = Context.ProductGroups
          .Where(g => g.Name.Contains("Group"))
          .Select(g => new
          {
            ProductGroup = g,
            Product = g.Products.FirstOrDefault()
          })
          .ToList();
				
			

As in the previous section, EF 3.1 executes just 1 SQL statement for loading all requested data.

				
					SELECT
    [p].[Id], [p].[Name], [p].[RowVersion],
    [t0].[Id], [t0].[GroupId], [t0].[Name], [t0].[RowVersion]
FROM [ProductGroups] AS [p]
LEFT JOIN
(
    SELECT [t].[Id], [t].[GroupId], [t].[Name], [t].[RowVersion]
    FROM
    (
        SELECT
            [p0].[Id], [p0].[GroupId], [p0].[Name], [p0].[RowVersion],
            ROW_NUMBER() OVER(PARTITION BY [p0].[GroupId] ORDER BY [p0].[Id]) AS [row]
        FROM [Products] AS [p0]
    ) AS [t]
    WHERE [t].[row] <= 1
) AS [t0] ON [p].[Id] = [t0].[GroupId]
WHERE CHARINDEX(N'Group', [p].[Name]) > 0
				
			

With respect to the N+1 Queries Problem, the new behavior of the SQL statement generator is an improvement. We can now use FirstOrDefault without producing dozens or even hundreds of SQL statements.

Usage of “FirstOrDefault” along with “GroupBy”

The usage of FirstOrDefault along with GroupBy in EF 2.1 works, kind of… The LINQ query below leads to client-side evaluation, which again leads to a massive performance loss due to the fact that the whole table Products is fetched into memory.

				
					var firstProducts = Context.Products
                  .GroupBy(p => p.GroupId)
                  .Select(g => g.FirstOrDefault())
                  .ToList();
				
			

As you may have read in my blog post Entity Framework Core 3.0 – “Hidden” GROUP BY Capabilities (Part 1)GroupBy has very limited support in EF 3.0/3.1. With the query above we get an InvalidOperationException.

Lazy Loading

Nothing has changed concerning Lazy Loading. Whether EF 2.1 or EF 3.1, we still have to be careful when using this feature.
Here is an example: for 100 Products and 10 ProductGroups the following LINQ query executes 11 SQL statements: 1 for loading the Products and 10 for loading the ProductGroups.

				
					var lookup = Context.Products
                    .ToLookup(p => p.Group.Name);

				
			

Summary

Personally, I like the behavior of the SQL generator in EF 3.1 better than in EF 2.1 because it gives me more control over SQL statement generation. Previously, if a LINQ query is not translatable to SQL, EF switched to client-side evaluation which could lead to bad performance if it was overlooked. Now, we get an exception and have to do something about it.
Let’ get back to my opening questions:

  • Is N+1 Queries Problem still a big issue in EF 3.1? – In short no. Sure, Lazy Loading may lead to more executed SQL statements than expected but that is the price we have to pay in order to use this feature.
  • Did the new behavior introduced new issues we have to be aware of? – Unfortunately yes, this new behavior inevitably causes the so-called Cartesian Explosion Problem
Free
Newsletter

Current articles, screencasts and interviews by our experts

Don’t miss any content on Angular, .NET Core, Blazor, Azure, and Kubernetes and sign up for our free monthly dev newsletter.

EN Newsletter Anmeldung (#7)
Related Articles
AI
sg
One of the more pragmatic ways to get going on the current AI hype, and to get some value out of it, is by leveraging semantic search. This is, in itself, a relatively simple concept: You have a bunch of documents and want to find the correct one based on a given query. The semantic part now allows you to find the correct document based on the meaning of its contents, in contrast to simply finding words or parts of words in it like we usually do with lexical search. In our last projects, we gathered some experience with search bots, and with this article, I'd love to share our insights with you.
17.05.2024
Angular
sl_300x300
If you previously wanted to integrate view transitions into your Angular application, this was only possible in a very cumbersome way that needed a lot of detailed knowledge about Angular internals. Now, Angular 17 introduced a feature to integrate the View Transition API with the router. In this two-part series, we will look at how to leverage the feature for route transitions and how we could use it for single-page animations.
15.04.2024
.NET
kp_300x300
.NET 8 brings Native AOT to ASP.NET Core, but many frameworks and libraries rely on unbound reflection internally and thus cannot support this scenario yet. This is true for ORMs, too: EF Core and Dapper will only bring full support for Native AOT in later releases. In this post, we will implement a database access layer with Sessions using the Humble Object pattern to get a similar developer experience. We will use Npgsql as a plain ADO.NET provider targeting PostgreSQL.
15.11.2023