Motivation

One of the main queries in my projects is to select multiple database records having a collection of some kind of identifiers. Most of the time it works very well using the method Contains().

List<Guid> ids = ...;

var customers = dbContext.Customers.Where(c => ids.Contains(c.Id));

The query above is translated by the Entity Framework Core (EF Core) to IN clause: WHERE Id IN ('id_1', ... 'id_n')

Most of the time using an IN clause is not a problem but when having a big collection of ids then the transmission of a huge SQL statement and parsing of it will have negative impact on the performance. This is just one use case of many.

In the second use case we have a collection of tuples, say a collection of (customerId, productId). In this case the method Contains can't help us, we have to use a JOIN but joining a query with an in-memory-collection (like a List<>) will lead to downloading all data from database because the JOIN will be done in memory.

List<(Guid customerId, Guid productId)> tuples = ...;

// can cause performance issues
var orderItems = ctx.OrderItems.Join(tuples,
                                     i => new { i.Order.CustomerId, i.ProductId },
                                     t => new {
                                                CustomerId = t.customerId, 
                                                ProductId = t.productId
                                              },
                                     (i, t) => i);

In both cases it would be better to Bulk Insert the ids into a temp table to perform a JOIN with it.

Solution

Use the following links to try out the demos:

Before using a temp table with EF Core we have to introduce it to our DbContext using the extension method ConfigureTempTable with desired column type(s).
The extension method returns an instance of QueryTypeBuilder for further configuration but most of the time there is no need for that.

Remarks: For different column types we have to make multiple calls of ConfigureTempTable. For example one temp table with Guid, the other with Guid? (i.e. nullable Guid), the third with a string, etc.

public class DemoDbContext : DbContext
{
   ...

   protected override void OnModelCreating(ModelBuilder modelBuilder)
   {
      ...

      modelBuilder.ConfigureTempTable<Guid>();        // for use case 1
      modelBuilder.ConfigureTempTable<Guid, Guid>();  // for use case 2
   }
}

After introduction of the temp tables we now may use them as often as needed.

The solution for 1st use case:

List<Guid> ids = ...;

var tempTableQuery = await ctx.BulkInsertTempTableAsync(ids);
var customers = await ctx.Customers
                         .Join(tempTableQuery, c => c.Id, t => t.Column1, (c, t) => c)
                         .ToListAsync();

Remarks: Each call of BulkInsertTempTableAsync is using a different temp table name by default so there are no conflicts when using this method multiple times. For more control over temp table creation and the bulk insert via SqlBulkCopy provide an instance of type SqlBulkInsertOptions as the 2nd parameter.

The solution for the 2nd use case:

List<(Guid customerId, Guid productId)> tuples = ...;

var tempTableQuery = await ctx.BulkInsertTempTableAsync(tuples);
var orderItems = await ctx.OrderItems.Join(tempTableQuery,
                                           i => new { i.Order.CustomerId, i.ProductId },
                                           t => new {
                                                      CustomerId = t.Column1, 
                                                      ProductId = t.Column2
                                                    },
                                           (i, t) => i)
                          .ToListAsync();

The NuGet package will be released in the near future, at the moment I'm gathering feedback.

Related Articles

entity framework
Entity Framework Core 3.0 - "Hidden" GROUP BY Capabilities (Part 2)
In the previous blog post we used a navigational property to work around the limitations of the extension method . The problem is, there is not always such a property we can use for grouping. Especially, when following domain driven design practices, a bidirectional navigation is…
Pawel Gerr
entity framework
Entity Framework Core 3.0 - "Hidden" GROUP BY Capabilities (Part 1)
With Entity Framework Core 3.0 (EF) the internal implementation of the LINQ query translation has been changed a lot. So, some queries that didn't work previously are working now and some that worked are not working anymore. :) The LINQ extension method is a special one…
Pawel Gerr
entity framework
Entity Framework Core - Getting more Deadlock Information with named Transactions
Whether in an application in production or in integration tests on CI, the deadlocks are virtually unavoidable. The behavior of a deadlock depends on the database (MS SQL Server, MySQL, etc)  and the isolation level (like Snapshot Isolation). Some of the databases are blocking…
Pawel Gerr
entity framework
Entity Framework Core - Beware of Lifetime Extension of TransactionScopes using C# 8.0
In the blog post Entity Framework Core: Use TransactionScope with Caution! we saw a few cases we have to be aware of when using . Here is another one that can lead to an exception when using the new C# 8.0 feature: . First, we look at a code snippet without C# 8.0 feature: Which…
Pawel Gerr