Incremental Roslyn Source Generators In .NET 6: Testing Source Generators, Analyzers & Code Fixes – Part 3

This article is the 3rd part of a series about Roslyn Source Generators & co. In the 1st article, of this series, we built an Incremental Source Generator to generate a new property Items of a Smart Enum. In the 2nd article, we added a Roslyn Analyzer and a Code Fix to prevent common mistake(s) and to help out the developers when using this Source Generator. All the code we have written so far was tested manually by executing the code and looking at the outcome. It is time to implement some automated tests to ensure the correct behavior.

In diesem Artikel:

Incremental Roslyn Source Generators In .NET 6: Testing Source Generators, Analyzers & Code Fixes – Part 3
Pawel Gerr ist Architekt und Consultant bei Thinktecture. Er hat sich auf .NET Core Backends spezialisiert und kennt Entity Framework von vorne bis hinten.

More information about the Smart Enums, the source code, and the documentation of Microsoft.CodeAnalysis.Testing can be found on GitHub:

Different Testing Approaches

When it comes to testing of Roslyn Source Generators, Analyzers, and Code Fixes then there are two different kinds of tests.

  • One is for testing the behavior of the generated code. In our case, it would be a test for the existence of the property Items and whether it returns all defined items.
  • The other kind of test verifies the emitted errors and warnings, and the generated code, not just the one produced by the Source Generator but also by the Code Fix. With such tests, we are able to tests all pieces we implemented in previous articles.

Preparation

Fist, create a new library project, DemoTests, which references both the DemoLibrary and the DemoSourceGenerator. Please note, that the reference to DemoSourceGenerator in DemoTests.csproj is missing the attribute ReferenceOutputAssembly="false" this time. For testing, we want both, the generated code, i.e. the standard functionality of a Source Generator, and a direct access to the classes DemoSourceGeneratorDemoAnalyzer and DemoCodeFixProvider.

We want to test 2 TargetFrameworks: .NET 5 and .NET 6.

				
					<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFrameworks>net5.0;net6.0</TargetFrameworks>
    </PropertyGroup>

    <ItemGroup>
        <ProjectReference Include="..\DemoLibrary\DemoLibrary.csproj" />
        <ProjectReference Include="..\DemoSourceGenerator\DemoSourceGenerator.csproj" 
                          OutputItemType="Analyzer" />
    </ItemGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Analyzer.Testing.XUnit"
                          Version="1.1.0" />
        <PackageReference Include="Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.XUnit"
                          Version="1.1.0" />
        <PackageReference Include="Microsoft.CodeAnalysis.CSharp.CodeRefactoring.Testing.XUnit"
                          Version="1.1.0" />
        <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces"
                          Version="4.0.1" />

        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.0.0" />
        <PackageReference Include="FluentAssertions" Version="6.3.0" />
        <PackageReference Include="xunit" Version="2.4.1" />
        <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3" />
    </ItemGroup>

</Project>
				
			

In the following tests, we will be using XUnit and Fluent Assertions.

Testing Behavior of Generated Code

In order to test the expected behavior, we need (at least) one enumeration. Copy the ProductCategory from the project DemoConsoleApplication to DemoTests or create a new one. These kinds of tests are very common, so one positive test should be enough as a starting point. In real scenarios, we should make negative tests and tests for the edge cases as well. An edge case would be an enumeration with no items.

Create a new class ItemsPropertyTests in the project DemoTests with the following content and let the test run.

				
					using FluentAssertions;
using Xunit;

namespace DemoTests;

public class ItemsPropertyTests
{
   [Fact]
   public void Should_return_all_known_items()
   {
      ProductCategory.Items.Should().HaveCount(2)
                     .And.BeEquivalentTo(new[]
                                         {
                                            ProductCategory.Fruits,
                                            ProductCategory.Dairy
                                         });
   }
}
				
			

The test should be green.

Testing Roslyn Source Generator

Create a new class DemoSourceGeneratorTests in the project DemoTests for testing the output of the DemoSourceGenerator. First, we make a helper method that executes the Source Generator directly and returns the generated output.

				
					using FluentAssertions;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Xunit;

namespace DemoTests;

public class DemoSourceGeneratorTests
{
   private static string? GetGeneratedOutput(string sourceCode)
   {
      var syntaxTree = CSharpSyntaxTree.ParseText(sourceCode);
      var references = AppDomain.CurrentDomain.GetAssemblies()
                                .Where(assembly => !assembly.IsDynamic)
                                .Select(assembly => MetadataReference
                                                    .CreateFromFile(assembly.Location))
                                .Cast<MetadataReference>();

      var compilation = CSharpCompilation.Create("SourceGeneratorTests",
                    new[] { syntaxTree },
                    references,
                    new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

      // Source Generator to test 
      var generator = new DemoSourceGenerator.DemoSourceGenerator();

      CSharpGeneratorDriver.Create(generator)
                           .RunGeneratorsAndUpdateCompilation(compilation,
                                                              out var outputCompilation,
                                                              out var diagnostics);

      // optional
      diagnostics.Where(d => d.Severity == DiagnosticSeverity.Error)
                 .Should().BeEmpty();

      return outputCompilation.SyntaxTrees.Skip(1).LastOrDefault()?.ToString();
   }
}
				
			

The method GetGeneratedOutput parses and compiles the provided sourceCode and executes the DemoSourceGenerator. The output generated by the Source Generator is the last one in the collection SyntaxTrees. Please note that the generator doesn’t produce any output under certain conditions. That’s why we Skip the first syntax tree which is the provided sourceCode.

With the helper method GetGeneratedOutput, the actual tests are reduced to the comparison of strings. A positive test looks like as following:

				
					   [Fact]
   public void Should_generate_Items_property_with_2_items()
   {
      var input = @"
using DemoLibrary;

namespace DemoTests
{
   [EnumGeneration]
   public partial class ProductCategory
   {
      public static readonly ProductCategory Fruits = new(""Fruits"");
      public static readonly ProductCategory Dairy = new(""Dairy"");

      public string Name { get; }

      private ProductCategory(string name)
      {
         Name = name;
      }
   }
}
";
      GetGeneratedOutput(input)
         .Should().Be(@"// <auto-generated />

using System.Collections.Generic;

namespace DemoTests
{
   partial class ProductCategory
   {
      private static IReadOnlyList<ProductCategory> _items;
      public static IReadOnlyList<ProductCategory> Items => _items ??= GetItems();

      private static IReadOnlyList<ProductCategory> GetItems()
      {
         return new[] { Fruits, Dairy };
      }
   }
}
");
   }

				
			

Testing Roslyn Analyzers and Code Fixes

For testing the Analyzers and Code Fixes, it is recommended to create a few helper classes. Depending on whether we have Code Fixes or not, we need either an AnalyzerVerifier or an AnalyzerAndCodeFixVerifier. In this article, we will create both helper classes but just use the 2nd one.

Here are the contents of the class AnalyzerVerifier for the sake of completeness.

				
					using DemoLibrary;
using Microsoft.CodeAnalysis.CSharp.Testing;
using Microsoft.CodeAnalysis.Diagnostics;
using Microsoft.CodeAnalysis.Testing;
using Microsoft.CodeAnalysis.Testing.Verifiers;

namespace DemoTests.Verifiers;

public static class AnalyzerVerifier<TAnalyzer>
   where TAnalyzer : DiagnosticAnalyzer, new()
{
   public static DiagnosticResult Diagnostic(string diagnosticId)
   {
      return CSharpAnalyzerVerifier<TAnalyzer, XUnitVerifier>.Diagnostic(diagnosticId);
   }

   public static async Task VerifyAnalyzerAsync(
      string source,
      params DiagnosticResult[] expected)
   {
      var test = new AnalyzerTest(source, expected);
      await test.RunAsync(CancellationToken.None);
   }

   private class AnalyzerTest : CSharpAnalyzerTest<TAnalyzer, XUnitVerifier>
   {
      public AnalyzerTest(
         string source,
         params DiagnosticResult[] expected)
      {
         TestCode = source;
         ExpectedDiagnostics.AddRange(expected);
#if NET6_0
         ReferenceAssemblies = new ReferenceAssemblies(
            "net6.0",
            new PackageIdentity("Microsoft.NETCore.App.Ref", "6.0.0"), 
            Path.Combine("ref", "net6.0"));
#else
         ReferenceAssemblies = ReferenceAssemblies.Net.Net50;
#endif

         TestState.AdditionalReferences.Add(typeof(EnumGenerationAttribute).Assembly);
      }
   }
}
				
			

What we need for this demo is the AnalyzerAndCodeFixVerifier, so create a new class with the following content:

				
					using DemoLibrary;
using Microsoft.CodeAnalysis.CodeFixes;
using Microsoft.CodeAnalysis.CSharp.Testing;
using Microsoft.CodeAnalysis.Diagnostics;
using Microsoft.CodeAnalysis.Testing;
using Microsoft.CodeAnalysis.Testing.Verifiers;

namespace DemoTests.Verifiers;

public static class AnalyzerAndCodeFixVerifier<TAnalyzer, TCodeFix>
   where TAnalyzer : DiagnosticAnalyzer, new()
   where TCodeFix : CodeFixProvider, new()
{
   public static DiagnosticResult Diagnostic(string diagnosticId)
   {
      return CSharpCodeFixVerifier<TAnalyzer, TCodeFix, XUnitVerifier>
                .Diagnostic(diagnosticId);
   }

   public static async Task VerifyCodeFixAsync(
      string source,
      string fixedSource,
      params DiagnosticResult[] expected)
   {
      var test = new CodeFixTest(source, fixedSource, expected);
      await test.RunAsync(CancellationToken.None);
   }

   private class CodeFixTest : CSharpCodeFixTest<TAnalyzer, TCodeFix, XUnitVerifier>
   {
      public CodeFixTest(
         string source,
         string fixedSource,
         params DiagnosticResult[] expected)
      {
         TestCode = source;
         FixedCode = fixedSource;
         ExpectedDiagnostics.AddRange(expected);
#if NET6_0
         ReferenceAssemblies = new ReferenceAssemblies(
            "net6.0",
            new PackageIdentity("Microsoft.NETCore.App.Ref", "6.0.0"),
            Path.Combine("ref", "net6.0"));
#else
         ReferenceAssemblies = ReferenceAssemblies.Net.Net50;
#endif

         TestState.AdditionalReferences.Add(typeof(EnumGenerationAttribute).Assembly);
      }
   }
}
				
			

The only thing worth mentioning is the last line where we add the assembly reference DemoLibrary to the testing class CSharpCodeFixTest, otherwise the EnumGenerationAttribute will not be resolved, and the test fails.

Having the helper classes, the actual tests become much easier to write.
Create a new test class DemoAnalyzerAndCodeFixTests to test our DemoAnalyzer and the DemoCodeFixProvider for correct handling of non-partial Smart Enums.

				
					using DemoSourceGenerator;
using Xunit;
using Verifier = DemoTests.Verifiers.AnalyzerAndCodeFixVerifier<
   DemoSourceGenerator.DemoAnalyzer,
   DemoSourceGenerator.DemoCodeFixProvider>;

namespace DemoTests;

public class DemoAnalyzerAndCodeFixTests
{
   [Fact]
   public async Task Should_trigger_on_non_partial_class()
   {
      var input = @"
using DemoLibrary;

namespace DemoTests
{
   [EnumGeneration]
   public class {|#0:ProductCategory|}
   {
   }
}";

      var expectedOutput = @"
using DemoLibrary;

namespace DemoTests
{
   [EnumGeneration]
   public partial class ProductCategory
   {
   }
}";

      var expectedError = Verifier.Diagnostic(DemoDiagnosticsDescriptors.EnumerationMustBePartial.Id)
                                  .WithLocation(0)
                                  .WithArguments("ProductCategory");
      await Verifier.VerifyCodeFixAsync(input, expectedOutput, expectedError);
   }
}
				
			

The magic lies in the using Verifier = ... at the top of the file and Verifier.VerifyCodeFixAsync at the bottom, which get the input source code and expects the expectedOutput along with the expectedError. The rather strange characters {|#0: and |} is markup, so the Verifier knows what location the compilation error will be pointing to.

We were talking about location in the 2nd article. Search for the text classDeclaration.Identifier to get more information on that.

Let’s run all three tests, which should be green.

Summary

This article concludes the introductory series about Roslyn Source Generators, Analyzer, and Code Fixes. As a matter of fact, writing tests turned out to be the easiest and fastest task compared to the previous two articles.

Kostenloser
Newsletter

Aktuelle Artikel, Screencasts, Webinare und Interviews unserer Experten für Sie

Verpassen Sie keine Inhalte zu Angular, .NET Core, Blazor, Azure und Kubernetes und melden Sie sich zu unserem kostenlosen monatlichen Dev-Newsletter an.

Diese Artikel könnten Sie interessieren
.NET
Incremental Roslyn Source Generators in .NET 6: Adapt Code Generation Based on Project Dependencies – Part 5

Incremental Roslyn Source Generators in .NET 6: Adapt Code Generation Based on Project Dependencies – Part 5

The Roslyn Source Generator, implemented in the previous articles of the series, emits some C# code without looking at the dependencies of the current .NET (Core) project. In this article our DemoSourceGenerator should implement a JsonConverter, but only if the corresponding library (e.g. Newtonsoft.Json) is referenced by the project.
08.07.2022
Unterschiede
.NET
Blazor WebAssembly vs. Blazor Server – Welche Unterschiede gibt es und wann wähle ich was?

Blazor WebAssembly vs. Blazor Server – Welche Unterschiede gibt es und wann wähle ich was?

Das Blazor Framework von Microsoft gibt es inzwischen in drei "Geschmacksrichtungen". Die erste ist Blazor WebAssembly, die zweite Blazor Server, und zu guter Letzt gibt es noch Blazor Hybrid. In diesem Artikel wollen wir uns die zwei "echten", also Browser-basierten, Web-Anwendungs-Szenarien WebAssembly und Server anschauen.
04.07.2022
Three different textured walls
.NET
Dependency Injection Scopes in Blazor

Dependency Injection Scopes in Blazor

The dependency injection system is a big part of how modern ASP.NET Core works internally: It provides a flexible solution for developers to structure their projects, decouple their dependencies, and control the lifetimes of the components within an application. In Blazor - a new part of ASP.NET Core - however, the DI system feels a bit odd, and things seem to work a bit differently than expected. This article will explain why this is not only a feeling but indeed the case in the first place and how to handle the differences in order to not run into problems later on.
31.05.2022
.NET
Asynchrone Operationen: Blazor WebAssembly für Angular-Entwickler – Teil 5 [Screencast]

Asynchrone Operationen: Blazor WebAssembly für Angular-Entwickler – Teil 5 [Screencast]

Eine Webanwendung will natürlich auch mit Daten gefüttert werden. Doch diese müssen irgendwo her kommen. Nichts liegt näher als diese von einer Web API zu laden. Dieser Screencast zeigt, wie asynchrone Operationen in Blazor funktionieren und welche gravierenden Unterschiede es zu Angular gibt.
26.05.2022
.NET
Typings: Blazor WebAssembly für Angular-Entwickler – Teil 4 [Screencast]

Typings: Blazor WebAssembly für Angular-Entwickler – Teil 4 [Screencast]

C# und TypeScript entstammen der Feder der selben Person. Doch sind sie deshalb auch gleich? In diesem Teil der Screencast-Serie erfahren Sie, wie mit Typen in den beiden Programmiersprachen verfahren wird und welche Unterschiede es gibt.
19.05.2022
.NET
Bindings: Blazor WebAssembly für Angular-Entwickler – Teil 3 [Screencast]

Bindings: Blazor WebAssembly für Angular-Entwickler – Teil 3 [Screencast]

Wer Komponenten einsetzt, steht früher oder später vor der Fragestellung, wie man Daten an die Komponente übergibt oder auf Ereignisse einer Komponente reagiert. In diesem Screencast wird gezeigt wie Bindings bei Komponenten funktionieren, also wie eine Komponente Daten von außerhalb benutzen und Rückmeldung bei Aktionen geben kann.
12.05.2022