Goodbye, NullReferenceException: Nullable Operators and References

In the previous post of this series, we covered two ideas to avoid the NullReferenceException: we should check for null before accessing the members of an object and check the input parameters of our methods.

Let’s see some new C# operators to simplify null checking and a new feature to better signal possible null references.

1. C# Nullable Operators

C# has three operators to simplify our null checks: ?., ??, and ??=. These operators don’t prevent us from having null in the first place, but they help us to easily write our null checks.

Without Nullable Operators

Let’s start with an example and refactor it to use these new operators.

string path = null;

if (HttpContext.Current == null 
    || HttpContext.Current.Request == null 
    || HttpContext.Current.Request.ApplicationPath  == null)
{
    path = "/some-default-path-here";
}
else
{
    path = HttpContext.Current.Request.ApplicationPath;
}

If you have worked with the old ASP.NET framework, you might have done something like that. If not, don’t worry. We’re only accessing a property down in a property chain, but any of those properties could be null.

Notice that to defend against null, we checked if every property was null to “fail fast” and use a default value.

With Nullable Operators

Now, let’s use the new nullable operators instead,

var path = HttpContext.Current?.Request?.ApplicationPath
//                           ^^^      ^^^
              ?? "/some-default-path-here";
//           ^^^^

More compact, right?

With the null-conditional operator (?.), we access the property or method of an object only if the object isn’t null. Otherwise, the entire expression evaluates to null. For our example, we retrieve ApplicationPath only if Request isn’t null and Current isn’t null, and HttpContext isn’t null.

Then, with the null-coalescing operator (??), we evaluate an alternative expression if the one on the left of the ?? operator isn’t null. For our example, if any of the properties in the chain to retrieve ApplicationPath is null, the whole expression is null, and the path variable gets assigned to the default string.

With the null-coalescing assignment operator (??=), we assign a new value to a variable only if it’s null. We could also write our example like this,

var path = HttpContext.Current?.Request?.ApplicationPath;
path ??= "/some-default-path-here";
//  ^^^^
//
// The same as
//if (path == null)
//{
//    path = "/some-default-path-here"; 
//}

Notice how we refactored our original example to only two lines of code with these three new operators. Again we could still have null values. These nullable operators make our lives easier by simplifying our null checks.

2. C# Nullable References

To solve the NullReferenceException, we should check for null. We got that! But the thing is knowing when we should do it or not. That’s precisely what C# 8.0 solves with Nullable References.

With C# 8.0, all reference variables are non-nullable by default. Accessing the member of nullable references results in compiler warnings or errors.

This is a breaking change. Therefore we need to turn on this feature at the project level in our csproj files. Like these,

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <!--             ^^^^^^ -->
    <!-- We could use netcoreapp3.1|net5.0|net6.0|.net7.0 too -->
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <!--      ^^^^^^ -->
    <!-- We could use enable|disable|warning too -->
    <WarningsAsErrors>nullable</WarningsAsErrors>
    <!--              ^^^^^^^^ -->
    <!-- We can take the extreme route -->

  </PropertyGroup>

</Project>

To use Nullable References, we need to target .NET Core 3 and upward. And inside the Nullable node in our csproj files, we could use: enable, disable, or warning. Even, we can take the extreme route and consider all nullable warnings as compilation errors.

With Nullable References On

Let’s see what our motivating example looks like with Nullable References turned on,

string path = null;
//            ^^^^
// CS8600: Converting null literal or possible null value to non-nullable type

if (HttpContext.Current == null
    || HttpContext.Current.Request == null
    || HttpContext.Current.Request.ApplicationPath == null)
{
    path = "/some-default-path-here";
}
else
{
    path = HttpContext.Current.Request.ApplicationPath;
}

// This isn't the real HttpContext class...
// We're writing some dummy declarations to prove a point
public class HttpContext
{
    public static HttpContext Current;
    //                        ^^^^^^^^
    // CS8618: Non-nullable field 'Current' must contain
    // a non-nullable value when exiting constructor

    public HttpContext()
    //     ^^^^^^^^^^^
    // CS8618: Non-nullable field 'Current' must contain
    // a non-nullable value when exiting constructor
    {
    }

    public Request Request { get; set; }
}

public record Request(string ApplicationPath);

Notice we have a warning when initializing path to null. And another one in the declaration of our HttpContext class if we don’t initialize any non-nullable fields. That’s not the real HttpContext by the way but bear with me. Also, we don’t need to check for null when retrieving the ApplicationPath since all our references aren’t nullable by definition.

To declare a variable that can be null, we need to add to its type declaration a ?. In the same way, we have always declared nullable primitive types like int?.

Without Null Checks

Let’s change our example to have nullable references and no null checks,

// Notice the ? symbol here
//   vvv
string? path = HttpContext.Current.Request.ApplicationPath;
//             ^^^^^^^^^^^
// CS8602: Deference of a possibly null value

// This isn't the real HttpContext class...
// We're writing some dummy declarations to prove a point
public class HttpContext
{
    public static HttpContext? Current;
    //                      ^^^
    // Notice the ? symbol here

    public HttpContext()
    //     ^^^
    // No more warnings here
    {
    }

    public Request Request { get; set; }
}

public record Request(string? ApplicationPath);

Notice this time, when declaring the variable path, we have a warning because we’re accessing the Current property, which might be null. Also, notice we changed, inside the HttpContext class, the Current property to have the HttpContext? type (with a ? at the end of the type).

Now with Nullable References, we have a way of telling when we should check for null by looking at the signature of our methods.

Voilà! Those are the new C# operators to simplify our null checks. We said “new” operators, but we have had them since C# 6.0. And that’s how we can tell if our references can be null or not using Nullable References.

We have these nullable operators available even if we’re using the old .NET Framework. But, to use Nullable References, we should upgrade at least to .NET Core 3.0.

In the next post, we will cover the Option type as an alternative to null and how to avoid the NullReferenceException when working with LINQ.

Happy coding!

Goodbye, NullReferenceException: What it is and how to avoid it

If you’re here, I bet you already have found the exception message: “Object reference not set to an instance of an object.”

In this series of posts, let’s see some techniques to completely eliminate the NullReferenceException from our code. Let’s start by understanding when NullReferenceException is thrown and a strategy to fix it.

NullReferenceException is thrown when we access a property or method of an uninitialized variable of a reference type. The easier way to solve this exception is to check for null before accessing the members of an object. But C# has introduced new operators and features to avoid this exception.

Let’s write an example that throws NullReferenceException,

var movie = FindMovie();
Console.Write(movie.Name);
//            ^^^^^
// System.NullReferenceException: 'Object reference not set to an instance of an object.'
//
// movie was null.

Console.ReadKey();

static Movie FindMovie()
{
    // Imagine this is a database call that might
    // or might not return a movie
    return null;
    //     ^^^^
}

record Movie(string Name, int ReleaseYear, float Rating);

Notice we returned null from FindMovie(). That caused the NullReferenceException. But, it could be a method that accessed a database and didn’t find anything or an API controller method that didn’t receive a required input parameter.

In our last example, we got a NullReferenceException when we returned null from a method. But, we could also find this exception when we try to loop through a null list or array, for example.

Speaking of returning null, one way to prevent the NullReferenceException is to never pass null between objects. Instead of returning null, let’s use empty lists and strings, or the Null Object pattern. And let’s use intention-revealing defaults for that.

1. Don’t catch the NullReferenceException

To fix the NullReferenceException, we might be tempted to write a try/catch block around the code that throws it. But, let’s not catch the NullReferenceException.

Let me say that again: don’t throw or catch NullReferenceException.

By any means, please, let’s not write something like this,

try
{
    AMethodThatMightThrowNullReferenceException();
}
catch (NullReferenceException)
{
    // ...
    // Beep, beep, boop
    // Doing something with the exception here
}

A NullReferenceException is a symptom of an unhandled and unexpected scenario in our code, and catching it won’t handle that. A NullReferenceException is a developer’s mistake.

2. Check for null

The solution for the NullReferenceException is to check for nulls and defense against them.

Let’s fix our previous example by adding a null check. Like this,

var movie = FindMovie();
if (movie != null)
//  ^^^^^
{
    // We're safe here...
    //
    // No more System.NullReferenceException
    // at least due to a movie being null 
    Console.Write(movie.Name);
}

Console.ReadKey();

static Movie FindMovie()
{
    // Imagine this is a database call that might
    // or might not return a movie
    return null;
    //     ^^^^
}

record Movie(string Name, int ReleaseYear, float Rating);

Notice we checked if the movie variable wasn’t null. As simple as that.

Alternatives to check for null

The lack of options isn’t an excuse to not check for null. Since some recent C# versions, and thanks to pattern matching, we have a couple of new ways to check for null. We can use any of these:

• if (movie != null) { //... },
• if (movie is not null) { //... },
• if (movie is { }) { //... }, and,
• if (movie is object) { //... }

Some of those don’t look like C# anymore. I still prefer the old if (movie != null) .... Which one do you prefer?

3. Defense against null

If the solution to NullReferenceException were to simply check for null, that wouldn’t be the Billion-Dollar mistake. And I wouldn’t be writing this series. But the thing is knowing when a reference might be null or not and, therefore, when we should check for it. That’s when we should protect ourselves against null.

To protect against null, inside our methods, let’s check our input parameters and throw a more detailed exception. C# has an exception for missing parameters: ArgumentNullException.

The ArgumentNullException stack trace contains the name of the parameter that was null. That will help us to better troubleshoot our code. Often, the NullReferenceException stack trace doesn’t include what was null in the first place. Happy debugging time!

Let’s check our input parameters,

public void DoSomething(Movie movie)
{
    if (movie == null)
    {
        throw new ArgumentNullException(nameof(movie))
    }
    // Since C# 10, we can also write:
    // ArgumentNullException.ThrowIfNull(movie);

    // Beep, beep, boop
    // Doing something here...
}

Notice that instead of waiting for NullReferenceException, we proactively prevented it by checking for a required parameter and throwing a controlled ArgumentNullException.

Voilà! That’s the NullReferenceException and how to fix it by checking for null. Remember, we shouldn’t catch this exception but prevent and prepare for it.

Don’t miss the other posts in this series! In the next post, we cover how to use Nullable Operators and Nullable References to prevent the NullReferenceException. In future posts, we’re covering the Option type and LINQ XOrDefault methods and a design technique to encapsulate optional state.

Happy coding!

Monday Links: Monday Links: Passions, Estimates, and Methodologies

Welcome to the first Monday Links of 2023. These are five reads I found interesting last month. This time, I found software methodologies was a recurring theme these days.

Why I’m Glad I Lack Passion to BE a Programmer

After a couple of years of working as a software engineer, I started to embrace simplicity. Software exists to satisfy a business need. Perfect software only exists in books. That’s why I started to see the big goal and only use libraries/tools/concepts when there’s a compelling reason to do so. Not all applications need to use Domain-Driven Design with Event Sourcing.

I liked this one: My ideal for software development is to find the simplest solution to the practical problem. I’m not a passionate programmer anymore either.

Read full article

11 Laws of Software Estimation for Complex Work

I can relate to this story. It happens to a friend of a friend of mine. One day his CEO came saying he just closed a really big deal but he didn’t know what they were going to do. Arrrggg!

Apart from a relatable story, this post contains 11 rules about estimations. Like all estimations are simply guesses and by the time developers have enough information to give more accurate estimations, close to the end of a project, it’s already too late.

Read full article

User Interface Design: 5 Rules of Thumb

This article contains some basic principles to design good UI. I don’t like those “are you sure you want to do something?” messages. It’s not a good idea based on this article.

Read full article

Why Do Many Developers Consider Scrum to Be an Evil Scam?

Like any widespread idea, it gets perverted over time. I’ve been in teams where SCRUM is only adopted to micromanage developers using daily meetings. And the next thing you know is that daily meetings become a narrative of how busy developers are to avoid getting fired.

Read full article

Why don’t software development methodologies work?

This is an old article I found on Hacker News front page. It showed, years ago, what everybody is complaining about these days. You only need to visit Hacker News or r/programming once every month.

I’ve been in small projects with no methodologies to larger projects with SCRUM as religion. I’ve been there.

I like this paragraph from the article: “My own experience, validated by Cockburn’s thesis and Frederick Brooks in No Silver Bullet, is that software development projects succeed when the key people on the team share a common vision, what Brooks calls ‘conceptual integrity.’“

Apart from the main article, it has really good comments. This is one that resonates with me about the one methodology:

“A single technical lead with full authority to make decisions, with a next tier assistant, associated technical staff, and a non-technical support person. the achievement of the team is then determined by the leadership of the team. the size of the team and project complexity is then limited by the leader and her ability to understand the problem and assign tasks.”

For me, the most successful project are the ones with a small team who knows each other, and everyone knows the main goal and what to do.

Read full article

Voilà! Another Monday Links. Do you consider yourself a passionate developer? What’s your experience with software methodologies?

In the meantime, don’t miss the previous Monday Links on 40-year Programmer, Work and Burnout.

Happy coding!

Best of 2022

In 2022, I wrote two major series of posts: one about SQL Server performance tuning and another one about LINQ.

Once one of my clients asked me to “tune” some stored procedures and that inspired me to take a close look at the performance tuning world. Last year, I took Brent Ozar Mastering courses and decided to share some of the things I learned.

On another hand, I updated my Quick Guide to LINQ to use new C# features and wrote a bunch of new posts about LINQ. In fact, I released one text-based course about LINQ on Educative: Getting Started with LINQ. That’s my favorite C# feature, ever.

I kept writing my Monday Links posts. And, I decided to have my own Advent of Code. I prefer to call it: Advent of Posts. I wrote 22 posts in December. One post per day until Christmas eve. I missed a couple of days. But I consider it a “mission accomplished.”

These are the 5 posts I wrote in 2022 you read the most. In case you missed any of them, here they are:

1. TIL: How to optimize Group by queries in SQL Server. This post has one of the lessons I learned after following Brent Ozar’s Mastering courses. Well, this one is about using CTEs to speed up queries with GROUP BY.
2. SQL Server Index recommendations: Just listen to them. Again, these are some of the lessons I learned in Brent Ozar’s Mastering Index Tuning course. I shared why we shouldn’t blindly create indexes recommendations from query plans. They’re only clues. We can do better than that. I have to confess I added every single index recommendation I got before learning these lessons.
3. Working with ASP.NET Core IDistributedCache Provider for NCache. This is a post I wrote in collaboration with Alachisoft, NCache creators. It introduces NCache and shows how to use it as a DistributedCache provider with ASP.NET Core.
4. Four new LINQ methods in .NET 6: Chunk, DistinctBy, Take, XOrDefault. After many years, with the .NET 6 release, LINQ received new methods and overloads. In this post, I show some of them.
5. How to use LINQ GroupBy method? Another post to get started with LINQ. It covers three GroupBy use cases.

Voilà! These were your 5 favorite posts. Hope you enjoy them as much as I did writing them. Probably, you found shorter versions of these posts on my dev.to account. Or the version of some random guy copy-pasted into his own website pretending I was an author on his programming site. Things you find out when you google your own user handle. Arrggg!

If any of my posts have helped you and you want to support my work, check my Getting Started with LINQ course on Educative or buy me a coffee, tea, or hamburger by downloading my ebooks from my Gumroad page.

Don’t miss my best of 2021.

Thanks for reading, and happy coding in 2023!

Let's refactor a test: Remove duplicated emails

This post is part of my Advent of Code 2022.

Recently, I’ve been reviewing pull requests as one of my main activities. This time, let’s refactor two tests I found on one code review session. The two tests check if an email doesn’t have duplicated addresses before sending it. But, they have a common mistake: testing private methods directly. Let’s refactor these tests to use the public facade of methods.

Always write unit tests using the public methods of a class or a group of classes. Don’t make private methods public and static to test them directly. Test the observable behavior of classes instead.

Here are the test to refactor

These tests belong to an email component in a Property Management Solution. This component stores all emails before sending them.

These are two tests to check we don’t try to send an email to the same addresses. Let’s pay attention to the class name and method under test.

public class SendEmailCommandHandlerTests
{
    [Fact]
    public void CreateRecipients_NoDuplicates_ReturnsSameRecipients()
    {
        var toEmailAddresses = new List<string>
        {
            "toMail1@mail.com", "toMail2@mail.com"
        };
        var ccEmailAddresses = new List<string>
        {
            "ccMail3@mail.com", "ccMail4@mail.com"
        };

        var recipients = SendEmailCommandHandler.CreateRecipients(toEmailAddresses, ccEmailAddresses);
        //                                       ^^^^^
        
        recipients.Should().BeEquivalentTo(
          new List<Recipient>
          {
              Recipient.To("toMail1@mail.com"),
              Recipient.To("toMail2@mail.com"),
              Recipient.Cc("ccMail3@mail.com"),
              Recipient.Cc("ccMail4@mail.com")
          });
    }

    [Fact]
    public void CreateRecipients_Duplicates_ReturnsRecipientsWithoutDuplicates()
    {
        var toEmailAddresses = new List<string>
        {
            "toMail1@mail.com", "toMail2@mail.com", "toMail1@mail.com"
        };
        var ccEmailAddresses = new List<string>
        {
            "ccMail1@mail.com", "toMail2@mail.com"
        };

        var recipients = SendEmailCommandHandler.CreateRecipients(toEmailAddresses, ccEmailAddresses);
        //                                       ^^^^^

        recipients.Should().BeEquivalentTo(
          new List<Recipient>
          {
              Recipient.To("toMail1@mail.com"),
              Recipient.To("toMail2@mail.com"),
              Recipient.Cc("ccMail1@mail.com"),
          });
    }
}

I slightly changed some names. But those are the real tests I had to refactor.

What’s wrong with those tests? Did you notice it? Also, can you point out where the duplicates are in the second test?

To have more context, here’s the SendEmailCommandHandler class that contains the CreateRecipients() method,

using MediatR;
using Microsoft.Extensions.Logging;
using MyCoolProject.Commands;
using MyCoolProject.Shared;

namespace MyCoolProject;

public class SendEmailCommandHandler : IRequestHandler<SendEmailCommand, TrackingId>
{
    private readonly IEmailRepository _emailRepository;
    private readonly ILogger<SendEmailCommandHandler> _logger;

    public CreateDispatchCommandHandler(
        IEmailRepository emailRepository,
        ILogger<CreateDispatchCommandHandler> logger)
    {

        _emailRepository = emailRepository;
        _logger = logger;
    }

    public async Task<TrackingId> Handle(SendEmailCommand command, CancellationToken cancellationToken)
    {
        // Imagine some validations and initializations here...

        var recipients = CreateRecipients(command.Tos, command.Ccs);
        //               ^^^^^
        var email = Email.Create(
            command.Subject,
            command.Body,
            recipients);

        await _emailRepository.CreateAsync(email);

        return email.TrackingId;
    }

    public static IEnumerable<Recipient> CreateRecipients(IEnumerable<string> tos, IEnumerable<string> ccs)
    //                                   ^^^^^
        => tos.Select(Recipient.To)
              .UnionBy(ccs.Select(Recipient.Cc), recipient => recipient.EmailAddress);
    }
}

public record Recipient(EmailAddress EmailAddress, RecipientType RecipientType)
{
    public static Recipient To(string emailAddress)
        => new Recipient(emailAddress, RecipientType.To);

    public static Recipient Cc(string emailAddress)
        => new Recipient(emailAddress, RecipientType.Cc);
}

public enum RecipientType
{
    To, Cc
}

The SendEmailCommandHandler processes all requests to send an email. It grabs the input parameters, creates an Email class, and stores it using a repository. It uses the free MediatR library to roll commands and command handlers.

Also, it parses the raw email addresses into a list of Recipient with the CreateRecipients() method. That’s the method under test in our two tests. Here the Recipient and EmailAddress work like Value Objects.

Now can you notice what’s wrong with our tests?

What’s wrong?

Our two unit tests test a private method directly. That’s not the appropriate way of writing unit tests. We shouldn’t test internal state and private methods. We should test them through the public facade of our logic under test.

In fact, someone made the CreateRecipients() method public to test it,

Making private methods public to test them is the most common mistake on unit testing.

For our case, we should write our tests using the SendEmailCommand class and the Handle() method.

Don’t expose private methods

Let’s make the CreateRecipients() private again. And let’s write our tests using the SendEmailCommand and SendEmailCommandHandler classes.

This is the test to validate that we remove duplicates,

[Fact]
public async Task Handle_DuplicatedEmailInTosAndCc_CallsRepositoryWithoutDuplicates()
{
    var duplicated = "duplicated@email.com";
    //  ^^^^^
    var tos = new List<string> { duplicated, "tomail@mail.com" };
    var ccs = new List<string> { duplicated, "ccmail@mail.com" };

    var fakeRepository = new Mock<IDispatchRepository>();

    var handler = new CreateDispatchCommandHandler(
        fakeRepository.Object,
        Mock.Of<ILogger<SendEmailCommandHandler>>());

    // Let's write a factory method that receives these two email lists
    var command = BuildCommand(tos: tos, ccs: ccs);
    //            ^^^^^
    await handler.Handle(command, CancellationToken.None);

    // Let's write some assert/verifications in terms of the Email object
    fakeRepository
        .Verify(t => t.CreateAsync(It.Is<Email>(/* Assert something here using Recipients */), It.IsAny<CancellationToken>());
    // Or, even better let's write a custom Verify()
    //
    // fakeRepository.WasCalledWithoutDuplicates();
}

private static SendEmailCommand BuildCommand(IEnumerable<string> tos, IEnumerable<string> ccs)
    => new SendEmailCommand(
        "Any Subject",
        "Any Body",
        tos,
        ccs);

Notice we wrote a BuildCommand() method to create a SendEmailCommand only with the email addresses. That’s what we care about in this test. This way we reduce the noise in our tests. And, to make our test values obvious, we declared a duplicated variable and used it in both destination email addresses.

To write the Assert part of this test, we can use the Verify() method from the fake repository to check that we have the duplicated email only once. Or we can use the Moq Callback() method to capture the Email being saved and write some assertions. Even better, we can create a custom assertion for that. Maybe, we can write a WasCalledWithoutDuplicates() method.

That’s one of the two original tests. The other one is left as an exercise to the reader.

Voilà! That was today’s refactoring session. To take home, we shouldn’t test private methods and always write tests using the public methods of the code under test. We can remember this principle with the mnemonic: “Don’t let others touch our private parts.” That’s how I remember it.

For more refactoring sessions, check these two: store and update OAuth connections and generate payment reports. Don’t miss my Unit Testing 101 series where I cover from naming conventions to best practices.

Happy coding!