Goodbye, NullReferenceException: Option and LINQ

In the previous post of this series, we covered three C# operators to simplify null checks and C# 8.0 Nullable References to signal when things can be null.

In this post, let’s learn a more “functional” approach to removing null and how to use it to avoid null when working with LINQ XOrDefault methods.

1. Use Option: A More Functional Approach to Nulls

Functional languages like F# or Haskell use a different approach for null and optional values. Instead of null, they use an Option or Maybe type.

With the Option type, we have a “box” that might have a value or not. It’s the same concept of nullable ints, for example. I bet you have already used them. Let’s see an example,

int? maybeAnInt = null;

var hasValue = maybeAnInt.HasValue;
// false

var dangerousInt = maybeAnInt.Value;
//                            ^^^^^
// Nullable object must have a value.

var safeInt = maybeAnInt.GetValueOrDefault();
// 0

With nullable ints, we have variable that either holds an interger or null. They have the HasValue and Value properties, and the GetValueOrDefault() method to access their inner value.

We can extend the concept of a box with possibly a value to reference types with the Option type. We can wrap our reference types like Option<int> or Option<Movie>.

A set of gift boxes
An Option is like a box. Photo by Sasun Bughdaryan on Unsplash

The Option Type

The Option type has two subtypes: Some and None. Some represents a box with a value inside it, and None, an empty box.

The Option has two basic methods:

  1. One method to put something into a box. Often we call it Unit. For this, we can use the constructor of Some and None.
  2. One method to open the box, transform its value and return a new one with the result. Let’s call this method Map.
Option Unit and Map functions
Option's Unit and Map functions

Let’s use the Optional library, a robust option type for C#, to see how to use the Some, None, and Map(),

using Optional;

Option<int> someInt = Option.Some(42);
Option<int> none = Option.None<int>();

var doubleOfSomeInt = someInt.Map(value => value * 2)
                             .ValueOr(-1);
// 84

var doubleOfNone = none.Map(value => value * 2)
                       .ValueOr(-1);
// -1

We created two optional ints: someInt and none. Then, we used Map() to double their values. Then, to retrieve the value of each optional, we used ValueOr() with a default value.

For someInt, Map() returned another optional with the double of 42 and ValueOr() returned the same result. And for none, Map() returned None and ValueOr() returned -1.

How to Flatten Nested Options

Now, let’s rewrite the HttpContext example from previous posts,

public class HttpContext
{
    public static Option<HttpContext> Current;
    //            ^^^^^

    public HttpContext()
    {
    }

    public Option<Request> Request { get; set; }
    //     ^^^^^
}

public record Request(Option<string> ApplicationPath);
//                    ^^^^^

Notice that instead of appending ? to type declarations like what we did with in the past post when we covered C# 8.0 Nullable References, we wrapped them around Option.

This time, Current is a box with Request as another box inside. And Request has the ApplicationPath as another box.

Now, let’s retrieve the ApplicationPath,

var path = HttpContext.Current
            .FlatMap(current => current.Request)
            // ^^^^^
            .FlatMap(request => request.ApplicationPath)
            // ^^^^^
            .ValueOr("/some-default-path-here");
            // ^^^^
            // Or
            //.Match((path) => path , () => "/some-default-path-here");

// This isn't the real HttpContext class...
// We're writing some dummy declarations to prove a point
public class HttpContext
{
    public static Option<HttpContext> Current;

    public HttpContext()
    {
    }

    public Option<Request> Request { get; set; }
}

public record Request(Option<string> ApplicationPath);

To get the ApplicationPath value, we had to open all boxes that contain it. For that, we used the FlatMap() method. It grabs the value in the box, transforms it, and returns another box. With FlatMap(), we can flatten two nested boxes.

Option FlatMap function
Option's FlatMap to flatten nested options

Notice we didn’t do any transformation with FlatMap(). We only retrieved the inner value of Option, which was already another Option.

This is how we read ApplicationPath:

  1. With FlatMap(), we opened the Current box and grabbed the Request box in it.
  2. Then, we used FlatMap() again to open Request and grab the ApplicationPath.
  3. Finally, with ValueOr(), we took out the value inside ApplicationPath if it had any. Otherwise, if the ApplicationPath was empty, it returned a default value of our choice.

“This is the way!” Sorry, this is the “functional” way! We can think of nullable ints like ints being wrapped around a Nullable box with more compact syntax and some helper methods.

2. Option and LINQ XOrDefault methods

Another source of NullReferenceException is when we don’t check the result of the FirstOrDefault, LastOrDefault, and SingleOrDefault methods.

These methods return null when the source collection has reference types, and there are no matching elements. In fact, this is one of the most common mistakes when working with LINQ.

There are some alternatives to prevent the NullReferenceException when working with XOrDefault methods.

.NET 6.0 released some new LINQ methods and overloads. With .NET 6.0, we can use a second parameter with the XOrDefault methods to pass a default value of our choice. Also, we can use the DefaultIfEmpty method instead of filtering collections with FirstOrDefault.

Use Optional’s XOrNone

But, let’s combine the XOrDefault methods with the Option type. We can make the XOrDefault methods return an Option<T> instead of null.

The Optional library has FirstOrNone(), LastOrNone() and SingleOrNone() instead of the usual XOrDefault methods.

This time, let’s use FirstOrNone() instead of FirstOrDefault(),

using Optional.Collections;
//    ^^^^^

var movies = new List<Movie>
{
    new Movie("Shrek", 2001, 3.95f),
    new Movie("Inside Out", 2015, 4.1f),
    new Movie("Ratatouille", 2007, 4f),
    new Movie("Toy Story", 1995, 4.1f),
    new Movie("Cloudy with a Chance of Meatballs", 2009, 3.75f)
};

var theBestOfAll = new Movie("My Neighbor Totoro", 1988, 5);

// With .NET FirstOrDefault()
var theBest = movies.FirstOrDefault(
                    movie => movie.Rating == 5.0,
                    theBestOfAll);
                    // ^^^^^

// With Optional's FirstOrNone()
var theBestAgain = movies.FirstOrNone(movie => movie.Rating == 5.0)
                    //    ^^^^^
                    .ValueOr(theBestOfAll);
                    // ^^^^^
Console.WriteLine(theBestAgain.Name);

record Movie(string Name, int ReleaseYear, float Rating);

By using the XOrNone methods, we’re forced to check if they return something before trying to use their result.

Voilà! That’s the functional way of doing null, with the Option or Maybe type. Here we used the Optional library, but there’s also another library I like: Optuple. It uses the tuple (bool HasValue, T Value) to represent the Some and None subtypes.

Even though we used a library to bring the Option type, we can implement our own Option type and its methods. It’s not that difficult. We need an abstract class with two child classes and a couple of extension methods to make it work.

Don’t miss the other posts in this series, what the NullReferenceException is and when it’s thrown, nullable operators and references, and separate optional state into separate objects.

Join my course C# NullReferenceException Demystified on Udemy and learn the principles, features, and strategies to avoid this exception in just 1 hour and 5 minutes.

Happy coding!

Goodbye, NullReferenceException: Nullable Operators and References

In the previous post of this series, we covered two ideas to avoid the NullReferenceException: we should check for null before accessing the members of an object and check the input parameters of our methods.

Let’s see some new C# operators to simplify null checking and a new feature to better signal possible null references.

1. C# Nullable Operators

C# has three operators to simplify our null checks: ?., ??, and ??=. These operators don’t prevent us from having null in the first place, but they help us to easily write our null checks.

Without Nullable Operators

Let’s start with an example and refactor it to use these new operators.

string path = null;

if (HttpContext.Current == null 
    || HttpContext.Current.Request == null 
    || HttpContext.Current.Request.ApplicationPath  == null)
{
    path = "/some-default-path-here";
}
else
{
    path = HttpContext.Current.Request.ApplicationPath;
}

If you have worked with the old ASP.NET framework, you might have done something like that. If not, don’t worry. We’re only accessing a property down in a property chain, but any of those properties could be null.

Notice that to defend against null, we checked if every property was null to “fail fast” and use a default value.

With Nullable Operators

Now, let’s use the new nullable operators instead,

var path = HttpContext.Current?.Request?.ApplicationPath
//                           ^^^      ^^^
              ?? "/some-default-path-here";
//           ^^^^

More compact, right?

With the null-conditional operator (?.), we access the property or method of an object only if the object isn’t null. Otherwise, the entire expression evaluates to null. For our example, we retrieve ApplicationPath only if Request isn’t null and Current isn’t null, and HttpContext isn’t null.

Then, with the null-coalescing operator (??), we evaluate an alternative expression if the one on the left of the ?? operator isn’t null. For our example, if any of the properties in the chain to retrieve ApplicationPath is null, the whole expression is null, and the path variable gets assigned to the default string.

With the null-coalescing assignment operator (??=), we assign a new value to a variable only if it’s null. We could also write our example like this,

var path = HttpContext.Current?.Request?.ApplicationPath;
path ??= "/some-default-path-here";
//  ^^^^
//
// The same as
//if (path == null)
//{
//    path = "/some-default-path-here"; 
//}

Notice how we refactored our original example to only two lines of code with these three new operators. Again we could still have null values. These nullable operators make our lives easier by simplifying our null checks.

Electronic devices in a workbench
What if we could tell when something is null? Photo by Nicolas Thomas on Unsplash

2. C# Nullable References

To solve the NullReferenceException, we should check for null. We got that! But the thing is knowing when we should do it or not. That’s precisely what C# 8.0 solves with Nullable References.

With C# 8.0, all reference variables are non-nullable by default. Accessing the member of nullable references results in compiler warnings or errors.

This is a breaking change. Therefore we need to turn on this feature at the project level in our csproj files. Like these,

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <!--             ^^^^^^ -->
    <!-- We could use netcoreapp3.1|net5.0|net6.0|.net7.0 too -->
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <!--      ^^^^^^ -->
    <!-- We could use enable|disable|warning too -->
    <WarningsAsErrors>nullable</WarningsAsErrors>
    <!--              ^^^^^^^^ -->
    <!-- We can take the extreme route -->

  </PropertyGroup>

</Project>

To use Nullable References, we need to target .NET Core 3 and upward. And inside the Nullable node in our csproj files, we could use: enable, disable, or warning. Even, we can take the extreme route and consider all nullable warnings as compilation errors.

With Nullable References On

Let’s see what our motivating example looks like with Nullable References turned on,

string path = null;
//            ^^^^
// CS8600: Converting null literal or possible null value to non-nullable type

if (HttpContext.Current == null
    || HttpContext.Current.Request == null
    || HttpContext.Current.Request.ApplicationPath == null)
{
    path = "/some-default-path-here";
}
else
{
    path = HttpContext.Current.Request.ApplicationPath;
}

// This isn't the real HttpContext class...
// We're writing some dummy declarations to prove a point
public class HttpContext
{
    public static HttpContext Current;
    //                        ^^^^^^^^
    // CS8618: Non-nullable field 'Current' must contain
    // a non-nullable value when exiting constructor

    public HttpContext()
    //     ^^^^^^^^^^^
    // CS8618: Non-nullable field 'Current' must contain
    // a non-nullable value when exiting constructor
    {
    }

    public Request Request { get; set; }
}

public record Request(string ApplicationPath);

Notice we have a warning when initializing path to null. And another one in the declaration of our HttpContext class if we don’t initialize any non-nullable fields. That’s not the real HttpContext by the way but bear with me. Also, we don’t need to check for null when retrieving the ApplicationPath since all our references aren’t nullable by definition.

To declare a variable that can be null, we need to add to its type declaration a ?. In the same way, we have always declared nullable primitive types like int?.

Without Null Checks

Let’s change our example to have nullable references and no null checks,

// Notice the ? symbol here
//   vvv
string? path = HttpContext.Current.Request.ApplicationPath;
//             ^^^^^^^^^^^
// CS8602: Deference of a possibly null value

// This isn't the real HttpContext class...
// We're writing some dummy declarations to prove a point
public class HttpContext
{
    public static HttpContext? Current;
    //                      ^^^
    // Notice the ? symbol here

    public HttpContext()
    //     ^^^
    // No more warnings here
    {
    }

    public Request Request { get; set; }
}

public record Request(string? ApplicationPath);

Notice this time, when declaring the variable path, we have a warning because we’re accessing the Current property, which might be null. Also, notice we changed, inside the HttpContext class, the Current property to have the HttpContext? type (with a ? at the end of the type).

Now with Nullable References, we have a way of telling when we should check for null by looking at the signature of our methods.

Voilà! Those are the new C# operators to simplify our null checks. We said “new” operators, but we have had them since C# 6.0. And that’s how we can tell if our references can be null or not using Nullable References.

We have these nullable operators available even if we’re using the old .NET Framework. But, to use Nullable References, we should upgrade at least to .NET Core 3.0.

In the next post, we will cover the Option type as an alternative to null and how to avoid the NullReferenceException when working with LINQ.

Join my course C# NullReferenceException Demystified on Udemy and learn the principles, features, and strategies to avoid this exception in just 1 hour and 5 minutes.

Happy coding!

Goodbye, NullReferenceException: What it is and how to avoid it

If you’re here, I bet you already have found the exception message: “Object reference not set to an instance of an object.”

In this series of posts, let’s see some techniques to completely eliminate the NullReferenceException from our code. Let’s start by understanding when NullReferenceException is thrown and a strategy to fix it.

NullReferenceException is thrown when we access a property or method of an uninitialized variable of a reference type. The easier way to solve this exception is to check for null before accessing the members of an object. But C# has introduced new operators and features to avoid this exception.

Let’s write an example that throws NullReferenceException,

var movie = FindMovie();
Console.Write(movie.Name);
//            ^^^^^
// System.NullReferenceException: 'Object reference not set to an instance of an object.'
//
// movie was null.

Console.ReadKey();

static Movie FindMovie()
{
    // Imagine this is a database call that might
    // or might not return a movie
    return null;
    //     ^^^^
}

record Movie(string Name, int ReleaseYear, float Rating);

Notice we returned null from FindMovie(). That caused the NullReferenceException. But, it could be a method that accessed a database and didn’t find anything or an API controller method that didn’t receive a required input parameter.

In our last example, we got a NullReferenceException when we returned null from a method. But, we could also find this exception when we try to loop through a null list or array, for example.

Speaking of returning null, one way to prevent the NullReferenceException is to never pass null between objects. Instead of returning null, let’s use empty lists and strings, or the Null Object pattern. And let’s use intention-revealing defaults for that.

Multicolored dream catcher
Don't catch it! Photo by David Restrepo Parrales on Unsplash

1. Don’t catch the NullReferenceException

To fix the NullReferenceException, we might be tempted to write a try/catch block around the code that throws it. But, let’s not catch the NullReferenceException.

Let me say that again: don’t throw or catch NullReferenceException.

By any means, please, let’s not write something like this,

try
{
    AMethodThatMightThrowNullReferenceException();
}
catch (NullReferenceException)
{
    // ...
    // Beep, beep, boop
    // Doing something with the exception here
}

A NullReferenceException is a symptom of an unhandled and unexpected scenario in our code, and catching it won’t handle that. A NullReferenceException is a developer’s mistake.

2. Check for null

The solution for the NullReferenceException is to check for nulls and defense against them.

Let’s fix our previous example by adding a null check. Like this,

var movie = FindMovie();
if (movie != null)
//  ^^^^^
{
    // We're safe here...
    //
    // No more System.NullReferenceException
    // at least due to a movie being null 
    Console.Write(movie.Name);
}

Console.ReadKey();

static Movie FindMovie()
{
    // Imagine this is a database call that might
    // or might not return a movie
    return null;
    //     ^^^^
}

record Movie(string Name, int ReleaseYear, float Rating);

Notice we checked if the movie variable wasn’t null. As simple as that.

Alternatives to check for null

The lack of options isn’t an excuse to not check for null. Since some recent C# versions, and thanks to pattern matching, we have a couple of new ways to check for null. We can use any of these:

  • if (movie != null) { //... },
  • if (movie is not null) { //... },
  • if (movie is { }) { //... }, and,
  • if (movie is object) { //... }

Some of those don’t look like C# anymore. I still prefer the old if (movie != null) .... Which one do you prefer?

3. Defense against null

If the solution to NullReferenceException were to simply check for null, that wouldn’t be the Billion-Dollar mistake. And I wouldn’t be writing this series.

But the thing is knowing when a reference might be null or not and, therefore, when we should check for it. That’s when we should protect ourselves against null.

To protect against null, inside our methods, let’s check our input parameters and throw a more detailed exception. C# has an exception for missing parameters: ArgumentNullException.

The ArgumentNullException stack trace contains the name of the parameter that was null. That will help us to better troubleshoot our code. Often, the NullReferenceException stack trace doesn’t include what was null in the first place. Happy debugging time!

Let’s check our input parameters,

public void DoSomething(Movie movie)
{
    if (movie == null)
    {
        throw new ArgumentNullException(nameof(movie))
    }
    // Since C# 10, we can also write:
    // ArgumentNullException.ThrowIfNull(movie);

    // Beep, beep, boop
    // Doing something here...
}

Notice that instead of waiting for NullReferenceException, we proactively prevented it by checking for a required parameter and throwing a controlled ArgumentNullException.

Voilà! That’s the NullReferenceException and how to fix it by checking for null. Remember, we shouldn’t catch this exception but prevent and prepare for it.

Don’t miss the other posts in this series! In the next post, we cover how to use Nullable Operators and Nullable References to prevent the NullReferenceException. In future posts, we’re covering why getting rid of null is a good idea, the Option type and LINQ XOrDefault methods, and a design technique to encapsulate optional state.

Join my course C# NullReferenceException Demystified on Udemy and learn the principles, features, and strategies to avoid this exception in just 1 hour and 5 minutes.

Happy coding!

Monday Links: Monday Links: Passions, Estimates, and Methodologies

Welcome to the first Monday Links of 2023. These are five reads I found interesting last month. This time, I found software methodologies was a recurring theme these days.

Why I’m Glad I Lack Passion to BE a Programmer

After a couple of years of working as a software engineer, I started to embrace simplicity. Software exists to satisfy a business need. Perfect software only exists in books. That’s why I started to see the big goal and only use libraries/tools/concepts when there’s a compelling reason to do so. Not all applications need to use Domain-Driven Design with Event Sourcing.

I liked this one: My ideal for software development is to find the simplest solution to the practical problem. I’m not a passionate programmer anymore either.

Read full article

11 Laws of Software Estimation for Complex Work

I can relate to this story. It happens to a friend of a friend of mine. One day his CEO came saying he just closed a really big deal but he didn’t know what they were going to do. Arrrggg!

Apart from a relatable story, this post contains 11 rules about estimations. Like all estimations are simply guesses and by the time developers have enough information to give more accurate estimations, close to the end of a project, it’s already too late.

Read full article

User Interface Design: 5 Rules of Thumb

This article contains some basic principles to design good UI. I don’t like those “are you sure you want to do something?” messages. It’s not a good idea based on this article.

Read full article

Why Do Many Developers Consider Scrum to Be an Evil Scam?

Like any widespread idea, it gets perverted over time. I’ve been in teams where SCRUM is only adopted to micromanage developers using daily meetings. And the next thing you know is that daily meetings become a narrative of how busy developers are to avoid getting fired.

Read full article

Why don’t software development methodologies work?

This is an old article I found on Hacker News front page. It showed, years ago, what everybody is complaining about these days. You only need to visit Hacker News or r/programming once every month.

I’ve been in small projects with no methodologies to larger projects with SCRUM as religion. I’ve been there.

I like this paragraph from the article: “My own experience, validated by Cockburn’s thesis and Frederick Brooks in No Silver Bullet, is that software development projects succeed when the key people on the team share a common vision, what Brooks calls ‘conceptual integrity.’“

Apart from the main article, it has really good comments. This is one that resonates with me about the one methodology:

“A single technical lead with full authority to make decisions, with a next tier assistant, associated technical staff, and a non-technical support person. the achievement of the team is then determined by the leadership of the team. the size of the team and project complexity is then limited by the leader and her ability to understand the problem and assign tasks.”

For me, the most successful project are the ones with a small team who knows each other, and everyone knows the main goal and what to do.

Read full article

Voilà! Another Monday Links. Do you consider yourself a passionate developer? What’s your experience with software methodologies?

In the meantime, don’t miss the previous Monday Links on 40-year Programmer, Work and Burnout.

Happy coding!

Best of 2022

In 2022, I wrote two major series of posts: one about SQL Server performance tuning and another one about LINQ.

Once one of my clients asked me to “tune” some stored procedures and that inspired me to take a close look at the performance tuning world. Last year, I took Brent Ozar Mastering courses and decided to share some of the things I learned.

On another hand, I updated my Quick Guide to LINQ to use new C# features and wrote a bunch of new posts about LINQ. In fact, I released one text-based course about LINQ on Educative: Getting Started with LINQ. That’s my favorite C# feature, ever.

I kept writing my Monday Links posts. And, I decided to have my own Advent of Code. I prefer to call it: Advent of Posts. I wrote 22 posts in December. One post per day until Christmas eve. I missed a couple of days. But I consider it a “mission accomplished.”

These are the 5 posts I wrote in 2022 you read the most. In case you missed any of them, here they are:

  1. TIL: How to optimize Group by queries in SQL Server. This post has one of the lessons I learned after following Brent Ozar’s Mastering courses. Well, this one is about using CTEs to speed up queries with GROUP BY.
  2. SQL Server Index recommendations: Just listen to them. Again, these are some of the lessons I learned in Brent Ozar’s Mastering Index Tuning course. I shared why we shouldn’t blindly create indexes recommendations from query plans. They’re only clues. We can do better than that. I have to confess I added every single index recommendation I got before learning these lessons.
  3. Working with ASP.NET Core IDistributedCache Provider for NCache. This is a post I wrote in collaboration with Alachisoft, NCache creators. It introduces NCache and shows how to use it as a DistributedCache provider with ASP.NET Core.
  4. Four new LINQ methods in .NET 6: Chunk, DistinctBy, Take, XOrDefault. After many years, with the .NET 6 release, LINQ received new methods and overloads. In this post, I show some of them.
  5. How to use LINQ GroupBy method? Another post to get started with LINQ. It covers three GroupBy use cases.

Voilà! These were your 5 favorite posts. Hope you enjoy them as much as I did writing them. Probably, you found shorter versions of these posts on my dev.to account. Or the version of some random guy copy-pasted into his own website pretending I was an author on his programming site. Things you find out when you google your own user handle. Arrggg!

If any of my posts have helped you and you want to support my work, check my Getting Started with LINQ course on Educative or buy me a coffee, tea, or hamburger by downloading my ebooks from my Gumroad page.

Don’t miss my best of 2021.

Thanks for reading, and happy coding in 2023!