Five common LINQ mistakes and how to fix them

It’s easy to start working with LINQ to replace for, foreach, and other loops. With a handful of LINQ methods, we have our backs covered.

But, often, we make some common mistakes when working with LINQ. Here are five common mistakes we make when working with LINQ for the first time and how to fix them.

Mistake 1: Use Count instead of Any

We should always prefer Any over Count to check if a collection is empty or has at least one element that meets a condition.

Let’s write,

movies.Any()

Instead of,

movies.Count() > 0

Any returns when it finds at least one element.

Count could use the size of the underlying collection. But, it could also evaluate the entire LINQ query for other collection types. And this could be a performance hit for large collections.

Mistake 2: Use Where followed by Any

We can use a condition with Any directly, instead of filtering first with Where to then use Any.

Let’s write,

movies.Any(movie => movie.Rating == 5)

Instead of,

movies.Where(movie => movie.Rating == 5).Any()

The same applies to the Where method followed by FirstOrDefault, Count, or any other method that receives a filter condition.

Let’s use the filtering condition directly instead of relying on the Where method first.

Mistake 3: Use FirstOrDefault instead of SingleOrDefault to find unique values

We should prefer SingleOrDefault over FirstOrDefault to find one and only one element matching a condition inside a collection.

Let’s write,

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1995, 4.6f),
    new Movie("Terminator 2", 1999, 4.7f),
    new Movie("Avatar", 2010, 5),
    //        ^^^^^
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
    //        ^^^^^
    // We have a tie here...
};

// SigleOrDefault expects only one element...but there are two of them
var theBest = movies.SingleOrDefault(movie => movie.Rating == 5);
//                   ^^^^^^
// System.InvalidOperationException: 'Sequence contains more than one matching element'
//
Console.WriteLine($"{theBest.Name}: [{theBest.Rating}]");

record Movie(string Name, int ReleaseYear, float Rating);

Instead of,

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1995, 4.6f),
    new Movie("Terminator 2", 1999, 4.7f),
    new Movie("Avatar", 2010, 5),
    //        ^^^^^
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
    //        ^^^^^
    // We have a tie here...
};

// FirstOrDefault remains quiet if there's more than one matching element...
var theBest = movies.FirstOrDefault(movie => movie.Rating == 5);
//                   ^^^^^

Console.WriteLine($"{theBest.Name}: [{theBest.Rating}]");

record Movie(string Name, int ReleaseYear, float Rating);

SigleOrDefault throws an exception when it finds more than one element matching a condition. But, with multiple matching elements, FirstOrDefault returns the first of them without signaling any problem.

Let’s pick between FirstOrDefault and SingleOrDefault to show the query’s intent. Let’s prefer SingleOrDefault to retrieve a unique matching element from a collection.

To guarantee that there is a single element in a collection, Single and SingleOrDefault have to evaluate the LINQ query over the entire collection. And, again, this could be a performance hit for large collections.

Mistake 4: Use FirstOrDefault without null checking

Let’s always check if we have a result when working with FirstOrDefault, LastOrDefault, and SingleOrDefault.

When any of those three methods don’t find results, they return the default value of the collection type.

For objects, the default value would be a null reference. And, do you know what happens when we access a property or method on a null reference?… Yes, It throws the fearsome NullReferenceException. Arrggg!

We have this mistake in the following code sample. We forgot to check if the worst variable has a value. An if (worst != null) would solve the problem.

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1995, 4.6f),
    new Movie("Terminator 2", 1999, 4.7f),
    new Movie("Avatar", 2010, 5),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

var worst = movies.FirstOrDefault(movie => movie.Rating < 2);

// We forgot to check for nulls after using FirstOrDefault
// It will break	
Console.WriteLine($"{worst.Name}: [{worst.Rating}]");
//                  ^^^^^^^^^^^^ 
// System.NullReferenceException: 'Object reference not set to an instance of an object.'
//
// worst was null.

record Movie(string Name, int ReleaseYear, float Rating);

We wrote a LINQ query with FirstOrDefault looking for the first movie with a rating lower than 2. But, we don’t have any movie that matches that condition. FirstOrDefault returned null, and we forgot to check if the worst variable was different from null before using it.

There are other alternatives to get rid of the NullReferenceException when using FirstOrDefault:

• C# 8.0 nullable references,
• LINQ DefaultIfEmpty method,
• .NET6 FirstOrDefault with a default value

Mistake 5: Expect LINQ queries to be cached

LINQ queries are lazy-evaluated.

It means the actual result of a LINQ query is evaluated when we loop through the result, not when we declare the query. And it’s evaluated every time we loop through it.

Let’s avoid looping through the result of a LINQ query multiple times expecting it to be cached the first time we run it.

var movies = new List<Movie>
{
    new Movie("The Fifth Element", 1995, 4.6f),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

var favorites = movies.Where(movie =>
{
    // Let's put a debugging message here...
    Console.WriteLine("Beep, boop...");
    return movie.Rating == 5;
});

// 1. Let's print our favorite movies
foreach (var movie in favorites)
{
    Console.WriteLine($"{movie.Name}: [{movie.Rating}]");
}
Console.WriteLine();

// 2. Let's do something else with our favorite movies
foreach (var movie in favorites)
{
    Console.WriteLine($"Doing something else with {movie.Name}");
}
Console.ReadKey();

record Movie(string Name, int ReleaseYear, float Rating);

// Output
// Beep, boop...
// Beep, boop...
// Beep, boop...
// My Neighbor Totoro: [5]
//
// Beep, boop...
// Beep, boop...
// Beep, boop...
// Doing something else with My Neighbor Totoro

We wrote a debugging statement inside the Where method, and we looped through the result twice. The output shows the debugging statements twice. One for every time we looped through the result.

There was no caching whatsoever. The LINQ query was evaluated every time.

Instead of expecting a LINQ query to be cached, we could use ToList or ToArray to break the lazy evaluation. This way, we force the LINQ query to be evaluated only once - when we declare it.

var movies = new List<Movie>
{
    new Movie("The Fifth Element", 1995, 4.6f),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

var favorites = movies.Where(movie =>
{
    // Let's put a debugging message here...
    Console.WriteLine("Beep, boop...");
    return movie.Rating == 5;
}).ToList();
// ^^^^^^
// We break the lazy evaluation with ToList

// 1. Let's print our favorite movies
foreach (var movie in favorites)
{
    Console.WriteLine($"{movie.Name}: [{movie.Rating}]");
}
Console.WriteLine();

// 2. Let's do something else with our favorite movies
foreach (var movie in favorites)
{
    Console.WriteLine($"Doing something else with {movie.Name}");
}
Console.ReadKey();

record Movie(string Name, int ReleaseYear, float Rating);

// Output
// Beep, boop...
// Beep, boop...
// Beep, boop...
// My Neighbor Totoro: [5]
//
// Doing something else with My Neighbor Totoro

Notice the output only shows the debugging messages once, even though we looped through the collection twice. We forced the query to be evaluated only once with the ToList method.

Voilà! Those are the five most common LINQ mistakes. They seem silly, but we often overlook them. Especially we often forget about the lazy evaluation of LINQ queries.

For more content about LINQ, check my Quick Guide to LINQ, these five common LINQ methods in Pictures, and how to use LINQ GroupBy method.

Happy coding!

How to use LINQ GroupBy method: Two more use cases

Last time, I showed five of the most common LINQ methods with pictures. Let’s take a deeper look at one of them: GroupBy.

The GroupBy method groups the elements of a collection based on a grouping key. This method returns a collection of “groups” or “buckets” organized by that key.

Let’s continue to work with our catalog of movies and group our movies by rating.

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1997, 4.6f),
    new Movie("Terminator 2", 1991, 4.7f),
    new Movie("Avatar", 2009, 5),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

// Group our catalog of movies based on their rating
var groupedByRating = movies.GroupBy(movie => movie.Rating);
//                           ^^^^^^^

foreach (var group in groupedByRating)
{
    // Each group or bucket has a Key property  
    Console.WriteLine($"Rating: {group.Key}");

    foreach (var movie in group)
    {
        Console.WriteLine($"{movie.Name}");
    }
    Console.WriteLine();
}
// Output:
//Rating: 4.5
//Titanic
//
//Rating: 4.6
//The Fifth Element
//
//Rating: 4.7
//Terminator 2
//
//Rating: 5
//Avatar
//My Neighbor Totoro
//
//Rating: 4
//Platoon

record Movie(string Name, int ReleaseYear, float Rating);

We used three recent C# features: the Top-level statements, records, and global using statements. Now we can write Console applications without the Main declaration. All boilerplate code is gone!

The GroupBy method receives a delegate as a parameter with the property to use as a key when grouping elements. In our previous example, we used the Rating property and wrote movie => movie.Rating.

What does GroupBy return?

The result of GroupBy is a collection of groups or buckets. It returns IEnumerable<IGrouping<TKey, TSource>> where TKey is the type of the grouping key and TSource is the type of the elements inside the collection.

The IGrouping interface is a wrapper for a collection and its grouping key. Each group or bucket has a Key property.

In our example, the type of the return collection was IEnumerable<IGrouping<float, Movie>>. That’s why we needed two foreach loops to print the movies in each group. One to print the ratings and another to print all movies in each rating.

How to transform every group

Also, the GroupBy method transforms each group or bucket.

Let’s count the movies with the same rating this time.

// This is a Console app without the Main class declaration
// and with Global using statements
var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1997, 4.6f),
    new Movie("Terminator 2", 1991, 4.7f),
    new Movie("Avatar", 2009, 5),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

// Transform every group into a RatingCount type
var countByRating = movies.GroupBy(movie => movie.Rating,
//                                  vvvvvvv
                                    (rating, groupedMovies) => new RatingCount(rating, groupedMovies.Count());

foreach (var group in countByRating)
{
    Console.WriteLine($"{group.Rating}: [{group.Count}]");
}
// Output:
//4.5: [1]
//4.6: [1]
//4.7: [1]
//5: [2]
//4: [1]

record Movie(string Name, int ReleaseYear, float Rating);
record RatingCount(float Rating, int Count);

We passed a second parameter to the GroupBy method. The first parameter was still the grouping key. But, the second one was a delegate that received the grouping key and the elements of each group,Func<TKey, IEnumerable<TSource>, TResult>. We named the two parameters: rating and groupedMovies.

Since we wanted to count the movies with the same rating, we used another LINQ method: Count.

As its name implies, the Count method returns the number of elements in a collection.

With the ratings and the movies per rating, we transformed every group of movies into a new object, RatingCount.

How to group by more than one property

In the two previous examples, we used the Rating property as the grouping key. But we can group the elements of a collection by more than one grouping property.

With the GroupBy method, to group a collection by more than one property, use a custom object as the grouping key.

Let’s group our movies by release year and rating.

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1997, 4.6f),
    new Movie("Terminator 2", 1991, 4.7f),
    new Movie("Avatar", 2009, 5),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5),
    new Movie("Life Is Beautiful", 1997, 4.6f),
    new Movie("Saving Private Ryan", 1998, 4.5f),
    new Movie("Léon: The Professional", 1994, 4.5f),
    new Movie("Forrest Gump", 1994, 4.5f)
};

// Group by our catalog of moves by release year and rating
var groupByReleasedYearAndRating = movies.GroupBy(movie => new { movie.ReleaseYear, movie.Rating });
//                                        ^^^^^^^

foreach (var group in groupByReleasedYearAndRating)
{
    var groupingKey = group.Key;
    Console.WriteLine($"Release Year/Rating: {groupingKey.ReleaseYear} - {groupingKey.Rating} ");

    foreach (var movie in group)
    {
        Console.WriteLine($"{movie.Name}");
    }
    Console.WriteLine();
}
// Output:
//Release Year/Rating: 1998 - 4.5
//Titanic
//Saving Private Ryan
//
//Release Year/Rating: 1997 - 4.6
//The Fifth Element
//Life Is Beautiful
//
//Release Year/Rating: 1991 - 4.7
//Terminator 2
//
//Release Year/Rating: 2009 - 5
//Avatar
//
//Release Year/Rating: 1986 - 4
//Platoon
//
//Release Year/Rating: 1988 - 5
//My Neighbor Totoro
//
//Release Year/Rating: 1994 - 4.5
//Léon: The Professional
//Forrest Gump

record Movie(string Name, int ReleaseYear, float Rating);

We used an anonymous object as the grouping key.

What are anonymous objects in C#?

An anonymous object is a temporary and immutable object defined without a class definition or a name.

In our example, we wrote an anonymous object like this,

new { ReleaseYear = movie.ReleaseYear, Rating = movie.Rating }

Inside anonymous objects, we can omit names while defining member properties if we want to keep the same names.

That’s why we only wrote,

new { movie.ReleaseYear, movie.Rating }

When we need to access properties from an anonymous object, we use their property names, as usual.

Voilà! That’s the GroupBy method. It creates groups or buckets with the elements of a collection and transforms each group.

If you noticed the output of our previous examples, the GroupBy method grouped the elements without sorting them. For that, we would need the OrderBy method.

To learn about LINQ and other methods, check my quick guide to LINQ and three set-like LINQ methods: Intersect, Union, and Except.

Happy coding!

Monday Links: Staging, Work, and Types

A career-ending mistake

Jumping from place to place until we retire? Hopefully, with good pay raises? Being in a team closing Jira tickets and issues until we get bored? Based on the article, the biggest mistake is “not planning the end of our careers.” The right time to decide the end of our careers is now. The article shows three career alternatives: independent, senior individual contributor (IC), and management. Read full article

Why we don’t use a staging environment

I have lived in almost all the situations described in this post. I worked for a company where we waited months to merge from our staging branch to the production branch. Merge conflicts were a nightmare! Patches going directly to the production environment made things more complicated. Often people forgot to merge patches back to the other environments. Arrrggg!

Did I mention that we had multiple staging environments? I remember our most-senior team member advocating for ideas like the ones in this post: Getting rid of the “just-in-case” staging environment, merging and publishing everything to production, and using feature flags.

Oh, I forgot to mention the “do-not-merge-or-touch-staging” times when the Sales team was demoing the product in the same environment. The whole development team had to wait for hours…If we only had had only one environment: production…Read full article

Algebraic Data Types in Haskell

This article might seem intimidating at first because of the Haskell syntax. But, it’s a good introduction to Sum and Product types. Custom types are useful when designing business-related entities in our domain. That’s precisely the main premise of Domain Modeling Made Functional: encode business rules, restrictions, and errors using the type system. Read full article

Maybe you should do less “work”

It contains good points about learning new things at work and making the most value of our working hours. Being efficient, developing other skills, and growing your network. Read full article

How to Quiet Your Mind Chatter

You just finished a Zoom call with one of your clients or your boss. But, you kept the conversation going in your head role-playing what you could have said differently.

We all have experienced that inner voice to imagine different endings to our conversations. Quoting the artcile, “what chatter does is take a stressful experience and prolong it… What makes stress bad is when it’s prolonged over time, and that’s what chatter really does.” The article shows two strategies to deal with chatter. Read full article

Voilà! Those are this month Monday Links. Do you have a career plan? How many environment do you have between developers’ machines and Production? Three? Do you use your work time to sharpen your skills?

In the meantime, check my Getting Started with LINQ course where I cover from what LINQ is to its most recent methods and overlaod in .NET6. Don’t miss the previous Monday Links on Blog, Error Messages and Recruiters.

Happy coding!

Five LINQ Methods in Pictures

If you’re learning LINQ for the first time, it can be daunting to learn all LINQ methods at once. Don’t try it.

Here are the five most common LINQ methods in pictures.

LINQ is the declarative, immutable, and lazy-evaluated way of working with collections in C#. And the most frequently used LINQ methods are Where, Select, Any, GroupBy, and FirstOrDefault.

Let’s work with a list of our favorite movies. Let’s write a Movie class with a name, release year, and a rating.

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1997, 4.6f),
    new Movie("Terminator 2", 1991, 4.7f),
    new Movie("Avatar", 2009, 5),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

1. Where

Where returns a new collection with only the elements that meet a given condition.

Where works like a filter on collections. Think of Where as a replacement for a foreach with an if in it.

Let’s filter our list of movies to keep only those with a rating greater than or equal to 4.5.

var favorites = movies.Where(movie => movie.Rating >= 4.5);

This query would be something like this,

We’re using arrows to display our LINQ queries. But, the output of a LINQ query is lazy-evaluated. It means the actual result of a LINQ query is evaluated until we loop through its result.

2. Select

Select applies a function to transform every element of a collection.

Let’s find only the names of our favorite movies.

var namesOfFavorites = movies.Where(movie => movie.Rating >= 4.5)
                             .Select(movie => movie.Name);

This query would be,

3. Any

Any checks if a collection has at least one element matching a condition. Unlike Where and Select, Any doesn’t return a new collection, but either true or false.

Let’s see if we have watched movies with a low rating.

var hasBadMovies = movies.Any(movie => movie.Rating < 2);

This query would be,

4. GroupBy

GroupBy returns a collection of “buckets” organized by a key. Also, GroupBy transforms each bucket of elements.

Let’s count the films with the same rating.

var groupedByRating = movies.GroupBy(movie => movie.Rating,
                            (rating, moviesWithSameRating) => new
                            {
                                Rating = rating,
                                Count = moviesWithSameRating.Count()
                            });

The second parameter of GroupBy is a Func with the grouping key and the elements of each group as parameters. This Func works like a mapping function to transform each group or bucket found.

This query would be,

GroupBy has other use-cases, like grouping by more than one property.

5. First & FirstOrDefault

First and FirstOrDefault return the first element in a collection or the first one matching a condition. Otherwise, First throws an exception and FirstOrDefault returns the default value of the collection type.

Let’s find the oldest film we have watched.

var oldest = movies.OrderBy(movie => movie.ReleaseYear)
                   .First();

This query would be,

Voilà! Those are five LINQ methods we use more often: Where, Select, Any, GroupBy, and FirstOrDefault.

Of course, LINQ has more methods like Aggreate, Intersect, Union, and Except, and new overloads from .NET6. But, you will get your back covered with those five methods.

To learn about LINQ and other methods, check my quick guide to LINQ with examples. All you need to know to start working with LINQ, in 15 minutes or less.

Happy LINQ time!

Brent Ozar’s Mastering Courses: My Honest Review

This is an honest review of Brent Ozar’s Mastering courses after finishing them all some months ago.

I couldn’t write this a couple of years ago. Working with databases was a subject I avoided at all costs. Even to the point where I traded database-related tasks with an ex-coworker at a past job.

Avoiding database concepts cost me painful lessons.

Like the day I wrote a query with a function around a column in the WHERE and it almost took the server to its knees. That query was poorly written, and the table didn’t have good indexes. My query ended up making SQL Server scan the whole table. Arrgggg!

But, it changed a couple of years later while working with one of my clients.

They asked me to investigate the performance of some critical parts of the app. The bottleneck was in the database and I ended up Googling what to do to speed up SQL Server queries and compiling six SQL Server performance tuning tips I found.

In that search for performance tuning advice, I found Brent Ozar and his Mastering courses. These are the three things I liked after taking them.

1. Realistic Labs and Workloads

As part of Brent’s courses, we work with a copy of the StackOverflow database. Yeap, the same StackOverflow we all know and use.

After every subject in each course, we have labs to finish. Labs with bad queries, no indexes, blocking issues, etc. For the last course, Mastering Server Tuning, we have an emergency to fix. A server is on fire, and we have to put down the fire and lay out a long-term fix.

Often, some labs have easier alternatives. Either focus on a particular issue or run a workload and assess the whole situation.

2. Constraints to Solve Labs

As we progress throughout the courses, we start to have constraints to solve the labs. For example, “no index changes allowed” or “only query-level fixes.”

But, the exercise I like the most is the “Let’s play being performance consultant.” We have to fix a workload under 30 minutes with as few changes as possible. The closest thing to a real-world emergency. That’s from Mastering Server Tuning again. My favorite course.

Of course, there are more courses. They’re four in total. There’s one course solely on indexes, another one about query tuning, one to fix parameter sniffing issues, and, my favorite, the one on server-level fixes. Each course sits on top of the previous ones.

3. Popular Wisdom and Guerilla Tactics

All over the courses, Brent shares his experience as a consultant. I have those tips and pieces of advice on my notes like “when working with clients.”

For example, he shares to build as few indexes as possible and provide rollback scripts for index creation, just in case. Also, to provide a prioritized list of actionable steps to make SQL Server fast.

Also, he shares personal anecdotes. Like the day he went to consult wearing jeans, and everybody at the place was wearing jackets and ties. That story didn’t have a happy ending for the company. But, I won’t tell you more so you can find out what happened by taking the courses.

Parting Thought

Voilà! These are the three things I liked. My biggest lessons are:

1. Focus all tuning efforts on the top-most wait type, and,
2. Make as few changes as possible to take you across the finish line.

Often, we start to push buttons and turn knobs expecting SQL Server to run faster, without noticeable improvements and making more harm than good.

I will take the second lesson to other parts of my work, even outside of performance tuning context. Focus on the few changes that make the biggest impact.

To learn how to make your SQL Server go faster with Brent—and support this blog—buy the Fundamentals Bundle here and the Fundamentals + Mastering Bundle here. I can’t recommend Brent’s courses enough. Learn from a real expert, with real queries.