Two new LINQ methods in .NET 9: CountBy and Index

LINQ doesn’t get new features with each release of the .NET framework. It just simply works. This time, .NET 9 introduced two new LINQ methods: CountBy() and Index(). Let’s take a look at them.

1. CountBy

CountBy groups the elements of a collection by a key and counts the occurrences of each key. With CountBy, there’s no need to first group the elements of a collection to count its occurrences.

For example, let’s count all movies in our catalog by release year, of course, using CountBy(),

var movies = new List<Movie>
{
    new Movie("Titanic", 1997, 4.5f),
    new Movie("The Fifth Element", 1997, 4.6f),
    new Movie("Forrest Gump", 1994, 4.3f),
    new Movie("Terminator 2", 1991, 4.7f),
    new Movie("Armageddon", 1998, 3.35f),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5),
    new Movie("Pulp Fiction", 1994, 4.3f),
};

var countByReleaseYear = movies.CountBy(m => m.ReleaseYear);
//                              ^^^^^^^
foreach (var (year, count) in countByReleaseYear)
{
    Console.WriteLine($"{year}: [{count}]");
}

// Output
// 1997: [2]
// 1994: [2]
// 1991: [1]
// 1998: [1]
// 1986: [1]
// 1988: [1]

Console.ReadKey();

record Movie(string Name, int ReleaseYear, float Rating);

CountBy() returns a collection of KeyValuePair with the key in the first position and the count in the second one.

By the way, if that Console application doesn’t look like one, it’s because we’re using three recent C# features: the Top-level statements, records, and global using statements.

Before .NET 9.0, we needed to use GroupBy() with a second parameter to transform each group, like this,

var countByReleaseYear = movies.GroupBy(
  x => x.ReleaseYear,
  (releaseYear, movies) => new
  // ^^^^^
  {
      Year = releaseYear,
      Count = movies.Count()
      //      ^^^^^
  });

CountBy() has the same spirit of DistinctBy, MinBy, MaxBy, and other LINQ methods from .NET 6.0. With these methods, we apply an action direcly on a collection using a key selector. We don’t need to filter or group a collection first to apply that action.

2. Index

Index projects every element of a collection alongside its position in the collection.

Let’s “index” our catalog of movies,

var movies = new List<Movie>
{
    new Movie("Titanic", 1998, 4.5f),
    new Movie("The Fifth Element", 1997, 4.6f),
    new Movie("Terminator 2", 1991, 4.7f),
    new Movie("Avatar", 2009, 5),
    new Movie("Platoon", 1986, 4),
    new Movie("My Neighbor Totoro", 1988, 5)
};

foreach (var (index, movie) in movies.Index())
//                                    ^^^^^
{
    Console.WriteLine($"{index}: [{movie.Name}]");
}

// Output
// 0: [Titanic]
// 1: [The Fifth Element]
// 2: [Terminator 2]
// 3: [Avatar]
// 4: [Platoon]
// 5: [My Neighbor Totoro]

Console.ReadKey();

record Movie(string Name, int ReleaseYear, float Rating);

Unlike CountBy(), Index() returns named tuples. It returns IEnumerable<(int Index, TSource Item)>.

Before, we had to use the Select() overload or roll our own extension method. In fact, this is one of the helpful extension methods I use to work with collections. But I call it Enumerated().

If we take a look at the Index source code on GitHub, it’s a foreach loop with a counter in its body. Nothing fancy!

Voilà! Those are two new LINQ methods in .NET 9.0: CountBy() and Index(). It seems the .NET team is bringing to the standard library the methods we needed to roll ourselves before.

To learn about LINQ and other methods, check my quick guide to LINQ, five common LINQ mistakes and how to fix them, and peeking into LINQ DistinctBy source code.

Happy coding!

How to Test Logging Messages with FakeLogger

Starting with .NET 8.0, we have a better alternative for testing logging and logging messages. We don’t need to roll our own mocks anymore. Let’s learn how to use the new FakeLogger<T> inside our unit tests.

.NET 8.0 introduces FakeLogger, an in-memory logging provider designed for unit testing. It provides methods and properties, such us LatestRecord, to inspect the log entries recorded inside unit tests.

Let’s revisit our post on unit testing logging messages. In that post, we used a Mock<ILogger<T>> to verify that we logged the exception message thrown inside a controller method. This was the controller we wanted to test,

using Microsoft.AspNetCore.Mvc;

namespace FakeLogger.Controllers;

[ApiController]
[Route("[controller]")]
public class SomethingController : ControllerBase
{
    private readonly IClientService _clientService;
    private readonly ILogger<SomethingController> _logger;

    public SomethingController(IClientService clientService,
                               ILogger<SomethingController> logger)
    {
        _clientService = clientService;
        _logger = logger;
    }

    [HttpPost]
    public async Task<IActionResult> PostAsync(AnyPostRequest request)
    {
        try
        {
            // Imagine that this service does something interesting...
            await _clientService.DoSomethingAsync(request.ClientId);

            return Ok();
        }
        catch (Exception exception)
        {
            _logger.LogError(exception, "Something horribly wrong happened. ClientId: [{clientId}]", request.ClientId);
            //      ^^^^^^^^
            // Logging things like good citizens of the world...

            return BadRequest();
        }
    }
}

// Just for reference...Nothing fancy here
public interface IClientService
{
    Task DoSomethingAsync(int clientId);
}

public record AnyPostRequest(int ClientId);

1. Creating a FakeLogger

Let’s test the PostAsync() method, but this time let’s use the new FakeLogger<T> instead of a mock with Moq.

To use the new FakeLogger<T>, let’s install the NuGet package: Microsoft.Extensions.Diagnostics.Testing first.

Here’s the test,

using FakeLogger.Controllers;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Testing;
//    ^^^^^
using Moq;

namespace FakeLogger.Tests;

[TestClass]
public class SomethingControllerTests
{
    [TestMethod]
    public async Task PostAsync_Exception_LogsException()
    {
        var clientId = 123456;

        var fakeClientService = new Mock<IClientService>();
        fakeClientService
            .Setup(t => t.DoSomethingAsync(clientId))
            .ThrowsAsync(new Exception("Expected exception..."));
        //               ^^^^^
        // 3...2...1...Boom...

        // Look, ma! No mocks here...
        var fakeLogger = new FakeLogger<SomethingController>();
        //                   ^^^^^
        var controller = new SomethingController(
                                fakeClientService.Object,
                                fakeLogger);
                                // ^^^^^

        var request = new AnyPostRequest(clientId);
        await controller.PostAsync(request);

        // Warning!!!
        //var expected = $"Something horribly wrong happened. ClientId: [{clientId}]";
        //Assert.AreEqual(expected, fakeLogger.LatestRecord.Message);
        //       ^^^^^^^^
        // Do not expect exactly the same log message thrown from PostAsync()
        
        // Even better:
        fakeLogger.VerifyWasCalled(LogLevel.Error, clientId.ToString());
        //         ^^^^^
    }
}

We needed a using for Microsoft.Extensions.Logging.Testing. Yes, that’s different from the NuGet package name.

We wrote new FakeLogger<SomethingController>() and passed it around. That’s it.

2. Asserting on FakeLogger

The FakeLogger<T> has a LatestRecord property that captures the last entry we logged. Its type is FakeLogRecord and contains a Level, Message, and Exception. And if no logs have been recorded, accessing LatestRecord will throw an InvalidOperationException with the message “No records logged.”

But, for the Assert part of our test, we followed the lesson from our previous post on testing logging messages: do not expect exact matches of logging messages in assertions. Otherwise, any change in the structure of our logging messages will make our test break, even if the underlying business logic remains unchanged.

Instead of expecting exact matches of the logging messages, we wrote an extension method VerifyWasCalled(). This method receives a log level and a substring as parameters. Here it is,

public static void VerifyWasCalled<T>(this FakeLogger<T> fakeLogger, LogLevel logLevel, string message)
{
    var hasLogRecord = fakeLogger
        .Collector
        // ^^^^^
        .GetSnapshot()
        // ^^^^^
        .Any(log => log.Level == logLevel
                    && log.Message.Contains(message, StringComparison.OrdinalIgnoreCase));
                    // ^^^^^

    if (hasLogRecord)
    {
        return;
    }

    // Output:
    //
    // Expected log entry with level [Warning] and message containing 'Something else' not found.
    // Log entries found:
    // [15:49.229, error] Something horribly wrong happened. ClientId: [123456]
    var exceptionMessage = $"Expected log entry with level [{logLevel}] and message containing '{message}' not found."
        + Environment.NewLine
        + $"Log entries found:"
        + Environment.NewLine
        + string.Join(Environment.NewLine, fakeLogger.Collector.GetSnapshot().Select(l => l));

    throw new AssertFailedException(exceptionMessage);
}

First, we used Collector and GetSnapshot() to grab a reference to the collection of log entries recorded. Then, we checked we had a log entry with the expected log level and message. Next, we wrote a handy exception message showing the log entries recorded.

Voilà! That’s how to write tests for logging messages using FakeLogger<T> instead of mocks.

If we only want to create a logger inside our tests without asserting anything on it, let’s use NullLogger<T>. But, if we want to check we’re logging exceptions, like good citizens of the world, let’s use the new FakeLogger<T> and avoid tying our tests to details like the log count and the exact log messages. That makes our tests harder to maintain. In any case, we can roll mocks to test logging.

If you want to read more about unit testing, check How to write tests for HttpClient using Moq, how to test an ASP.NET Authorization Filter and how to write good unit tests: Use simple test values. Don’t miss my Unit Testing 101 series where we cover from what a unit test is, to fakes and mocks, to other best practices.

Happy testing!

How I used AI to launch my new testing course

I ran an experiment. Maybe it was fear of missing out. I decided to use AI to help me launch a new course. This is how I used Copilot and the prompts I used.

I got this idea after watching one of Brent Ozar’s Office Hours videos on YouTube where he shared he keeps ChatGPT opened all the time and uses it as a junior employee. I decided to run a similar experiment, but for launching a new course on unit testing, one of my favorite subjects.

1. Lesson content and materials

I planned the lesson content and recorded and edited all video lessons myself. #madebyahuman

For the editing part, I used Adobe Podcast to remove background noise. My neighbor’s dog started to bark every time I hit record. And the other day at home, somebody made a smoothie with a loud blender while I was recording. Arrrggg! The only downside is that my voice sounds auto-tuned, especially in word endings.

I used Copilot for its convenience. I don’t need to create an account. Even I made Microsoft Edge open Copilot as the default tab. I only need to press the Windows key, type “Edge,” and I’m right there.

These are the prompts I used.

Generate a list of title ideas for my course

You’re an expert on course creation, programming, and SEO, give me a list of titles for a course to teach insert subject here

Rewrite my draft for a landing page

Now you’re an expert on online writing, SEO, marketing, and copywriting, please help me improve this landing page for an online course to increase sales and conversions. Make sure to use a friendly and conversational tone.

This is my landing page:

insert landing page here

Turn a landing page into a script for an introductory video

Now turn that last version of the landing page into a list of paragraphs and sentences I can use to create a PowerPoint presentation. Keep it short and to the point. Use only 10 paragraphs. I will turn each paragraph into a slide

2. Online Marketing

Once I got all the lesson content and a landing page ready, I moved to the promotion part.

These are the prompts I used.

Write an email inviting readers to buy this new course

You’re an expert on copywriting and email marketing, give me n ideas for a short email to invite a reader who already took some action to buy my new video course: insert course name here. In that course, I insert brief description here. Offer a promo code for a limited time. Use friendly and engaging language.

Write a call to action for my posts

You’re an expert copywriter, give me n ideas for a two-sentence paragraph to promote my new course insert course name here. I’d like to use that paragraph at the end of posts on my blog to invite my readers to join the course. Use a friendly and conversational tone.

Write a launching post on LinkedIn

You’re an expert on copywriting, LinkedIn, and personal branding. Give me n ideas for a LinkedIn post to promote my new course insert title here based on the landing page of the course. Use a friendly and conversational tone.

This is the course landing page of the course:

insert landing page here

Nothing fancy. I followed the pattern: “Act as X, do Y for me based on some input. This is the input.”

I don’t use the exact same words Copilot gives me. I change the words I don’t use to make it sound like me.

Voilà! That’s how I used AI, especially Copilot, to help me launch my new testing course. On a Saturday morning, I ended up with a landing page and the script for an intro video for the course. What a productive morning!

I use AI the same way Jim Kwik, the brain coach, recommends in one of his YouTube videos: “AI (artificial intelligence) to enhance HI (human intelligence), not to replace it.” I don’t want AI to take out the pleasure of doing what I like to do. It’s my assistant.

In case you’re curious, this is the end result: Mastering C# Unit Testing with Real-world Examples.

For more productivity content (but not AI-related), check how to color a website based on its URL and how to declutter sites with uBlock Origin filters.

Happy coding!

Extreme Ownership: How U.S. Navy SEALs Lead and Win. Takeaways

By the title, I guess you infer this isn’t a book about software engineering or programming. It’s about leadership. By any means, I’m advocating for war. But I think the military has to teach a lot about management and leadership. For years, they have been leading large organizations with complex tasks in changing environments. That sound a lot like software engineering, right?

Extreme Ownership is a book about the stories of two Navy SEALs officers and the lessons they learned while in service. In every chapter, they tell a story about their time in service, the leadership principle behind it, and its application to the corporate world. It’s an easy read. Probably, you could finish it in a weekend, too.

These are some of my takeaways.

Extreme Ownership & Leadership

Extreme Ownership is the principle that a leader “owns” or is responsible for everything that happens to the mission or his team. Even to take the blame when things go wrong. “The leader is truly and ultimately responsible for everything.”

Leaders, who accept their responsibility when things go wrong, inspire respect and trust. They show and teach that attitude to the team.

A leader is responsible even for the underperformers. His job is to train and mentor them to level up their skills.

A leader should lead “up” too. He should speak up and ask questions when things could be better. He is also responsible for the communication with his own leaders.

A leader should create a simple plan and make sure everyone understands it clearly. A leader should first understand and believe in the mission. And when priorities change, he should communicate those changes and pass “situational awareness.”

A leader should put a system and mentor people so he can be the “tactical genius” looking at the bigger picture.

Voilà! Those are some of my takeaways. The book uses memorable stories to tell these principles. I like how the authors extrapolated the principles they learned while on duty to the corporate world. It turns out that the authors, as SEAL officers, had to deal with some of the same bureaucracy from the corporate world too. Even PowerPoint presentations! PowerPoint presentations! I didn’t see that coming.

As someone who influences decision-making inside teams, I enjoyed the book, and I will take lots of these ideas into my daily activities.

I’d like to close with this quote from the book: “there are no bad teams, only bad leaders.”

After reading this book, I connected the dots with some of the lessons I learned after a failed project. I needed Extreme Onwership in that project. Too late.

If you want to read my takeaways from proramming books, read Unit Testing Principles, Practices, and Patterns and Hands-on Domain-Driven Design with .NET Core.

Happy reading!

Monday Links: Notebooks, Learning, and KPIs

Five interesting reads from the past month.

Lab Notebooks

I like the idea of keeping “lab” notebooks, especially to record the thought process for consulting clients. I’m a big fan of plain text. But the article recommends using pen and paper.

Read full article

Git Things

Git and Version Control are one of the subjects we have to learn ourselves. I prefer short Pull Requests with only one or two commits. I name my commits with a long and descriptive title that becomes a good title on PR descriptions.

This article shows “tactics” to better use Git. I really like these two:

1. start by writing the commit message before the actual work and then amending it. Commit Message Driven Development I guess, to follow the XDD trend
2. splitting refactorings into two commits: changing the API and then changing the calling sites

Read full article

10 Things Software Developers Should Learn about Learning

This article contains ten findings, backed by research, about learning and software engineering. Totally worth reading if you’re into learning about learning.

These are some of my favorite lessons:

• “Expert programmers may have low or high working memory capacity but it is the contents of their long-term memory that make them experts.”
• “Expert developers can reason at a higher level by having memorized (usually implicitly, from experience) common patterns in program code, which frees up their cognition.”
• “The brain needs rest to consolidate the new information that has been processed so that it can be applied to new problems.”
• “There is a key distinction between a beginner who has never learned the details and thus lacks the memory connections, and an expert who has learned the deeper structure but searches for the forgotten fine details.”
• “There is no reliable correspondence between problem-solving in the world of brain teasers and problem-solving in the world of programming. If you want to judge programming ability, assess programming ability.”

If you don’t want to read the whole article, jump to the Summary section at the end.

Read full article

Why Most Software Engineering KPIs are BS and What to Do About it in 2024

Does your team measure story points too? The truth is we don’t know how to measure our work. I worked with a company that measured the percentage of completed tasks. It was a killing pressure. And once we have a metric, we start to game it. On the last day of the sprint, we saw people removing tasks so as not to mess with the final percentage.

From the article: “Engineering leaders (and sometimes non-technical executives) have often made the poor choice to measure output rather than impact”. The author recommends: “separate engineering health metrics to KPIs.” This is how the team feels vs the business impact of the work done.

Read full article

27 Years Ago, Steve Jobs Said the Best Employees Focus on Content, Not Process.

This article tells Steve Jobs’s opinion on office politics and bureaucracy and how often we think “paperwork” is the real work. Process vs Output. Also, this article shares how to keep the best team members around, those who are good at identifying the output.

From the article: “Success is rarely based solely on process. Success mostly comes down to what your business accomplishes.”

Voilà! What are the KPIs of your team? Do you measure story points or percentage of completed vs planned tasks? Until next Monday Links.

In the meantime, don’t miss the previous Monday Links on NDC Copenhagen.

Happy coding!