Pipeline pattern: An assembly line of steps

You need to do a complex operation made of smaller consecutives tasks. These tasks might change from client to client. This is how you can use the Pipeline pattern to achieve that. Let’s implement the Pipeline pattern in C#.

With the Pipeline pattern, a complex task is divided into separated steps. Each step is responsible for a piece of logic of that complex task. Like an assembly line, steps in a pipeline are executed one after the other, depending on the output of previous steps.

TL;DR Pipeline pattern is like the enrich pattern with factories. Pipeline = Command + Factory + Enricher

When to use the Pipeline pattern?

You can use the pipeline pattern if you need to do a complex operation made of smaller tasks or steps. If a single task of this complex operation fails, you want to mark the whole operation as failed. Also, the tasks in your operation vary per client or type of operation.

Some common scenarios to use the pipeline pattern are booking a room, generating an invoice or creating an order.

Let’s use the Pipeline pattern

A pipeline is like an assembly line in a factory. Each workstation in an assembly adds a part until the product is assembled. For example, in a car factory, there are separate stations to put the doors, the engine and the wheels.

With the pipeline pattern, you can create reusable steps to perfom each action in your “assembly line”. Then, you run these steps one after the other in a pipeline.

For example, in an e-commerce system to sell an item, you need to update the stock, charge a credit card, send a delivery order and notify the client.

Let’s implement our own pipeline

First, create a command/context class for the inputs of the pipeline.

public class BuyItemCommand : ICommand
{
    // Item code, quantity, credit card information, etc
}

Then, create one class per each workstation of your assembly line. These are the steps.

In our e-commerce example, steps will be UpdateStockStep, ChargeCreditCardStep, SendDeliveryOrderStep and NotifyClientStep.

public class UpdateStockStep : IStep<BuyItemCommand>
{
    public Task ExecuteAsync(BuyItemCommand command)
    {
        // Put your own logic here
        return Task.CompletedTask;
    }
}

Next, we need a builder to create our pipeline with its steps. Since the steps may vary depending on the type of operation or the client, you can load your steps from a database or configuration files.

For our e-commerce example, we don’t need to create a delivery order when we sell an eBook. In that case, we need to build two pipelines: BuyPhysicalItemPipeline for products that require shipping and BuyDigitalItemPipeline for products that don’t.

But, let’s keep it simple. Let’s create a BuyItemPipelineBuilder.

public class BuyItemPipelineBuilder : IPipelineBuilder
{
    private readonly IStep<BuyItemCommand>[] Steps;

    public BuyItemPipelineBuilder(IStep<BuyItemCommand>[] steps)
    {
        Steps = steps;
    }

    public IPipeline CreatePipeline(BuyItemCommand command)
    {
      // Create your pipeline here...
      var updateStockStep = new UpdateStockStep();
      var chargeCreditCardStep = new ChargeCreditCard();
      var steps = new[] { updateStockStep, chargeCreditCardStep };
      return new BuyItemPipeline(command, steps);
    }
}

Now, create the pipeline to run all its steps. It will have a loop to execute each step.

public class BuyItemPipeline : IPipeline
{
    private readonly BuyItemCommand Command;
    private readonly IStep<BuyItemCommand>[] Steps;

    public BuyItemPipeline(BuyItemCommand command, IStep<BuyItemCommand>[] steps)
    {
        Command = command;
        Steps = steps;
    }

    public async Task ExecuteAsync()
    {
        foreach (var step in Steps)
        {
            await step.ExecuteAsync(Command);
        }
    }
}

Also, you can use the Decorator pattern to perform orthogonal actions on the execution of the pipeline or every step. You can run the pipeline inside a database transaction, log every step or measure the execution time of the pipeline.

Now everything is in place, let’s run our pipeline.

var command = new BuyItemCommand();
var builder = new BuyItemPipelineBuilder(command);
var pipeline = builder.CreatePipeline();

await pipeline.ExecuteAsync();

Some steps of the pipeline can be delayed for later processing. The user doesn’t have to wait for some steps to finish his interaction with the system. You can schedule the execution of some steps in background jobs for later processing. For example, you can use Hangfire or roll your own queue mechanism (Kiukie…Ahem, ahem)

Conclusion

Voilà! This is the Pipeline pattern. You can find it out there or implement it on your own. Depending on the expected load of your pipeline, you could use Azure Functions or any other queue mechanism to run your steps.

I have used and implemented this pattern before. I used it in an invoicing platform to generate documents. Each document and client type had a different pipeline.

Also, I have used it in a reservation management system. I had separate pipelines to create, modify and cancel reservations.

PS: You can take a look at Pipelinie to see more examples. Pipelinie offers abstractions and default implementations to roll your own pipelines and builders.

All ideas and contributions are more than welcome!

Clean Code: Takeaways

Clean Code will change the way you code. It doesn’t teach how to code in a particular language. But, it teaches how to produce code easy to read, grasp and maintain. Although code samples are in Java, all concepts can be translated to other languages.

The Clean Code starts defining what it’s clean code by collecting quotes from book authors and other well-known people in the field. It covers the subject of Clean Code from variables to functions to architectural design.

The whole concept of Clean Code is based on the premise that code should be optimized to be read. It’s true we, as programmers, spend way more time reading code than actually writing it.

These are the three chapters I found instructive. The whole book is instructive. But, if I could only read a few chapters, I would read the next ones.

Naming

The first concept after the definition of Clean Code is naming things. This chapter encourages names that reveal intent and are easy to pronounce. And, to avoid punny or funny names.

Instead of writing int d; // elapsed time in days, write int elapsedTimeInDays.

Instead of writing, genymdhms, write generationTimestamp.

Instead of writing, HolyHandGrenade, write DeleteItems.

Comments

Clean Code is better than bad code with comments.

We all have heard that commenting our code is the right thing to do. But, this chapter shows what actually needs comments.

Have you seen this kind of comment before? i++; // Increment i Have you written them? I did once.

Don’t use a comment when a function or variable can be used.

Don’t keep the list of changes and authors in comments at the top of your files. That’s what version control systems are for.

Functions

There is one entire chapter devoted to functions. It recommends to write short and concise functions.

“Functions should do one thing. They should do it well”.

This chapter discourages functions with boolean parameters. They will have to handle the true and false scenarios. Then, they won’t do only one thing.

Voilà! These are the three chapters I find the most instructive and more challenging. If you could only read a few chapters, read those ones. Clean Code should be an obligatory reading for every single developer. Teachers should, at least, point students to this book. This book doesn’t deserve to be read, it deserves to be studied. If you’re new to the Clean Code concept, grab a copy and study it.

If you’re interested in my takeaways of other books, take a look at Clean Coder and The Art of Unit Testing.

Happy reading!

Tips and Tricks for Better Code Reviews

Are you new to code reviews? Do you know what to look for in a code review? Do you feel frustrated with your code review? I’ve been there too. Let’s see some tips I learned and found to improve our code reviews.

Code review is a stage of the software development process where a piece of code is examined to find bugs, security flaws, and other issues. Often reviewers follow a coding standard and style guide while reviewing code.

TL;DR

1. For the reviewer: Be nice. Remember you are reviewing the code, not the writer.
2. For the reviewee: Don’t take it personally. Every code review is an opportunity to learn.
3. For all the dev team: Reviews take time too. Add them to your estimates.

Advantages of Code Reviews

Code reviews are a great tool to identify bugs before the code gets shipped to end users. Sometimes we only need another pair of eyes to spot unnoticed issues in our code.

Also, code reviews ensure that the quality of the code doesn’t degrade as the project moves forward. They help to spread knowledge inside a team and mentor new members.

Now that we know what code reviews are good for, let’s see what to look for during code reviews and tips for each role in the review process.

What to look for in a code review?

If we’re new to code reviews and we don’t know what it’s going to be reviewed in our code…or if we have been asked to review somebody else code and we don’t know what to look for, we can start looking at this:

Does the code:

• Compile in somebody else machine? If you have a Continuous Integration/Continuous Deployment (CI/CD) tool, we can easily check if the code is compiling and all tests are passing.
• Include unit or integration tests?
• Introduce new bugs?
• Follow current standards?
• Reimplement things? Is some logic already implemented in the standard library or an extension method?
• Build things the hard way?
• Kill performance?
• Have duplication? Has the code been copied and pasted?

It’s a good idea to have a checklist next to us while reviewing the code. We can create our own checklist or use somebody else as a reference. Like Doctor McKayla Code Review Checklist.

For the reviewer

Before we start any review, let’s understand the context around the code we’re about to review.

A good idea is to start by looking at the unit tests and look at the “diff” of the code twice. One for the general picture and another one for the details.

If we’re a code reviewer, let’s:

• Be humble. We all have something to learn.
• Take out the person when giving feedback. We are reviewing the code, not the author.
• Be clear. We may review code from juniors, mid-level, or seniors, even from non-native speakers of our language. Everybody has different levels of experience. Obvious things for us aren’t obvious for somebody else.
• Give actionable comments. Let’s not use tricky questions to make the author change something. Let’s give clear and actionable comments instead. For example, what do you think about this method name? vs I think X would be a better name for this method. Could we change it?
• Always give at least one positive remark. For example: It looks good to me (LGTM), good choice of names.
• Use questions instead of commands or orders. For example, Could this be changed? vs Change it.
• Use “we” instead of “you”. We’re part of the development process too. We’re also responsible for the code we’re reviewing.
• Instead of showing an ugly code, teach. Let’s link to resources to explain even more. For example, blog posts and StackOverflow questions.
• Review only the code that has changed. Let’s stop saying things like Now you’re here, change that method over there too.
• Find bugs instead of style issues. Let’s rely on linters, compiler warnings, and IDE extensions to find styling issues.

Recently, I found out about Conventional Comments. With this convention, we start our comments with labels to show the type of comments (suggestion, nitpick, question) and their nature (blocking, non-blocking, if-minor).

I use Conventional Comments to avoid tricky or pushy questions during code reviews.

For the reviewee

Before asking someone to review our code, let’s review our own code. For example, let’s check if we wrote enough tests and followed the naming conventions and styling guidelines.

It’s a good idea to wait for the CI/CD to build and run all tests before asking someone to review our changes or assign reviewers in a web tool. This will save time for our reviewers and us.

If we’re a reviewee, let’s:

• Stop taking it personally. It’s the code under review, not us.
• Find in every code review an opportunity to learn. Let’s identify frequent comments and avoid them in the future. For example, I use one Git hook to add ticket numbers to every commit message and another hook to format SQL files.
• Give context. Let’s give enough context to our reviews. We can write an explanatory title and a description of what our code does and what decisions we made.
• Keep your work short and focused. Let’s not make reviewers go through thousands of lines of code in a single review session. For example, we can separate changes in business logic from formatting/styling.
• Keep all the discussion online. If we contact reviewers by chat or email, let’s bring relevant comments to the reviewing tool for others to see them.

For team management

If we’re on the management side, let’s:

• Make code reviews have the highest priority. We don’t want to wait days until we get our code reviewed.
• Remember code reviews are as important as writing code. They take time too. Let’s add them to our estimates.
• Have as a reviewer someone familiar with the code being reviewed. Otherwise, we will get styling and formatting comments. People judge what they know. That’s a cognitive bias.
• Have at least two reviewers. For example, as reviewees, let’s pick the first reviewer. Then he will choose another one until the two of them agree.

Voilà! These are the tips I’ve learned while reviewing other people’s code and getting mine reviewed too. Code reviews can be frustrating. Especially when they end up being a discussion about styling issues and naming variables. I know, I’ve been there.

One of my lessons as a code reviewer is to use short and focused review sessions. I prefer to have short sessions in a day than a single long session that drains all my energy. Also, I include a suggestion or example of the change to be made in every comment. I want to leave clear and actionable comments on every code review.

Of course, I didn’t come up with some of these tips. These are the resources I used to learn how to do better code reviews: NDC’s Code Review Etiquettes 101, Russell Cohen’s How to code review, Smashing Magazine’s Bringing A Healthy Code Review Mindset To Your Team, and FreeCodeCamp’s A Zen Manifesto for Effective Code Reviews.

Happy coding!

Two C# idioms

This post is part of the series "C# idioms"

1. Two C# idioms This post
2. Another two C# idioms
3. Two C# idioms: On Dictionaries
4. Two C# idioms: On defaults and switch

You have heard about Pythonic code? Languages have an expressive or “native” way of doing things. But, what about C#? Is there C-Sharpic code?

In this series of posts, I will attempt to present some idioms or “expressions” to write more expressive C# code. I collected these idioms after reviewing code and getting mine reviewed too.

In this first part, you have two useful C# idioms on conditionals and its alternative solutions.

Instead of lots of or’s, use an array of possible values

Use an array of known or valid options, instead of a bunch of comparison inside an if statement.

Use this idiom when checking preconditions or validating objects.

Before, we used comparison with || inside an if statement.

if (myVar == 2 || myVar == 5 || myVar == 10)
{
    DoSomeOperation();
}

After, we can use an array of valid options.

var allowedValues = new int[] { 2, 5, 10 };
if (allowedValues.Any(t => myVar == t))
{
    DoSomeOperations();
}

If you need to check for a new value, you add it in the array instead of adding a new condition in the if statement.

Instead of lots of if’s to find a value, use an array of Func

Replace consecutive if statements to find a value with an array of Func or small choice functions. Then, pick the first result different from a default value or null.

Use this idiom when finding a value among multiple choices.

Before, we used consecutive if statements.

var someKey = FindKey();
if (someKey == null)
    someKey = FindAlternateKey();
if (someKey == null)
    someKey = FindDefaultKey();

After, we use a list of Func.

var fallback = new List<Func<SomeObject>>
{
    FindKey(),
    FindAlternateKey(),
    FindDefaultKey()
};
var someKey = fallback.FirstOrDefault(t => t != null);

You can take advantage of the Null-coalescing operator (??) if these choice functions return null when a value isn’t found.

The Null-coalescing operator returns the expression on the left it it isn’t null. Otherwise, it evaluates the expression on the right.

var someKey = FindKey() ?? FindAlternateKey() ?? FindDefaultKey();

Similarly, if you need to add a new alternative, either you add it in the array or nest it instead of adding the new alternative in the if statement.

Validate a complex object with an array of Func

Also, this last idiom is useful when validating an object against a list of rules or conditions. Create a function for every rule, then use All() LINQ method to find if the input object or value satisfy all required rules.

var toValidate = new ComplexObject();

Func<ComplexObject, bool> Validation = (obj) => /* Validate something here...*/;
Func<ComplexObject, bool> AnotherValidation = (obj) => /* Validate something else here...*/;

var validations = new List<Func<ComplexObject, bool>>
{
    Validation,
    AnotherValidation
}
var isValid = validations.All(validation => validation(toValidate));

Voilà! These are our first two idioms on conditionals. I have found these two idioms more readable in some scenarios. But, don’t start to rewrite or refactor your code to follow any convention you find online. Make sure to follow the conventions in your own codebase, first.

These two first idioms rely on Func and LINQ, if you need to learn more on these subjects, check What the Func, Action! and Quick Guide to LINQ.

Happy coding!

Git guide for TFS users

Dear developer, you’re working in a project that uses Team Foundation Server, TFS. You’re used to check-out and check-in your files. Now you have to use Git. Do you know how the two relate to each other? This is what makes TFS and Git different.

Team Foundation Server (TFS) and Git belong to two different types of Version Control Systems. TFS is centralized and Git is distributed. This distinction makes collaborating with others, syncing changes and branching different.

Centralized vs Distributed

Centralized version control systems need a server in place to operate. To version control a file, view the history of a file or perform any other operation you need to communicate to a server.

But, distributed version control systems don’t need a server. Each operation is performed in a local copy of the project. You could choose to sync your local copy with a server later on.

TFS is a centralized version control system and Git a distributed one.

TFS vs Git

This distinction between Centralized and Distributed Version Control Systems brings some differences between TFS and Git. These are some of them.

Exclusive checkout

Have you ever needed to modify a file and you couldn’t because a co-worker was working on that file too? That’s Exclusive Checkout.

Exclusive Checkout is a feature you can turn on or off in TFS. But, it isn’t available in Git. Because, there is no lock on files to have only one user working in a file at a time.

With Git, each developer works in his own copy of the project.

No mappings

Before starting to work with a project under TFS, you need some “mappings”. A place where the files in TFS server will be in your computer.

With Git, you can clone and keep a project anywhere in your computer. There’s no such a thing as mappings.

Checking and Pushing

After modifying some files, you want to version control them. With TFS you “check-in” your files. This means, those files are kept in the history of the project and synced with the server.

With Git, you don’t “check-in” your files, you “commit” them. And, your committed files live only in your local copy of the project, until you sync or push them to a server.

Branching

With Git, branches have a new meaning. You could have lots of light-weight and short-lived branches to try things out, solve bugs or do your everyday work. By convention, all Git repositories start with a branch called master.

Starting from Git 2.28, Git will use the configuration value init.defaultBranch to name the default branch. Other alternatives for master are: main, primary or default.

Gitflow, a branching model

Maybe, you have worked with one branch per environment: Dev, QA, Beta and Production. You start every new task directly on the Dev branch.

There is another branch model: Gitflow.

Gitflow suggests that the master branch mirrors the Production environment. The develop branch is where everyday work happens. But, every new task starts in a new branch taken from develop. Once you’re done with your task, you merge this branch back to develop.

In case of bugs or issues in Production, you create a new branch from master. And you merge it back to master and develop once you’re done fixing the issue.

Every release has a separate branch, too. This release branch is also merged to master and develop. For more details, see Introducing GitFlow.

Pull or Merge Requests

Some projects adopt another convention. Nobody pushes or syncs directly to the master or develop branch.

Instead, every new task goes through a code review phase using a pull or merge request. During code review, your code is examined to find bugs and to stick to coding conventions.

A pull or merge request means that “the author of the task asks to merge his changes from a branch into the current codebase”.

Most of the time, this code review is done through a web tool or web interface. After one or two team members review your changes, the same tool will allow the reviewer to merge the changes to the destination branch.

Reference

This is how TFS and Git actions relate to each other.

Changesets = Commits

In TFS, a changeset is a set of changes that have been version-controled and synced to the server.

With Git, you have commits. Your commits can live only on your local copy, until you upload them to a server.

If you want to collaborate with others or host your code somewhere else, you can use a server as a single point of authority.

Have you ever heard about GitHub, GitLab or Bitbucket? These are third-party hosting solutions for Git.

Check-in = Commit + Push

With TFS, you “check-in” your files. But, with Git, from the command line, you need three commands: add, commit and push.

$ git add .
$ git commit -m 'A beatiful commit message'
$ git push origin my-branch

Get latest version = Pull

To get the most recent changes from a server, with TFS, you “get latest version”.

With Git, you need to “pull” from a remote branch on a server. Use git pull origin my-branch.

By convention, the name associated to the sync server is origin. You can add other remotes if needed.

Branch = Branch + Checkout

With TFS, you create a new branch from the source branch using the “Branch” option. If you have a large project, this could take a while.

With Git, you need to create a branch and switch to it. Use the commands: branch and checkout, respectively. But, you can combine these two actions into a single one with checkout and the -b flag.

$ git checkout -b a-new-branch

Shelve = stash

If you want to temporary suspend your work and resume it later, you use a shelve in TFS or an stash with Git.

With Git, to create an stash, use git stash -u. And, git stash apply to bring back your changes.

Integration

Please, do not be afraid of using the command line and memorizing lots of commands. There are Git clients and out-of-the-box integrations (or plugins) for Git in most popular IDE’s. For example, Visual Studio and Visual Studio Code support Git out-of-the-box, and SourceTree is a Git client you can download for free.

Voilà! That’s what makes Git and TFS different. Now you know where your changesets are, how to check-out to files and why you don’t need any mappings with Git.

For a more in-depth guide to Git, read my beginner’s guide to Git and GitHub. If you’re interested in code reviews, check my post on Better Code Reviews.

Happy Git time!