ExceptionHandler Needed

As a follow-up to the Create a Custom ProblemDetailsFactory post, it has been discovered that a custom Exception Handler must be defined in order to use the ProblemDetailsFactory. The Exception Handler can be incredibly simple, but it must produce an IActionResult that will trigger the calling of a ProblemDetailsFactory. This can be accomplished with something as simple as this:

	using Microsoft.AspNetCore.Mvc;
	namespace Your.Namespace
	{
	[ApiExplorerSettings(IgnoreApi = true)]
	public class YourBussExceptionHandlerController : ControllerBase
	{
	[Route("/your-buss-exception-handler")]
	public IActionResult ExceptionHandler() => Problem();
	}
	}

view raw YourBussExceptionHandlerController.cs hosted with ❤ by GitHub

The controller only needs to create a Problem IActionResult to trigger the usage your ProblemDetailsFactory.
To ensure that YourBussProblemDetailsFactory is used, you can create two extension functions to use during Startup.cs’s ConfigureServices and Configure methods. That might look like this:

	namespace Your.Namespace
	{
	public class Startup
	{
	public void ConfigureServices(IServiceCollection services)
	{
	services.AddControllers();
	services.AddYourBussErrorAndValidationHandling();
	}
	public void Configue(IApplicationBuilder app, IWebHostEnvironment env)
	{
	app.UseYourBussErrorAndValidationHandling();
	}
	}
	}

view raw Startup.cs hosted with ❤ by GitHub

And the extensions functions that provide the wiring would look like the code sample below. Some noticeable pieces in the example are:

• Within the IServiceCollection extension method, .AddMvc().AddApplicationPart(thisAssembly) is used to ensure that the controller from above is included within the top level application. This is how you can add controllers from sub-libraries.
• Within the IApplicationBuilder extension method, the final wiring to setup the global exception handler is created to “/your-buss-exception-handler”.

	using System;
	using System.Collections.Generic;
	using System.Reflection;
	using System.Text;
	using Microsoft.AspNetCore.Builder;
	using Microsoft.AspNetCore.Mvc.Infrastructure;
	using Microsoft.Extensions.Configuration;
	using Microsoft.Extensions.DependencyInjection;
	using Ucsb.Sa.Enterprise.AspNetCore.Mvc.Core;
	using Ucsb.Sa.Enterprise.AspNetCore.Mvc.Hosting;
	namespace Microsoft.Extensions.DependencyInjection
	{
	public static class YourBussErrorAndValidationExtensions
	{
	public static IServiceCollection AddYourBussErrorAndValidationHandling(this IServiceCollection services)
	{
	services.AddTransient<IYourBussApiExceptionDetailConverter, YourBussApiExceptionDetailConverter>();
	services.AddTransient<ProblemDetailsFactory, YourBussProblemDetailsFactory>();
	var thisAssembly = Assembly.GetAssembly(typeof(YourBussExceptionHandlerController));
	services.AddMvc().AddApplicationPart(thisAssembly);
	return services;
	}
	}
	}
	namespace Microsoft.Extensions.Configuration
	{
	public static class YourBussErrorAndValidationExtensions
	{
	public static IApplicationBuilder UseYourBussErrorAndValidationHandling(this IApplicationBuilder app)
	{
	app.UseExceptionHandler("/your-buss-exception-handler");
	return app;
	}
	}
	}

view raw YourBussErrorAndValidationExtensions.cs hosted with ❤ by GitHub

There's an example Exception Detail Converter in a follow-up post (ExceptionDetailConverter Example).

SSH Key Auth to GitHub on Win 10 w/ VSCode

My work computer has been using SSH keys to authenticate to GitHub for a while. But I’ve slept a few nights since I set that up and I have no clear memory of what I did.

I wanted to setup my home computer the the same way and struggled to figure out how to do it. So, I thought it might be worth documenting.

The secret (I think) … install git 2.20.0 or higher

In the end, the final change that made SSH key authentication work was updating my git installation from version 2.15.0 to 2.25.0. My work computer has 2.20.0 on it, so I figure that should be the minimum level.

Here’s an outline of the things I tried and notes about them:

	Work Computer	Home Computer	Notes
git (version)	2.20.0	2.15.0 –> 2.25.0	Didn’t work with 2.15.0. Finally worked with 2.25.0.
SSH keys	I don’t remember how I generated them. I think I generated the keys using ubuntu WSL.	Copied them from my work computer to my home computer using normal NTFS system (didn’t need Git Bash, WSL, or any of those). I did register the keys using ssh-add in `Git Bash`, ‘wsl’, and using the Windows 10 ssh-add (see Notes). But, in the end, I turned off Win10’s ssh-agent service and the SSH keys continued to be used for authentication.	On my current version of Win 10, you can start an ssh-agent service in windows. Which means you don’t need to to use a bash command prompt to execute `ssh-keygen` or `ssh-add` commands. Reminder: use `ssh-add –l` to list already registered keys.
Github PAT	I never created one for this machine.	I created one for this machine, and it would work for an individual commit (username: normal github account name, password: PAT)	I don’t think this is needed.

Reducing Noise in Error Logs / Removing PS Errors

I have a nightly scheduled job which will send out a notification email if the job has an error occur anywhere within it (even when the error is handled). This job infrequently sends out the error email. However, my long history of reviewing these emails has brought me to the point where I assume the error is always:

• There was a file lock on file X when the file was being save; the function detected the error, waited a brief time period for the lock to clear and then retried the save operation successfully.

I can't remember a time when that wasn't the case. Because of this, I am finding myself less interested in actually reading the error message and desiring to simply ignore the email. But, I know that is going to lead to a situation where something unexpected will happen and I'll ignore the warning emails. Which would be a failure of the entire warning system.

So, what I have is a very narrowly defined and well known case of when the exception occurs, and I have a desire to ignore it. If I setup the code to simply suppress this error after the save operation successfully completes, then I should be able to safely reduce the amount of noise in the error messages that are sent to me. (It should still report the error if the retries never complete successfully)

This is a very common scenario: Teams setup a warning mechanism that is highly effective when a system is first built. At that time, there are a myriad of possible unforeseen errors that could occur. There also hasn’t been enough operational history to feel that the system is stable, so being notified on every potential problem is still a welcome learning experience. As those problems are reduced or eliminated it builds trust in the new system. However, it’s also very common that once a team completes a project and does a moderate amount of post deployment bug fixes, they are asked to move on and prioritize a new project. Which gives no devoted / allocated time to maintaining small and inconsistent issues that arise in the previous project(s).

Unfortunately, the side effect of not giving the time needed to maintain and pay down the technical debt on the older projects is that you can become used to “little” problems that can occur on them; including ignoring the warning messages that they send out. And this creates an effect where you can start to distrust that the warning messages coming from a system are important, because you believe that you know the warning is “little” or “no big deal”.

The best way to instill confidence in the warning and error messages produced by a system is to ensure that the systems only send out important messages, separating the Signal from the Noise.

For my scenario above, the way I’m going to do this is to prevent these handled errors from sending out notification emails. This goes against best practices because I will need to alter the global error monitor in Powershell, $global:Error. But, given that my end goal is to ensure that I only receive important error messages, this seems like an appropriate time to go against best practices.

Below is a snippet of code which can be used to remove error records from $global:Error that fit a given criteria. It will only remove the most recent entries of that error, in order to try and keep the historical error log intact.

You need to be careful with this. If the error you’re looking for occurs within a loop with a retry policy on it, then you need to keep the errors which continued to fail beyond the retry policy, and only remove future errors when the retry policy succeeded. You can better handle the retry policy situation by using the –Last 1 parameter.

	<#
	.SYNOPSIS
	Removes errors from $global:Error which match the given type or message. The errors will be removed
	from the end of the $global:Error "queue" (most recently occurred), back towards the front of
	the queue (the oldest errors). If an element in the queue does not match the given type or
	message, then it will stop removing errors from the queue. This also means, that if the most
	recent error doesn't match the type or message given, then no errors will be removed. This will
	search inner exceptions for matches by default.
	.PARAMETER All
	Clears all occurrences of the error record instead of just the last ones
	#>
	function Clear-LatestGlobalErrors {
	param(
	[string] $Type = [string]::Empty,
	[string] $Message = [string]::Empty,
	[switch] $IgnoreInnerExceptions,
	[switch] $All,
	[int] $Last = -1
	)
	if([string]::IsNullOrWhiteSpace($Type) -and [string]::IsNullOrWhiteSpace($Message)) {
	throw (
	"Either parameter Type or Message are required, please specify one of them for usage. ",
	"If you would like to clear all global errors, use `$global:Error.Clear() instead."
	) -join ""
	}
	$hasType = -not [string]::IsNullOrWhiteSpace($Type)
	$hasMessage = -not [string]::IsNullOrWhiteSpace($Message)
	$hasLastCount = $Last -gt 0
	# regexify the message
	$MessageRegex = $Message
	if($hasMessage) {
	$MessageRegex = $MessageRegex.Replace(".", "\.").Replace("*", ".*");
	}
	$count = $global:Error.Count
	if($count -le 0) {
	# there are no errors, stop early
	return;
	}
	# create a copy of the current error arraylist, to ensure it doesn't change size/shape
	$globalErrorClone = $global:Error.Clone()
	# search and remove errors
	$i = 0
	$removeMatch = $false
	$toRemove = @()
	do {
	$removeMatch = $false;
	<#
	In VSCode debugging, using $global:Error may cause a bunch of
	CmdletInvocationExceptions, with a message of
	'The scope number 'X' exceeds the number of active scopes.'
	These values can be safely ignored.
	The errors that we are really interested in all have Exceptions
	associated with them
	#>
	$errorRecord = $null
	$testi = $i
	do {
	$errorRecord = $globalErrorClone[$testi]
	if($null -eq $errorRecord.Exception) { $testi++ }
	} while($null -eq $errorRecord.Exception -and ($testi -le $globalErrorClone.Count))
	$i = $testi
	$current = $errorRecord.Exception
	if($null -eq $current) {
	# Related to the CmdletInvocationException note above;
	# when the error record has no Exception associated with it, we will break the loop
	# however, if All messages are supposed to be cleared out, then continue the loop
	if(-not $All) {
	break;
	} else {
	continue;
	}
	}
	do {
	if($hasType) {
	$etype = $current.GetType()
	if( $etype.Name -eq $Type `
	-or $etype.FullName -eq $Type
	) {
	$removeMatch = $true
	}
	}
	if($hasMessage) {
	if($current.Message -match $MessageRegex) {
	$removeMatch = $true
	}
	}
	$current = $current.InnerException
	} while( # stop when you run out of inner exceptions or you find a match
	(-not $IgnoreInnerExceptions -and $null -ne $current) `
	-and $removeMatch -eq $false
	);
	if($removeMatch) {
	$toRemove += @($errorRecord)
	}
	$i++
	} while( # stop when you no longer find matches or run out of errors to search through
	# if all matching error records are to be removed, only stop when you
	# run out of errors to search through
	# it can also stop if we've reached the maximum number of entries they wanted to remove ($Last)
	($removeMatch -or $All) `
	-and (-not $hasLastCount -or $Last -lt $toRemove.Count) `
	-and ($i -lt $globalErrorClone.Count)
	);
	# finally, remove the entries
	foreach($r in $toRemove) {
	$global:Error.Remove($r)
	}
	}

view raw Clear-LatestGlobalErrors.ps1 hosted with ❤ by GitHub

	# http://stackoverflow.com/questions/1183183/path-of-currently-executing-powershell-script
	$root = Split-Path $MyInvocation.MyCommand.Path -Parent;
	if($global:CoreUcsbInvokePester -eq $null -or $global:CoreUcsbInvokePester -eq $false) {
	Import-Module CoreUcsb -Force
	Import-Module Pester
	}
	Describe -Tag "Unit","Public" -Name "Clear-LatestGlobalErrors" {
	function Get-ExceptionTypeCount {
	param([string] $Type)
	$matching =
	$global:Error |? {
	if($null -eq $_.Exception) { return $false }
	if($null -eq $_.Exception.InnerException) { return $false }
	return $_.Exception.InnerException.GetType().Name -eq $Type
	}
	return $matching.Count
	}
	function Get-ExceptionMessageCount {
	param([string] $Message)
	$matching =
	$global:Error |? {
	if($null -eq $_.Exception) { return $false }
	if($_.Exception.Message -match $Message) { return $true }
	if($null -eq $_.Exception.InnerException) { return $false }
	return $_.Exception.InnerException.Message -match $Message
	}
	return $matching.Count
	}
	It "Prompts user to use `$global:Error.Clear()" {
	$global:Error.Clear()
	{ Clear-LatestGlobalErrors } |
	Should -Throw "Error.Clear"
	}
	It "Clears the last error by type (DivideByZeroException)" {
	$global:Error.Clear()
	try { 1 / 0 } catch { <# surpress #> }
	(Get-ExceptionTypeCount -Type "DivideByZeroException") | Should -Be 1
	Clear-LatestGlobalErrors -Type DivideByZeroException
	(Get-ExceptionTypeCount -Type "DivideByZeroException") | Should -Be 0
	}
	It "Only clears the last error if it really was the last error (DivideByZeroException)" {
	$global:Error.Clear()
	try { 1 / 0 } catch { <# surpress #> }
	try { throw "something something" } catch { <# surpress #> }
	(Get-ExceptionTypeCount -Type "DivideByZeroException") | Should -Be 1
	Clear-LatestGlobalErrors -Type DivideByZeroException
	(Get-ExceptionTypeCount -Type "DivideByZeroException") | Should -Be 1
	}
	It "Clears the last error by message (DivideByZeroException)" {
	$global:Error.Clear()
	try { 1 / 0 } catch { <# surpress #> }
	$ex.InnerException.GetType().Name | Should -Be DivideByZeroException
	(Get-ExceptionMessageCount -Message "Attempted to divide by zero.") | Should -Be 1
	Clear-LatestGlobalErrors -Message "Attempted to divide by zero."
	(Get-ExceptionMessageCount -Message "Attempted to divide by zero.") | Should -Be 0
	}
	It "Clears powershell thrown errors (`"something something`")" {
	$global:Error.Clear()
	try { throw "something something" } catch { <# surpress #> }
	(Get-ExceptionMessageCount -Message "something something") | Should -Be 1
	Clear-LatestGlobalErrors -Message "something something"
	(Get-ExceptionMessageCount -Message "something something") | Should -Be 0
	}
	It "Clears all of an exception message (`"something something`")" {
	$global:Error.Clear()
	try { throw "something something" } catch { <# surpress #> }
	try { 1 / 0 } catch { <# surpress #> }
	try { throw "something something" } catch { <# surpress #> }
	(Get-ExceptionMessageCount -Message "something something") | Should -Be 2
	Clear-LatestGlobalErrors -Message "something something" -All
	(Get-ExceptionMessageCount -Message "something something") | Should -Be 0
	}
	It "Clears Last 1 of an exception message (`"something something`")" {
	$global:Error.Clear()
	try { throw "something something" } catch { <# surpress #> }
	try { 1 / 0 } catch { <# surpress #> }
	try { throw "something something" } catch { <# surpress #> }
	(Get-ExceptionMessageCount -Message "something something") | Should -Be 2
	Clear-LatestGlobalErrors -Message "something something" -Last 1
	(Get-ExceptionMessageCount -Message "something something") | Should -Be 1
	(Get-ExceptionTypeCount -Type DivideByZeroException) | Should -Be 1
	}
	}

view raw Clear-LatestGlobalErrors.Unit.Tests.ps1 hosted with ❤ by GitHub

Book Review?: The Unicorn Project

The Unicorn Project (amazon, audible, supplements: itrevolution) is a new book/follow up of The Phoenix Project by Gene Kim.

And, it’s much more inline with what I was expecting the The Phoenix Project to be. The Phoenix Project focused on The 3 Ways with a strong emphasis on it’s connection to Lean Management. This was done intentionally as the book was supposed to be a retelling of The Goal done with DevOps in mind. In order for The Phoenix Project to tell it’s story it needed to be told from the perspective of someone who was required to see the whole picture of the company, to facilitate understanding of The First Way. To do that, the protagonist is a high level CIO type which has overview of all IT operations in the company. This means that a lot of the day-to-day aspects of a mid-level manager or front-line implementer are glossed over. I would even describe the book as mostly focusing on The First Way (taking more than half the book to explain) and The Second and Third Way also get a bit glossed over. But, in the context of that book, it’s fine. Because “the goal” of that book is to introduce The 3 Ways and give practical examples to help them stick with the reader.

This book continues to build upon the information given in The Phoenix Project, but it presents the information in two modified ways:

• The book is from the point of view of someone who is really a mid-level manager, but the book needs to force her into a front-line implementer position from time to time. This is done to allow for more tangible day-to-day examples to be presented of what can be done.
• The details of the external world are updated to more closely match the current state of DevOps and IT work in 2018/2019. The book references some of the newer capabilities in NoSQL databases, functional programming, and automated testing.

If The Phoenix Project was about describing The 3 Ways. Then this book is about describing The Five Ideals (which are still Lean aligned + some other ideas) :

• The First Ideal – Locality and Simplicity (Remove Mudas/Wastes)
• The Second Ideal – Focus, Flow, and Joy (A lot of things have to work together to get to Joy, Project to Product)
• The Third Ideal – Improvement of Daily Work (Kaizen)
• The Fourth Ideal – Psychologic Safety (Generative Culture, Complex Systems Fail)
• The Fifth Ideal – Customer Focus (Lean Startup, Lean Six Sigma, OKR)

They are all very useful ideals, but the book seemed to fall prey to glossing over details on how to achieve them. As mentioned earlier, there was a similar problem in The Phoenix Project. An example in this book is that our protagonist, Maxine, worked with her team to help define that a Continuous Integration (build) system needs to run Unit Tests in order to verify that each check-in of code doesn’t break the overall functionality. This is introduced as a new concept for their team. The night she introduces the idea, she falls ill and is sick for the next three days. When she returns to work, everyone on the team is writing well designed unit tests and the system has full code coverage. What?! To get a team that has never used unit tests to (a) embrace the value that unit tests provide, (b) take the time to learn a unit testing pattern that isn’t brittle, (c) create meaningful code coverage takes weeks and (d) involves a great deal of mentoring, code review, and will cause frustrations about where your teams time is most valuably spent. But, for this book, it can happen overnight with no negative consequences or trade-offs.

One thing that I really like about this book is that it is trying to take years and years of knowledge and distil it into an easily understandable and entertaining format that might get someone interested in learning more. Hopefully, it encourages anyone that enjoys the book to continue reading. The books publisher, itrevolution.com,  has a number of other books that dive deeper into the subject matter of DevOps and Business Management. By reading or listening to any of their books, you will find a long list of referenced material to continue learning from.