XProj/SDK … or use

In the previous post, Use XProj/SDK project files with ASP.NET Web Apps, I describe creating a custom AfterBuild Target which would copy over the files from the standard `/bin/{configuration}/{architecture}` dll output location to the standard `/bin` webapp output location.

A coworker pointed output can be controlled with the `<OutDir>` build property. Which is way easier to use. So, here’s an updated .csproj file:

	<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup Label="Globals">
	<GenerateDocumentationFile>true</GenerateDocumentationFile>
	<NoWarn>$(NoWarn);CS1591;CA1724;CS1574</NoWarn>
	<OutDir>$(MSBuildProjectDirectory)\bin\</OutDir>
	</PropertyGroup>
	<PropertyGroup>
	<TargetFramework>net48</TargetFramework>
	</PropertyGroup>
	<ItemGroup>
	<PackageReference Include="Microsoft.AspNet.Mvc" Version="5.2.7" />
	<PackageReference Include="Microsoft.AspNet.TelemetryCorrelation" Version="1.0.8" />
	<PackageReference Include="Microsoft.AspNet.WebApi" Version="5.2.7" />
	<PackageReference Include="Microsoft.CodeDom.Providers.DotNetCompilerPlatform" Version="3.6.0" />
	<PackageReference Include="Microsoft.CSharp" Version="4.5.0" />
	<PackageReference Include="Microsoft.Owin" Version="3.0.1" />
	<PackageReference Include="Newtonsoft.Json" Version="12.0.1" />
	</ItemGroup>
	<ItemGroup>
	<Content Include="Web.Int.config">
	<DependentUpon>Web.config</DependentUpon>
	</Content>
	<Content Include="Web.QA.config">
	<DependentUpon>Web.config</DependentUpon>
	</Content>
	<Content Include="Web.Prod.config">
	<DependentUpon>Web.config</DependentUpon>
	</Content>
	</ItemGroup>
	</Project>

view raw Net48WebApp.OutDir.csproj hosted with ❤ by GitHub

Use XProj/SDK project files with ASP.NET Web Apps

The new SDK style project files (sometimes called VS2017 project files, and at one point referred to as xproj files) were not designed to work with ASP.NET Web Application projects. If you’re looking to use the newer SDK project files, then Microsoft is hoping you would use them with ASP.NET Core web apps. However, the SDK project format is soo much easier to work with than the older style, that it’s painful to go back to the old files and their associated packages.configs once you’ve moved to the new style.

So, if you were to convert a .NET 4.8 Web App’s .csproj file to an SDK style project file what are the problems that now occur:

• You can’t target a webapp as an output type with the SDK style project file. The closest you have is the ability to target framework net48 with a library/dll output type (the default type).
  
• I think that might be it?

How do you overcome that challenge:

• If your output type is a library/dll and you set your targetFramework to net48, then you will create an output directory at /bin/{Debug|Release|Xxxx}/net48 which contains all the dlls and other references that would have normally gone into the web app’s /bin folder. So, you are producing the files that you need.
  
• You just need to copy those files into the root /bin folder for IIS/IIS Express to run the website normally. To do that you can add a “CopyToBin” Target to you .csproj file. This target will run after the build completes.
  
• After that, you will want to directly modify the .csproj file  to associate files which are commonly grouped together; such as Web.*.config files.
  

Here is an example:

	<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup Label="Globals">
	<GenerateDocumentationFile>true</GenerateDocumentationFile>
	<NoWarn>$(NoWarn);CS1591;CA1724;CS1574</NoWarn>
	</PropertyGroup>
	<PropertyGroup>
	<TargetFramework>net48</TargetFramework>
	</PropertyGroup>
	<ItemGroup>
	<PackageReference Include="Microsoft.AspNet.Mvc" Version="5.2.7" />
	<PackageReference Include="Microsoft.AspNet.TelemetryCorrelation" Version="1.0.8" />
	<PackageReference Include="Microsoft.AspNet.WebApi" Version="5.2.7" />
	<PackageReference Include="Microsoft.CodeDom.Providers.DotNetCompilerPlatform" Version="3.6.0" />
	<PackageReference Include="Microsoft.CSharp" Version="4.5.0" />
	<PackageReference Include="Microsoft.Owin" Version="3.0.1" />
	<PackageReference Include="Newtonsoft.Json" Version="12.0.1" />
	</ItemGroup>
	<ItemGroup>
	<Content Include="Web.Int.config">
	<DependentUpon>Web.config</DependentUpon>
	</Content>
	<Content Include="Web.QA.config">
	<DependentUpon>Web.config</DependentUpon>
	</Content>
	<Content Include="Web.Prod.config">
	<DependentUpon>Web.config</DependentUpon>
	</Content>
	</ItemGroup>
	<!-- xproj builds write to /bin/{configuration}/{targetFramework}, and for webapps those files need to be written back to /bin -->
	<Target Name="CopyToBin" AfterTargets="AfterBuild">
	<ItemGroup>
	<BuiltFiles Include="$(OutputPath)\*" />
	</ItemGroup>
	<MakeDir Directories="bin" />
	<Copy SourceFiles="@(BuiltFiles)" DestinationFolder="bin\%(RecursiveDir)" SkipUnchangedFiles="true" />
	</Target>
	</Project>

view raw Net48WebApp.csproj hosted with ❤ by GitHub

Unfortunately, if you do this; it will help makes things work on your local machine. But, it won’t really help for your build process. If you use a third party tool to do builds for you, you’ll need to create a custom script which will run after your build completes, but before the results are packaged for deployment. This would need to be a custom solution for your environment. But, the basic outline would look something like this:

• Have your build system check that the .csproj file is (a) building a “web app” (however you define that), (b) a net4X application, and (c) using an SDK style csproj file.
  
  With that many checks needed before performing an action; you know this isn’t a great idea.
  
• Once verified, you’ll want to copy all the normal content files from the source code to a designated output location (css, js, imgs, views?) and then recreate the /bin directory using the output from the build.
  

Microsoft.Extensions.DependencyInjection - ASP.NET

There is an fantastic Stackoverflow answer on how to use Microsoft.Extensions.DependencyInjection inside of a WebAPI 2 project (ASP.NET Full Framework). While it’s not cutting edge, it is a good middle ground solution when rewriting an entire ASP.NET application to ASP.NET Core seems out of the realm of possibility.

I took the code snippet and broke it apart a little bit to create a reusable project to house it. It’s not great, so I don’t really think it’s worth creating a github repo or a nuget package, but if you want to drop it into a project in your code base it could help out.

Here’s an example usage in a .NET 4.8 ASP.NET MVC / WebApi 2 based project:

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Threading;
	using System.Threading.Tasks;
	using System.Web;
	using System.Web.Http;
	using System.Web.Http.Controllers;
	using System.Web.Http.Dependencies;
	using System.Web.Http.ExceptionHandling;
	using System.Web.Http.Filters;
	using System.Web.Mvc;
	using System.Web.Optimization;
	using System.Web.Routing;
	using Microsoft.Extensions.Configuration;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.DependencyInjection.Extensions;
	using DependencyInjection.AspNet.WebApi;
	using IDependencyResolver = System.Web.Http.Dependencies.IDependencyResolver;
	using IExceptionFilter = System.Web.Http.Filters.IExceptionFilter;
	namespace YourNamespace
	{
	/// <summary>
	/// https://stackoverflow.com/questions/50358349/using-microsoft-extension-dependencyinjection-in-asp-net-web-api2
	/// </summary>
	public class WebApiApplication : System.Web.HttpApplication
	{
	protected void Application_Start()
	{
	AreaRegistration.RegisterAllAreas();
	GlobalConfiguration.Configure(WebApiConfig.Register);
	FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
	RouteConfig.RegisterRoutes(RouteTable.Routes);
	BundleConfig.RegisterBundles(BundleTable.Bundles);
	GlobalConfiguration.Configuration.AddMicrosoftDependencyInjectionProvider(services =>
	{
	services.AddControllersAsServices(System.Reflection.Assembly.GetExecutingAssembly());
	});
	}
	}

view raw WebApiApplication.cs hosted with ❤ by GitHub

And, it relies on a DependencyInjection.AspNet.WebApi library, which is targeting framework net48 (here’s the .csproj):

	<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
	<TargetFramework>net48</TargetFramework>
	</PropertyGroup>
	<ItemGroup>
	<PackageReference Include="Microsoft.AspNet.WebApi.Core" Version="5.2.7" />
	<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="3.1.3" />
	</ItemGroup>
	<ItemGroup>
	<Reference Include="System.Web" />
	</ItemGroup>
	</Project>

view raw DependencyInjection.AspNet.WebApi.csproj hosted with ❤ by GitHub

And, here’s the original stackoverflow posts code, just slightly modified:

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Reflection;
	using System.Web;
	using System.Web.Http;
	using System.Web.Http.Controllers;
	using System.Web.Http.Dependencies;
	using Microsoft.Extensions.DependencyInjection;
	namespace DependencyInjection.AspNet.WebApi
	{
	public static class HttpConfigurationExtensions
	{
	public static IServiceProvider AddMicrosoftDependencyInjectionProvider(
	this HttpConfiguration config,
	Action<IServiceCollection> configure = null
	) {
	var provider = IServiceCollectionExtensions.BuildServiceProvider(configure);
	var resolver = new DefaultDependencyResolver(provider);
	config.DependencyResolver = resolver;
	return provider;
	}
	}
	public static class HttpApplicationExtensions
	{
	public static IServiceProvider BuildServiceProvider(this HttpApplication application, Action<IServiceCollection> configure = null)
	{
	var provider = IServiceCollectionExtensions.BuildServiceProvider(configure);
	return provider;
	}
	}
	public static class IServiceCollectionExtensions
	{
	public static IServiceProvider BuildServiceProvider(Action<IServiceCollection> configure = null)
	{
	var services = new ServiceCollection();
	configure?.Invoke(services);
	var serviceProvider = services.BuildServiceProvider();
	return serviceProvider;
	}
	public static IServiceCollection AddControllersAsServices(this IServiceCollection services,
	Assembly controllersAssembly)
	{
	var controllerTypes = GetControllers(controllersAssembly);
	foreach (var type in controllerTypes)
	{
	services.AddTransient(type);
	}
	return services;
	}
	public static IEnumerable<Type> GetControllers(Assembly controllersAssembly)
	{
	var controllers = controllersAssembly.GetExportedTypes()
	.Where(t => !t.IsAbstract && !t.IsGenericTypeDefinition)
	.Where(t => typeof(IHttpController).IsAssignableFrom(t)
	|| t.Name.EndsWith("Controller", StringComparison.OrdinalIgnoreCase));
	return controllers;
	}
	}
	public class DefaultDependencyResolver : IDependencyResolver
	{
	protected IServiceProvider ServiceProvider { get; set; }
	public DefaultDependencyResolver(IServiceProvider serviceProvider)
	{
	this.ServiceProvider = serviceProvider;
	}
	public object GetService(Type serviceType)
	{
	return this.ServiceProvider.GetService(serviceType);
	}
	public IEnumerable<object> GetServices(Type serviceType)
	{
	return this.ServiceProvider.GetServices(serviceType);
	}
	public IDependencyScope BeginScope()
	{
	return new DefaultDependencyResolver(this.ServiceProvider.CreateScope().ServiceProvider);
	}
	public void Dispose()
	{
	// you can implement this interface just when you use .net core 2.0
	// this.ServiceProvider.Dispose();
	}
	}
	}

view raw Extensions.cs hosted with ❤ by GitHub

Powershell Range Operator Performance

This is a truly silly experiment, but it caught my interest. I was discussing Iron Scripter Challenges with thedavecarroll and he was using switch statements with range operators (PSGibberish.psm1):

	function Get-RandomConjunction {
	[CmdletBinding()]
	param()
	$Chance = Get-RandomPercent
	switch ($Chance) {
	{ 1..45 -contains $_ } { 'and' } # 45% chance for 'and'
	{ 46..75 -contains $_ } { 'or' } # 25% chance for 'or'
	{ 76..85 -contains $_ } { ',' } # 10% chance for 'comma'
	{ 86..95 -contains $_ } { ';' } # 10% chance for 'semi-colon'
	{ 96..100 -contains $_ } { ':' } # 5% chance for 'colon'
	}
	}

view raw Get-RandomConjunction.ps1 hosted with ❤ by GitHub

What struck me as odd was the idea that the range operators might be calculating each of their ranges at runtime, on each execution of the function.

So, I ran a couple of experiments and the range operators are pretty neat. Here’s what I think (with no real definitive proof to support) is happening with them:

• Range Operators used within Switch statements, that are contained within Functions are Cached.
• It seems like when the function is JIT’d, the Range Operator value is calculated and Cached.
• So, there’s no reason to pre-calculate the values and reference them within the function.
• And, if you do reference variables from outside the function, looking up variables that require a scope lookup can also be time consuming. (Although, performance isn’t why people turn to powershell in the first place.)
• Range Operators used within a Switch statement outside of a Function are not cached (like a code block).

To determine this, I ran a series of test against a function which focused on executing the switch statement which used range operators:

	function ffff($Value) {
	switch($Value) {
	{0..24 -contains $_} { $a = 0 }
	{25..49 -contains $_} { $a = 1 }
	{50..74 -contains $_} { $a = 2 }
	{75..99 -contains $_} { $a = 3 }
	}
	}

view raw RangeOperatorTestFunction.ps1 hosted with ❤ by GitHub

To determine how much time was spent making the function call and setting the $a variable, this function was used. This is noted as “Calling a Function Overhead”.

	function BaselineFunctionWithoutSwitch($Value) {
	$a = 0
	}

view raw BaselineFunctionWithoutSwitch.ps1 hosted with ❤ by GitHub

Switch Avg Execution Time = Total Avg Execution Time – Calling a Function Overhead

The results were:

The results indicate that both the Range Operator when run inside of a Function, and the Explicitly Scoped Cached Values have about the same running time. Which might indicate that when the function is JIT’d, it calculates the Range Operator values and caches them.

The large increase in running time between Range Operator and Cached Values not in Func might indicate that searching for variables outside of the function scope has a relatively costly penalty by comparison.

And, finally the Range Operator that was run outside of a Function was mostly likely calculated on each execution. While relatively expensive, its surprisingly fast. C# usually uses 10,000 ticks per millisecond, so that’s ~0.19 milliseconds for compilation and execution.

Full Test Script:

	function RangeOperatorInFunc($Value) {
	switch($Value) {
	{0..24 -contains $_} { $a = 0 }
	{25..49 -contains $_} { $a = 1 }
	{50..74 -contains $_} { $a = 2 }
	{75..99 -contains $_} { $a = 3 }
	}
	}
	$gr1 = 0..24;
	$gr2 = 24..49;
	$gr3 = 50..74;
	$gr4 = 75..99;
	function CachedValuesNotInScope($Value) {
	switch($Value) {
	{$gr1 -contains $_} { $a = 0 }
	{$gr2 -contains $_} { $a = 1 }
	{$gr3 -contains $_} { $a = 2 }
	{$gr4 -contains $_} { $a = 3 }
	}
	}
	$global:hr1 = 0..24;
	$global:hr2 = 24..49;
	$global:hr3 = 50..74;
	$global:hr4 = 75..99;
	function CachedValuesWithScopedPrefixed($Value) {
	switch($Value) {
	{$global:hr1 -contains $_} { $a = 0 }
	{$global:hr2 -contains $_} { $a = 1 }
	{$global:hr3 -contains $_} { $a = 2 }
	{$global:hr4 -contains $_} { $a = 3 }
	}
	}
	Write-Host "All Times in Ticks (from Measure-Command)`r`n"
	$r = New-Object System.Random
	#$r.next(100)
	function BaselineFunctionWithoutSwitch($Value) {
	$a = 0
	}
	$baseFuncTicks = @()
	for($i = 0; $i -lt 10000; $i++) {
	$k = $r.Next(100)
	$m = Measure-Command { BaselineFunctionWithoutSwitch($k) }
	$baseFuncTicks += @($m)
	}
	$baseAvgTicks = ($baseFuncTicks.Ticks | Measure-Object -Average).Average
	"Calling a Function Overhead = $baseAvgTicks`r`n"
	$timeComparisons = @()
	function New-TC($Description, $Measure, $BaseAvgTicks) {
	$ticks = ($Measure.Ticks | Measure-Object -Average).Average
	$switchTicks = $ticks - $BaseAvgTicks
	return [PSCustomObject] @{
	Description = $Description
	'Switch Avg Execution Time' = $switchTicks
	'Total Avg Execution Time' = $ticks
	}
	}
	$ftimes = @()
	for($i = 0; $i -lt 10000; $i++) {
	$k = $r.Next(100)
	$m = Measure-Command { RangeOperatorInFunc($k) }
	$ftimes += @($m)
	}
	$timeComparisons += @(New-TC -Description "Range Operator in Func" -Measure $ftimes -BaseAvgTicks $baseAvgTicks)
	$gtimes = @()
	for($i = 0; $i -lt 10000; $i++) {
	$k = $r.Next(100)
	$m = Measure-Command { CachedValuesNotInScope($k) }
	$gtimes += @($m)
	}
	$timeComparisons += @(New-TC -Description "Cached Values not in Func Scope" -Measure $gtimes -BaseAvgTicks $baseAvgTicks)
	$htimes = @()
	for($i = 0; $i -lt 10000; $i++) {
	$k = $r.Next(100)
	$m = Measure-Command { CachedValuesWithScopedPrefixed($k) }
	$htimes += @($m)
	}
	$timeComparisons += @(New-TC -Description "Explicitly Scoped Cached Values" -Measure $htimes -BaseAvgTicks $baseAvgTicks)
	$itimes = @()
	for($i = 0; $i -lt 10000; $i++) {
	$k = $r.Next(100)
	$m = Measure-Command {
	switch($Value) {
	{0..24 -contains $_} { $a = 0 }
	{25..49 -contains $_} { $a = 1 }
	{50..74 -contains $_} { $a = 2 }
	{75..99 -contains $_} { $a = 3 }
	}
	}
	$itimes += @($m)
	}
	$timeComparisons += @(New-TC -Description "Range Operator with No Func" -Measure $times -BaseAvgTicks 0)
	$timeComparisons

view raw SwitchPerformanceTest.ps1 hosted with ❤ by GitHub

Diagnosing Slow node builds on Win2016

In a move a from a Windows 2012 R2 build server to a Windows 2016 build server, the nodejs build step nearly doubled in it’s execution time. This seemed odd, since everything else was pretty much the same on the new server. So, what could the difference be?

Fortunately, a coworker point me towards the Windows Performance Recorder from Microsoft’s Assessment and Deployment Kit (Windows ADK). This worked really well in troubleshooting the issue, and I just wanted to drop in some screen grabs to show it’s visualizations.

The build was on-premise, so I did have access to install the kit** and control the execution of the Windows Performance Recorder to coincide with the execution of the problematic step. This would have been much more difficult on a hosted build server.

Getting the visualization comes by way of a two-step process.

• First Windows Performance Recorder is used to track analysis information from all over your system while the issue is occurring. You can track different profiles, or record more detailed information in particular areas through manual configuration.
  
• Once the problem has been recorded, the analysis information can then be pulled up in Windows Performance Analyzer. Which has a pretty nice interface.

First, here’s a screenshot of Windows Performance Analyzer from the “dotnet publish” (ie. npm init/build) step on the older Windows 2012 R2 server. In the screenshot, the step started by running node.exe and performing the init command. Which would copy over the npm packages from the local npm-cache. This would take about 60 seconds to complete.

However, when performing the same build/same step on the new Windows Server 2016 instance, node.exe wasn’t the dominant process during npm’s init phase. Instead another process was dominant (greyed out in the screenshot), which ran for nearly the same length of time as node.exe and seemed to mirror the process. Because the other process was competing for CPU time with node.exe, the node process took nearly 200 seconds to complete (up from 60 seconds).

So, what was the other process?

MsMpEng.exe, aka. Windows Defender, the classic anti-virus software. On the Windows Server 2016 image I was using, Windows Defender was pre-installed and doing it’s job.

I didn’t take a screenshot of it, but using the Disk IO dashboard I was able to drill into what files MsMpEng.exe was reading and something struck me as odd. It almost looked as if Windows Defender was virus checking the file before it was read before the copy, and read again at the destination after the copy. I’m not sure if that’s the case, but it did seem odd.

For the resolution, I added some Path Exclusion rules to the realtime file scanning capability of Windows Defender. These were specific paths used by the build system and we know those files should be coming from trusted sources. I still left on realtime process scanning and also ensured the scheduled scans were setup, which would look through all the files.

The final result of adding the excluded paths reduced the overall time for the npm init section down to 80s (down from 200s, but also up from 60s on the old, not bad); and MsMpEng.exe was still reporting that is was performing realtime virus scans on the process itself.

A quick sidenote: The offline installer is kind of odd. To do the offline installer, you run the online installer and direct it to download it’s files/resources to the same directory where the online installer’s adksetup.exe is at. The next time you run adksetup.exe, it will detect the the files have already been downloaded and present a different set of options when it runs.

Best Practices should not be the end of a conversation

Sometimes, Best Practices can be used as an end all to a conversation. No more needs to be said, because Best Practices have laid out the final statement … and that doesn’t really feel right.

Best practices weren’t always best practices. At some point a new technology came around and people started working with it to create their own practices. And the practices that worked stuck around. Over time, those practices might be written down as suggested practices for a particular technology stack. And, when coming from the authoritative source for a technology stack, they might be labeled as Best Practices.

But, usually, when I hear Best Practices used as an end all to a conversation, it’s not in reference to a particular technology stack. It’s used as a generalization, to explain guidance to approach an area. The guidance is supposed to help people who haven’t done something before start off in the right direction. It’s supposed to be a starting point. And I think you’re supposed to continue to study the usage of those practices, to determine what the right practices are for your environment and your technology stack. Maybe even setup criteria to evaluate if a practice is working successfully in your environment. And, then change a practice if it doesn’t meet your needs.

That isn’t a trivial thing to do. You have to first understand where the practices came from and what they were accomplishing. But, once you do, you should be able to see where their limitations are and where they can be expanded. Sometimes a technology stack wasn’t available when a practice was written, and that changes the possible ways a desired outcome can be achieved. To change a practice, you have to be knowledgeable of what outcomes are trying to be achieved, and the pitfalls that come with them; and then make a decision based on the trade-offs of going to a new practice.

The only way to create a new practice, is if Best Practices are the start of a conversation, not the end of one.

(Maybe we could also drop the word “Best”, and just make them practices?)

More Andon Cord

I was listening to Gene Kim’s new podcast, Idealcast, interview with Dr. Steven Spear (Decoding the DNA of the Toyota Production System), and the subject of Andon Cords came up. Since I had recently written a post on Andon Cords, I was very curious if their conversation would line up with my thoughts or if it would show a different angle or new depths. The text of the conversation was (from Ep 5):

---

Gene Kim

The notion of the Andon Cord is that anyone on the front line would be thanked for exposing an ignorance/deficiency for trying to solve a problem that the dominant architecture or current processes didn't foresee.

Dr. Steven Spear

That's right. Basically, the way the Andon Cord works is you, Gene, have asked me, Steve, to do something and I can't. And, I'm calling it to your attention. Such that the deficiencies in my doing become a reflection of the deficiencies in your thinking / your planning / your designing. And we're going to use the deficiencies in my doing as a trigger to get together and improve on 'your thinking and my doing' or 'our thinking and our doing'.

Gene Kim

And the way that's institutionalized amplifies signals in the system to self correct.

---

To me, that definitely feels like a new angle or view point on Andon Cords. It still feels like it aligns with the “popular definition”, but it’s more clear in it’s description. It seems to follow a line of thinking that “We’ve got a problem occurring right now; let’s use this as an opportunity to look at the problem together. And, then let’s think about ways to improve the process in the future.” Which feels like a more directed statement than “the capability to pause/halt any manufacturing line in order to ensure quality control and understanding.”

But, the definition Gene (Mr. Kim?) and Dr. Spear’s give does imply something I want to point out: Gene’s scenario is one where the person using the Andon Cord is someone directly involved in the processing line. It’s by someone who is directly on the line and seeing a problem as it’s happening. The cord isn’t being pulled by someone who wasn’t asked to be involved in the process.

I wonder if there are any texts on recognizing when an Andon Cord is being used inappropriately? Is that even a thing?

Record Request Body in ASP.NET Core 3.0–Attempt 2

In the original post (Record Request Body in ASP.NET Core 3.0), the ITelemetryInitializer was creating some unexpected behavior. It was preventing the Operation ID associated with each request from changing/being updated. So, all the requests that were going through the system were being displayed on the same Performance graph. This created a nearly unusable performance graph as all the request were squished together and unreadable for timing purposes.

So, I needed to remove the ITelemetryInitializer from the code. But, I still wanted to record the JsonBody. The work around I used (which isn’t great) was to create a fake dependency on the request and record the body within the properties of the dependency.

Here’s the code:

	using System.IO;
	using System.Text;
	using System.Threading.Tasks;
	using Microsoft.ApplicationInsights;
	using Microsoft.ApplicationInsights.DataContracts;
	using Microsoft.AspNetCore.Mvc.Filters;
	namespace YourNamespace
	{
	public class TrackConstants
	{
	public const string JsonBody = "JsonBody";
	}
	public class TrackRequestBodyAttribute : ActionFilterAttribute
	{
	private readonly TelemetryClient _telemtry;
	public TrackRequestBodyAttribute(
	TelemetryClient telemetry = null
	)
	{
	_telemtry = telemetry;
	}
	public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
	{
	var request = context.HttpContext.Request;
	request.Body.Position = 0;
	using var stream = new StreamReader(request.Body, Encoding.UTF8, false, 1024, true);
	var body = await stream.ReadToEndAsync();
	request.Body.Position = 0;
	context.HttpContext.Items.Add(TrackConstants.JsonBody, body);
	var toperation = _telemtry.StartNewOperation<DependencyTelemetry>("JsonBody");
	toperation.TData.Add(TrackConstants.JsonBody, body);
	toperation.Stop();
	await base.OnActionExecutionAsync(context, next);
	}
	}
	}

view raw TrackRequestBodyAttribute.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Net.Http;
	using Microsoft.ApplicationInsights.Extensibility;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.DependencyInjection.Extensions;
	namespace YourNamespace
	{
	public static class IServiceCollectionExtensions
	{
	public static IServiceCollection AddYourDependencies(
	this IServiceCollection services
	)
	{
	// add other dependencies
	services.TryAddTransient<TrackRequestBodyAttribute>();
	return services;
	}
	}
	}

view raw IServiceCollectionExtensions.cs hosted with ❤ by GitHub

	using System.Collections.Generic;
	using System.IO;
	using System.Text;
	using System.Threading.Tasks;
	using Microsoft.ApplicationInsights;
	using Microsoft.AspNetCore.Builder;
	using Microsoft.AspNetCore.Hosting;
	using Microsoft.AspNetCore.Http;
	using Microsoft.Extensions.Configuration;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.Hosting;
	namespace YourNamespace
	{
	public class Startup
	{
	// This method gets called by the runtime. Use this method to add services to the container.
	public void ConfigureServices(IServiceCollection services)
	{
	services.AddControllers();
	services.AddYourDependencies()
	}
	// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
	public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
	{
	app.UseMiddleware<EnableBufferingMiddleware>();
	// use that middleware before everything else
	}
	}
	}

view raw Startup.cs hosted with ❤ by GitHub

	using System.IO;
	using System.Threading.Tasks;
	using Microsoft.AspNetCore.Http;
	using Microsoft.AspNetCore.Http.Features;
	namespace YourNamespace
	{
	public static class HttpRequestExtensions
	{
	public static bool CanReadBody(this HttpRequest request)
	{
	return (
	request.Method == HttpMethods.Post
	|| request.Method == HttpMethods.Put
	)
	&& request.Body.CanRead;
	}
	}
	public class EnableBufferingMiddleware
	{
	private readonly RequestDelegate _next;
	public EnableBufferingMiddleware(
	RequestDelegate next
	)
	{
	_next = next;
	}
	public async Task InvokeAsync(HttpContext context)
	{
	var request = context.Request;
	if (request.CanReadBody())
	{
	request.EnableBuffering();
	}
	await _next(context);
	}
	}
	}

view raw EnableBufferingMiddleware.cs hosted with ❤ by GitHub

	using System.Linq;
	using System.Threading.Tasks;
	using Microsoft.ApplicationInsights;
	using Microsoft.AspNetCore.Mvc;
	namespace YourNamespace
	{
	[ApiController]
	[Route("[controller]")]
	public class ExampleController : ControllerBase
	{
	public CodePushedController() {}
	[ServiceFilter(typeof(TrackRequestBodyAttribute))]
	public async Task<IActionResult> PostAsync([FromBody] CodePushedRequest request)
	{
	// your code
	}
	}
	}

view raw ExampleController.cs hosted with ❤ by GitHub

Baseline C# Objects to Populate Jira DevInfo Pt. 2

From the previous post, Baseline C# Objects to Populate Jira DevInfo Pt. 1:

Jira has this great “Development Information” (DevInfo) that can be associated with your work items. Which has an API described here. The information provided in the development tabs are for Branches, Commits, Pull Requests, Builds, Deployments and Feature Flags. Which is a way to have visibility into all the development/code activity that is related to a particular work item. It’s a great way to connect everything together.

On the previous post, there’s also a list of “gotcha’s” with the Jira documentation and a list of things could be improved.

But, this post is about the baseline C# objects which can be used to push information to the Atlassian/Jira DevInfo API.

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Threading.Tasks;
	using Atlassian.Jira;
	namespace YourProject
	{
	public class JiraDevInfoBulkRequest
	{
	public List<JiraRepository> Repositories { get; set; }
	public bool PreventTransitions { get; set; }
	public Dictionary<string,string> Properties { get; set; }
	public JiraProviderMetadata ProviderMetadata { get; set; }
	}
	public class JiraRepository
	{
	public string Id { get; set; }
	public string Name { get; set; }
	public string Description { get; set; }
	public string Url { get; set; }
	public string Avatar { get; set; }
	public string AvatarDescription { get; set; }
	public string ForkOf { get; set; }
	public List<JiraCommit> Commits { get; set; } = new List<JiraCommit>();
	public List<JiraPullRequest> PullRequests { get; set; } = new List<JiraPullRequest>();
	public List<JiraBranch> Branches { get; set; } = new List<JiraBranch>();
	public long UpdateSequenceId { get; set; }
	}
	public class JiraCommit
	{
	public string Id { get; set; }
	public string Hash { get; set; }
	public string Message { get; set; }
	public List<string> IssueKeys { get; set; }
	public string Url { get; set; }
	public string DisplayId { get; set; }
	public DateTime AuthorTimestamp { get; set; }
	public JiraAuthor Author { get; set; }
	public int FileCount { get; set; }
	public List<string> Flags { get; set; }
	public List<JiraCommitFile> Files { get; set; }
	public long UpdateSequenceId { get; set; }
	}
	public class JiraAuthor
	{
	public string Name { get; set; }
	public string Email { get; set; }
	public string Username { get; set; }
	public string Url { get; set; }
	public string Avatar { get; set; }
	}
	public class JiraCommitFile
	{
	public string Path { get; set; }
	public string Url { get; set; }
	public string ChangeType { get; set; }
	public string LinesAdded { get; set; }
	public string LinesRemoved { get; set; }
	}
	public class JiraProviderMetadata
	{
	public string Product { get; set; }
	}
	public class JiraPullRequest
	{
	public string Id { get; set; }
	public List<string> IssueKeys { get; set; }
	public string Status { get; set; }
	public string Title { get; set; }
	public string DisplayId { get; set; }
	public string Url { get; set; }
	public JiraAuthor Author { get; set; }
	public int CommentCount { get; set; }
	public string SourceBranch { get; set; }
	public string SourceBranchUrl { get; set; }
	public DateTime LastUpdate { get; set; }
	public string DestinationBranch { get; set; }
	public List<JiraReviewer> Reviewers { get; set; }
	public string Avatar { get; set; }
	public string AvatarDescription { get; set; }
	public long UpdateSequenceId { get; set; }
	}
	public class JiraReviewer
	{
	public string Name { get; set; }
	public string ApprovalStatus { get; set; }
	public string Url { get; set; }
	public string Avatar { get; set; }
	}
	public class JiraBranch
	{
	public string Id { get; set; }
	public List<string> IssueKeys { get; set; }
	public string Name { get; set; }
	public JiraCommit LastCommit { get; set; }
	public string CreatePullRequestUrl { get; set; }
	public string Url { get; set; }
	public long UpdateSequenceId { get; set; }
	}
	}

view raw JiraDevInfoBulkRequestModels.cs hosted with ❤ by GitHub

	using System.Collections.Generic;
	namespace YourProject
	{
	public class JiraDevInfoResponse
	{
	public Dictionary<string, JiraDevInfoResponseEntitySet> AcceptedDevInfoEntities { get; set; }
	public Dictionary<string, JiraDevInfoResponseEntitySet> FailedDevInfoEntities { get; set; }
	public List<string> UnknownIssueKeys { get; set; }
	}
	public class JiraDevInfoResponseEntitySet
	{
	public List<string> Branches { get; set; }
	public List<string> Commits { get; set; }
	public List<string> PullRequests { get; set; }
	}
	}

view raw JiraDevInfoResponse.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Net.Http;
	using System.Net.Http.Formatting;
	using System.Text;
	using System.Text.Json;
	using System.Threading.Tasks;
	using Microsoft.TeamFoundation.SourceControl.WebApi;
	using Newtonsoft.Json;
	using Newtonsoft.Json.Serialization;
	using JsonSerializer = System.Text.Json.JsonSerializer;
	namespace YourProject
	{
	public class JiraDevInfoHttpClient : IJiraDevInfoHttpClient
	{
	private readonly HttpClient _client;
	private readonly IUpdateSequenceIdManager _idManager;
	public JiraDevInfoHttpClient(
	HttpClient client,
	IUpdateSequenceIdManager idManager
	)
	{
	_client = client;
	_idManager = idManager;
	_client.SetApiBaseUri(AtlassianConstants.YourInstanceCloudId, "devinfo");
	}
	private JiraDevInfoBulkRequest BuildBulkRequest(JiraRepository repository)
	{
	var bulkRequest = new JiraDevInfoBulkRequest()
	{
	Repositories = new List<JiraRepository>() { repository },
	PreventTransitions = true,
	ProviderMetadata = AtlassianConstants.AzDProviderMetadata
	};
	return bulkRequest;
	}
	public async Task<JiraDevInfoResponse> BulkUpdateAsync(
	JiraRepository repository
	) {
	var updateSequenceId = await _idManager.NextAsync();
	repository.UpdateSequenceId = updateSequenceId;
	foreach (var c in repository.Commits)
	{
	c.UpdateSequenceId = updateSequenceId;
	}
	foreach (var b in repository.Branches)
	{
	b.UpdateSequenceId = updateSequenceId;
	}
	foreach (var pr in repository.PullRequests)
	{
	pr.UpdateSequenceId = updateSequenceId;
	}
	var bulkRequest = BuildBulkRequest(repository);
	var content = bulkRequest.ToJsonContent();
	var response = await _client.PostAsync("bulk", content);
	var result = await response.DeserializeAsync<JiraDevInfoResponse>();
	return result;
	}
	}
	public static class HttpClientExtensions
	{
	public static void SetApiBaseUri(this HttpClient client, string cloudId, string entityType)
	{
	client.BaseAddress = ApiBaseUri(cloudId, entityType);
	}
	public static Uri ApiBaseUri(string cloudId, string entityType)
	{
	// example:
	// https://api.atlassian.com/jira/devinfo/0.1/cloud/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/bulk
	var baseUrl = string.Format("{0}jira/{1}/0.1/cloud/{2}/",
	AtlassianConstants.AtlassianApiRootUrl,
	entityType,
	cloudId
	);
	var baseUri = new Uri(baseUrl);
	return baseUri;
	}
	}
	public static class JiraOAuthHttpClientExtensions
	{
	public static readonly JsonSerializerOptions SerializerOptions =
	new JsonSerializerOptions() {PropertyNamingPolicy = JsonNamingPolicy.CamelCase};
	public static HttpContent ToJsonContent<T>(this T value)
	{
	var json = JsonSerializer.Serialize(value, SerializerOptions);
	var content = new StringContent(json, Encoding.UTF8, "application/json");
	return content;
	}
	}
	}

view raw JiraDevInfoHttpClient.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Threading.Tasks;
	using Microsoft.EntityFrameworkCore;
	using Nito.AsyncEx;
	namespace YourProject
	{
	public interface IUpdateSequenceIdManager // you need to write your own implementation of this
	{
	Task<long> NextAsync();
	Task<long> CurrentAsync();
	}
	// This uses a MSSQL database backend with a very simple table to manage sequence number generation.
	// But, you need to write your own implementation of this. This one is not great.
	// CREATE TABLE [dbo].[tbl_UpdateSequenceIds](
	// [Name] [varchar](50) NULL,
	// [Id] [bigint] NULL
	// ) ON [PRIMARY]
	// GO
	// INSERT INTO [dbo].[tbl_UpdateSequenceIds] ([Name], [Id]) VALUES ('Current', 0)
	// GO
	public class UpdateSequenceIdManager : IUpdateSequenceIdManager
	{
	private readonly AzDPullRequestsDbContext _dbContext;
	private readonly AsyncReaderWriterLock _seqLock = new AsyncReaderWriterLock();
	public UpdateSequenceIdManager(YourProjectsDbContext dbContext)
	{
	_dbContext = dbContext;
	}
	public async Task<long> NextAsync()
	{
	using (await _seqLock.WriterLockAsync())
	{
	var sequenceRecord = await _dbContext.UpdateSequenceIds.FirstAsync(i => i.Name == "Current");
	var next = ++sequenceRecord.Id;
	await _dbContext.SaveChangesAsync();
	return next;
	}
	}
	public async Task<long> CurrentAsync()
	{
	using (await _seqLock.ReaderLockAsync())
	{
	var sequenceRecord = await _dbContext.UpdateSequenceIds.FirstAsync(i => i.Name == "Current");
	return sequenceRecord.Id;
	}
	}
	}
	}

view raw UpdateSequenceIdManager.cs hosted with ❤ by GitHub

Baseline C# Objects to Populate Jira DevInfo Pt. 1

Jira has this great “Development Information” (DevInfo) that can be associated with your work items. Which has an API described here. The information provided in the development tabs are for Branches, Commits, Pull Requests, Builds, Deployments and Feature Flags. Which is a way to have visibility into all the development/code activity that is related to a particular work item. It’s a great way to connect everything together.

It really is great, but there are some pieces of the puzzle that can be improved. For example:

• Currently, github also associates code commits, pull requests, and build/action information into an issues history. But, github’s layout intermingles those changes within the history of an issue to create a timeline. This allows for a reviewer of an issue, to visually see when a change was made, what the context was surrounding that change. Maybe there was a compelling argument made by a team member half way through an issue being worked on. And, that argument resulted in 5 files changing; you can see that in the github history.
  
  But, Jira’s history won’t show you that because the conversation history (Jira Comments) do not intermingle the code commits, pull requests, builds or deployments to create a timeline history.
  
  That would be a really nice improvement. And, on a personal note, if would make reviewing my coworkers work items a lot easier.
  
• The Commits display (screenshot above) has a weird little bug in it. The Files Count Column (right side) should be able to display a count of all the files within a commit. The File Details Display, the list of files associated with a commit (“MODIFIED  PullRequest/Send-AzDPRStatusUpdates.ps1/” in the screenshot), will only show the first 10 files from the commit. But the File Count Column isn’t showing the total count of files, it only showing the count of files in the File Details Display that number (“1 file” in the screenshot). This seems to be a bug, but I haven’t reported it yet.
  
  (PS. Why is there a ‘/’ on the end of “PullRequest/Send-AzDPRStatusUpdates.ps1/”? The information submitted to the API did not have a slash on the end.)
  
• The Documentation is REALLY CONFUSING when it comes urls. All of the examples in the documentation present url structures that look like this:
  
  https://your-domain.atlassian.net/rest/devinfo/0.10/bulk
  
  Except, that’s not the right url!!
  
  All the APIs have an “Authorization” section in their documentation, which has a link to Integrate JSW Cloud with On-Premise Tools. And BURIED in that documentation is this quick note:
  
  The root URL for OAuth 2.0 operations is: https://api.atlassian.com/jira/<entity type>/0.1/cloud/<cloud ID>/

Note that this URL is different to the URLs used in the documentation. However, you easily translate from one to the other. For example, POST /rest/builds/0.1/bulk translates to POST https://api.atlassian.com/jira/builds/0.1/cloud/<cloud ID>/bulk.

And I agree that it’s easy to translate. But, you have to first know that you need to translate it. Maybe, an alternative direction to take is to update the OAuth 2.0 APIs documentation to use the correct urls? Or, explicitly state it on all the API documentation, so that you don’t have to find it in a separate page?


Atlassian/Jira does provide this really great C# SDK for working/reading Jira issues. But, the SDK doesn’t contain any objects/code to work with the “DevInfo”. So, I want to post a couple baseline objects which can be used in an aspnetcore/netstandard application to push information to the DevInfo endpoint in Atlassian/Jira Cloud …

But, before doing that, this post will be of a few baseline objects to authenticate with the Atlassian/Jira OAuth 2.0 endpoint.

The next post will contain objects to use with the “DevInfo” API.

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Net.Http;
	using System.Threading.Tasks;
	using Atlassian.Jira;
	using Microsoft.ApplicationInsights.Extensibility;
	using Microsoft.AspNetCore.Http;
	using Microsoft.AspNetCore.Http.Features;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.DependencyInjection.Extensions;
	namespace YourProject
	{
	public static class IServiceCollectionExtensions
	{
	public static IServiceCollection AddSaEnterpriseAzDPullRequestsWeb(
	this IServiceCollection services,
	Action<AzDPullRequestOptions> configure = null
	)
	{
	services.TryAddTransient<JiraOAuthDelegatingHandler>();
	services.TryAddScoped<IAtlassianOAuthenticator, AtlassianOAuthenticator>();
	services.AddHttpClient();
	services.TryAddTransient<HttpClient>(s =>
	{
	var factory = s.GetService<IHttpClientFactory>();
	return factory.CreateClient();
	});
	services.AddHttpClient<IJiraDevInfoHttpClient, JiraDevInfoHttpClient>()
	.AddHttpMessageHandler<JiraOAuthDelegatingHandler>();
	return services;
	}
	}
	}

view raw IServiceCollectionExtensions.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Net;
	using System.Net.Http;
	using System.Net.Http.Headers;
	using System.Threading;
	using System.Threading.Tasks;
	namespace YourProject
	{
	public class JiraOAuthDelegatingHandler : DelegatingHandler
	{
	private readonly IAtlassianOAuthCredentialsManager _manager;
	public JiraOAuthDelegatingHandler(
	IAtlassianOAuthCredentialsManager manager
	)
	{
	_manager = manager;
	}
	protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	{
	var headers = request.Headers;
	if (string.IsNullOrWhiteSpace(headers.Authorization?.Parameter))
	{
	var jwt = await _manager.GetJwtAsync();
	headers.Authorization = new AuthenticationHeaderValue("Bearer", jwt);
	}
	var retries = 0;
	HttpResponseMessage response;
	do
	{
	response = await base.SendAsync(request, cancellationToken);
	// unauthorized responses should have the JWT token refreshed and then try again.
	if (response.StatusCode == HttpStatusCode.Unauthorized)
	{
	await _manager.RefreshJwtAsync();
	var jwt = await _manager.GetJwtAsync();
	headers.Authorization = new AuthenticationHeaderValue("Bearer", jwt);
	}
	retries++; // this could go in the if statement, but it works here too
	} while (response.StatusCode == HttpStatusCode.Unauthorized && retries < 2);
	if (!response.IsSuccessStatusCode)
	{
	var body = await response.Content.ReadAsStringAsync();
	var message = $"{(int)response.StatusCode} ({response.StatusCode}) - {body}";
	throw new HttpRequestException(message);
	}
	return response;
	}
	}
	}

view raw JiraOAuthDelegatingHandler.cs hosted with ❤ by GitHub

	using System;
	using System.Threading;
	using System.Threading.Tasks;
	using Microsoft.IdentityModel.JsonWebTokens;
	using Nito.AsyncEx;
	namespace YourProject
	{
	public interface IAtlassianOAuthCredentialsManager
	{
	Task<string> GetJwtAsync();
	Task RefreshJwtAsync();
	}
	private readonly IAtlassianOAuthenticator _authenticator;
	private readonly AsyncReaderWriterLock _jwtLock = new AsyncReaderWriterLock();
	private AtlassianOAuthTokenResponse _tokenResponse = null;
	private DateTime? _validTo = null;
	public AtlassianOAuthCredentialsManager(
	IAtlassianOAuthenticator authenticator
	)
	{
	_authenticator = authenticator;
	}
	public async Task<string> GetJwtAsync()
	{
	if (await ShouldRefreshJwtAsync())
	{
	await RefreshJwtAsync();
	}
	string result;
	using (await _jwtLock.ReaderLockAsync())
	{
	result = _tokenResponse.access_token; // should make an immutable copy
	}
	return result;
	}
	public async Task<AtlassianOAuthTokenResponse> GetCurrentTokenResponse()
	{
	using (await _jwtLock.ReaderLockAsync())
	{
	var result = new AtlassianOAuthTokenResponse()
	{
	access_token = _tokenResponse.access_token,
	scope = _tokenResponse.scope,
	expires_in = _tokenResponse.expires_in,
	token_type = _tokenResponse.token_type
	};
	return result;
	}
	}
	private async Task<bool> ShouldRefreshJwtAsync()
	{
	using(await _jwtLock.ReaderLockAsync()) {
	if (_tokenResponse == null) return true;
	if (_validTo == null) return true;
	if (DateTime.Now >= _validTo.Value) return true;
	return false;
	}
	}
	public async Task RefreshJwtAsync()
	{
	using(await _jwtLock.WriterLockAsync()) {
	var response = await _authenticator.AuthenticateAsync();
	SetJwt(response);
	}
	}
	private void SetJwt(AtlassianOAuthTokenResponse tokenResponse)
	{
	_tokenResponse = tokenResponse;
	var jwtHandler = new JsonWebTokenHandler();
	var token = jwtHandler.ReadJsonWebToken(_tokenResponse.access_token);
	// https://developer.atlassian.com/cloud/jira/software/integrate-jsw-cloud-with-onpremises-tools/
	// Write your code to anticipate that a granted token might no longer work. For example,
	// track the expiration time and request a new token before the existing one expires
	// (at least 30-60 seconds prior).
	_validTo = token.ValidTo.AddMinutes(-1);
	}
	}
	}

view raw AtlassianOAuthCredentialsManager.cs hosted with ❤ by GitHub

	using System;
	using System.Net.Http;
	using System.Threading.Tasks;
	using Microsoft.Extensions.Options;
	namespace YourProject
	{
	public interface IAtlassianOAuthenticator
	{
	HttpClient Client { get; }
	Task<AtlassianOAuthTokenResponse> AuthenticateAsync();
	}
	public interface IPasswordRetriever // you have to implement this class
	{
	string GetPassword(string key);
	}
	public class AtlassianOAuthenticator : IAtlassianOAuthenticator
	{
	private readonly YourProjectOptions _options;
	private readonly IPasswordRetriever _passwordRetriever;
	public HttpClient Client { get; }
	public AtlassianOAuthenticator(
	IOptions<YourProjectOptions> options,
	IPasswordRetriever passwordRetriever,
	HttpClient client
	)
	{
	_options = options.Value;
	_passwordRetriever = passwordRetriever;
	Client = client;
	client.BaseAddress = AtlassianConstants.AtlassianApiRootUri;
	}
	internal string GetPassword(string nameExt)
	{
	var name = _options.ApplicationName + nameExt;
	var password = _passwordRetriever.GetPassword(name); // you write this one
	return password;
	}
	public async Task<AtlassianOAuthTokenResponse> AuthenticateAsync()
	{
	var credentials = new AtlassianOAuthCredentials()
	{
	audience = AtlassianConstants.YourInstanceRootUrl,
	grant_type = "client_credentials",
	client_id = GetPassword(".JiraOAuth2ClientId"),
	client_secret = GetPassword(".JiraOAuth2ClientSecret")
	};
	var response = await Client.PostAsJsonAsync("oauth/token", credentials);
	response.EnsureSuccessStatusCode();
	var result = await response.DeserializeAsync<AtlassianOAuthTokenResponse>();
	return result;
	}
	}
	}

view raw AtlassianOAuthenticator.cs hosted with ❤ by GitHub

	using System;
	using Ucsb.Sa.Enterprise.AzDPullRequests.Web.Models.Atlassian;
	namespace Ucsb.Sa.Enterprise.AzDPullRequests.Web.Models
	{
	public static class AtlassianConstants
	{
	public static string YourInstanceRootUrl = "https://your-domain.atlassian.net/";
	public static string AtlassianApiRootUrl = "https://api.atlassian.com/";
	public const string YourInstanceCloudId = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
	public static Uri YourInstanceRootUri = new Uri(YourInstanceRootUrl);
	public static Uri AtlassianApiRootUri = new Uri(AtlassianApiRootUrl);
	public static JiraProviderMetadata YourInstanceProviderMetadata = new JiraProviderMetadata()
	{
	Product = "<Where you get your data from. Bitbucket? AzureDevOps?>"
	};
	}
	}

view raw AtlassianConstants.cs hosted with ❤ by GitHub

Powershell for working with AzureDevOps Web Hooks

Recently, I was working with the AzureDevOps Web Hooks infrastructure and I really enjoyed the tooling that they had provided. In particular, I liked the interface they provide which shows:

• The number of successful and failed requests sent to a web hook/service,
• The history of those requests,
• And the details of what was actually sent and received (the payloads)

This visibility provided by the tooling was tremendously helpful in developing, debugging and monitoring.

There’s also a fair amount of documentation put together by the AzureDevOps team to make using these services easier.

• Integrate with service hooks
• Create a service hooks subscription programmatically
• Web Hooks
• .NET Client Libraries for Azure DevOps and TFS
• Azure DevOps Services REST API Reference

However, I have two gripes about the documentation.

• It took me a while to find the documentation because I didn’t know the exact search terms to look for. A generalized search for “azuredevops api .net client” will land you in the “Integrate Application” area of the documentation, rather than the desired “Service hooks” area.
• When it got down to the details of which libraries to use and how to use them, the documentation became a bit thin. For example, the AzureDevOps website has a really cool piece of functionality where they hide the “git” event types when the project has no git repositories within it. So, I was thinking about “querying the possible input values” on a particular project to see if the response limited the values similarly. To do that, I originally looked in the .NET API browser for a ServiceHooks*HttpClient object. But, that doesn’t exist. So, I looked at the Publishers – Query Input Values api from the REST documentation. And, the first property I should put in the request body is “currentValues”, which is of type object (what are the details on that?), and has no description given. If I’m trying to query for the current values, what should I put in for “currentValues” and in what format? It becomes apparent that the request and response are using the same object, and that property is only used in the response. So you can ignore that property on the request. But, why doesn’t the documentation state that?

Enough with my tribulations, here’s the results:

• These are a set of functions for working with Web Hooks (even though it’s misnamed as ServiceHook).
• These Web Hooks are created for the very specific needs that I wanted. Which were to create Web Hook integrations with a webservice I built for analyzing code commits, pull requests, and builds.
• The powershell is a wrapper aroound the .NET Client library (nuget) Microsoft.VisualStudio.Service.ServiceHooks.WebApi version 15.131.1, which isn’t the latest version. Since it’s not the latest version of the library, that difference might account for the documentation on the .NET Client libraries not lining up with it. For example, the ServiceHooksPublisherHttpClient does not exist in the .NET API browser.

	# see https://docs.microsoft.com/en-us/azure/devops/integrate/concepts/dotnet-client-libraries?view=azure-devops
	# for links to some of the nuget packages which contain these (this code was written against version 15.131.1)
	Add-Type -Path "C:\YourPath\Newtonsoft.Json.dll"
	Add-Type -Path "C:\YourPath\Microsoft.TeamFoundation.Client.dll"
	Add-Type -Path "C:\YourPath\System.Net.Http.Formatting.dll"
	Add-Type -Path "C:\YourPath\Microsoft.VisualStudio.Services.Common.dll"
	Add-Type -Path "C:\YourPath\Microsoft.VisualStudio.WebApi.dll"
	Add-Type -Path "C:\YourPath\Microsoft.VisualStudio.Services.ServiceHooks.dll"
	$pat = "your-personal-access-token"
	$global:AzDModule = @{}
	$global:AzDModule.Uri = "https://dev.azure.com/{your-organization}"
	$vssBasicCredential = New-Object -TypeName Microsoft.VisualStudio.Services.Common.VssBasicCredential "", $pat
	$global:AzDModule.VssConnection = New-Object `
	-TypeName Microsoft.VisualStudio.Services.WebApi.VssConnection `
	-ArgumentList @($global:AzDModule.Uri, $vssBasicCredential)
	$global:AzDModule.ServiceHooksPublisherClient = $global:AzDModule.VssConnection.GetClient([Microsoft.VisualStudio.Services.ServiceHooks.WebApi.ServiceHooksPublisherHttpClient])

view raw AzDModule.psm1 hosted with ❤ by GitHub

	function Get-AzDServiceHook {
	[CmdletBinding(DefaultParameterSetName = "ProjectName")]
	param(
	[Parameter(ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[string] $ProjectName = $null,
	[Parameter(ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[string] $ProjectId = $null,
	[Parameter(ValueFromPipelineByPropertyName = $true)]
	[ValidateSet("git.push","git.pullrequest.created","git.pullrequest.updated","build.complete")]
	[string] $EventType = $null,
	[Parameter(ValueFromPipelineByPropertyName = $true)]
	[string] $ToUrl = $null,
	[Parameter(ValueFromPipelineByPropertyName = $true)]
	[switch] $Exact
	)
	$shpClient = $global:AzDModule.ServiceHooksPublisherClient
	$inputFilters = New-Object 'System.Collections.Generic.List[Microsoft.VisualStudio.Services.FormInput.InputFilter]'
	$response = $shpClient.QuerySubscriptionsAsync("tfs", $inputFilters, $null)
	if($null -ne $response.Exception) {
	if($null -ne $response.Exception.InnerException) {
	throw $response.Exception.InnerException.Message
	} else {
	throw $response.Exception.Message
	}
	}
	$serviceHooks = $response.Result
	if(-not [string]::IsNullOrWhitespace($EventType)) {
	$serviceHooks = $serviceHooks |? { $_.EventType -eq $EventType }
	}
	if(-not [string]::IsNullOrWhitespace($ToUrl)) {
	if(-not $Exact) {
	if(-not $ToUrl.StartsWith("*")) { $ToUrl = "*" + $ToUrl }
	if(-not $ToUrl.EndsWith("*")) { $ToUrl = $ToUrl + "*" }
	}
	$serviceHooks = $serviceHooks |? { $_.ConsumerInputs.url -like $ToUrl }
	}
	if(-not [string]::IsNullOrWhitespace($ProjectName)) {
	$project = Get-AzDProject -ProjectName $ProjectName
	$ProjectId = $project.Id.ToString()
	}
	if(-not [string]::IsNullOrWhitespace($ProjectId)) {
	$serviceHooks = $serviceHooks |? { $_.PublisherInputs.projectId -eq $ProjectId }
	}
	return $serviceHooks
	}

view raw Get-AzDServiceHook.ps1 hosted with ❤ by GitHub

	function New-AzDServiceHook {
	[CmdletBinding(DefaultParameterSetName = "ProjectName")]
	param(
	[Parameter(Mandatory = $true, ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[string] $ProjectId,
	[Parameter(Mandatory = $true, ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[string] $ProjectName,
	[Parameter(Mandatory = $true, ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[Parameter(Mandatory = $true, ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[ValidateSet("git.push","git.pullrequest.created","git.pullrequest.updated","build.complete")]
	[string] $EventType,
	[Parameter(Mandatory = $true, ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[Parameter(Mandatory = $true, ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[string] $ToUrl,
	[Parameter(Mandatory = $true, ParameterSetName = "SubscriptionObject", ValueFromPipelineByPropertyName = $true)]
	[Microsoft.VisualStudio.Services.ServiceHooks.WebApi.Subscription] $Subscription
	)
	if($PSCmdlet.ParameterSetName -ne "SubscriptionObject") {
	$uri = New-Object System.Uri $ToUrl
	if([string]::IsNullOrEmpty($ProjectId)) {
	$project = Get-AzDProject -ProjectName $ProjectName
	$projectId = $project.Id.ToString()
	}
	$newSub = New-Object Microsoft.VisualStudio.Services.ServiceHooks.WebApi.Subscription
	$newSub.PublisherId = "tfs"
	$newSub.EventType = $eventType
	$newSub.EventDescription = "Any branch on any repository."
	$newSub.ResourceVersion = "1.0"
	$newSub.ConsumerId = "webHooks"
	$newSub.ConsumerActionId = "httpRequest"
	$newSub.ActionDescription = "To host $($uri.Host)"
	$publisherInputs = New-Object 'System.Collections.Generic.Dictionary[System.String, System.String]'
	$publisherInputs.Add("projectId", $projectId)
	$publisherInputs.Add("tfsSubscriptionId", "")
	switch($EventType) {
	"git.push" {
	$publisherInputs.Add("branch", "")
	$publisherInputs.Add("repository", "")
	$publisherInputs.Add("pushedBy", "")
	}
	"git.pullrequest.created" {
	$publisherInputs.Add("branch", "")
	$publisherInputs.Add("repository", "")
	$publisherInputs.Add("pullrequestCreatedBy", "")
	$publisherInputs.Add("pullrequestReviewersContains", "")
	}
	"git.pullrequest.updated" {
	$publisherInputs.Add("branch", "")
	$publisherInputs.Add("repository", "")
	$publisherInputs.Add("pullrequestCreatedBy", "")
	$publisherInputs.Add("pullrequestReviewersContains", "")
	$publisherInputs.Add("notificationType", "")
	}
	"build.complete" {
	$publisherInputs.Add("definitionName", "")
	$publisherInputs.Add("buildStatus", "")
	}
	}
	$newSub.PublisherInputs = $publisherInputs
	$securityHeaderValue = Get-AzDSecurityHeaderValue # you'll come up with something
	$consumerInputs = New-Object 'System.Collections.Generic.Dictionary[System.String, System.String]'
	$consumerInputs.Add("acceptUntrustedCerts", "true")
	$consumerInputs.Add("httpHeaders", "your-security-header:$securityHeaderValue")
	$consumerInputs.Add("url", $ToUrl)
	$newSub.ConsumerInputs = $consumerInputs
	$Subscription = $newSub
	}
	$output = ($Subscription | ft -AutoSize Id, EventType, ConsumerId, Status, ActionDescription | Out-String) -split [Environment]::NewLine
	Write-Verbose "Creating: $($output[3])"
	$shpClient = $global:AzDModule.ServiceHooksPublisherClient
	$response = $shpClient.CreateSubscriptionAsync($Subscription)
	if($null -ne $response.Exception) {
	if($null -ne $response.Exception.InnerException) {
	throw $response.Exception.InnerException.Message
	} else {
	throw $response.Exception.Message
	}
	}
	$result = $response.Result
	return $result
	}

view raw New-AzDServiceHook.ps1 hosted with ❤ by GitHub

	function Update-AzDServiceHook {
	param(
	[Parameter(Mandatory = $true, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true)]
	[Microsoft.VisualStudio.Services.ServiceHooks.WebApi.Subscription] $Subscription,
	[Parameter(ValueFromPipelineByPropertyName = $true)]
	[switch] $Enable,
	[Parameter(ValueFromPipelineByPropertyName = $true)]
	[switch] $Disable
	)
	if($Enable) {
	$Subscription.Status = [Microsoft.VisualStudio.Services.ServiceHooks.WebApi.SubscriptionStatus]::Enabled
	}
	if($Disable) {
	$Subscription.Status = [Microsoft.VisualStudio.Services.ServiceHooks.WebApi.SubscriptionStatus]::DisabledByUser
	}
	$output = ($Subscription | ft -AutoSize Id, EventType, ConsumerId, Status, ActionDescription | Out-String) -split [Environment]::NewLine
	Write-Verbose "Updating: $($output[3])"
	$shpClient = $global:AzDModule.ServiceHooksPublisherClient
	$response = $shpClient.UpdateSubscriptionAsync($Subscription, $null)
	if($null -ne $response.Exception) {
	if($null -ne $response.Exception.InnerException) {
	throw $response.Exception.InnerException.Message
	} else {
	throw $response.Exception.Message
	}
	}
	return $response.Result
	}

view raw Update-AzDServiceHook.ps1 hosted with ❤ by GitHub

	function Remove-AzDServiceHook {
	[CmdletBinding(DefaultParameterSetName = "ProjectName")]
	param(
	[Parameter(ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[Parameter(ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[ValidateSet($null, "git.push","git.pullrequest.created","git.pullrequest.updated","build.complete")]
	[string] $EventType = $null,
	[Parameter(ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[Parameter(ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[string] $ToUrl = $null,
	[Parameter(ParameterSetName = "ProjectId", ValueFromPipelineByPropertyName = $true)]
	[string] $ProjectId = $null,
	[Parameter(ParameterSetName = "ProjectName", ValueFromPipelineByPropertyName = $true)]
	[string] $ProjectName = $null,
	[Parameter(Mandatory = $true, ParameterSetName = "SubscriptionObject", ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true)]
	[Microsoft.VisualStudio.Services.ServiceHooks.WebApi.Subscription] $Subscription
	)
	$subscriptions = @()
	if($PSCmdlet.ParameterSetName -eq "SubscriptionObject") {
	$subscriptions = @($Subscription)
	} else {
	$subscriptions = Get-AzDServiceHook @PSBoundParameters
	if($null -eq $subscriptions) { $subscriptions = @() }
	}
	foreach($sub in $subscriptions) {
	$output = ($sub | ft -AutoSize Id, EventType, ConsumerId, Status, ActionDescription | Out-String) -split [Environment]::NewLine
	Write-Verbose "Removing: $($output[3])"
	$shpClient = $global:AzDModule.ServiceHooksPublisherClient
	$response = $shpClient.DeleteSubscriptionAsync($sub.Id, $null)
	if($null -ne $response.Exception) {
	if($null -ne $response.Exception.InnerException) {
	throw $response.Exception.InnerException.Message
	} else {
	throw $response.Exception.Message
	}
	}
	}
	}

view raw Remove-AzDServiceHook.ps1 hosted with ❤ by GitHub

Record Request Body in ASP.NET Core 3.0

Application Insights is a great tool, but it doesn’t record the body of a request by default. This is for good reason, payloads can be large and can sometimes contain sensitive information. But … sometimes you just need to record them.

When you do a google search there’s a great StackOverflow post (View POST request body in Application Insights) which gives you a lot of hope that it can be setup easily. But, with all the advancements in ASP.NET Core 3.0, it’s not quiet as easy as that post makes it look.

Here’s the obstacles you may need to overcome:

• In ASP.NET Core 3.0, the request’s Body is no longer available to be read after a particular point in the application pipeline. You will need to create a middleware component to “EnableBuffering”. (This was done for purposes of speed as the underlying layers of the stack were replaced with Span’s. There’s a new Request.BodyReader that works with the spans for high performance, but it’s also not available to be read after a particular point in the application pipeline.)
  
• The ITelemetryInitializer runs after a request completes. This means that the request’s body is disposed of by the time the initializer runs and records the event. You will have to record the body somewhere after “EnableBuffering” is enabled and before the Action completes. Like inside of an IActionFilter.
  
• You may not want to record the body of everything that flows through your website, so an ActionFilterAttribute can make it easy to select which action you would like to record.

So, here’s some code that can help accomplish that:

	using System.IO;
	using System.Threading.Tasks;
	using Microsoft.AspNetCore.Http;
	using Microsoft.AspNetCore.Http.Features;
	namespace YourNamespace
	{
	public class EnableBufferingMiddleware
	{
	private readonly RequestDelegate _next;
	public EnableBufferingMiddleware(
	RequestDelegate next
	)
	{
	_next = next;
	}
	public async Task InvokeAsync(HttpContext context)
	{
	var request = context.Request;
	if (request.CanReadBody())
	{
	//var syncIOFeature = context.Features.Get<IHttpBodyControlFeature>();
	//if (syncIOFeature != null)
	//{
	// syncIOFeature.AllowSynchronousIO = true;
	//}
	request.EnableBuffering();
	}
	await _next(context);
	}
	}
	}

view raw EnableBufferingMiddleware.cs hosted with ❤ by GitHub

	using System.Collections.Generic;
	using System.IO;
	using System.Text;
	using System.Threading.Tasks;
	using Microsoft.ApplicationInsights;
	using Microsoft.AspNetCore.Builder;
	using Microsoft.AspNetCore.Hosting;
	using Microsoft.AspNetCore.Http;
	using Microsoft.Extensions.Configuration;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.Hosting;
	namespace YourNamespace
	{
	public class Startup
	{
	// This method gets called by the runtime. Use this method to add services to the container.
	public void ConfigureServices(IServiceCollection services)
	{
	services.AddControllers();
	}
	// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
	public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
	{
	app.UseMiddleware<EnableBufferingMiddleware>();
	// use that middleware before everything else
	}
	}
	}

view raw Startup.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Threading.Tasks;
	namespace YourNamespace
	{
	public class TrackConstants
	{
	public const string JsonBody = "JsonBody";
	}
	}

view raw TrackConstants.cs hosted with ❤ by GitHub

	using System.IO;
	using System.Text;
	using System.Threading.Tasks;
	using Microsoft.AspNetCore.Mvc.Filters;
	namespace YourNamespace
	{
	public class TrackRequestBodyAttribute : ActionFilterAttribute
	{
	public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
	{
	var request = context.HttpContext.Request;
	request.Body.Position = 0;
	using var stream = new StreamReader(request.Body, Encoding.UTF8, false, 1024, true);
	var body = await stream.ReadToEndAsync();
	request.Body.Position = 0;
	context.HttpContext.Items.Add(TrackConstants.JsonBody, body);
	await base.OnActionExecutionAsync(context, next);
	}
	}
	}

view raw TrackRequestBodyAttribute.cs hosted with ❤ by GitHub

	using Microsoft.ApplicationInsights.Channel;
	using Microsoft.ApplicationInsights.DataContracts;
	using Microsoft.ApplicationInsights.Extensibility;
	using Microsoft.AspNetCore.Http;
	namespace YourNamespace
	{
	// https://stackoverflow.com/questions/42686363/view-post-request-body-in-application-insights
	public class TrackRequestBodyInitializer : ITelemetryInitializer
	{
	private readonly IHttpContextAccessor httpContextAccessor;
	public TrackRequestBodyInitializer(IHttpContextAccessor httpContextAccessor)
	{
	this.httpContextAccessor = httpContextAccessor;
	}
	public void Initialize(ITelemetry telemetry)
	{
	if (telemetry is RequestTelemetry requestTelemetry)
	{
	var context = httpContextAccessor.HttpContext;
	var request = context.Request;
	if (request.CanReadBody())
	{
	const string jsonBody = TrackConstants.JsonBody;
	if (!context.Items.ContainsKey(jsonBody))
	{
	return;
	}
	if (requestTelemetry.Properties.ContainsKey(jsonBody))
	{
	return;
	}
	var body = context.Items[jsonBody] as string;
	requestTelemetry.Properties.Add(jsonBody, body);
	}
	}
	}
	}
	}

view raw TrackRequestBodyInitializer.cs hosted with ❤ by GitHub

	using System.Threading.Tasks;
	using Microsoft.AspNetCore.Mvc;
	using Microsoft.Extensions.Logging;
	namespace YourNamespace
	{
	[ApiController]
	[Route("[controller]")]
	public class YourController : ControllerBase
	{
	private readonly ILogger<YourController> _logger;
	public PullRequestUpdatedController(
	ILogger<YourController> logger
	)
	{
	_logger = logger;
	}
	[TrackRequestBody]
	public async Task<IActionResult> PostAsync([FromBody] YourObjectModel model)
	{
	// do stuff
	return Ok();
	}
	}
	}

view raw YourController.cs hosted with ❤ by GitHub

What’s an Andon Cord?

I’ve seen a couple of great explanations of an Andon Cord, but I feel like there’s another side to them. Something that hasn’t already been written about in John Willis’ The Andon Cord, Six Sigma Daily’s The Andon Cord: A Way to Stop Work While Boosting Productivity, or even Amazon’s Andon Cord for their distributed supply chain (example 1, 2).

Personally, I’ve heard two descriptions of the Andon Cord and one of them makes a lot of sense to me, and the other one is the popular definition. The popular definition is that an Andon Cord is the capability to pause/halt any manufacturing line in order to ensure quality control and understanding. Anyone can pull the Andon Cord and review a product that they are unsure about. Which intuitively makes sense.

The second definition I heard was from Mike Rother’s book, Toyota Kata, and wasn’t so much of a definition as a small glimpse into the history the Andon Cord. The concept that we know today started at Toyota before they made cars. This was back when they were making sewing machines. And, you need to imagine that the sewing machine manufacturing line was modeled off of Henry Ford’s Model T production lines. So there was a belt that ran across the factory floor and in between stations that people worked at. The Sewing Machines would be mounted to the line and would move from one station to the next on a timed interval (lets say every 5 minutes). So, each person would be working at their station and they would have 5 minutes to complete the work of their station; which was usually installing a part and then some sort of testing. If the person on the line felt that they couldn’t complete their work on time, then they should pull the Andon Cord to freeze the line. This ensured that no defective part/installation continued on down the factory line. The underlying purpose of not having a bad part go down the line is that disassembling and reassembling a machine to replace a defective part is very expensive compared to stopping the line and fixing it while the machine is at the proper assembly level. This makes complete sense to me.

The second definition makes a lot more sense to me than the first because of one unspoken thing:

Anyone on the assembly line can pull the Andon Cord. The Andon Cord can be pulled by anyone, but it’s supposed to be for the people who are actually on the assembly and have expert knowledge about their step within the overall process. It’s their experience and knowledge on that particular product line which makes them the correct person to pull the cord. It’s not for people from other product lines to come over and pull their lines cord.

This is a classic problem that I’ve run into time and again. On multiple occassions, I have seen the Ops and Management side of businesses hear that “anyone can pull the Andon Cord” and immediately start contemplating how they can use the cord to add Review Periods into process lines and allow anyone to put the brakes on a production deployment if anyone doesn’t understand it.

But those ideas seems counter-productive to the overall goals of DevOps. You don’t want to add a Review Period as that just delays the business value from getting to the end customer. And you don’t want to stop a release because someone who isn’t an expert on a product has a question about it; you want that person to go ask the experts, and then you want the experts on a Product Line to pull the Andon Cord.

Now, in an idealized world, all the people involved in a products deployment process would all be on the same Product Team. That team would be made up of Dev, Ops, and other team members. And all of those team members would be experts on the product line and would be the right people to pull the cord.

However, the majority of businesses I’ve talked with have separate Devs and Ops/Engineering teams. Simply because that structure has been lauded as a very cost effective way to reduce the companies expenditure on Ops and allow for their knowledge to be centralized and therefore non-redundant. But, when the Ops team is separate from the Dev team then the Ops team has a sense that they are a part of every product line; and they should have ownership over allowing any release to go to production. Even when they are not experts on the product line and have no knowledge of what a change actually does.

This sense of ownership that Ops (and to some degree Management) have often manifest in the form of asking for a review period between the time a deployment has passed all of it’s testing requirements and it actually goes out to production. This review period should start with a notification to the customers and usually end a few hours/or a day later so that Operations, Management, and Customers all have time to review the change and pull the cord if they have concerns about the change. Except, the Customer and the Product Team are the only ones on that line who are really experts on the product. And for customers that work alongside their product teams, they usually know what’s coming long before scheduling; and customers that don’t work alongside their product teams usually won’t be involved at all at this point.

So, if the above is true, then Operations (and Management) wouldn’t be experts on the product line at this point in the release process, so why would they be pulling the cord at this point?

For Management, I’m not sure. But, for Operations, they are experts on the Production environment that the deployment will be going into. Ops should be aware if there are any current issues in the production environment and be able to stop a deployment from making a bad situation worse. But, that isn’t a product line Andon Cord. That’s an Andon Cord for an entire environment (or a subsection of an environment). The Operations team should have an Andon Cord to pause/halt all deployments from going into Production if something is wrong with that environment. Once the environment has been restored to a sense of stability, then Operations should be able to release the cord and let the queued deployments roll out again. (sidenote: Many companies that are doing DevOps have communication channels setup where everyone should be aware of a Production environment problem; this should allow for “anyone” to pull the Environment Andon Cord and pause deployments for a little while.)

Finally, in the popular definition of the Andon Cord there is a lot of attention paid to human beings pulling the Andon Cord, but not a lot of explicit statements about machines pulling the Andon Cord. For me, I see it as both groups can pull the Andon Cord. It seems like everyone intuitively understands that if unit tests, or smoke tests, or a security scan fails then the process should stop and the product should go back to the developer to fix it. What I don’t think people connect is that that’s an Andon Cord pull. It’s an automated pull to stop the process and send the product back to the station that can fix the problem with the least amount of rework required. To see that though, you have to first recognize that a CI/CD automated build and deployment process is the digital transformation of a factory floors product line. Your product moves from station to station through human beings and automated tooling alike (manual commit, CI build, CI unit tests, manual code review, manual PR approval, CI merge, CI packaging, CD etc.), and at every station there is a possibility of an Andon Cord pull.

AzureDevOps PR Policies with Powershell

After tinkering with AzureDevOps Pull Request Statuses, I started thinking about automating the creation of the policies. The build policies need to be setup on an individual repository basis because it requires a build definition to be associated with it. Some policies could be moved to the Project level (like the Minimum Number of Reviewers or Merge Strategy policies). But, I’m not sure if you can override a Project level policy in an individual branch.

So, I went searching to see if anyone had already done it and Jesse Houwing had: Configuring standard policies for all repositories in Azure Repos. I really liked his approach to exploring the API and using the json created by AzureDevOps to help construct his solution. I also was really interested by the idea of using the Azure CLI to perform the updates.

However, I was still pretty enamored with how easy the .NET SDK libraries were to use and thought I could continue to use them with Powershell to perform similar updates. So, here’s some sample code which can create a simple policy like: At least one reviewer is required on a Pull Request and that reviewer can be the requestor.

	# Use the connection code from http://stevenmaglio.blogspot.com/2020/05/update-azuredevops-pr-status-with.html
	# to setup your $projectName, $repoName, and $vssConnection variable
	$gitClient = $vssConnection.GetClient([Microsoft.TeamFoundation.SourceControl.WebApi.GitHttpClient])
	$policyClient = $vssConnection.GetClient([Microsoft.TeamFoundation.Policy.WebApi.PolicyHttpClient])
	##### Get the RepoId guid
	# https://docs.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.sourcecontrol.webapi.githttpclientbase.getrepositoriesasync?view=azure-devops-dotnet#Microsoft_TeamFoundation_SourceControl_WebApi_GitHttpClientBase_GetRepositoriesAsync_System_String_System_Nullable_System_Boolean__System_Nullable_System_Boolean__System_Nullable_System_Boolean__System_Object_System_Threading_CancellationToken_
	$repositories = $gitClient.GetRepositoriesAsync($projectName).Result
	$repo = $repositories |? { $_.Name -eq $repoName }
	Write-Host "RepositoryId: $($repo.Id)"
	$project = $repo.ProjectReference
	##### Get the list of policie types for a Project
	# https://docs.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.policy.webapi.policyhttpclient.getpolicytypesasync?view=azure-devops-dotnet#Microsoft_TeamFoundation_Policy_WebApi_PolicyHttpClient_GetPolicyTypesAsync_System_Guid_System_Object_System_Threading_CancellationToken_
	$policyTypes = $policyClient.GetPolicyTypesAsync($project.Id).Result
	$minReviewersPolicyType = $policyTypes |? { $_.DisplayName -eq "Minimum number of reviewers" }
	### Create a new minimum number of reviewers policy
	# https://docs.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.policy.webapi.policyconfiguration?view=azure-devops-dotnet
	$policyConfig = New-Object Microsoft.TeamFoundation.Policy.WebApi.PolicyConfiguration
	$policyConfig.IsEnabled = $true
	$policyConfig.IsBlocking = $true
	# validDuration = 720, is 12 hours in minutes
	$settingsJson = @"
	{
	"minimumApproverCount": 1,
	"creatorVoteCounts": false,
	"allowDownvotes": false,
	"resetOnSourcePush": false,
	"blockLastPusherVote": false,
	"scope": [
	{
	"refName": "refs/heads/master",
	"matchKind": "Exact",
	"repositoryId": "$($repo.Id)"
	}
	]
	}
	"@
	# https://www.newtonsoft.com/json/help/html/ParseJsonObject.htm
	$settings = [Newtonsoft.Json.Linq.JObject]::Parse($settingsJson)
	$policyConfig.Settings = $settings
	$policyConfig.Type = $minReviewersPolicyType
	$policyClient = $global:VstsUcsb.PolicyClient
	# https://docs.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.policy.webapi.policyhttpclient.createpolicyconfigurationasync?view=azure-devops-dotnet#Microsoft_TeamFoundation_Policy_WebApi_PolicyHttpClient_CreatePolicyConfigurationAsync_Microsoft_TeamFoundation_Policy_WebApi_PolicyConfiguration_System_Guid_System_Nullable_System_Int32__System_Object_System_Threading_CancellationToken_
	$policy = Invoke-AzDRequest -ScriptBlock { $policyClient.CreatePolicyConfigurationAsync($policyConfig, $project.Id) }
	Write-Host "Created Minimum policy for the master branch of Repository '$($repo.Name)'"

view raw gistfile1.txt hosted with ❤ by GitHub

Update AzureDevOps PR Status’ with Powershell

AzureDevOps integrates third party CI pipeline components through a Pull Request Status API (example usage with node.js). The end goal of the status API is to allow you to integrate any number of 3rd party tooling into your pull request validation processes. Hopefully creating something that looks like this:

The Build succeeded link comes from the built-in Build Policy of Azure DevOps Branch Protection. But, for any other service, you’ll need to integrate it’s results using an external service. That’s how the Deploy succeeded link was created in the screenshot above.

There are examples on how to use the API for external services (like the node.js example), but I didn’t run across any examples for powershell. So, here’s a quick example.

The example code:

• Load’s up some dlls from AzureDevOps .NET Client Libraries.
• Loads up the VSTeam powershell module.
• Signs into Azure DevOps using your Personal Access Token (both VSTeam and the .NET Client libraries need to be signed into separately).
• Uses VSTeam to retrieve your Project’s internal Guid.
• You can do this with the .NET libraries too. I just want to support the usage of the VSTeam module.
• Uses the .NET Libraries to retrieve your Repositories internal Guid.
• Creates a GitPullRequestStatus object and populates it with some information.
• Uses CreatePullRequestStatusAsync to update the status.
• The great part of CreatePullRequestStatusAsync is that the display will always show the most recently supplied status for an integration component. So, you don’t have to worry about “updating” the status. Just call CreatePullRequestStatusAsync repeatedly throughout your process to update the status.

	Import-Module VsTeam # https://github.com/DarqueWarrior/vsteam
	##### Variables you'll need
	$pat = "your-azuredevops-personal-access-token"
	$org = "your-azuredevops-org-name (ie. https://dev.azure.com/your-azuredevops-org-name/)"
	$projectName = "your-azuredevops-projectname" # ie. https://dev.azure.com/your-azuredevops-org-name/your-azuredevops-projectname)
	$repoName = "your-repository-name" # ie. https://dev.azure.com/your-azuredevops-org-name/your-azuredevops-projectname/_git/your-repository-name)
	[int] $pullRequestId = 143
	# you can find the pull request id using GetPullRequestsAsync
	# https://docs.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.sourcecontrol.webapi.githttpclientbase.getpullrequestsasync?view=azure-devops-dotnet
	##### Load up some dlls (you may need to get some of these from nuget)
	Write-Verbose "Loading ordered list of dlls"
	$orderedLoadList = @(
	"Newtonsoft.Json.dll",
	"Microsoft.TeamFoundation.Client.dll",
	"System.Net.Http.Formatting.dll",
	"Microsoft.TeamFoundation.Build.Common.dll",
	"Microsoft.TeamFoundation.Build.Client.dll",
	"Microsoft.TeamFoundation.VersionControl.Client.dll"
	)
	foreach($dll in $orderedLoadList) {
	$dllPath = Join-Path -Path $resourcesDir -ChildPath $dll
	Write-Verbose "`t$dllPath"
	[System.Reflection.Assembly]::LoadFrom($dllPath) | Out-Null
	}
	##### Setup your connections to AzureDevOps
	Set-VSTeamAccount -Level Process -PersonalAccessToken $pat -Account $org -Version VSTS
	$vssBasicCredential = New-Object `
	-TypeName Microsoft.VisualStudio.Services.Common.VssBasicCredential `
	-ArgumentList @("", $pat)
	$url = "https://dev.azure.com/{0}" -f $org
	$vssConnection = New-Object `
	-TypeName Microsoft.VisualStudio.Services.WebApi.VssConnection `
	-ArgumentList @($url, $vssBasicCredential)
	$git = $vssConnection.GetClient([Microsoft.TeamFoundation.SourceControl.WebApi.GitHttpClient])
	##### Get the ProjectId guid
	$project = Get-VsTeamProject -Name $projectName
	Write-Host "ProjectId: $($project.Id)"
	##### Get the RepoId guid
	$repositories = $git.GetRepositoriesAsync($project.Id).Result
	$repo = $repositories |? { $_.Name -eq $repoName }
	Write-Host "RepositoryId: $($repo.Id)"
	##### Create Status Update Object
	$status = New-Object Microsoft.TeamFoundation.SourceControl.WebApi.GitPullRequestStatus
	# https://docs.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.sourcecontrol.webapi.gitstatusstate?view=azure-devops-dotnet
	$status.State = [Microsoft.TeamFoundation.SourceControl.WebApi.GitStatusState]::Succeeded
	# $status.State = [Microsoft.TeamFoundation.SourceControl.WebApi.GitStatusState]::Pending
	# $status.State = [Microsoft.TeamFoundation.SourceControl.WebApi.GitStatusState]::Failed
	$description = ""
	switch($GitStatusState) {
	Succeeded { $description = "Deploy succeeded" }
	Pending { $description = "Deploy running" }
	Failed { $description = "Deploy failed" }
	}
	$status.Description = $description
	$status.TargetUrl = "https://link-to-your-external-services/build-or-results"
	$statusContext = New-Object Microsoft.TeamFoundation.SourceControl.WebApi.GitStatusContext
	$statusContext.Name = "your-external-service-name"
	$statusContext.Genre ="ci"
	$status.Context = $statusContext
	Write-Host @"
	---------------------------------------
	Status Update:
	$($status | Out-String)
	"@
	##### Perform Status Update
	$response = $git.CreatePullRequestStatusAsync($status, $project.Id, $repo.Id, $pullRequestId)
	if($response.Exception) {
	Write-Host @"
	--------------------------------------------------------
	Exception:
	$($response | Out-String)
	"@
	throw $response.Exception.Message
	}
	Write-Host @"
	---------------------------------------------------------
	Result:
	$($response.Result | Out-String)
	"@

view raw Update-AzDWithExternalServiceStatus.ps1 hosted with ❤ by GitHub

Book Review? A Seat at the Table

Mark Schwartz’ A Seat at the Table (amazon, audible) is as close of description of DevOps to the one I have in my head as I’ve ever read. It definitely doesn’t have all the answers, and it asks some questions where the offered answers don’t feel like they satisfy all aspects of the question. But, it’s always an earnest answer which tries to address as many aspects as it can while still being very coherent and relatable.

Some pieces that I haven’t heard from other books (which seemed to match my own thoughts):

• Projects which are bound by start and ends dates aren’t enough for long term success. By placing an artificial end date on a project it only ensures that the product will become stale and unmaintained after that end date. Even if you were to have product teams (Project to Product) it still isn’t enough, because that only secures knowledgeable resources to be available as future needs are determined. You also need to flatten the management structure to give the product team full agency over the product direction. So that the team can pull feedback directly from their customers and determine the future needs themselves. This is very hard to achieve because of the need for the team to be knowledgeable about the long term needs of the company and it’s strategies.
• In a lot of Lean Management books (like James P. Womack’s works) there are incredibly vivid and tangible descriptions of how to implement Lean Principles within a Product line. But, they feel somewhat vague about the role of management within Lean production. Even Gemba Walks (2nd Edition) description felt like it was saying “the leadership should set a direction and strategize on how to execute in that direction without creating confusion”. It’s very vague and kind of hand wavy.
  
  But, in this book, Mark Schwartz very clearly states that management is a form of Waste. “If your not coding, your a waste” (<—take that with a grain of salt, it’s taken out of context). The goal of management is to set a direction and remove as many impediments as possible from the path to achieve that goal. Management adds value by removing the impediments and reducing the number of interruptions. And you don’t need multiple levels of management to do that. For me, that helps explain why Lean Management books are soo thin on the topic of what Managements role is: Management just isn’t a massively important part of product creation in Lean.

Of course no two people ever see eye-to-eye on everything. There was one statement that really threw me for a loop and I’m puzzling to better understand his view point:

In a chapter describing Quality in Software development he stated that all bugs should fall into two categories: either they should be fixed immediately, or they should be accepted as acceptable and never recorded/addressed as backlog items.

Now, I’m definitely on board with the idea that not all bugs are equal and there are some that should be fixed right away: Build is broken (test failure, deployment failure), production bug, practically anything that prevents work from being completed. But, when a bug isn’t an immediate fix, I feel like it should still be added to the backlog to be fixed during the next sprint. Maybe I need to read/listen to the chapter again and see if he made some caveats around when you should change it from a bug report to a feature request, or if the bug report comes from an end user it’s always an immediate fix. I don’t know.

I really hope that one day he might write another book that tackles real world problem scenarios that he’s run into and how he overcame them. I feel like the majority of DevOps books bring up that there are almost always conflicts between colleagues surrounding producing functionality vs meeting security and compliance goals; but then the authors wave a magic wand by saying “and this leads to discussions that ultimately result in value being created.” But, I haven’t read a book that digs in and describes an actual situation that occurred, getting into the details of the difficulties (from conflicts in opinion, conflicts over policy interpretation, conflicts in ideology, etc) and then describes all the tools and strategies that we used to overcome each of the difficulties. Maybe I have read that book and I’m just too dense to have noticed it.

camelCase Enums in Swashbuckle

In an earlier version of Swashbuckle.AspNetCore, SwaggerGenOptions came with two extensions method that helped facilitate returning camelCase values from your enum data types.

	services.AddSwaggerGen(c =>
	{
	c.DescribeAllEnumsAsStrings();
	c.DescribeStringEnumsInCamelCase();
	});

view raw SwaggerIServiceCollectionExtensions.cs hosted with ❤ by GitHub

However, in newer versions of Swashbuckle, those methods have been deprecated and they ask you to use the built-in features of your preferred JsonSerializer (this example is for Newtonsoft’s Json.NET):

	services.AddMvc()
	.AddNewtonsoftJson(options => {
	var strategy = new CamelCaseNamingStrategy();
	options.SerializerSettings.Converters.Add(new StringEnumConverter() { NamingStrategy = strategy });
	});

view raw Startup.cs hosted with ❤ by GitHub

These options will produce the expected camelCased data to flow across the wire. But, it won’t update the documentation generated by Swashbuckle to show the camelCased values. Instead Swashbuckle will continue to show the enum values:

The Unchase.Swashbuckle.AspNetCore.Extensions collection has an Enum’s extension which will provide the documentation that you’re looking for, but it still expects you to return numerical values (instead of the camelCased names).

I was looking for both:

• Return camelCased values across the wire, and
• Update the swagger/OpenApi documentation to display the camelCased values

Of course, I couldn’t be the only one looking for something like this and there was a nice piece of starting code on stackoverflow (Swagger UI Web Api documentation Present enums as strings), which grew into an EnumDocumentFilter.cs class (its specific to Newtonsoft’s Json.NET). The document filter produces output that looks like this:

	using System;
	using System.Collections;
	using System.Collections.Generic;
	using System.Linq;
	using System.Reflection;
	using System.Reflection.Metadata;
	using System.Text.Json;
	using Microsoft.AspNetCore.Mvc;
	using Microsoft.Extensions.Options;
	using Microsoft.OpenApi.Any;
	using Microsoft.OpenApi.Models;
	using Newtonsoft.Json.Converters;
	using Newtonsoft.Json.Serialization;
	using Swashbuckle.AspNetCore.SwaggerGen;
	namespace YourNamespace
	{
	/// <summary>
	/// https://stackoverflow.com/questions/36452468/swagger-ui-web-api-documentation-present-enums-as-strings
	/// </summary>
	public class EnumDocumentFilter : IDocumentFilter
	{
	private MvcNewtonsoftJsonOptions _jsonOptions;
	private bool _isCamelCase = true;
	public EnumDocumentFilter(
	IOptions<MvcNewtonsoftJsonOptions> jsonOptions
	) {
	_jsonOptions = jsonOptions.Value;
	var stringEnumConvertor = _jsonOptions.SerializerSettings.Converters.FirstOrDefault(c =>
	{
	return c.GetType() == typeof(StringEnumConverter);
	}) as StringEnumConverter;
	if (stringEnumConvertor != null)
	{
	_isCamelCase = stringEnumConvertor.NamingStrategy.GetType() == typeof(CamelCaseNamingStrategy);
	}
	}
	public void Apply(OpenApiDocument openApiDoc, DocumentFilterContext context)
	{
	var schemas = openApiDoc.Components.Schemas;
	foreach (var schemaNode in schemas.Where(x => x.Value?.Enum?.Count > 0))
	{
	IList<IOpenApiAny> propertyEnums = schemaNode.Value.Enum;
	if (propertyEnums != null && propertyEnums.Count > 0)
	{
	var name = schemaNode.Key;
	var schema = schemaNode.Value;
	schema.Description += DescribeEnum(propertyEnums, name, context);
	schema.Enum = GenerateOpenApiEnumAsString(name, context);
	schema.Type = "string";
	schema.Format = null;
	}
	}
	// add enum descriptions to input parameters
	foreach (var pathItem in openApiDoc.Paths.Values)
	{
	DescribeEnumParameters(pathItem.Operations, openApiDoc, context);
	}
	}
	public List<IOpenApiAny> GenerateOpenApiEnumAsString(string typeName, DocumentFilterContext context)
	{
	var type = GetEnumTypeByName(typeName, context);
	var enumNames = Enum.GetNames(type);
	var openApiEnumNames = new List<IOpenApiAny>();
	foreach (var n in enumNames)
	{
	var formattedName = FormatName(n);
	openApiEnumNames.Add(new OpenApiString(formattedName));
	}
	return openApiEnumNames;
	}
	private string FormatName(string name)
	{
	if (!_isCamelCase) return name;
	if (name.Length == 1) return name.ToLower();
	var camelCase = name.Substring(0, 1).ToLower() +
	name.Substring(1);
	return camelCase;
	}
	private void DescribeEnumParameters(
	IDictionary<OperationType, OpenApiOperation> operations,
	OpenApiDocument openApiDoc,
	DocumentFilterContext context
	) {
	if (operations != null)
	{
	foreach (var oper in operations)
	{
	foreach (var param in oper.Value.Parameters)
	{
	var schemaNode = openApiDoc.Components.Schemas.FirstOrDefault(x => x.Key == param.Name);
	if (schemaNode.Value != null)
	{
	var name = schemaNode.Key;
	var schema = schemaNode.Value;
	param.Description += DescribeEnum(schema.Enum, name, context);
	param.Schema.Enum = GenerateOpenApiEnumAsString(name, context);
	schema.Type = "string";
	schema.Format = null;
	}
	}
	}
	}
	}
	private Type GetEnumTypeByName(string enumTypeName, DocumentFilterContext context)
	{
	var matches = AppDomain.CurrentDomain
	.GetAssemblies()
	.SelectMany(x => x.GetTypes())
	.Where(x => enumTypeName.EndsWith(x.Name ));
	foreach(var m in matches)
	{
	string schemaId;
	context.SchemaRepository.TryGetIdFor(m, out schemaId);
	if (schemaId == enumTypeName) return m;
	}
	throw new ArgumentException();
	}
	private string DescribeEnum(IList<IOpenApiAny> enums, string proprtyTypeName, DocumentFilterContext context)
	{
	List<string> enumDescriptions = new List<string>();
	var enumType = GetEnumTypeByName(proprtyTypeName, context);
	if (enumType == null)
	return null;
	foreach (OpenApiInteger enumOption in enums)
	{
	int enumInt = enumOption.Value;
	var name = Enum.GetName(enumType, enumInt);
	var formattedName = FormatName(name);
	enumDescriptions.Add(string.Format("{0} = {1}", enumInt, formattedName));
	}
	return string.Join(", ", enumDescriptions.ToArray());
	}
	}
	}

view raw EnumDocumentFilter.cs hosted with ❤ by GitHub

Swashbuckle Duplicate SchemaId Detected

I’m sure this is rarely run into, but every once in a while, you might reuse the name of  class between two namespaces. For example:

• YourCompany.CommonComponentLibary.SystemOperation
• YourCompany.ProjectXXXXX.SystemOperation

If both of those classes are then exposed as return types from an API endpoint, there is a chance that Swagger might throw an error that looks something like this:

The Swashbuckle team has run into this / thought about this and there is a function called CustomSchemaIds to handle it. The function’s usage looks a little like this (default implementation):

	services.ConfigureSwaggerGen(options =>
	{
	//your custom configuration goes here
	...
	// UseFullTypeNameInSchemaIds replacement for .NET Core
	options.CustomSchemaIds(x => x.FullName);
	});

view raw CustomSchemaIds.cs hosted with ❤ by GitHub

As best as I can tell the intention behind this function is to generate out the expected name for a given Type, completely agnostic of any external information.

Not using external information creates helps ensure the result should always be the same, because their is no “state of the system” information used to create unique names. This makes the names consistent despite processing order or updates to the underlying code base. So, it’s a good thing.

But … it can make for rather large names. Using the above code snippet as a example, the FullName is a unique name, but it contains a lot of information about how you internally generated the name. I’m not looking to have that information concealed for any security or risk purpose, it can just be hard to read in a json/yaml format or even in a Swagger UI display.

So, it might be easier to create a new Custom Schema Id which would try to stick with the default implementation, but alter it to return a longer name if conflicts occur. For example, what if the rules were:

• Use the default implementation of the Class Name (without Namespace) when possible.
• If the Class Name has already been used, then start prefixing Namespace names (from closest to the Class Name, back down to the root Namespace last).

An example of this might be:

• YourCompany.CommonComponentLibary.SystemOperation –> SystemOperation
• YourCompany.ProjectXXXXX.SystemOperation –> ProjectXXXXX.SystemOperation

But, to do that you would need to know about the current registered types (ie. external information).

Interestingly enough, the code which calls the CustomSchemaId/SchemaIdSelector function has access to a SchemaRepository class which contains exactly that information. But, it doesn’t pass the SchemaRepository into the function. The SchemaRespository was available within the code at the time of the SchemaIdSelector’s introduction (Swashbuckle.AspNetCore.SwaggerGen-5.0.0-rc2), so it could have been passed in. But, sometimes it’s just hard for see weird use cases like the one I’m describing.

There is a way to implement the use case I’m describing, by replicating the SchemaRepository within your own code. It doesn’t take a lot of effort, but it can feel like you’re doing something wrong. Here’s what that can look like:

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using Microsoft.AspNetCore.Builder;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.DependencyInjection.Extensions;
	using Microsoft.OpenApi.Models;
	namespace YourCompany.Web
	{
	public interface ISwaggerSchemaIdSelector
	{
	string Select(Type type);
	}
	public class SwaggerSchemaIdSelector : ISwaggerSchemaIdSelector
	{
	public class UsedInfo
	{
	public string Name { get; set; }
	public string FullName { get; set; }
	public Type Type { get; set; }
	}
	private Dictionary<string, UsedInfo> _usedNames;
	private Dictionary<Type, string> _nameLookups;
	public SaSwaggerSchemaIdSelector(Dictionary<string, UsedInfo> usedNames = null)
	{
	if (usedNames != null)
	{
	_usedNames = usedNames;
	_nameLookups = new Dictionary<Type, string>();
	foreach (var name in _usedNames.Keys)
	{
	var info = _usedNames[name];
	_nameLookups.Add(info.Type, name);
	}
	}
	else
	{
	_usedNames = new Dictionary<string, UsedInfo>();
	_nameLookups = new Dictionary<Type, string>();
	}
	}
	public string Select(Type type)
	{
	if (ContainsMappedType(type))
	{
	return _nameLookups[type];
	}
	var name = GetName(type);
	var fullname = type.FullName;
	if (ContainsName(name))
	{
	if (ContainsUsedType(name, type))
	{
	return name;
	}
	var names = fullname.Split(".");
	var i = names.Length - 2;
	while (i >= 0 && ContainsName(name))
	{
	name = names[i] + "." + name;
	i--;
	}
	if (ContainsUsedType(name, type) == false)
	{
	AddusedType(name, type);
	return name;
	}
	}
	if (ContainsName(name) == false)
	{
	AddusedType(name, type);
	}
	return name;
	}
	private string GetName(Type type)
	{
	if (!type.IsConstructedGenericType) return type.Name;
	var prefix = type.GetGenericArguments()
	.Select(genericArg => GetName(genericArg))
	.Aggregate((previous, current) => previous + current);
	return prefix + type.Name.Split('`').First();
	}
	private void AddusedType(string name, Type type)
	{
	var info = new UsedInfo {Name = name, FullName = type.FullName, Type = type};
	_usedNames.Add(name, info);
	_nameLookups.Add(info.Type, name);
	}
	private bool ContainsMappedType(Type type)
	{
	if (_nameLookups.ContainsKey(type)) return true;
	return false;
	}
	private bool ContainsUsedType(string name, Type type)
	{
	if (_usedNames.ContainsKey(name))
	{
	var usedBy = _usedNames[name];
	var containsType = usedBy.Type == type;
	return containsType;
	}
	return false;
	}
	private bool ContainsName(string name)
	{
	return _usedNames.ContainsKey(name);
	}
	}
	public static class SwaggerExtension
	{
	public static void AddSwagger(this IServiceCollection services)
	{
	services.TryAddSingleton<ISwaggerSchemaIdSelector, SwaggerSchemaIdSelector>();
	var provider = services.BuildServiceProvider();
	var schemaBuilder = provider.GetService<ISwaggerSchemaIdSelector>();
	services.AddSwaggerGen(c =>
	{
	c.SwaggerDoc("v1", new OpenApiInfo
	{
	Version = "v1",
	Title = "YourCompany.Web",
	Description = "YourCompany.Web",
	Contact = new OpenApiContact() { Name = "Joe Developer", Email = "joe.developer@email.com" }
	});
	c.DescribeAllEnumsAsStrings();
	c.DescribeStringEnumsInCamelCase();
	c.CustomSchemaIds(schemaBuilder.Select);
	});
	}
	public static void UseCustomSwagger(this IApplicationBuilder app)
	{
	app.UseSwagger();
	app.UseSwaggerUI(c =>
	{
	c.SwaggerEndpoint("swagger/v1/swagger.json", "YourCompany v1");
	});
	}
	}
	}

view raw SwaggerExtensions.cs hosted with ❤ by GitHub

As a sidenote, I think I’ll touch on is the large change that came with version 5.0.0-rc3. That version introduced/switch over to OpenApi 3.0 and the Microsoft.OpenApi libraries. It was a great change, but it also changed the way that the SchemaIdSelector worked. In the 5.0.0-rc2 version (the first version that introduced the selector), the selector would only be called once per type. In 5.0.0-rc3+, it started to be called on every type lookup. This means when you’re writing a custom selector, the selector needs to detect if a type has been selected before, and return the same value as it did previously.

Debug Builds with xprojs

I recently switched an older project that was using the VS2015 .csproj file to the newer VS2017/“xproj” style file that came about with dotnet. I love the new xproj style files because they are so much cleaner/simpler and they integrate the nuget package information. It’s also fantastic that you can open and edit those files in Visual Studio without have to unload the project. Just update the file and watch as Visual Studio dynamically reloads the interface with the new information. Fantastic!

But, something that I didn’t expect to happen was losing line number information in my stack traces when exceptions occurred. So, it was surprising when our Test environment didn’t return the line numbers of an exception after switching over to the xproj style file.

What was missing?

I didn’t use an auto-conversion tool to do the conversion, so I mistakenly dropped a few important details from the file. The critical piece that dropped was the <DebugSymbols> and <DebugType> flags. The older style csproj file looked like this:

	<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Int|AnyCPU'">
	<DebugSymbols>true</DebugSymbols>
	<DefineConstants>DEBUG;TRACE</DefineConstants>
	<DebugType>full</DebugType>
	<PlatformTarget>AnyCPU</PlatformTarget>
	<!-- omitted -->
	</PropertyGroup>

view raw old.csproj hosted with ❤ by GitHub

One of the key things in that Conditional is that the BuildConfiguration is named “Int” (rather than “Debug”).

This just needed to be replicated in the new csproj:

	<!-- pretty much the same -->
	<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Int|AnyCPU'">
	<DebugSymbols>true</DebugSymbols>
	<DefineConstants>DEBUG;TRACE</DefineConstants>
	<DebugType>full</DebugType>
	</PropertyGroup>

view raw new.csproj hosted with ❤ by GitHub

As you can see they are pretty much the same.

What if you don’t want to set those values in your .csproj?

When I was trying to figure out what happened, I ran across a really cool piece of functionality that is in the new msbuild system. If you set the BuildConfiguration to “Debug” (instead of “Int”, like I had), it will automatically set the <DebugSymbols> and <DebugType> to values that will produce line numbers in your StackTraces/Exceptions. Very friendly!

ExceptionDetailConverter Example

Thanks to Alan’s sharp eyes, he saw that I had forgotten to put in examples for the IYourBussApiExceptionDetailConverter and YourBussApiExceptionDetailConverter into the post called ExceptionHandler Needed. Which was a follow-up post to Create a Custom ProblemDetailsFactory.

So, here are some examples, which should definitely be customized to fit your environments needs:

	namespace Your.Namespace
	{
	/// <summary>
	/// Serializable information from an exception. This will contain the Exceptions Message, Type, StackTrace and InnerException (if any).
	/// </summary>
	public class YourBussApiExceptionDetail
	{
	/// <summary>
	/// The message of the original exception.
	/// </summary>
	public string Message { get; set; }
	/// <summary>
	/// The type of the exception.
	/// </summary>
	public string Type { get; set; }
	/// <summary>
	/// The stacktrace of the exception.
	/// </summary>
	public string StackTrace { get; set; }
	/// <summary>
	/// The inner exception, if any. If there is no inner exception, then this is null.
	/// </summary>
	public SaApiExceptionDetail InnerException { get; set; }
	}
	}

view raw YourBussApiExceptionDetail.cs hosted with ❤ by GitHub

	using System;
	using System.Collections.Generic;
	using System.Runtime.CompilerServices;
	using System.Text;
	using Microsoft.Extensions.Logging;
	namespace Your.Namespace
	{
	/// <summary>
	/// Convertor for <see cref="Exception" />s into <see cref="SaApiExceptionDetail" /> object.
	/// </summary>
	public interface IYourApiExceptionDetailConverter
	{
	/// <summary>
	/// Will convert an <see cref="Exception" /> into an <see cref="SaApiExceptionDetail" /> object.
	/// </summary>
	YourBussApiExceptionDetail Convert(Exception e);
	}
	/// <summary>
	/// Convertor for <see cref="Exception" />s into <see cref="SaApiExceptionDetail" /> object.
	/// </summary>
	public class YourBussApiExceptionDetailConverter : IYourBussApiExceptionDetailConverter
	{
	private ILogger<YourBussApiExceptionDetailConverter> _logger;
	public YourBussApiExceptionDetailConverter(ILogger<YourBussApiExceptionDetailConverter> logger)
	{
	_logger = logger;
	}
	[MethodImpl(MethodImplOptions.AggressiveInlining)]
	internal void LogConverting(Exception e)
	{
	_logger.LogDebug(
	$"Converting '{e.GetType().Name}' ({e.Message})",
	new { Type = e.GetType().Name, Message = e.Message }
	);
	}
	/// <inheritdoc />
	public YourBussApiExceptionDetail Convert(Exception e)
	{
	var result = new YourBussApiExceptionDetail();
	var currentException = e;
	var currentDetail = result;
	do
	{
	LogConverting(currentException);
	currentDetail.Message = currentException.Message;
	currentDetail.Type = currentException.GetType().AssemblyQualifiedName;
	currentDetail.StackTrace = currentException.StackTrace;
	currentDetail.InnerException = null;
	if (currentException.InnerException != null)
	{
	currentDetail.InnerException = new YourBussApiExceptionDetail();
	currentDetail = currentDetail.InnerException;
	}
	currentException = currentException.InnerException;
	} while (currentException != null);
	return result;
	}
	}
	}

view raw IYourBussApiExceptionDetailConverter.cs hosted with ❤ by GitHub

Again, Thanks for catching that!

Book Review: The Art of Business Value

Mark Schwartz’s The Art of Business Value (itrevolution, audio) doesn’t feel as similar to other DevOps books because it focus’ is on a different aspect, the CIO/MBA analysis of Business Value. It’s a shorter book than others, but that’s mostly because it takes a hard look at mathematical analysis of value propositions. And, he also has a way of bringing together ideas from multiple sources that is compact and directly associates there values for comparison.

Because the book is focused on the definition of Business Value, it needs to discuss the history of prior works which define and guide thought leadership on that topic. He goes into details on ROI, NVB, PV and others. Giving examples of how they can be applied, and in what situations they can be misleading. It can feel a bit like when The Machine That Changed the World’s started describing the equations that could put dollar values to every step and aspect in the supply chain. This is important to do, and difficult without having seen it done before. But, ultimately, his point is that the prior works don’t quiet fit right because of lost opportunity costs. And, that he hopes the work being done on new models, like Beyond Budgeting, can help teams/companies figure out the model that works for them. (There is no one size fits all approach.)

The part of the book that stuck with me was “Learning Bureaucracy vs Authoritative Bureaucracy”. Bureaucracy isn’t a bad thing, it can just go bad really fast. You have to have a balancing act between the stability and efficiencies that Bureaucracy provides and creativity and improvements that Generative cultures provide. When bureaucracy’s define standards and best practices they remove some of the chaos, but it soo easy to over-step just a little bit and create unnecessary slow downs (in the forms of approvals, reviews, audits, testing overhead) which don’t provide Business Value at the cost that they come for.

(picture from Scaling Agile @ Spotify with Henrik Kniberg)

In the end, Mark Schwartz does give some prescriptions of what could be used to help improve the understanding of business value. And, I interpret that advice as creating a Learning Culture and building knowledgeable team members (he has a lot more detail). There seems to be an overarching theme of moving away from a single person that defines what Business Value is and into a team of people that can define and test for Business Value; he even mentions Eric Reis’ Lean Startup as describing a starting model. These are some of the ways to help connect the implementers understanding of what creates business value with managements understanding. Which he notes is one of the areas that Lean Management and Agile are lacking in detail; how does management best perform it’s role in leadership and guidance and connect that vertical information flow into the horizontal business process flows?

(PS. Also check out Henrik Kniberg’s notes on Trust. In my opinion, Trust is the most important factor in creating efficiencies. And Trust vs Security is the classic debate that just does not seem to have a good solution.)

Reading ApiVersion with ASP.NET Core Versioning

ASP.NET Core Versioning is the red headed step child of the ASP.NET Core family. Unlike Dependency Injection, Routing, and HealthChecks, Versioning doesn’t have a page on the official ASP.NET MSDN documentation. What makes that odd is that Microsoft has internal REST Versioning Guidelines that they follow (AzureDevOps REST API). And, most API companies strongly suggest using versioning with APIs. But, again, Microsoft doesn’t have a page for it in their official MSDN documentation?

So, what does the ASP.NET Core API Versioning team have in the way of documentation? Their github repository, a good wiki guide, and a set of examples that can help you get started right away.

While the examples can help you get started, it does get a bit tougher when you want to do a little more advanced scenario. Here’s an example:

• Url: https://somesite.company.com/apis/v1/hr/jobs
  
• Optional header: x-comp-api-version: 1.2
  
• In the example, the /v1 part of the url segment indicates the major version of the API to use. And, the API will return the result of the latest minor version implementation of that major version. Optionally, you can override the minor version used with the optional header.

So, the ASP.NET Core Versioning team did setup a way to do a fallback. Where if the url segment didn’t contain the api version, then it could fall back and inspect the headers for an api version. But, they didn’t foresee using one convention to override or refine another convention. But, really, who would pre-plan for that scenario?

What they did do was setup an interface, IApiVersionReader, which you can use to implement your own custom logic.

But, here’s where the real problem comes in. The ASP.NET Core Versioning Team started their work back when ASP.NET was the standard language of the day. So, the .NET Core side feels a lot more like a bolt on for an ASP.NET Versioning system, rather than a subsystem specifically designed to work with the Dependency Injection subsystem of ASP.NET Core.

One of the places where this “after-thought” seems to stick out is when you try to implement an IApiVersionReader in ASP.NET Core. In ASP.NET Core …

• You do not define your IApiVersionReader within the Dependency Injection System
• You do not tell the ApiVersioning system of the type that you would like generated for each request
• Nor do you create a factory
• You do define your IApiVersionReader as a singleton instance within the ApiVersionOptions class
• And, you can only define your ApiVersionOptions class during ConfigureServices with AddApiVersioning, not in the Configure method with UseApiVersioning.

So, what if your IApiVersionReader needs a new instance of another class for each request that’s processed? Well … you can pass an IServiceProvider to your singleton instance at creation time. Just make sure you’ve got everything wired up in your IServiceCollection before creating your provider:

	using YourCompany;
	namespace Microsoft.Extensions.DependencyInjection {
	public static class IServiceCollectionExtensions {
	public static IServiceCollection AddYourCompanyApiVersioning(this IServiceCollection services) {
	var provider = services.BuildServiceProvider();
	services.AddApplicationInsightsTelemetry(o => {
	o.ApiVersionReader = new YourCompany.DefaultApiVersionReader(provider);
	}
	return services;
	}
	}
	}

view raw IServiceCollectionExtensions.cs hosted with ❤ by GitHub

	using Microsoft.Extensions.DependencyInjection;
	namespace YourCompany {
	public class DefaultApiVersionReader : IApiVersionReader {
	private IServiceProvider _serviceProvider;
	public DefaultApiVersionReader(IServiceProvider provider) {
	_serviceProvider = provider;
	}
	public string Read(HttpRequest request) {
	var neededInstance = provider.GetService<YourCompany.YourClass>();
	/* the rest of the logic */
	return null;
	}
	}
	}

view raw DefaultApiVersionReader.cs hosted with ❤ by GitHub

PAGELATCH_UP and tempdb Contention

The first thing to remember about PAGELATCH_UP is that while it’s not a lock, it can block.

The screenshot above is from the SQL Observability Tool: Idera SQL Diagnostics Manager for SQL Server. It’s well worth the price.

An observability tool that takes to time to highlight particular activity with bright red backgrounds is definitely giving you a hint that an improvement can be made. In the screenshot above, the boxes with the red background are queries that are blocked by another query. And the reason for the blocking in outlined in the red blocks on the right side: PAGELATCH_UP on wait resources 2:1:1, 2:5:32352, and 2:3:48528.

So, what does that mean?

Well … I first needed to know, what is PAGELATCH_UP? Here are some resources to get started (at the time of this writing these links work; I sure hope they continue to work in the future):

• Diagnosing and Resolving Latch Contention on SQL Server (PDF)
  
  A 2011 whitepaper from Microsoft’s SQL Experts on how to work through latch problems.
  
  It starts out with a description of all the different latches, and describes PAGELATCH_UP and a type of hold on a page in the memory to read the page with a potential for updating it.
  
  A similar description can be found on the MSDN Doc, Q&A on Latches in SQL Server Engine.
  
  Near the beginning of the Diagnosing PDF, there is a section on likely scenario’s that will produce a PAGELATCH_UP contention. One of those scenarios is “High degree of concurrency at the application level”, which is exactly the situation that produced the screenshot above. That screenshot was produced by having around 1000 simultaneous users trying to read the from same table at the same time, repeatedly.
  
  For a “High degree of concurrent at the application level” problem, the whitepaper suggested reading Table-Valued Function and tempdb Contention (broken link: http://go.microsoft.com/fwlink/p/?LinkID=214993).
  
• Table-Valued Function and tempdb Contention / SQLCAT’s Guide To: Relational Engine (PDF)
  
  So, “Table-Valued Function and tempdb Contention” was a blog post on the Microsoft websites at some point in time. And, it has been referenced in a number of other blog posts. However, the original post is no longer available.
  
  But, in 2013 the Microsoft SQLCAT team took the blog post in it’s entirety and copied it into a new whitepaper they were writing, SQLCAT’s Guide To: Relational Engine. This is a fantastic blessing, but using google searches, it really felt like that knowledge had been lost.
  
  Within the whitepaper it describes some key information, PAGELATCH_UPs, PFS and SGAM.
  
  Less detailed information about these components can be found in TempDB Monitoring and Troubleshooting: Allocation Bottleneck.
  
  The important information that the Table-Valued Function and tempdb Contention whitepaper describes is that a PAGELATCH_UP block on Wait Resource 2:1:1 is a block that is waiting to read the first line of the first page of the Page Free Space (PFS) page. In fact, all of those Wait Resource reference are PFS pages. You can determine if a Wait Resource is a PFS page by dividing the 3rd number by 8088. If it is evenly divisible by 8088, then it is a PFS page. (32352 / 8088 = 4, 48528 / 8088 = 6).
  
  The guide then goes on to describe what is happening in the background: The system is looking to allocate some tempdb space in order to hold temporary results as it is processing through a query. In order for the tempdb to reserve some free space it will need to request the amount of space it needs from SGAM, which will in turn ask the PFS system where free space is available. At that point in time, the PFS is read and if it finds free space it will mark that space as allocated (requiring a PAGELATCH_UP handler to read/write the PFS information).
  
  The way the original PFS handler algorithm was written, it first searches from free space at the very first PFS page (2:1:1) and then sequentially looks through PFS pages until it finds free space.
  
• Recommendations to reduce allocation contention in SQL Server tempdb database
  
  SQLCAT’s Guide To: Relational Engine goes on to give a few suggestions on how to improve tempdb PFS contention. Contention has been around for a while and one of the ways that performance can be improved is to split tempdb into multiple files. In the wait resource above the 2:5:32352 number can be read as:
  
  2 = Database Id, 2 is always tempdb
  5 = File 5, databases can be split into multiple physical files on disk
  32352 = Page 32352, pages are 8-KB in size and they are the smallest allocated size of memory/space the system creates. Page 1 and every page evenly divisible by 8088 are PFS pages.
  
  The very commonly suggested way to approach to splitting tempdb in multiple files is to make a temp physical file for each logical core on the server. If you 8 cores, then make split tempdb into 8 evenly sized files. The reason for this, is that it can allow each core to search a different physical file looking for free space. This reduces contention on the files and their associated latches.
  
• PFS page round robing algorithm improvement in SQL Server 2014, 2016 and 2017
  
  Around May of 2018 the SQL Server team also came up with an improved way to search PFS files in order to reduce contention. Instead of always starting out with 2:X:1 as the first page read in each file, instead it would remember the last page it found space in and start the next search in the first page following that previously stopping point. This has an overall affect of evenly distributing tempdb allocations over the whole of the allocated space.
  
  Many years ago, spreading data out of the disk would have been a performance loss. Especially on spindle hard drives. This is the reason that defrag’s and dbcc shrink push the data next to each other on disk.
  
  However, I don’t know if using regular spindle drives on a SQL server is a very common practice anymore. If you’re running in the cloud or in an on-premise data center, it’s likely that the backing storage is a SAN array of some kind. It might be a SAN array of spindle disks, but it’s still probably a group of disks working together to make a high performance array. At that point, does it really matter if the data is grouped together? (That would be pretty interesting to look up.)
  
  If it doesn’t need to be grouped together, then leaving the tempdb allocations evenly spread seem like a performance increase without any downsides.

Hopefully, that long wall of text helps lower the amount of research needed to find out what PAGELATCH_UP is and why it might be blocking threads in a SQL Server database.

Functional Testing Harness in ASP.NET Core

The ASP.NET Core team has made testing a first class system within their ecosystem, and it really shows. One of the aspects they made a few steps easier is functional testing of web applications. Both ASP.NET and ASP.NET Core have similar goals of creating a TestClient (GetTestClient) which can be used to perform the actual functional testing, but the ASP.NET Core teams HostBuilder pattern makes it just a touch easier to configure your TestServer.

One of the very cool things with ASP.NET Core is that if you are writing functional tests for your web application, you can use your actual Startup.cs class to configure your TestServer. This saves a lot of configuration overhead that’s involved in setting up unit tests (which still should be created). With functional tests, you usually want to see how many parts of the system work together, but you still want to stub/mock some of the external connections.

So, how can you let the system wire itself up and then mock out just the external connections? The HostBuilder makes it pretty easy to do just that. You can use your normal  Startup.cs class to configure the system and then add on an extra .ConfigureServices() function which will add in your mocks.

Here’s what that might look like:

	using System;
	using System.Collections.Generic;
	using System.Threading.Tasks;
	using Microsoft.AspNetCore.Hosting;
	using Microsoft.AspNetCore.TestHost;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.Hosting;
	using Moq;
	namespace YourCompany.Web.Tests.Functional
	{
	public class FunctionalTestLoader
	{
	public static IHost BuildTestHost<TStartup>(Action<IServiceCollection> configureServices)
	where TStartup : Startup
	{
	var args = new string[] { };
	var hostBuilder = Host.CreateDefaultBuilder(args)
	.ConfigureWebHostDefaults(webBuilder =>
	{
	webBuilder.UseTestServer();
	webBuilder.UseStartup<TStartup>()
	.ConfigureServices(configureServices);
	});
	return hostBuilder.Start();
	}
	}
	}

view raw FunctionalTestLoader.cs hosted with ❤ by GitHub

And here’s what some code that uses it could look like:

	using System.Collections.Generic;
	using System.Text.Json;
	using System.Threading.Tasks;
	using Microsoft.AspNetCore.TestHost;
	using Microsoft.Extensions.DependencyInjection;
	using Microsoft.Extensions.Diagnostics.HealthChecks;
	using Microsoft.Extensions.Hosting;
	using Moq;
	using YourCompany;
	using Xunit;
	namespace YourCompany.Web.Tests.Functional
	{
	public class StartupTests
	{
	public IHost BuildAlphaControllerHost()
	{
	var testHost = FunctionalTestLoader.BuildTestHost<YourCompany.Startup>(services =>
	{
	var repo = new Mock<YourCompany.IAlphasInfoRepository>();
	var a = new YourCompany.Alpha()
	{
	Id = 1,
	Name = "FunctionalTestAlpha",
	Status = "FunctionalTestStatus",
	Betas = new List<YourCompany.BetaInfo>() { new BetaInfo() { Id = 1, Property1 = "FunctionalTestProperty1" } }
	};
	repo.Setup(i => i.GetAsync(It.IsAny<int>())).Returns(Task.FromResult(a));
	services.RebindSingleton<YourCompany.IAlphasInfoRepository>(repo.Object);
	});
	return testHost;
	}
	[Fact]
	public async Task Controllers_Load()
	{
	var testHost = BuildAlphaControllerHost();
	var client = testHost.GetTestClient();
	var response = await client.GetAsync("/v1/alphas/1");
	Assert.True(response.IsSuccessStatusCode, $"Did not get a successful response back:`r`n{response.Content.ReadAsStringAsync().Result}");
	var contentStream = await response.Content.ReadAsStreamAsync();
	var options = new JsonSerializerOptions() { PropertyNameCaseInsensitive = true };
	var alpha = await JsonSerializer.DeserializeAsync<YourCompany.Alpha>(contentStream, options);
	Assert.Equal("FunctionalTestStatus", alpha.Status);
	Assert.Equal("FunctionalTestProperty1", alpha.Betas[0].Property1);
	}
	}
	}

view raw StartupTests.cs hosted with ❤ by GitHub

Octopus Deploy Configuration Transforms

For the last couple of years, I’ve worked with Octopus Deploy as our main Deployment system. Prior to Octopus Deploy, we used custom made Powershell scripts that were built as extensions to TFS XAML builds.

The nice part of having all of our deployment logic in Powershell scripts is that we were able to reuse those scripts with Octopus Deploy. However, there are many features within Octopus Deploy which we were happy to ditch the scripts for and use what “came out of the box”. One of those places is Octopus Deploy’s Configuration Transforms.

ASP.NET (Web.Config)

Configuration Transforms are more of a legacy feature at this point, since they are designed to perform XML transforms for ASP.NET Web.Config files. ASP.NET Core’s appSettings.json files (and it’s corresponding Configuration system) are fine without performing environment specific transformations.

To use the Configuration Transforms feature, you only need to update your Deploy a Package step to use the Configuration Transforms feature and then setup a search pattern to find the files you want transformed.

ASP.NET Core (appSettings.json)

Of course, with ASP.NET Core you don’t really need to do configuration transforms anymore. However, if you do need to provide the functionality to transform appSettings.json files you can do that with a Powershell script.

Quick sidenote: Using Octopus Deploy, you can use the JSON Configuration Variables to perform substitutions in JSON files. However, the feature is designed to have the substitution values provided by Octopus variables.

Here’s a quick Powershell script which will:

• Uses the Microsoft.VisualStudio.Jdt nuget package (within a Transforms.Console.exe application, Thanks Jason La!) to transform the appSettings.json file.
• Removes the environment specifics transform files.

	$unzipWebStep = "UCSB - Unzip Web Package"
	$unzipDir = $OctopusParameters["Octopus.Action[$unzipWebStep].Output.Package.InstallationDirectoryPath"]
	$transformConfigEnv = $OctopusParameters["TransformConfigEnv"]
	Write-Host @"
	*************************************************************
	`$unzipDir = $unzipDir
	`$transformConfigEnv = $transformConfigEnv
	*************************************************************
	"@
	$transforms = dir -Path $unzipDir -Filter "appSettings.*.json"
	if($transforms -eq $null) {
	Write-Host "No transforms found. Skipping."
	return
	} else {
	Write-Host @"
	Found $(@($transforms).Count) transforms:
	$($transforms.Name | Out-String)
	"@
	}
	## transform the appSettings.json file with the environment specific transform
	$transformerExe = "D:\AllContent\Scripts\WebApps\Resources\Ucsb.Sa.Enterprise.NetCore.Transforms.Console.exe"
	$srcJson = "$unzipDir\appSettings.json"
	$transformJson = "$unzipDir\appSettings.$transformConfigEnv.json"
	Write-Host @"
	********************************************************
	Transforming ...
	`$transformerExe = $transformerExe
	`$srcJson = $srcJson
	`$transformJson = $transformJson
	output = $srcJson
	"@
	. $transformerExe --source "$srcJson" --transform "$transformJson" --output "$srcJson"
	## Remove the appSettings.{env}.json files that don't match the current environment
	$transforms | Remove-Item -Verbose

view raw Merge-AppSettingsJson.ps1 hosted with ❤ by GitHub

Book Review: Mindset: The New Psychology of Success

This wasn’t the normal DevOps book that I enjoy reading, but it wasn’t too far from it either. The focus of the book was on creating a mindset such that your goal is to learn from the activities you do, rather than focus on winning or achieving a prize. This isn’t a new concept, but the history she relates helps paint a more detailed picture of how people can fall into a mindset where completing a task is the goal, rather than gaining the deeper understanding of how you complete a task.

Her description of how people fall into this mindset comes from a tremendous amount of research and first hand experience educating children. Her theory is that education systems that put pressure on teachers to teach towards a test often set an unfortunate precedence that passing the test is the highest priority. This changes the goals, or mindsets, of the teachers so that they can rationalize that if they get the children to memorize the answers and regurgitate them, then they have completed their task. That if the children pass the test, it corresponds with the false belief that the children have learned. Extending from that idea, this sets up classrooms where children are given tests throughout the year and they either pass or fail the test, and then they move on to the next subject. Which creates a psychological barrier that the children either know it or they don’t, and there’s no way that can possibly change. Creating a false impression that the test has decided, this is all they will ever know.

Instead, research from Dr. Dweck and others have shown that if you change the goal from testing into fostering positive learning experiences that can take root in children, then the children are self motivated to tackle hard problems using hard work, overcome discouragement through a desire to improve their own knowledge and abilities, and create a virtuous cycle by looking at knowledge gain as the real reward. This is very similar to the Lean practices of Continuous Improvement (Toyota Kata stresses continuous improvement as does Lean Startup), Dr. Westrum’s Generative Culture, and W. Edwards Deming’s thoughts on education (The New Economics: For Industry, Government, Education).

I truly enjoyed the many examples that she had working with children, and the advice she gave on how to encourage children to learn is equally applicable to adults; so there is a lot of value within those pages.

However, I would encourage others to skip the chapters which use sports examples. Dr. Dweck can be a bit single minded in her focus to connect success with a learning mindset. While a willingness to continually grow and improve are necessary to achieve great success, it’s often a breakthrough within a particular field or a combination of improvements that create a new framework which actually creates the success; it doesn’t come from just having a learning mindset.

Reference Microsoft.AspNetCore.Routing in Library

So, I’m confused by the Migrate from ASP.NET Core 2.2 to 3.0 documentation when it comes to the deprecated libraries.

In the Remove obsolete package references section, there is a long list of packages hidden under “Click to expand the list of packages no longer being produced”. The package from that list that I’m going to focus on is Microsoft.AspNetCore.Routing. That’s the package that contains IEndpointRouteBuilder.

The article explains that the removed packages are now available through the shared framework Microsoft.AspNetCore.App. So, let’s test out if that works. I’m going to:

• Create a new class library which will reference IEndpointRouteBuilder.
• Attempt to get that class library to successfully compile.

The examples in this post can be found on github at IEndpointRouteBuilderDemo.

Here’s the sample IEndpointRouteBuilderExtensions.cs class that will be used in our test:

	using System;
	using Microsoft.AspNetCore.Routing;
	namespace IEndpointRouteBuilderDemo
	{
	public static class IEndpointRouteBuilderExtensions
	{
	public static IEndpointRouteBuilder MapSaHealthChecks(this IEndpointRouteBuilder endpoints)
	{
	// do stuff
	return endpoints;
	}
	}
	}

view raw IEndpointRouteBuilderExtensions.cs hosted with ❤ by GitHub

So, let’s try it with the suggested .csproj file settings:

	<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
	<TargetFramework>netstandard2.1</TargetFramework>
	</PropertyGroup>
	<ItemGroup>
	<FrameworkReference Include="Microsoft.AspNetCore.App" />
	</ItemGroup>
	</Project>

view raw IEndpointRouteBuilderDemo-DoesntCompile-SharedFramework.csproj hosted with ❤ by GitHub

And the error we get back is:

C:\Program Files\dotnet\sdk\3.1.102\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(283,5): error NETSDK1073: The FrameworkReference 'Microsoft.AspNetCore.App' was not recognized

Okay … so, what can we do? Well, there is a closed issue on github about this problem: AspNetCore Issue #16638, Cannot find the AspNetCore Nuget packages for 3.0 (specifically routing). The response to that issue is to do what was demonstrated above. So, what else can we try.

That issue accurately describes that the Microsoft.AspNetCore.Routing dll is embedded in Microsoft.AspNetCore.App, which is located on disk at C:\Program Files\dotnet\packs\Microsoft.AspNetCore.App.Ref\3.1.2\ref\netcoreapp3.1\Microsoft.AspNetCore.Routing.dll. But, how can you get it referenced propely?

One way to get it referenced is to change the .csproj file to use the SDK of Microsoft.NET.SDK.Web and change the TargetFramework to netcoreapp3.1. Like this:

	<Project Sdk="Microsoft.NET.Sdk.Web">
	<PropertyGroup>
	<TargetFramework>netcoreapp3.1</TargetFramework>
	</PropertyGroup>
	</Project>

view raw IEndpointRouteBuilderDemo-DoesntCompile-WebSdk.csproj hosted with ❤ by GitHub

But, when you do that, you get a new error:

CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point

Which, kind of makes sense. The project has been changed over to a netcoreapp, so it kind of expects to create an executable. The great part about executables is that they also create .dlls, which is what we are looking for in the first place.

We just need to get it to compile in order to get the .dll. To do that, let’s create a DummyMain.cs class which will provide the required static ‘Main’ method:

	namespace IEndpointRouteBuilderDemo
	{
	public class DummyMain
	{
	public static void Main(string[] args)
	{
	}
	}
	}

view raw DummyMain.cs hosted with ❤ by GitHub

Which provides a successful compile:

Done building project "IEndpointRouteBuilderDemo-Compiles.csproj".

Build succeeded.
    0 Warning(s)
    0 Error(s)

Of course, this isn’t the ideal result. And, it would be hard to believe that the ASP.NET Core team expected this to occur. So, it’s most likely my misunderstanding of how to reference the Shared Framework correctly in order to prevent the need for the rest of these workarounds to occur.

Hopefully the Microsoft team will be able to shed more light on this in AspNetCore Issue #19481, Reference IEndpointRouteBuilder in a class libary (ASP.NET Core 3.1).

Which they did!

So, the documentation from Migrate from ASP.NET Core 2.2 to 3.0 doesn’t use the Microsoft.NET.SDK.Web SDK reference in the .csproj file. It uses the Microsoft.NET.SDK reference instead. Along with the Shared Framework reference, this allows for the class library to be compiled without needing the DummyMain method:

	<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
	<TargetFramework>netcoreapp3.1</TargetFramework>
	</PropertyGroup>
	<ItemGroup>
	<FrameworkReference Include="Microsoft.AspNetCore.App" />
	</ItemGroup>
	</Project>

view raw IEndpointRouteBuilderDemo-Compiles.csproj hosted with ❤ by GitHub

Convenient Rebinds for ASP.NET Core DI

ASP.NET Core’s Dependency Injection system is a solid implementation and can provide solutions for most DI needs. But, like most DI systems, it is an opinionated API (and rightly so). I moved to the ASP.NET Core implementation from Ninject and, in doing so, there were a couple of Ninject methods that I really missed. Especially the .Rebind functions.

These are the functions that will take a interface-to-implementation binding in the system, and remove the old binding and set up a new one; with new lifetime scoping and new configuration/implementation details. With ASP.NET Core’s system, they really want the developer of the application to setup exactly the bindings that they desire at the very beginning of the program. And, the first binding that’s put in place should be the last binding made for that interface-to-implementation approach.

Their approach is well reasoned and it has it’s merits. It should lower overall confusion and needed knowledge when trying to figure out what bindings are being used. If you start reading Startup.cs’s ConfigureServices and you find a binding declaration, that should be the correct binding which will be resolved at runtime.

However, because of Ninject’s .Rebind functions, I am stuck in the mind set that bindings should be flexible as new subsystems are added. If you make a library, MyLib, that has a default caching implementation that uses InMemory caching, then your library will most likely setup a binding of IMyLibCache to MyLibInMemoryCache. If I then create an add-on library that implements caching using redis, MyLib.Redis, then I want to be able to swap out the binding of IMyLibCache with a new binding to MyLibRedisCache.

With the prescribed API of ASP.NET Core’s DI system, the way you would do this in code would look something like this:

	services.AddMyLibRedis(); // binds IMyLibCache to MyLibRedisCache
	services.AddMyLib(); // would internally use services.TryAddTransient<IMyLibCache, MyLibInMemoryCache>() which would not succeed

view raw ms-di-backwards.cs hosted with ❤ by GitHub

But, that just feels backwards. When you were writing your original code, you would have to know upfront that someone in the future would have a need to use a different caching system. So, you would have to have the forethought to create the binding using .TryAddTransient() instead of .AddTransient().

It would feel much more natural if it was written like this:

	services.AddMyLib(); // would do a default binding of services.AddTransient<IMyLibCache, MyLibInMemoryCache>()
	services.AddMyLibRedis(); // should do a rebind with services.RebindTransient<IMyLibCache, MyLibRedisCache>()

view raw convenience-di.cs hosted with ❤ by GitHub

So, that’s the Ninject thinking that is stuck in my head. And, because of it, here are a few convenience overloads which can make working with IServiceCollection a little bit easier:

	using System;
	using System.Linq;
	namespace Microsoft.Extensions.DependencyInjection
	{
	/// <summary>
	/// Extensions methods that for the IServiceCollection object that can be used across many different projects types (Web, Exe, Batch, etc).
	/// </summary>
	public static class ConvenienceIServiceCollectionExtensions
	{
	/// <summary>
	/// Tests if the service collection contains a "binding" for the given type.
	/// </summary>
	/// <param name="services">The collection to read through.</param>
	/// <param name="serviceType">The type to find.</param>
	/// <returns>True if the type is found. False otherwise.</returns>
	public static bool Contains(this IServiceCollection services, Type serviceType)
	{
	var serviceDescriptor = services.FirstOrDefault(descriptor => descriptor.ServiceType == serviceType);
	return serviceDescriptor != null;
	}
	/// <summary>
	/// Tests if the service collection contains a "binding" for the given type.
	/// </summary>
	/// <param name="services">The collection to read through.</param>
	/// <param name="serviceType">The type to find.</param>
	/// <returns>True if the type is found. False otherwise.</returns>
	public static bool Contains<T>(this IServiceCollection services)
	{
	return Contains(services, typeof(T));
	}
	/// <summary>
	/// Removes a "binding" from the service collection. This functionality is also available from the standard
	/// IServiceCollection (<see cref="IServiceCollection.Remove" />), but you can to find the underlying
	/// <see cref="ServiceDescriptor" /> object which contains the binding. The is a convenience overload
	/// that does that search and removal for you.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The type binding to remove.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection Remove(this IServiceCollection services, Type serviceType)
	{
	var serviceDescriptor = services.FirstOrDefault(descriptor => descriptor.ServiceType == serviceType);
	if (serviceDescriptor != null) services.Remove(serviceDescriptor);
	return services;
	}
	/// <summary>
	/// Removes a "binding" from the service collection. This functionality is also available from the standard
	/// IServiceCollection (<see cref="IServiceCollection.Remove" />), but you can to find the underlying
	/// <see cref="ServiceDescriptor" /> object which contains the binding. The is a convenience overload
	/// that does that search and removal for you.
	/// </summary>
	/// <typeparam name="T">The type binding to remove.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection Remove<T>(this IServiceCollection services)
	where T : class
	{
	return services.Remove(typeof(T));
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it in the new scope.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindTransient(
	this IServiceCollection services,
	Type serviceType
	)
	{
	services.Remove(serviceType);
	services.AddTransient(serviceType);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it to type <typeparamref name="TImplementation" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationType">The type to bind to.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindTransient(
	this IServiceCollection services,
	Type serviceType,
	Type implementationType
	)
	{
	services.Remove(serviceType);
	services.AddTransient(serviceType, implementationType);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it in the new scope.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <typeparam name="TService">The service/interface type to bind for lookup.</typeparam>
	/// <typeparam name="TImplementation">The concrete/implementation to bind to.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindTransient<TService>(this IServiceCollection services)
	where TService : class
	{
	return services.RebindTransient(typeof(TService));
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it to type <typeparamref name="TImplementation" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <typeparam name="TService">The service/interface type to bind for lookup.</typeparam>
	/// <typeparam name="TImplementation">The concrete/implementation to bind to.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindTransient<TService, TImplementation>(this IServiceCollection services)
	where TService : class
	where TImplementation : class, TService
	{
	return services.RebindTransient(typeof(TService), typeof(TImplementation));
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationFactory">A factory to generate the requested service/interface type.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindTransient(
	this IServiceCollection services,
	Type serviceType,
	Func<IServiceProvider, object> implementationFactory
	)
	{
	services.Remove(serviceType);
	services.AddTransient(serviceType, implementationFactory);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="implementationFactory">A factory to generate the requested service/interface type.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindTransient<T>(
	this IServiceCollection services,
	Func<IServiceProvider, object> implementationFactory
	)
	{
	return RebindTransient(services, typeof(T), implementationFactory);
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it in the new scope.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddSingleton{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationType">The type to bind to.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton(
	this IServiceCollection services,
	Type serviceType
	)
	{
	services.Remove(serviceType);
	services.AddSingleton(serviceType);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to type <paramref name="implementationType" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddSingleton{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationType">The type to bind to.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton(
	this IServiceCollection services,
	Type serviceType,
	Type implementationType
	)
	{
	services.Remove(serviceType);
	services.AddSingleton(serviceType, implementationType);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it in the new scope.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddSingleton{TService, TImplementation}" />.
	/// </summary>
	/// <typeparam name="TService">The service/interface type to bind for lookup.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton<TService>(this IServiceCollection services)
	where TService : class
	{
	return services.RebindSingleton(typeof(TService));
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it to type <typeparamref name="TImplementation" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddSingleton{TService, TImplementation}" />.
	/// </summary>
	/// <typeparam name="TService">The service/interface type to bind for lookup.</typeparam>
	/// <typeparam name="TImplementation">The concrete/implementation to bind to.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton<TService, TImplementation>(this IServiceCollection services)
	where TService : class
	where TImplementation : class, TService
	{
	return services.RebindSingleton(typeof(TService), typeof(TImplementation));
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddSingleton{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationFactory">A factory to generate the requested service/interface type.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton(
	this IServiceCollection services,
	Type serviceType,
	Func<IServiceProvider, object> implementationFactory
	)
	{
	services.Remove(serviceType);
	services.AddSingleton(serviceType, implementationFactory);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="implementationFactory">A factory to generate the requested service/interface type.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton<T>(
	this IServiceCollection services,
	Func<IServiceProvider, object> implementationFactory
	)
	{
	return RebindSingleton(services, typeof(T), implementationFactory);
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="implementationInstance">A instance of type T.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindSingleton<T>(
	this IServiceCollection services,
	T implementationInstance
	) where T : class
	{
	services.Remove(typeof(T));
	services.AddSingleton<T>(implementationInstance);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it in the new scope.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddScoped{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindScoped(
	this IServiceCollection services,
	Type serviceType
	)
	{
	services.Remove(serviceType);
	services.AddScoped(serviceType);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to type <paramref name="implementationType" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddScoped{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationType">The type to bind to.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindScoped(
	this IServiceCollection services,
	Type serviceType,
	Type implementationType
	)
	{
	services.Remove(serviceType);
	services.AddScoped(serviceType, implementationType);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it in the new scope.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddScoped{TService, TImplementation}" />.
	/// </summary>
	/// <typeparam name="TService">The service/interface type to bind for lookup.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindScoped<TService>(this IServiceCollection services)
	where TService : class
	{
	return services.RebindScoped(typeof(TService));
	}
	/// <summary>
	/// Removes a "binding" for the type <typeparamref name="TService" /> and rebinds it to type <typeparamref name="TImplementation" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddScoped{TService, TImplementation}" />.
	/// </summary>
	/// <typeparam name="TService">The service/interface type to bind for lookup.</typeparam>
	/// <typeparam name="TImplementation">The concrete/implementation to bind to.</typeparam>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindScoped<TService, TImplementation>(this IServiceCollection services)
	where TService : class
	where TImplementation : class, TService
	{
	return services.RebindScoped(typeof(TService), typeof(TImplementation));
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddScoped{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="serviceType">The service/interface type to bind for lookup.</param>
	/// <param name="implementationFactory">A factory to generate the requested service/interface type.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindScoped(
	this IServiceCollection services,
	Type serviceType,
	Func<IServiceProvider, object> implementationFactory
	)
	{
	services.Remove(serviceType);
	services.AddScoped(serviceType, implementationFactory);
	return services;
	}
	/// <summary>
	/// Removes a "binding" for the type <paramref name="serviceType" /> and rebinds it to the type
	/// returned from <paramref name="implementationFactory" />.
	/// This is a convenience overload which calls <see cref="Remove{T}" /> and then
	/// <see cref="ServiceCollectionServiceExtensions.AddTransient{TService, TImplementation}" />.
	/// </summary>
	/// <param name="services">The <see cref="IServiceCollection" /> object to update.</param>
	/// <param name="implementationFactory">A factory to generate the requested service/interface type.</param>
	/// <returns>The <see cref="IServiceCollection" /> object which was referenced/passed in.</returns>
	public static IServiceCollection RebindScoped<T>(
	this IServiceCollection services,
	Func<IServiceProvider, object> implementationFactory
	)
	{
	return RebindScoped(services, typeof(T), implementationFactory);
	}
	}
	}

view raw ConvenienceIServiceCollectionExtensions.cs hosted with ❤ by GitHub

Implementation Efficiency Frustration Tiers

For me, a lot of stress and anxiety about working efficiently comes from the momentary feelings of being ineffective. If I need to accomplish a work item, how long will it take to complete that work item? How many sub-tasks do I need to complete before I can complete the work item? How many of those do I understand and can do with a minimal amount of effort, and how many do I need to do research for before I can even begin implementing them? The more time, energy, and amount of knowledge that has to be gained to complete a work item, the more stressful it becomes to complete it.

So, I wanted to take a moment and start to break down those feelings into categories. I read Scott Hanselman’s Yak Shaving post years ago, and it has become a part of the shared language among the development teams I work with. Before reading that post, I had described the act of Yak Shaving as “speed bumps”; but I would have to explain it every time I used it. Hopefully, getting this written down can help me define a language so I can communicate this feeling more easily.

At the moment, the feeling of implementation efficiency can be broken down as:

Tier 3

This is when you need to implement something, but in order to do it you are going to need to learn a new technology stack or a new paradigm in order to complete it. The task you’re trying to complete could be something as trivial as adding Exception Handling to an application, but in order to do it, you’re going to research APM solutions, determine which best fits your needs and then implement the infrastructure and plumbing that will allow you to use the new tool.

An example of this might be your first usage of Azure Application Insights in an ASP.NET Core application. Microsoft has put in a tremendous amount of work to make it very easy to use, but you’ll still need to learn how to Create an Application Insights resource, add Application Insights into an ASP.NET Core application, re-evaluate if you created your Application Insights resource correctly to handle multiple environments, and then most likely reimplement it with Dev, Test, Prod in mind, determine what common parameters which are unique to your company should always be recorded, and then work with external teams to setup firewall rules, develop risk profiles and work through all the other details necessary to get a working solution.

Tier 3 is the most frustrating because you have learn so much yourself just to get to your end value. So, for me, it’s the one that I also feel the most nervous about taking on because it can feel like I’m being incredibly inefficient at doing so much work to produce something that feels so small.

Tier 2

This is when you already have all the knowledge of how to do something and you understand what configuration needs to take place, but you are going to have to do the configuration yourself. When you know at the beginning exactly how much work it will take to complete, there is a lot less frustration because you can rationalize the amount of time spent for the end value that’s achieved. The moment this becomes frustrating is when the extra work that you’re putting in is a form of Yak Shaving. For example, when you are dealing with a production issue, and you realize that you’re going to need to implement component X in order to get the necessary information in order to solve the problem, that’s the moment you heavily sigh because you realize the amount of hand work you’re going to have to put in place just to get component X working.

This level of efficient usually happens when your working on the second or third project that you’ve used a particular technology stack with. Let’s use Application Insights as the example again. You’ve probably already developed some scripts which can automatically create the Application Insights instances, and you’re comfortable installing the nuget packages that you need, but you still need to run those scripts by hand, and setup permissions by hand, and maybe even request firewall rules to be put in place. None of these tasks will really take up too much time, but it feels like wasted time because your not producing the real end value that you had in mind in the first place.

Tier 1

This is when the solution is not only well known to yourself, but your organization has developed the tooling and infrastructure to rigorously minimize the amount of time spent on implementing the solution. This doesn’t come cheap, but the peace of mind that comes with having an instantaneous solution to a problem are the moments that make work enjoyable. The ability to stumble upon a problem and think, “Oh, I can fix that”, and within moments you’re back to working on whatever you were originally doing creates a sense that any problem can be overcome. It removes the feeling that you’re slogging through mud with no end in sight, and instead that feeling is replaced with confidence that you can handle whatever is thrown at you.

It’s rare that you can get enough tooling and knowledge built up in an organization that Tier 1 can be achieved on a regular and on going basis. It requires constant improvement of work practices and investment into people’s knowledge, skillsets, and processes to align the tooling and capabilities of their environment with their needs.

When creating working environments, everyone starts out with a goal of creating a Tier 1 scenario. But, it seems pretty difficult to get there and maintain it.

This one of the pieces I can find very frustrating about security. There is a lot of information available about what could go wrong, and different risk scenarios, but their just isn’t a lot of premade tooling which can get you to a Tier 1 level of Implementation Efficiency. People are trying though: OWASP has the Glue Docker image, and Github automated security update scanner is fantastic, and NWebSec for ASP.NET Core is a step in the right direction. But, overall, their needs to be a better way to get security into that Tier 1 of Implementation Efficiency zone.