Telemetry in PowerShellForGitHub

The PowerShellForGitHub module has a number of interesting things about it. One interesting aspect is it’s implementation for collecting telemetry data, Telemetry.ps1. Telemetry is very useful to help answer the questions of:

• What aspects of your service are being used most frequently?
• What aspects are malfunctioning?
• Are there particular flows of usage which can be simplified?

But, to be able to collect the data necessary to answer those questions you have to make the collection process incredibly easy. The goal would be to boil down the collection process to single line of code. And that’s what PowerShellForGitHub tried to do.

The level of data collection the module provides does take more than a single of line of code to use, but it’s so easily done as a part of the development process, it doesn’t feel like “extra work” or “overhead”. Here’s a snippet from GitHubRelease.ps1’s Get-GitHubRelease:

Looking through the function you can see a hashtable called $telemtryProperties created earlier on and it’s properties are slowly filled in as the function continues. Eventually, it gets to the point where a common function, Invoke-GHRestMethodMultipleResults is called and the telemetry information is passed off the underlying provider.

All of the hard work of setting up where the information will be collected and how it’s collected it abstracted away, and it boils down to be a subfeature of a single function, Invoke-GHRestMethodXYZ. Boiling everything down to that single line of code is what makes the telemetry in that module soo useful: it’s approachable. It’s not a headache that you have to go setup yourself or get permissions too, it just works.

To make it work at that level was no easy feat though! The code which makes all of that possible, including the amazing supplementary function’s like Get-PiiSafeString, is really long and involves the usage of nuget.exe to download and load Microsoft’s Application Insights and Event Tracing .NET libraries. These are hidden away in Telemetry.ps1 and NugetTools.ps1.

So, given the idea that “Telemetry is an incredibly useful and necessary piece of software development in order to answer the questions asked at the top of this article”, the new question becomes “How can you refactor the telemetry code from PowerShellForGitHub to be an easy to reuse package that any PowerShell module could take advantage of?”

Monitor and Download Latest .Net Core Installer

.NET Core has changed it’s IIS server installation model compared to .NET Full Framework. Full Framework updates were installable in offline installers, but they were also available through Windows Update/SCCM. However, with .NET Core the installer the IIS server installer, the “Hosting Bundle”, is only an offline installer that can be found by following these steps: (example for 2.2.7)

• Watching the github AspNetCore/releases page for new stable releases.
• Following the Download and Install link to the appropriate download.md.
• Finding the Downloads section and within the ASP.NET Core Runtime column, downloading the Hosting Bundle.

Even though this process is really quick, it can feel repetitive and feel like something you shouldn’t need to do. This feeling can be compounded if you missed that a new release was created weeks or months before and someone else points outs the update to you.

So, a small solution to these two minor inconveniences is to setup a watcher script which will monitor the AspNetCore teams github repository for new releases, and upon finding a new one will download the Hosting Bundle and notify you of the update. This solution could run on a nightly basis.

In this example script those previously mentioned pieces of functionality are provided by:

• PowerShellForGithub\Get-GitHubRelease
  
  This function can be used to pull back all the releases for a github repository and the results can be filtered to find the latest stable release.
  
  In order to know if the latest release is newer than the release you currently have on your servers, you could simply check the date stamp for that release was the current day. However, for this sample script, it’s going to assume you can query an external system to find out the latest installed version.
  
  
• Selenium \ SeleniumUcsb
  
  Selenium is used to navigate through the github release pages and find links to the download. It then downloads the Hosting Bundle installer by parsing through the pages to find the appropriate link. The parsing is actually a bit difficult to figure out sometimes, so an XPath Helper/tester for your browser can be really handy.
  
  
• Emailing
  
  Send-MailMessage … yeah, that’s pretty straight forward.

.NET Core Installer Wait Loop

.NET Core has a pretty fast release cycle and the team is not offering their hosting bundles through the Windows Update/SCCM Update channels. So, you may find yourself need to install the bundle on your system yourself. (I envy all the Docker folks that don’t need to think about this ever again.)

But, if you are looking to create a small installation script for .NET Core that can run remotely on your servers using Powershell, this snippet might help you get started. The script relied on the command Get-InstalledNetCoreVersion which can be found in a previous post.

The installer is really quick, always under 3 minutes, but the tricky piece is waiting until the installation completes. This particular wait loop can actually exit before the installer completes. But, the 60 second wait period between “checks” hasn’t created any race conditions yet.

Basic PowerShell Convertor for MSTest to xUnit

This is just a quick script that can help convert a .Tests.csproj which was originally written for MS Test over to using xUnit. It probably doesn’t cover every conversion aspect, but it can get you moving in the right direction.

What it will convert:

• Replace using Microsoft.VisualStudio.TestTools.UnitTesting; with using Xunit;
• Remove [TestClass]
• Replace [TestMethod] with [Fact]
• Replace Assert.AreEqual with Assert.Equal
• Replace Assert.IsTrue with Assert.True
• Replace Assert.IsFalse with Assert.False
• Replace Assert.IsNull with Assert.Null
• Replace Assert.IsNotNull with Assert.NotNull

Azure/Elastic Kubernetes Service and gMSA

I’ve written previously about how Docker Containers Are Not Domain Joined and all of the difficulties that it creates.

This post simply adds to that previous article with a little more links and information.

When I first heard of Docker, I imagined a system where you would throw a container at a service and it would figure out everything that was needed to run the container and just make it happen. Obviously that’s extremely difficult to do and as I learnt more about Docker the larger and more engrossing the problem became. My current understanding is no where near complete but here’s some more info on the problem.

In 2018, around the time I looked at AWS’ ALB prices, I looked into a price comparison of a Dockerized Web Farm vs an a IIS EC2 Web Farm. When developing out the system architecture for the Dockerized Web Farm I ran into two major issues:

• Theoretically, it looks like, Windows containers use an absolute limit (search for “CPU limit is enforced as an absolute limit”) when allocating CPU utilization to the container.
  
  NOTE: I have not gotten to the point where I can prove or disprove the above statement; and OLDER Docker documentation doesn’t seem to indicate that Windows has this problem.
  
  What this means is that if you have a 2 CPU Host system, and you were to allocate .5 CPU to a Windows Container, then the Windows container would be given that .5 CPU for it’s sole usage. No other container could use the .5 CPU and the allocating container would be hard-capped at .5 CPU.
  
  In Linux containers this is not an issue. You can allocate dozens of containers on a single host to use .5 CPU and they would (a) all share the full 100% CPU resources available, (b) never be hard-capped, and (c) only use the .5 CPU hard cap once the CPU reached 100% utilization and it needed to share the CPU between two containers that were fighting over the CPUs time.
  
• The gMSA issue that was brought up in previous Is SQL Server looking to Dockerize on Windows? post.

Even with those issues, I was curious about what AWS was doing with containers in hopes that they had the same idea that I did: We should be able to give a container image to a service and the service just figures out everything needed to run it and maked it happen. And they did: AWS Fargate.

But!! …

They were also frustrated with the permissions and gMSA security issues that the Windows OS introduced into the equation. And, as such, they don’t support Windows Containers on Fargate. They don’t directly say that they don’t support it because of the gMSA/permissions issues, but when you look at what needs to be done to support gMSA it becomes an easily rationalized conclusion. Here’s what it looks like to use a gMSA account on a Windows Container (with all the secret/password storage and management removed):

1. Create a gMSA account in Active Directory.
2. Select the Docker Host that will host the new container instance.
3. Update Active Directory to register the gMSA to be usable on that Docker Host.
4. Register the gMSA on the Docker Host (checks with Active Directory to validate the request).
5. Start the container, and you’re now able use the gMSA account within the container.
6. You’ll need to reapply the registrations (steps 2-4) for each Docker Host that the container will run on.

With a fully automated provisioning process, that’s not that difficult. It’s really doable in fact. However, here’s the list of difficult specifics that a fully managed Kubernetes infrastructure (like Fargate) would have to deal with:

1. Where is the Active Directory located?
2. Are the networking routes open for the ECS/Fargate infrastructure to it?
3. Are there other security requirements?
4. What versions of Active Directory are supported?
5. etc, etc, etc …

I don’t know at what bullet point you just *facepalm* and say “We’re not supporting this.”

But!! …

Figuring out all the details of this should be in the wheel house of Azure, right? It’s the Microsoft OS and platform, they are probably working on this problem.

So, here’s what the landscape looks like today with AKS:

• AKS does not support gMSA accounts (along with a number of other limitations).
• However it does have some interesting integrations with Azure Active Directory.
  
  The caveat to using Azure Active Directory with gMSA accounts is that not all resources used in an application will be running in Azure. For many applications, you need the gMSA to be in the companies on-premise Active Directory and replicated into the Azure AD. And, I don’t know if registering a gMSA account with a Host name within the on-premise Active Directory flows that registration into the Azure AD.
  
  And then, does that lead back to the same set of “Are the networking routes open for the AKS infrastructure to the on-premise AD” questions?
• There seems to be something going on with Open Service Broker API (OSBA) that is trying to tackle these these permissions problems, but I don’t fully understand it yet.
• I don’t see direct documentation on the CPU limiting issue, so maybe they have that solved?

So there’s the update.