Basic PowerShell Convertor for MSTest to xUnit

on Monday, September 9, 2019

This is just a quick script that can help convert a .Tests.csproj which was originally written for MS Test over to using xUnit. It probably doesn’t cover every conversion aspect, but it can get you moving in the right direction.

What it will convert:

  • Replace using Microsoft.VisualStudio.TestTools.UnitTesting; with using Xunit;
  • Remove [TestClass]
  • Replace [TestMethod] with [Fact]
  • Replace Assert.AreEqual with Assert.Equal
  • Replace Assert.IsTrue with Assert.True
  • Replace Assert.IsFalse with Assert.False
  • Replace Assert.IsNull with Assert.Null
  • Replace Assert.IsNotNull with Assert.NotNull

Azure/Elastic Kubernetes Service and gMSA

on Friday, September 6, 2019

I’ve written previously about how Docker Containers Are Not Domain Joined and all of the difficulties that it creates.

This post simply adds to that previous article with a little more links and information.

When I first heard of Docker, I imagined a system where you would throw a container at a service and it would figure out everything that was needed to run the container and just make it happen. Obviously that’s extremely difficult to do and as I learnt more about Docker the larger and more engrossing the problem became. My current understanding is no where near complete but here’s some more info on the problem.

In 2018, around the time I looked at AWS’ ALB prices, I looked into a price comparison of a Dockerized Web Farm vs an a IIS EC2 Web Farm. When developing out the system architecture for the Dockerized Web Farm I ran into two major issues:

  • Theoretically, it looks like, Windows containers use an absolute limit (search for “CPU limit is enforced as an absolute limit”) when allocating CPU utilization to the container.

    NOTE: I have not gotten to the point where I can prove or disprove the above statement; and OLDER Docker documentation doesn’t seem to indicate that Windows has this problem.

    What this means is that if you have a 2 CPU Host system, and you were to allocate .5 CPU to a Windows Container, then the Windows container would be given that .5 CPU for it’s sole usage. No other container could use the .5 CPU and the allocating container would be hard-capped at .5 CPU.

    In Linux containers this is not an issue. You can allocate dozens of containers on a single host to use .5 CPU and they would (a) all share the full 100% CPU resources available, (b) never be hard-capped, and (c) only use the .5 CPU hard cap once the CPU reached 100% utilization and it needed to share the CPU between two containers that were fighting over the CPUs time.
  • The gMSA issue that was brought up in previous Is SQL Server looking to Dockerize on Windows? post.

Even with those issues, I was curious about what AWS was doing with containers in hopes that they had the same idea that I did: We should be able to give a container image to a service and the service just figures out everything needed to run it and maked it happen. And they did: AWS Fargate.

But!! …

They were also frustrated with the permissions and gMSA security issues that the Windows OS introduced into the equation. And, as such, they don’t support Windows Containers on Fargate. They don’t directly say that they don’t support it because of the gMSA/permissions issues, but when you look at what needs to be done to support gMSA it becomes an easily rationalized conclusion. Here’s what it looks like to use a gMSA account on a Windows Container (with all the secret/password storage and management removed):

  1. Create a gMSA account in Active Directory.
  2. Select the Docker Host that will host the new container instance.
  3. Update Active Directory to register the gMSA to be usable on that Docker Host.
  4. Register the gMSA on the Docker Host (checks with Active Directory to validate the request).
  5. Start the container, and you’re now able use the gMSA account within the container.
  6. You’ll need to reapply the registrations (steps 2-4) for each Docker Host that the container will run on.

With a fully automated provisioning process, that’s not that difficult. It’s really doable in fact. However, here’s the list of difficult specifics that a fully managed Kubernetes infrastructure (like Fargate) would have to deal with:

  1. Where is the Active Directory located?
  2. Are the networking routes open for the ECS/Fargate infrastructure to it?
  3. Are there other security requirements?
  4. What versions of Active Directory are supported?
  5. etc, etc, etc …

I don’t know at what bullet point you just *facepalm* and say “We’re not supporting this.”

But!! …

Figuring out all the details of this should be in the wheel house of Azure, right? It’s the Microsoft OS and platform, they are probably working on this problem.

So, here’s what the landscape looks like today with AKS:

So there’s the update.

A somewhat scientific analysis of a quality issue

on Monday, August 26, 2019

I was recently working through a concern over the quality of data being entered manually which was causing a number of reprocessing requests. The person that performed the actual task of processing the incoming requests was noticing that there was a number of requests which were nearly duplicates of previous work and they needed to remove / clean-up the original work along with processing the new work. This was classic Muda and I wanted to figure out why.

The person that had reported the issue was convinced that the solution to prevent the reprocessing was to add a manual review step in the process; where the original request would be routed back to another group for review before continuing to their station for execution. The description he gave of the problem made sense, but I had had some direct involvement with some of the people putting in the requests. And when the requests were being submitted I was actually somewhat involved. So, something didn’t feel quite right and I wanted to dig into the data.

Background: The process was the addition of a database role onto a service account. The input data for the process were: Database Name, Service Account Name, Role Name, CreateRole (true/false), and Environment (dev/test/prod/all).

After getting ahold of the data this is what it looked like:

44584 - Db1, ISVC_Acct1, Role1 (all)
44582 - Db1, IUSR_Acct2, Role1 (all)
44536 - Db2, ISVC_Acct3, Role2 (all)
44504 - Db3, ISVC_Acct4, Role3Role (all) - Reprocessing (Bad Name) - Pulled name from Documentation. Docs were later corrected. Docs written by maglio-s. see 44447
44449 - Db4, ISVC_Acct4, Role3 (all)
44448 - Db3, ISVC_Acct4, Role3 (all) - Reprocessing (Wrong Database) - Developer didn't read documentation closely enough. Docs written by maglio-s. see 44447
44447 - Db1, ISVC_Acct4, Role3 (all)
44360 - Db5, ISVC_Acct1, Role4 (all)
44359 - Db6, ISVC_Acct5, Role5 (all)
44358 - Db6, ISVC_Acct1, Role6 (all)
43965 - Db1, IUSR_Acct6, Role1 (all) - Reprocessing (Bad Name) - Pulled name from Documentation. Docs were later corrected. Docs written by maglio-s. see 43960
43964 - Db7, IUSR_Acct6, Role7 (all)
43963 - Db7, IUSR_Acct6, Role8 (all)
43962 - Db7, IUSR_Acct6, Role9 (all)
43961 - Db7, IUSR_Acct6, Role1Role (all)
43960 - Db1, IUSR_Acct6, Role1Role (all)
43959 - Db8, IUSR_Acct6, Role10 (all) - Extra Message - Db8 didn't yet exist in Prod. This wasn't a problem that affected the results or required reprocessing.
43585 - Db9, IUSR_Acct7, Role11 (dev) - Extra Processing - Detected problem with script (updated bot after next Deployments / Dev Support meeting)
43295 - Db11, SVC_Acct8, Role12 (prod)
43294 - Db11, SVC_Acct8, Role12 (test)
43256 - Db7, IUSR_Acct9, Role8 (all)
43255 - Db7, IUSR_Acct9, Role9 (all)
43254 - Db7, IUSR_Acct9, Role7 (all)
43144 - Db3, ISVC_Acct10, Role3Role (all)
43088 - Db10, SVC_Acct11, Role13 (all)
43087 - Db1, SVC_Acct11, Role1 (all)
43086 - Db1, SVC_Acct11, Role14 (all)
43063 - Db11, SVC_Acct12, Role15 (prod)
42918 - Db11, SVC_Acct12, Role15 (test)
42920 - Db12, SVC_Acct12, Role16 (all) - Reviewed Before Running / Reprocessing (Bad Name), see also 42919
42921 - Db12, SVC_Acct13, Role16 (all) - Reviewed Before Running - CJ determined it wasn't necessary (requestor: maglio-s)

(*maglio-s = me; I figured I might as well out myself as the guilty party for a lot of these.)

It doesn’t look like too much reprocessing, until you look at and break down of the overall defect rates:

image

Overall there were 6 defects: 4 reprocessing needed, 1 reviewed and rejected, and 1 bug during processing. That’s 20% defects, with 13.3% reprocessing.

Upon the first review, there did seem to be a data quality issue, but more of an issue with my documentation and people trusting my documentation. If the engineer that was reporting this data quality issue was trying to get me to improve my thoroughness without pointing a finger at me; then good job!

But, when I was talking with the engineer that reported the issue, they were adamant that it wasn’t a single person but an overall quality issue. I couldn’t totally agree with them, but there was definitely was a quality problem. Now, how do we improve the quality?

As mentioned earlier, for the engineer, the solution was to add a manual review step by another group before it got to him for processing. But, that was something I was adamantly trying to avoid. I wanted to avoid it because:

  • It would take a manual process and move that manual labor to another group, rather than replace it.
  • The other group would need to be consulted because it was going to increase their workload, and they would need to add their own ideas and solutions into the conversation.
  • It wasn’t creating a smaller feedback loop for the requestor to figure out if they had submitted bad input.

I’m a fan Henrik Kniberg’s saying, “Manage for the normal, treat the exceptions as exceptional.

Each of these reprocessing issues seemed (to me) to be exceptions. And I wanted to deal with each one as an exceptional case rather than implement a new review step that would become part of the normal process.

The easy part of dealing with each one as an exception, is that you don’t have to change the overall process. And, because I had already been involved in resolving some of them earlier the implementation cost of correcting the documentation and “fixing the bug in the bot” were already taken care of.

However, neither of these approaches really seemed like they were going to be a sure fire way to ensure the quality of the data increased. They both felt like they required a “let’s wait and see if this fixes the problem” approach. And the reporting engineer had a really good point that we need to improve the quality and lower the amount of reprocessing work.

But then something new started to stand out. At the top of this article I mentioned the inputs to the system. One of the inputs to the system that didn’t make it into the analysis data was the parameter CreateRole. In the original implementation of the system, if the role in the database didn’t exist, the script which added the database role would fail. The CreateRole flag was asked for by the development team, so they could indicate to the engineering team that the role would need to be created. The engineering team looked at this problem and fixed the system by ALWAYS creating the role if it didn’t exist. And this is where the heart of the confusion occurred. The development team thought that if CreateRole was set to ‘false’, and the role didn’t exist, then the system would throw an error. The assumption was that even if they got the name wrong, it would be fine because the system wouldn’t create a new role that wasn’t asked for.

After looking at the new information, 3 out of the 4 reprocessed requests (75%) we’re all attributable to the CreateRole flag being ignored. So how do we improve the system?

Multifold:

  • Hold myself to a higher standard when writing documentation in order to prevent downstream team members from using the wrong names.
  • Ensure that Role names are unique enough to not be confused with each other. (The ones that needed to be reprocessed had Role names that were really similar to other Role names.)
  • Add a fast feedback loop, by setting up the input mechanism to verify if a role exists at the time the request is put in (if the CreateRole flag is set to false).

The most important change that came from the data analysis was introducing a new fast feedback loop. And, I don’t think we would have found it without analyzing the data. It’s a hard discipline to gather the metrics, but we need to start doing it much more frequently and with greater detail.

Promoting Vision and Behavior through Validation

on Monday, August 19, 2019

Building out a new architecture requires many processes and behaviors to be re-evaluated and modified. Those changes take time to refine, communicate and become embedded within a departments teams. And, when it comes to changes within the understanding of how processes works, it’s a very slow process which takes a large amount of repetition of intentions in order to build a communal understanding of a larger vision. And, usually, a lot of the details get lost when you’re trying to shift your mindset from one large vision over to a new large vision. The small stuff doesn’t really sink in until the large stuff has taken root.

But, there are minor opportunities to help remind team members of those smaller details when implementing validation checks within your systems. Software developers do this all the time when developing websites for their end customers. But, it isn’t done as often for internal facing applications or infrastructure pieces. For example, Domino’s requires you to put in a telephone number when you order a pizza for delivery, because if their driver becomes lost they will need to call you. The error message on the Domino’s order form does say the field is required, but it also has a little pop-up tip of why it’s required. When was the last time you saw an internal facing application explain why it wants you to do something in a validation message?

Its difficult to know when it will be useful to put in the extra time to adjust an error message in order for it to be more informative. So being on the look out for anyone that is reporting an issue and directly saying, “I didn’t know we should do it that way” is very helpful in locating places where a little more attention could go a long way.

Here’s an example. This was the original error message:

Service Accounts should start with ISVC_, IUSR_, or SVC_. Domain Account = '{0}'. AD User and Group accounts can also be used. Service Accounts should not start with '{domain name}\'

And after the update, here’s the new error message:

Service Accounts should start with ISVC_, IUSR_, or SVC_. Domain Account = '{0}'. AD User and Group accounts can also be used. Service Accounts should not start with '{domain name}\'. Please see `serviceaccount guidelines` for more information.

And this is what `serviceaccount guidelines` provides:

Service Accounts

Service accounts cannot be over 20 characters long including the endings of 'Dev' and 'Test'. Because of that, the maximum allowed length is 16 characters, which should include the prefix.

Prefixes

    IUSR_   Stands for IIS User Account. Used with websites that are seen/used by end users.
    ISVC_   Stands for IIS Service Account. Used with webservices and webjobs.
    SVC_    Stands for Win Services Account. Used with Windows Services and Scheduled Tasks.

Example

    ISVC_OrgAppName{env}


When choosing names for accounts, try to avoid redundancies within the name. For example, IUSR_DoorWebApp{env} would have 2 redundancies. 'Web' is redundant because the prefix of 'IUSR_' indicates it's a top level web application. And, 'App' is redundant because the prefix of 'IUSR_' indicates it's a web application. Another example of redundancy would be to add 'Svc' in an 'ISVC_' account name, eg. 'ISVC_DoorSvc{env}'.

It’s a small addition, but it has two effects. First, it’s communicating out a group of standardized application types and how to signal to other’s what the role of your application is. It’s also empowering the team members to have the information necessary to make solid decisions without needing to reach out to other teams (who may be busy with other work).

It’s extra overhead to put together the extra documentation, but it can definitely be worth it.

Powershell: Using a file hash to test for a change

on Monday, August 12, 2019

The PowershellForGitHub module is great! But … sometimes it can be a bit verbose when it’s trying to help out new users/developers of the module. This isn’t a bad thing in any way, just a personal preference thing. And, the module owner, Howard Wolosky, is really open to suggestions. Which is great!

So, I opened a ticket (PowerShellForGitHub Issue #124) to explain my confusion over the warning messages. And, to be fair, I explained my confusion in a very confusing way. But, he was nice enough to work through it with me and we found that something we needed was a way to tell if someone had updated a settings file after downloading the module on their machine.

Enter, Get-FileHash.

This command looks like it’s been around for quite a while, but it does the classic job of creating a hash of a file. And, that hash can be stored in code, so that it can be used to check if a change in the file has occurred.

So, how to use the check.

Here’s the original code:

And, here’s the updated code using Get-FileHash:

PowerShellForGitHub–Adding Get-GitHubRelease

on Monday, August 5, 2019

PowerShellForGitHub is an awesome powershell module for interacting with the GitHub API. It has a wide set of features that are already implemented and its supported by Microsoft(!!). You can also tell the amount of care the maintainer, Howard Wolosky, has put into it when you dig into the code and read through the inline documentation, contributing documentation and telemetry support(!!). BTW, if you ever need to create a PII transmittable string, check it out: Get-PiiSafeString.

One of the features I was looking for the other day was the ability to retrieve a list of releases for a repository. (I was building a script to monitor the dotnet/core releases; in order to build an ASP.NET Core Hosting Bundle auto-installer.)

I submitted a pull request of the update last week and was really impressed with all the automation in the pull request processing. The first thing that surprised me was the integrated msftclas bot (Microsoft Contribution License Agreements), which posted a legal agreement form that I (or the company I represent) consent to give Microsoft ownership of the code we contribute. It was soo smooth and easy to do.

Next was the meticulous level of comments and review notes on the pull request. If he made all those comments by hand, holy moly! That would be amazing and I would want to praise him for his patience and level of detail. Hopefully, some of the comments were stubbed out by a script/bot; which would be a really cool script to know about.

So, I’m gonna go through the comments and see if I can update this pull request.

  • *facepalm* Wow, I really missed changing the name GitHubLabels.ps1 to GitHubReleases.ps1 in the .tests.ps1 file.
  • White space in .tests.ps1: Ahhh … I can better see the white space formatting style now.
  • Examples missing documentation: Hahaha! My mistake. It looks like I started writing them and got distracted.
  • Telemetery: I loved the note:

    For these, I think it's less interesting to store the encrypted value of the input, but more so that the input was provided (simply in terms of tracking how a command is being used).

    Thank you for pointing that out! It makes complete sense.

In summary, a big Thank You to Howard Wolosky and the Microsoft team for making this module! It was a huge time saver and really informative on how to write Powershell code in a better way.

Pester Testing Styles

on Monday, July 29, 2019

Pester is a great testing framework for Powershell. And it can be used in a variety of different testing styles: TDD, BDD, etc. I’m going to look at two different styles, both of which are perfectly good to use.

TDD’ish with BeforeAll / AfterAll

Lines 4 through 7 are used to ensure that module don’t get repeatable imported, when this tests are run as part of a Test Suite. However, they will allow modules to be reloaded if you are running the individual test file within VSCode. For the most part, they can be ignored.

In this more Test Driven Development style test

  • The Describe blocks name is the function under test
  • And each It test is labelled to describe a specific scenario it is going to test
  • All the logic for setting up the test and executing the test are contained within the It block
  • This relies on the Should tests to have clear enough error messages that when reading through the unit tests output you can intuit what was the failing condition

This is a very straight forward approach and it’s really easy to see how all the pieces are setup. It’s also very easy for someone new to the project to add a test to it, because everything is so isolated. One thing that can really help future maintainers of a project is to write much lengthier and more descriptive It block names than the ones in the example, in order to help clarify what is under test.

Some things to note:

In this setup, the BeforeAll script is used to configure the environment to be ready for tests that are about to be run. Over time, this function has been replaced with BeforeEach, but for this example I’m using BeforeAll. The BeforeAll is setting up some values that I want available when the test is run, or a variable I want available when the test is run. I put a prefix of $script: on the variable created within the BeforeAll function because I have seen behavior where the variable was no longer defined outside of the scope of BeforeAll.

The AfterAll is a corresponding block to the BeforeAll, and is pretty self explanatory. The interesting part of the these two blocks is that they have to be declared within the Describe block and not within the InModuleScope block. They will not be run if they are declared in the InModuleScope block.

BDD’ish with try / finally

Lines 10 and 11 are used to ensure that that module has been configured correctly (for normal usage … not specific to the tests) and ensuring that the module isn’t being reloaded when being run in a Test Suite.

In this more Behavior Driven Development style test

  • Uses the Describe block to outline the preconditions for the tests
  • Immediately following the declaration of the Describe block, it has the code which will setup the preconditions
  • Uses the Context block to outline the specific scenario the user would be trying
  • And, immediately following the declaration, it has the code which will execute that scenario
  • Uses the It blocks to outline the specific condition that is being tested.
  • This requires more code, but makes it clearer what condition actually failed when reviewing unit test output

This is not as straight forward of an approach, as different areas of the code create the conditions which are being tested. You might have to search around a bit to fully understand the test setup. It also adds a little more overhead when testing multiple conditions as you will be writing more It block statements. The upside of that extra work is that the unit test output is easier to understand.

Some things to note:

In this setup, variable scope is less of an issue because variables are defined at the highest scope needed to be available in all tests.

The BeforeAll/AfterAll blocks have also been replaced with try/finally blocks. This alternative approach is better supported by Pester, and it can also help new developers make a key insight into the way Pester tests are run: They are not run in parallel, but instead are run in order from top to bottom. Because of this, you can use some programming tricks to mock and redefine variables in particular sections of the code without having to worry about affecting the results of other tests.


Creative Commons License
This site uses Alex Gorbatchev's SyntaxHighlighter, and hosted by herdingcode.com's Jon Galloway.