Replacing Invoke-WebRequest with HttpClient

I’ve written before about how frustrating the Error handling for Invoke-WebRequest and Invoke-RestMethod can be. But, there is another way to make web requests which will never update the global $Error object: write your own wrapper around HttpClient.

This method is much much more complicated than using Invoke-WebRequest, Invoke-RestMethod, or even Invoke-WebServiceProxy. But, it will give you complete control over the request and the response. And as a nice side effect, it’s cross platform compatible (runs on linux and windows).

Below is an example use of HttpClient to call Apigee’s OAuth Login endpoint.

(PS. The idea for using an HttpClient came from David Carroll’s PoShDynDnsApi powershell module. Which works with two implementations that use HttpClient (.NET Core and .NET Full Framework). The reason for two implementations is that DynDns requires one of their calls to perform a non-standard GET request with a body. Microsoft’s HttpClient implementation is pretty strict about following the rules and does not allow a body to be sent in GET requests. So, he had to use reflection to inject a body into his request. Each version of .NET had a different internal class structure that had to be set differently. It’s a pretty amazing work around.)

	<#
	.SYNOPSIS
	Makes a call to the Apigee OAuth login endpoint and gets access tokens to use.
	This should be used internally by the ApigeePs module. But, it shouldn't be needed by
	the developer.
	.EXAMPLE
	$result = Get-ApigeeAccessTokens
	#>
	Function Get-ApigeeAccessTokens {
	[CmdletBinding()]
	[OutputType([PSCustomObject])]
	Param ()
	$attempt = 1
	$retry = $false
	$content = $null
	do {
	$retry = $false
	# You'll need to set teh Shared Secret and Username safely with your account information
	$otp = Get-ApigeeOTP `
	-SharedSecret $global:ApigeePs.OAuthLogin.OTPSharedSecret `
	-Email $global:ApigeePs.OAuthLogin.Username
	# You'll need to set the Username and Password safely with your account information
	$body = "username={0}&password={1}&mfa_token={2}&grant_type={3}" -f `
	$global:ApigeePs.OAuthLogin.Username, `
	$global:ApigeePs.OAuthLogin.Password, `
	$otp.OTP, `
	"password"
	$httpClient = [System.Net.Http.Httpclient]::new()
	$httpClient.Timeout = [System.TimeSpan]::new(0, 0, 90)
	$httpClient.DefaultRequestHeaders.TransferEncodingChunked = $false
	$accept = [System.Net.Http.Headers.MediaTypeWithQualityHeaderValue]::new("application/json")
	$accept.CharSet = "utf-8"
	$httpClient.DefaultRequestHeaders.Accept.Add($accept)
	# that's a hard coded value. it's literally in their documentation:
	# https://docs.apigee.com/api-platform/system-administration/management-api-tokens
	# (it makes complete sense when you think about it, but it's really concerning when you first see it)
	$authorization = [System.Net.Http.Headers.AuthenticationHeaderValue]::new("Basic", "ZWRnZWNsaTplZGdlY2xpc2VjcmV0")
	$httpClient.DefaultRequestHeaders.Authorization = $authorization
	$httpClient.BaseAddress = [Uri] ($global:ApigeePs.OAuthLogin.Url)
	$httpMethod = [System.Net.Http.HttpMethod] "POST"
	$httpRequest = [System.Net.Http.HttpRequestMessage]::new($httpMethod, "token")
	if($body -ne $null) {
	$httpRequest.Content = [System.Net.Http.StringContent]::new($body, [System.Text.Encoding]::UTF8, "application/x-www-form-urlencoded")
	}
	$httpResponseMessage = $httpClient.SendAsync($httpRequest)
	if ($httpResponseMessage.IsFaulted) {
	$PsCmdlet.ThrowTerminatingError($httpResponseMessage.Exception)
	}
	$result = $httpResponseMessage.Result
	$content = Get-ApigeeRequestContent -Response $result
	# check for
	if($result.StatusCode -eq 401) {
	if($content -eq "{`"error`":`"unauthorized`",`"error_description`":`"Error: Invalid MFA code.`"}") {
	# handle authentication retry (see note above about why this isn't implemented)
	$retry = $true
	}
	}
	if($result.StatusCode -ge 400 -and $retry -eq $false) {
	$uri = "{0}/{1}" -f $global:ApigeePs.ApiUrl, $ApiPath
	Write-Verbose "Error calling $uri ($Method)"
	if($Body -ne $null) {
	Write-Verbose "`tBody: $Body"
	}
	Write-Verbose "Response: Status = $([int]$result.StatusCode) $($result.ReasonPhrase)"
	Write-Verbose "Resposne: Headers"
	foreach($key in $result.Headers) {
	Write-Verbose ("`t{0}`t`t{1}" -f $key, $result.Headers[$key])
	}
	Write-Verbose "Response: Content"
	$contentOutput = [string]::Empty
	if([string]::IsNullOrWhiteSpace($content) -eq $false) { $contentOutput = $content }
	Write-Verbose $contentOutput
	return $result
	}
	$attempt++
	} while( $retry )
	# parse the result and set headers
	$json = ConvertFrom-Json -InputObject $content
	$authorization = [System.Net.Http.Headers.AuthenticationHeaderValue]::new("bearer", $json.access_token)
	$global:ApigeePs.AuthHeader = $authorization
	return $json
	}

view raw Get-ApigeeAccessTokens.ps1 hosted with ❤ by GitHub

	<#
	.SYNOPSIS
	Implementation of the Time-based One-time Password Algorithm used by Google Authenticator.
	.DESCRIPTION
	As described in http://tools.ietf.org/id/draft-mraihi-totp-timebased-06.html, the script generates a one-time password based on a shared secret key and time value.
	This script generates output identical to that of the Google Authenticator application, but is NOT INTENDED FOR PRODUCTION USE as no effort has been made to code securely or protect the key. For demonstration-use only.
	Script code is essentially a transation of a javascript implementation found at http://jsfiddle.net/russau/uRCTk/
	Output is a PSObject that includes the generated OTP, the values of intermediate calculations, and a URL leading to a QR code that can be used to generate a corresponding OTP in Google Authenticator applications.
	The generated QR code contains a URL that takes the format "otpauth://totp/<email_address_here>?secret=<secret_here>", for example: otpauth://totp/tester@test.com?secret=JBSWY3DPEHPK3PXP
	The generated OTP is (obviously) time-based, so this script outptu will only match Google Authenticator output if the clocks on both systems are (nearly) in sync.
	The acceptable alphabet of a base32 string is ABCDEFGHIJKLMNOPQRSTUVWXYZ234567.
	Virtually no parm checking is done in this script. Caveat Emptor.
	.PARAMETER sharedSecretKey
	A random, base32 string shared by both the challenge and reponse side of the autheticating pair. This script mandates a string length of 16.
	.EXAMPLE
	.\Get-OTP.ps1 -sharedSecret "JBSWY3DPEHPK3PXP" | Select SharedSecret, Key, Time, HMAC, URL, OTP
	.NOTES
	FileName: Get-OTP.ps1
	Author: Jim Nelson nelsondev1
	#>
	Function Get-ApigeeOTP {
	[CmdletBinding()]
	param
	(
	[Parameter(Mandatory=$true,ValueFromPipeline=$true)]
	[ValidateLength(16,16)]
	[string] $SharedSecret,
	[Parameter(Mandatory=$true,ValueFromPipeline=$true)]
	[string] $Email
	)
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Converts the supplied Int64 value to hexadecimal.
	#------------------------------------------------------------------------------
	function Convert-DecimalToHex($in)
	{
	return ([String]("{0:x}" -f [Int64]$in)).ToUpper()
	}
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Converts the supplied hexadecimal value Int64.
	#------------------------------------------------------------------------------
	function Convert-HexToDecimal($in)
	{
	return [Convert]::ToInt64($in,16)
	}
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Converts the supplied hexadecimal string to a byte array.
	#------------------------------------------------------------------------------
	function Convert-HexStringToByteArray($String)
	{
	return $String -split '([A-F0-9]{2})' | foreach-object { if ($_) {[System.Convert]::ToByte($_,16)}}
	}
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Converts the supplied base32 string to a hexadecimal string
	#------------------------------------------------------------------------------
	function Convert-Base32ToHex([String]$base32)
	{
	$base32 = $base32.ToUpper()
	$base32chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567"
	$bits = ""
	$hex = ""
	# convert char-by-char of input into 5-bit chunks of binary
	foreach ($char in $base32.ToCharArray())
	{
	$tmp = $base32chars.IndexOf($char)
	$bits = $bits + (([Convert]::ToString($tmp,2))).PadLeft(5,"0")
	}
	# leftpad bits with 0 until length is a multiple of 4
	while ($bits.Length % 4 -ne 0)
	{
	$bits = "0" + $bits
	}
	# convert binary chunks of 4 into hex
	for (($tmp = $bits.Length -4); $tmp -ge 0; $tmp = $tmp - 4)
	{
	$chunk = $bits.Substring($tmp, 4);
	$dec = [Convert]::ToInt32($chunk,2)
	$h = Convert-DecimalToHex $dec
	$hex = $h + $hex
	}
	return $hex
	}
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Get the currentUnix epoch (div 30) in hex, left-padded with 0 to 16 chars
	#------------------------------------------------------------------------------
	function Get-EpochHex()
	{
	# this line from http://shafiqissani.wordpress.com/2010/09/30/how-to-get-the-current-epoch-time-unix-timestamp/
	$unixEpoch = ([DateTime]::Now.ToUniversalTime().Ticks - 621355968000000000) / 10000000
	$h = Convert-DecimalToHex ([Math]::Floor($unixEpoch / 30))
	return $h.PadLeft(16,"0")
	}
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Get the HMAC signature for the supplied key and time values.
	#------------------------------------------------------------------------------
	function Get-HMAC($key, $time)
	{
	$hashAlgorithm = New-Object System.Security.Cryptography.HMACSHA1
	$hashAlgorithm.key = Convert-HexStringToByteArray $key
	$signature = $hashAlgorithm.ComputeHash((Convert-HexStringToByteArray $time))
	$result = [string]::join("", ($signature | % {([int]$_).toString('x2')}))
	$result = $result.ToUpper()
	return $result
	}
	#------------------------------------------------------------------------------
	#------------------------------------------------------------------------------
	# Get the OTP based on the supplied HMAC
	#------------------------------------------------------------------------------
	function Get-OTPFromHMAC($hmac)
	{
	$offset = Convert-HexToDecimal($hmac.Substring($hmac.Length -1))
	$p1 = Convert-HexToDecimal($hmac.Substring($offset*2,8))
	$p2 = Convert-HexToDecimal("7fffffff")
	[string]$otp = $p1 -band $p2
	$otp = $otp.Substring($otp.Length - 6, 6)
	return $otp
	}
	# -------------------------------------------------------------------------------------------------------
	# -------------------------------------------------------------------------------------------------------
	# -------------------------------------------------------------------------------------------------------
	# MAIN PROGRAM
	# -------------------------------------------------------------------------------------------------------
	# -------------------------------------------------------------------------------------------------------
	$params = @{
	"SharedSecret" = "";
	"Key" = "";
	"Time" = "";
	"HMAC" = "";
	"OTP" = "";
	"URL" = "";
	}
	$reportObject = New-Object PSObject -Property $params
	# google can generate a QR code of the secret for their authenticator app at this url...
	$url = ('https://chart.googleapis.com/chart?chs=200x200&cht=qr&chl=200x200&chld=M|0&cht=qr&chl=otpauth://totp/' + $Email + '%3Fsecret%3D' + $SharedSecret)
	$key = Convert-Base32ToHex $sharedSecret
	$time = Get-EpochHex
	$hmac = Get-HMAC $key $time
	$otp = Get-OTPFromHMAC $hmac
	$reportObject.SharedSecret = $sharedSecret
	$reportObject.Key = $key
	$reportObject.Time = $time
	$reportObject.HMAC = $hmac
	$reportObject.OTP = $otp
	$reportObject.URL = $url
	return $reportObject
	}

view raw Get-ApigeeOTP.ps1 hosted with ❤ by GitHub

	function Get-ApigeeRequestContent {
	param(
	[System.Net.Http.HttpResponseMessage] $Response
	)
	$content = $Response.Content.ReadAsStringAsync().Result
	return $content
	}

view raw Get-ApigeeRequestContent.ps1 hosted with ❤ by GitHub