Apigee Response CORS Headers using Javascript

Apigee provides a quick “Add CORS Headers” to responses when creating a new API Proxy. It’s straight forward and will get you started to add CORS headers to the replies from your first API endpoints. The problem with that is that CORS headers are used in “preflight” and aren’t that useful after the call has successfully completed. Apigee OPTIONS Response for Preflight/CORS can help you set up preflight responses.

But, it’s still useful to add in CORS headers to your responses in order to ensure that your endpoints are communicating their security requirements. To do this you can use javascript to inspect the responses and add in missing CORS headers. This sample javascript will:

• Ensure Access-Control-Allow-Origin is defined. Sets the default value to ‘*’.
• Ensure Access-Control-Allow-Headers is defined. Sets the default value to ‘origin, x-requested-with, accept, my-api-key, my-api-version, authorization, content-type’.
  • my-api-key and my-api-version are custom headers specific to the Apigee endpoints this script is used with. If the Resource Service doesn’t return these headers, then they will be added in.
• Ensure Access-Control-Max-Age is defined. Sets the default value to ‘3628800’ seconds (42 days … I have no idea why that was chosen.)
• Ensure Access-Control-Allow-Methods is defined. Sets the default value to ‘GET, PUT, POST, DELETE’. This should really be set by the Resource Service, so use it only if you feel comfortable.

This should be created as a Shared Flow and applied to Proxy Endpoint's Postflow.

//  Access-Control-Allow-Origin
var accessControlAllowOrigin = context.getVariable("response.header.Access-Control-Allow-Origin.values").toString();
if(accessControlAllowOrigin.startsWith('[')) { accessControlAllowOrigin = accessControlAllowOrigin.substring(1, accessControlAllowOrigin.length() - 1); }
if(accessControlAllowOrigin.endsWith('[')) { accessControlAllowOrigin = accessControlAllowOrigin.substring(0, accessControlAllowOrigin.length() - 1); }
if(accessControlAllowOrigin.length() === 0) {
    accessControlAllowOrigin = "*";
}
context.setVariable("response.header.Access-Control-Allow-Origin", accessControlAllowOrigin);

//  Access-Control-Allow-Headers
var accessControlAllowHeaders = context.getVariable("response.header.Access-Control-Allow-Headers.values").toString();
if(accessControlAllowHeaders.startsWith('[')) { accessControlAllowHeaders = accessControlAllowHeaders.substring(1, accessControlAllowHeaders.length() - 1); }
if(accessControlAllowHeaders.endsWith('[')) { accessControlAllowHeaders = accessControlAllowHeaders.substring(0, accessControlAllowHeaders.length() - 1); }
if(accessControlAllowHeaders.length() === 0) {
    accessControlAllowHeaders = "origin, x-requested-with, accept, my-api-key, my-api-version, authorization, content-type";
}
if(accessControlAllowHeaders.indexOf("my-api-key") === -1) {
    accessControlAllowHeaders += ", my-api-key";
}
if(accessControlAllowHeaders.indexOf("my-api-version") === -1) {
    accessControlAllowHeaders += ", my-api-version";
}
context.setVariable("response.header.Access-Control-Allow-Headers", accessControlAllowHeaders);

//  Access-Control-Max-Age
var accessControlMaxAge = context.getVariable("response.header.Access-Control-Max-Age.values").toString();
if(accessControlMaxAge.startsWith('[')) { accessControlMaxAge = accessControlMaxAge.substring(1, accessControlMaxAge.length() - 1); }
if(accessControlMaxAge.endsWith('[')) { accessControlMaxAge = accessControlMaxAge.substring(0, accessControlMaxAge.length() - 1); }
if(accessControlMaxAge.length() === 0) {
    accessControlMaxAge = "3628800";
}
context.setVariable("response.header.Access-Control-Max-Age", accessControlMaxAge);

//  Access-Control-Allow-Methods
var accessControlAllowMethods = context.getVariable("response.header.Access-Control-Allow-Methods.values").toString();
if(accessControlAllowMethods.startsWith('[')) { accessControlAllowMethods = accessControlAllowMethods.substring(1, accessControlAllowMethods.length() - 1); }
if(accessControlAllowMethods.endsWith('[')) { accessControlAllowMethods = accessControlAllowMethods.substring(0, accessControlAllowMethods.length() - 1); }
if(accessControlAllowMethods.length() === 0) {
    accessControlAllowMethods = "GET, PUT, POST, DELETE";
}
context.setVariable("response.header.Access-Control-Allow-Methods", accessControlAllowMethods);

Apigee Key Value Maps (KVM) To Store Passwords

Cloud based computing has broken some of the molds of traditional security models. Things like IP whitelisting on a firewall sometimes aren’t even an option. And, because of that, some older techniques are back and can really work wonders for simple authentication security.

A quick note: Apigee’s Business option (and above) actually comes with static IP addresses, so this technique can be used in conjunction with IP whitelisting.

Apigee is an API Management system, so it has the capability to handle many authentication protocols for clients to connect to it’s cloud based endpoints. But, we’re gonna look at the other half to the communication path. We’re gonna look at when the API Gateway has to call down to the resource service. And, this is a technique to inform the resource server that it is the API Gateway which is making the call to it.

The technique is Basic Authentication. It’s been around for a long time and it’s basically a magic string you put in the header of your requests. Your resource service will inspect the header and make sure it’s talking to a client that knows the shared secret. Since this is a shared secret we need a way to store the secret in Apigee that’s secure. And, it’s pretty darn secure.

Put the Shared Secret in the KVM

Basic Authentication is a username and password joined together by a colon and then base 64 encoded. The header looks like this:

Authorization:    Basic   {base64encoded(“username:password”)}

So, we’re going to store both the username and password into Apigee’s KVM. The first thing we need to do is select the KVM level we want to store it at.

• Organization Level
• If you’re going to reuse the same username/password on multiple APIs in multiple environments, then this works well.
• Environment Level
• If you’re going to reuse the same username/password on multiple APIs, but you want to use a different secret between Prod and everything else.
• API Proxy Level
• If you’re looking for a secret defined to a single API Proxy, but used in all environments.

In this example, were going to do an API Proxy Level KVM.

$adminUser = "tom@place.com"   # apigee.com/edge username
$adminPass = "tommyspass"      # apigee.com/edge password
$org = "org1"                  # apigee.com/edge organization
$apiName = "my-api"            # an api proxy name

# bump up TLS to 1.2 (.NET defaults to SSL3, which isn't supported on Apigee management endpoints)
[System.Net.ServicePointManager]::SecurityProtocol = 
				[System.Net.SecurityProtocolType]::Tls12 + [System.Net.SecurityProtocolType]::Tls11 + [System.Net.SecurityProtocolType]::Tls;

# this isn't the KVM, this is Apigee security
$bytes = [System.Text.Encoding]::ASCII.GetBytes($adminUser + ":" + $adminPass)
$encodedText = [Convert]::ToBase64String($bytes)
$adminHeader = @{ Authorization = "Basic $encodedText"; "Content-Type" = "application/json" }

$rootUrl = "https://api.enterprise.apigee.com/v1/organizations/$org"
$apikvmUrl = "$rootUrl/apis/$apiName/keyvaluemaps"

$kvms = Invoke-RestMethod -Method GET -Uri $apikvmUrl -Headers $adminHeader
# currently $kvms is most likely empty

# so, let's add one
$kvmName = "my-customKVM"
$kvmEntry = @{
    name = $kvmName
    encrypted = "true"    # this is important and will come back later
    entry = @(
        @{ name = "username"; value = "sooo" },
        @{ name = "password"; value = "secret" }
    )
}
$json = ConvertTo-Json $kvmEntry

$newKvm = Invoke-RestMethod -Method POST -Uri $apikvmUrl -Headers $adminHeader -Body $json
$newKvm | fl
# this is the last time you can see the unencrypted secret values
# make sure to store the values in a password safe before clearing these values

$kvms = Invoke-RestMethod -Method GET -Uri $apikvmUrl -Headers $adminHeader
$kvms # $kvms should now list "new-customKVM"

$apiKvmEntryUrl = "$apikvmUrl/$kvmName"
$kvm = Invoke-RestMethod -Method GET -Uri $apiKvmEntryUrl -Headers $adminHeader
$kvm | fl
# this time the values are hidden (******)

## And, of course the delete
#Invoke-RestMethod -Method Delete -Uri $apiKvmEntryUrl -Headers $adminHeader

Retrieve the Shared Secret from the KVM

Now we have my-customKVM setup at the API Proxy level. So, let’s use the value in the flow to create a Basic Authorization header and populate the value. To do this, we are going to use the KeyValueMapOperations Policy to retrieve the credentials. In this policy, it’s very important to store the credentials to a variable that starts with private.. The KVM entry that we made was encrypted. And, you can only read an encrypted KVM value into a variable that is scoped to private.. The reason for this is security. private. variables will never appear in the Trace tool, nor will they be logged. However, you can look at them by using javascript callouts (this is important for debugging).

You can only read an encrypted KVM value into a variable that is scoped to private.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="Retrieve-Credentials" mapIdentifier="my-customKVM">
    <DisplayName>Retrieve Credentials</DisplayName>
    <Properties/>
    <ExclusiveCache>false</ExclusiveCache>
    <ExpiryTimeInSecs>300</ExpiryTimeInSecs>
    <Scope>apiproxy</Scope>
    <Get assignTo="private.ba.username" index="1">
        <Key>
            <Parameter>username</Parameter>
        </Key>
    </Get>
    <Get assignTo="private.ba.password" index="1">
        <Key>
            <Parameter>password</Parameter>
        </Key>
    </Get>
</KeyValueMapOperations>

And, Assign the Header to the Request

So, we’ve now loaded the username and password from the KVM into private.ba.username and private.ba.password. We are now going to use the BasicAuthentication Policy to set the Authorization header.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<BasicAuthentication async="false" continueOnError="false" enabled="true" name="Add-BasicAuth-Header">
    <DisplayName>Add BasicAuth Header</DisplayName>
    <Operation>Encode</Operation>
    <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
    <User ref="private.ba.username"/>
    <Password ref="private.ba.password"/>
    <AssignTo createNew="false">request.header.Authorization</AssignTo>
</BasicAuthentication>

Troubleshooting private. variables

When working with private. variables, it’s very useful to use Javascript policies to inspect the value of the variables (because they don’t appear in the Trace tool).

print(context.getVariable("private.ba.username"))
print(context.getVariable("private.ba.password"))

Alter PathSuffix in Apigee with/out Load Balancer

Apigee’s API Gateway is by many measures a proxy server with some really nice bells and whistles attached. But, it’s still a proxy server at it’s core. Which means it should be able to transform an incoming request before it’s sent to the backend/target resource servers. It can do that, but it’s not as easy as you might hope.

With an API Gateway, a common trasnformation would be to remove a version number from a url before sending the request to the backend server. This scenario crops up when the developer of the resource API didn’t design their system with version numbers in mind. The scenario looks like this:

So, in this scenario, the Body API Proxy has a BasePath of /body (proxy.basepath). And the PathSuffix would be /v1/wheels?drive=4WD (proxy.pathsuffix). The developer of the resource service didn’t have version built into the url path, and is expecting a url without it.

Without a Load Balancer Configuration

To make this transformation, we are going to need to artificial create the target endpoints url during the flow process. Seeing that the API Gateway is a proxy server, you would think that you would just need overwrite the request or proxy variables, but most of those are actually read only. Here’s what you’ll need to do:

1. You’ll use the request.uri and proxy.basepath to figure out the full path suffix.
2. If the path contains a version number (/v1/) then you will …
3. Set target.copy.pathsuffix to false. (At the moment, you have to use a Javascript Callout. There is a bug with using an AssignMessage Policy).
1. This must occur in the Target Endpoint flows (most likely the PreFlow). You can’t do this in the Proxy Endpoint, because the target variables haven’t been created yet. So, they aren’t “in scope”.
4. You’ll then remove the version number (/v1/) to get the “new” path suffix.
5. And, finally, you will set the target.url to constructed path. (target.url is one of the few read/write variables.)

Target Endpoint with No Load Balancer

With a Load Balancer Configuration

Apigee uses a Load Balancer variable in the Target Endpoint configuration to allow for Resource Server DNS hostnames to be dynamic between the environments. Unfortunately, when this is used, the target.url variable is no longer used. And, you need to set target.copy.queryparams to false as well.

To this, you’ll follow the same steps above, but this time you’ll …

5. And, finally, you will set the target.url to constructed path. (target.url is one of the few read/write variables.)
6. Set target.copy.queryparams to false.
7. Set the {newpathsuffix} variable, which will be configured on the Target Endpoint’s Path.

Target Endpoint with Load Balancer

Javascript Callout for Target Endpoint PreFlow: (note that variable {newpathsuffix} isn’t needed when no Load Balancer is involved. It’s being used to make both implementations look similar.)

//  parses the original request to remove the version piece ("/v1", etc)
var basepath = context.getVariable("proxy.basepath")
print("basepath: " + basepath);
var uri = context.getVariable("request.uri");
print("uri: " + uri);
var pathsuffix = uri.substring(basepath.length)
var regex = /(.*)\/v[0-9]+\/(.*)/
var found = regex.exec(pathsuffix)
print("found: " + found)
if(found !== null) {
    //  prevents the request to the backend server from using the original "request.pathSuffix"
    //  this is very important!
    //  the original "request.path" will overwrite whatever we do here if this isn't set
    context.setVariable("target.copy.pathsuffix", false)
    
    // remove the "/v1" part
    var newPathSuffix = found[1]
    if(newPathSuffix.length > 0) { newPathSuffix += "/" }
    newPathSuffix += found[2]
    
    print("newPathSuffix: " + newPathSuffix)
    context.setVariable("newpathsuffix", newPathSuffix)
    
    var targetUrl = context.getVariable("target.url")
    print("target url: " + targetUrl)
    if(targetUrl !== null) {
        
        var pathSuffixRegex = /(.*){newpathsuffix}(.*)/
        var pathSuffixFound = pathSuffixRegex.exec(targetUrl)
        print("pathSuffixFound: " + pathSuffixFound)
        
        if(pathSuffixFound !== null) {
            
            var newUrl = pathSuffixFound[1] + newPathSuffix + pathSuffixFound[2]
            print("new url (replace): " + newUrl)
            context.setVariable("target.url", newUrl);
            
        } else {
            var newUrl = targetUrl + newPathSuffix
            print("new url (append): " + newUrl)
            context.setVariable("target.url", newUrl);
            
        }
    } else {
        // using load balancer
        context.setVariable("target.copy.queryparams", "false") // needed on load balancer
        // the load balancer can use the variable substitution on the  innerText
    }
} else {
    print("newpathsuffix: [empty string]")
    context.setVariable("newpathsuffix", "")
}

target.copy.pathsuffix and target.copy.queryparams

So, these are the key variables that make overwriting the target path possible. The creation of these variables probably has good reasoning behind it, but from an outside perspective they seem really odd. Apigee’s internal system allows you to do a variety of alterations and checks through the Proxy and Target Endpoint flows. These flows can alter most things within the system at the time they execute within the pipeline. BUT, the proxy.pathsuffix and proxy.queryparams are (a) readonly and (b) will overwrite any changes you make to the target.url value. They just ignore everything that happened in the pipeline and override it. This behavior seems to conflict with the way the “flow” system was designed.

Apigee OPTIONS Response for Preflight/CORS

Apigee comes with the ability to add CORS headers to responses right out of the box. This really isn't that useful though. And, it instills a false sense that it’s actually providing valuable CORS information so the developer doesn't have to think about it.

CORS is really implemented into browsers to prevent requests from going to unauthorized endpoints. To do this, many browsers (like Chrome) use a “Preflight” request to pull back a couple of headers which let the browser know that a web service does allow requests from other “origins” (or, DNS names). Essentially CORS headers state:

These websites can use this web service. And, this web service allows these methods (GET, POST, etc) to be called from that website for the resource in question. (With web services, a lot of the time, “These websites” is actually “All websites.”)

Back to Apigee’s initial setup: The problem with adding CORS headers on all responses is that the Preflight request isn’t going to match one of the normal endpoints on an API. So, the response will most likely be a 404 Not Found. And, browsers will consider that an error, and they won’t allow the real request to go through.

This is a big deal for https://editor.swagger.io/. The basic Swagger UI Tester uses fetch, which does the Preflight request/check. The swagger editor and tester are used all over the place and most browsers will try to do a Preflight check which will result in this error message (the image is from Chromes developer tools):

In Chrome’s Network tab it will look like this:

Take note that the Preflight request is asking the server not only if it’s okay if http://editor.swagger.io is calling, but it wants to know if the ‘ucsb-api-version’ header is acceptable. This means the preflight request doesn’t actually send across any security information. It’s asking if it’s okay to send across security information.

So, if you want to have an Apigee web service that is compatible with the standard Swagger UI editor and tester you need to watch for the OPTIONS preflight request and return an acceptable response. Luckily, this can be done by taking the original CORS headers response and turning it into a PreFlow response. Start out by creating a Shared Flow that looks for OPTIONS requests:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SharedFlow name="default">
    <Step>
        <Name>OPTIONS-CORS-Headers-Response</Name>
        <Condition>request.verb = "OPTIONS"</Condition>
    </Step>
</SharedFlow>

Then add a RaiseFault Policy that will return all the CORS headers and successful status code:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RaiseFault async="false" continueOnError="false" enabled="true" name="OPTIONS-CORS-Headers-Response">
    <DisplayName>OPTIONS CORS Headers Response</DisplayName>
    <Properties/>
    <FaultResponse>
        <Set>
            <Headers>
                <Header name="Access-Control-Allow-Origin">*</Header>
                <Header name="Access-Control-Allow-Headers">origin, x-requested-with, accept, ucsb-api-key, ucsb-api-version, authorization</Header>
                <Header name="Access-Control-Max-Age">3628800</Header>
                <Header name="Access-Control-Allow-Methods">GET, PUT, POST, DELETE</Header>
            </Headers>
            <Payload contentType="text/plain"/>
            <StatusCode>200</StatusCode>
            <ReasonPhrase>OK</ReasonPhrase>
        </Set>
    </FaultResponse>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</RaiseFault>

Now, all you need to do is add the Shared Flow as the very first Step in your API Proxies’ Preflow Proxy Endpoint steps. The Shared Flow step must come before the API Key verification step because the OPTIONS request will not contain security authorization information. It should look something like this:

Once this is all setup, preflight requests from https://editor.swagger.io/ will pass without error. And now you should get a successful response:

But, this isn’t a perfect solution. There are still faults with it because your API Proxy is now blindly stating that it will take requests from almost anywhere and for a variety of different METHOD types.

The best possible solution would be to allow for OPTIONS Preflight requests to be detected by looking for the OPTIONS method and checking if the 3 required headers exist. If all of those conditions are met, then flow the request down to the resource server and let it determine what the exact CORS response it can serve. But, that’s all configuration for another day.

Apigee’s security is NOT Default Deny

Apigee makes a great API Gateway product. But, one thing that’s been surprising is that the security system is not Deny Access by Default. It’s a very forward thinking design, but it surprised many of us who assumed the common security practice of Default Deny was the starting point.

For an application to have access to an API, the application must first be approved to use an API Product. The piece that’s surprising is that if an API Product has no API Proxies or Resource Path restrictions applied to it, then it gives full access to all API Proxies.

Don’t do this. Always attach at least one API Proxy to your API Products.

Once you have an API Product setup with an API Proxy, you have restricted access to just that API Proxy’s endpoint. Which is a good step forward. But, it gives you access to the entire endpoint, with no filtering on the “known paths”.

Beware of not applying resource path restrictions. Without resource restrictions, everything going through your API Proxy’s endpoint is passed through.

For example, if an API Proxy has a base path of:

https://{org}-{env}.apigee.net/firstapiproxy

And, that API Proxy has “known paths” (eg. flows) of

• GET /cars
• GET /trucks
• GET /vans

Because there are no resource path restrictions, these urls will also work:

• https://{org}-{env}.apigee.net/firstapiproxy/garbledjunkpath
• https://{org}-{env}.apigee.net/firstapiproxy/internalhealthcheckaddress

To apply the Resource Path restrictions use the API Product interface:

Or, for even stricter security, create a DefaultNotFound Flow within your API Proxy. Like the Send404NotFoundResponse used in the oauth2/proxy example:

Apigee OAuth Tester in Powershell

New Apigee instances/organizations come with a built in OAuth 2.0 server. Their default security mechanism is an API Key, but they fully support OAuth 2.0 right out of the box.

A new instance will come with an active OAuth 2.0 endpoint deployed to your Dev, Test, and Prod instances.

The default OAuth 2.0 endpoint is very similar to this proxy example. But, the tutorial on Apigee’s website is to send the grant_type as a form parameter. So, a quick swap can change the grant_type lookup:

Once that’s changed over, you’ll need to request an access token from the endpoint. To do this go into one of your applications and get the client_id and client_secret:

And now we can throw this info into a powershell script to get back our bearer token:

$apigeeHost = "{organization}-{environment}.apigee.net"
$clientId = "{your client id}"
$clientSecret = "{your client secret}"

$authUrl = "https://$apigeeHost/oauth/client_credential/accesstoken"
$authHeaders = @{
    "Content-Type" = "application/x-www-form-urlencoded"
}
$authBody = "grant_type=client_credentials" + `
            "&client_id=$clientId" + `
            "&client_secret=$clientSecret"

$authResponse = Invoke-WebRequest -Method POST -Headers $headers -Body $body -Uri $loginUrl

if($response.StatusCode -ne 200) {
    throw ("Authorization Failure`r`n" + $response)
}

$authInfo = ConvertFrom-Json $response.Content

$authInfo

Before making a call to a resource, make sure to setup the resource API Proxy with an OAuth Verification:

You actually only need the <Operation>VerifyAccessToken</Operation>, but it doesn’t hurt to leave the rest.

Now that we have a bearer token, we can use it as an authorization header to make a call to our resource:

# use your resource url here
$resourceUrl = "https://$apigeeHost/sa/quartercalendar/oauth/v1/quarters?quarter=20154"
$resourceHeaders = @{
    Authorization = "Bearer $($authInfo.access_token)"
}
$resourceResponse = Invoke-WebRequest -Method GET -Uri $resourceUrl -Headers $resourceHeaders
ConvertFrom-Json $resourceResponse.Content