Python Partials · Embrace the Suboptimal

This one's for you, Nick Robbins.

One of my favorite odd little corners of Python is the functools module, and in particular the partial() function.

According to the docs: "partial() is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature."

The two magic words in that definition are "simplified signature". Let's say you are using a vendor's REST API and start out with a bunch of CRUD functions.

def listClouds():
    resp = api.get("cloud")
    print(resp.status_code)
    for vs in resp.json()['results']:
        print('%s\t%s' % (vs['name'],vs['uuid']))

def listTenants():
    resp = api.get("tenant")
    print(resp.status_code)
    for vs in resp.json()['results']:
        print('%s\t%s' % (vs['name'],vs['uuid']))

When you're kicking the tires and exploring the API it's easy to end up with a bunch of functions which are mostly identical boilerplate and begging to be refactored. So you do that.

def listResources(resource):
    resp = api.get(resource)
    print(resp.status_code)
    for vs in resp.json()['results']:
        print('%s\t%s' % (vs['name'],vs['uuid']))

Now if you still want to have specific functions because you're going to call them frequently (or you just like explanatory names) you can recreate them using partial functions.

from functools import partial

# specialized functions to list specific resources
listClouds = partial(listResources, 'cloud')
listNetworks = partial(listResources, 'network')
listCerts = partial(listResources, 'sslkeyandcertificate')
listSSHKeys = partial(listResources, 'cloudconnectoruser')
listIpamDnsProviders = partial(listResources, 'ipamdnsproviderprofile')

What's happening there is partial() takes a function (listResources), and list of arguments (only one here) and returns a function-like object which you can then call without specifying the arguments you provided. It's sort of like filling in a template for function application.

Now you can still use dedicated functions with specific names, but without any code duplication. Of course you can also accomplish this with regular functions.

def listClouds():
    return listResources("cloud")

def listTenants():
    return listResources("tenant")

There's nothing wrong with that, but I'm partial (cough) to the first method, as it is more concise. The difference isn't that dramatic in that simple example, but the simplification really starts to shine when dealing with functions with many arguments.

Let's take another CRUD example, using POST methods. We're going to configure a load balancer with automation, and we've already refactored a generic POST function to do it.

def postIt(endpoint, template, params):
    resp = api.post(endpoint, data=template % params)
    print(resp.status_code)
    print(resp.text)

We can use this function in the normal manner to configure our load balancer.

def configLoadBalancer(config):
    postIt("network", networkTemplate, config)
    postIt("ipam", ipamTemplate, config)
    postIt("cacert", caCertTemplate, config)
    postIt("cert", certTemplate, config)
    postIt("cloud", cloudTemplate, config)

That does exactly what we want, but I find it a bit difficult to tell at a glance what's happening. Notice that when calling postIt for a given resource type, the endpoint (tail of URL) and template (parameterized JSON) will always be the same. We can codify that with partial functions.

createCloud = partial(postIt, 'cloud', cloudTemplate)
createNetwork = partial(postIt, 'network', networkTemplate)
createCaCert = partial(postIt, 'sslkeyandcertificate', caCertTemplate)
createCert = partial(postIt, 'sslkeyandcertificate', certTemplate)
createIPAM = partial(postIt, 'ipamdnsproviderprofile', ipamTemplate)

Which makes our function look like this.

def configLoadBalancer(config):
    createNetwork(config)
    createIPAM(config)
    createCaCert(config)
    createCert(config)
    createCloud(config)

I find that much easier to comprehend. I'd still love to get rid of the duplicated config argument, but minimalism has its limits.

In summary, partial functions encourage you to write more generic functions which you can specialize concisely in a self-documenting manner.

They aren't something you should reach for the first time you write a module, but after you've been living in a chunk of code for a while and gotten a feel for what can be made cleaner, they are a good tool to have in the box.

If you find this example interesting, check out the api docs and see what they can do with mixed positional and keyword arguments. It gets even better.

Here's wishing you partial success in your python endeavors.

Menu