/ python

Fun With Types

tl;dr

Python has slowly been adding type annotations to the language. This allows for static type checking and the such. But I believe that we will see (are seeing) the emergence of new patterns that take advantage of these syntax additions.

Setting the scene

With PEP 3107 python added function annotations. With PEP 484 typing was standardized. With tools such as mypy, these annotation become useful beyond a pure documentation aspect.

However, I believe annotations will bring some interesting new patterns to the language. Especially with variable annotations added in PEP 526. These kind of changes have started making their way into the language. Notably PEP 557 Data Classes, that I find particularly interesting.

Here are few things I have been toying around with.

API Validation

With some small additions to Flask, it's possible to get runtime type validation for your API using annotations.

Here's what it looks like:

@app.post("/<user_id>", max_version=1)
def endpoint(request, user_id: int, body: EndpointInput) -> Response:
    success = api_producer(body)

    return Response(
        message=success
    )

Here EndpointInput is a schematics object, and validation is handled by the library.

The nice thing is this will also get static type checking with mypy.

Data Wrapper

When creating Python objects there is a certain amount of boilerplate. Especially if you're writing a small class to carry data around. You'll have to write at least an __init__. It will probably look something like this:

class MyData:
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c

This is pretty redundant. And because of this, more often than not, people will use a dict. This is unfortunate, as we now have no idea what the intended structure of the object is. Making it harder to reason about the structure of our programs.

Wouldn't it be nice if declaring a new type was this simple:

class Message(BaseType):
    target_id: int
    sender_id: int
    body: str
    metadata: Dict[str, Any]
    message_id: int
    rabbit: Dict[str, Any]
    api_response: Optional[Dict[str, Any]]

In python 3.6, we get pretty close with the namedtuple provided by the typing module. However, we don't yet have support for optionals.

This Meta Class is my attempt to get something pretty close.

class AnnotationMeta(type):

    def __new__(cls, name, bases, attrs):

        def init(self, **k):

            for k, v in k.items():
                if k in attrs or k in self.__annotations__:
                    setattr(self, k, v)

            required = {
                k
                for k, v in self.__annotations__.items()
                if not (
                    ('Union' in str(type(v)) and type(None) in v.__args__)
                    or 'Opt' in str(type(v))
                )
            }
            if len(required) > len(self.keys):
                miss = required - set(self.keys)
                raise TypeError(f'Missing keys {miss}')

        if '__init__' not in attrs:
            attrs['__init__'] = init

        return super().__new__(cls, name, bases, attrs)

The good news is the Data Class PEP was accepted and it should give us something more useful than namedtuple. I still need to read the PEP in details, but these are exciting times.

Serializer

The last use I've found is describing serializers.

class MessageSerializer(BaseSerializer):
    __model__ = Message

    id: str
    message_channel: List[MessageChannelSerializer]
    recipient_u_id: str
    sender_u_id: str

Pretty simple, but it's has a nice declarative feel to it.

Conclusion

I've found the new annotations to be particularly useful. Being able to introspect the __annotations__ variable let's you write some nice little modules. I believe we'll see more python libraries do this in the future.
Now that we have a standardized way of marking collection types (Dict, List, etc), it would be really nice to have a runtime module that let us manipulate and check those types. Currently, asserting that your variable is the correct time has proven somewhat ... hacky.

Mathieu Sabourin

Mathieu Sabourin

I codez. I do the pythonz at WayUp, previously of NerdWallet and Sauce Labs. Contrary to what banner would lead to think, not into bitcoin or crypto(currencies).

Read More