My favorite parts of Effective Python by Brett Stalkin

Feb 23, 2020
book
7 min read

I programmed in Python on and off for the last 15 years but haven’t looked systematically at the language in the least 10 years. It was time to sharpen my saw so I picked up Effective Python by Brett Stalkin. The book is organized into 90 items show casing Python best practices with their rationale and code examples. This post covers my favorite items, the ones I intend to integrate in my use of Python.

Item 4: Interpolated F-Strings

This is the item that made me pick the book in the first place. I was confused by the many ways someone can format a string in Python and have been alternating between these:

s = "foo"
i = 42
print("A string %s and an int %d" % (s, i))
print("A string {} and an int {}".format(s, i))

A string foo and an int 42
A string foo and an int 42

The book explains that Python has been searching itself in terms of string formatting but that it converged to Interpolated Strings:

print(f"A string {s} and an int {i}")

A string foo and an int 42

The syntax inside the curly braces is as follow:

{ <expression> <optional !s, !r, or !a> <optional : format specifier> }

Where !s, !r and !a are conversion specifier to print using str, repr or ascii:

value = "allô"
print(f"s: {value!s}, r: {value!r}, a: {value!a}")

s: allô, r: 'allô', a: 'all\xf4'

For more details on what is being supported, see the Format Specification Mini-Language. Here is an example aligning text to the left, center or right or a cell:

# e.g. < ^ > to align left, center or right followed by a field width
print(f"|{i:<5}|{i:^5}|{i:>5}|")

|42   | 42  |   42|

Item 10: Assignment expressions (a.k.a. the walrus operator)

Assignment Expressions (:=) were introduced in Python 3.8. They allow the assignment of a variable within an expression:

import re

# long version
m = re.search(r"[0-9]+", "abc123def")
if m is not None:
  print(f"Found {m.group()} without assignment expressions")

# using the walrus operator
if (m := re.search(r"[0-9]+", "abc123def")) is not None:
  print(f"Found {m.group()} with assignment expressions")

Found 123 without assignment expressions
Found 123 and saved one line with assignment expressions

This one is pure syntactic sugar, but everything that saves me keystrokes without sacrificing on readability is a win in my book.

Item 25: Keyword-only arguments

There are times when we need to create functions taking a gazillion parameters. Good practices is to pass keyword arguments to these functions:

# What does this call do?
draw_rectangle(0, 0, 10, 10, "blue", 45)

# Is this one easier to undersand?
draw_rectangle(x=0, y=0, width=0, height=0, color="blue", rotation=45)

With keyword-only arguments, we can force the callers use keyword parameters. The syntax is as follow:

# all arguments after the * must be named
def draw_rectangle(x, y, width, height, *, color, rotation)
  pass

# this call will fail
draw_rectangle(0, 0, 10, 20, "blue", 15)

# this call will succeed
draw_rectangle(0, 0, 10, 20, color="blue", rotation=15)

This makes the code easier to read and easier to evolve. For example, deprecating the color parameter in favor of an rgb parameter could be done this way:

import warnings

def draw_rectangle(x, y, width, height, *, rotation, rgb=None, color=None):
  assert not color or not rgb, "You can't specify both the color and rgb parameters"
  if color:
    warnings.warn("The color parameter is now deprecated and will be removed in version X")
    rgb = to_rgb(color)
  # ...

Item 60: Co-routines

The book has a whole section no parallelism, multi-threading and process management in Python. It also present an introduction to co-rountine and how they are supported in Python. This was my favorite discovery of the whole book.

Co-routines allow a program to suspend and resume its operations. We can use them to inform Python to stay busy with something else when doing blocking operations. They are supported using two keywords: async to annotate bloc of codes that can be suspended and await to suspend the execution.

To get a feel of how co-rountine works, let’s start from this example checking if websites are up or not by looking at http status codes:

import requests
from datetime import datetime
from pprint import pprint

def is_up(url):
  """Fecth the status code for a given url."""
  return requests.head(url).status_code

urls = ["https://www.google.com", "https://twitter.com", "https://www.bing.com"]

# We check the status of 3 websites and print the time it takes
start_time = datetime.now()
status_codes = {url: is_up(url) for url in urls}
end_time = datetime.now()

pprint(status_codes)
print(f"Duration: {end_time - start_time}")

{'https://twitter.com': 200,
 'https://www.bing.com': 200,
 'https://www.google.com': 200}
Duration: 0:00:00.773243

We can rewrite this example using co-routines using aiohttp:

import aiohttp
import asyncio
from datetime import datetime
from pprint import pprint

async def async_is_up(url):
  """Fecth the status code for a given url asynchronously.

  Note that this function is different from `is_up` because we use
  an async API to check the status code.
  """
  async with aiohttp.ClientSession() as session:
    async with session.head(url) as response:
      return response.status

urls = ["https://www.google.com", "https://twitter.com", "https://www.bing.com"]

start_time = datetime.now()

# When using co-routines, we construct a graph of tasks and then
# execute them.
#
# 1. Create the tasks
tasks = [async_is_up(url) for url in urls]

# 2. Assemble them in a graph and make them run in parallel
results = asyncio.gather(*tasks)

# 3. Execute the graph
loop = asyncio.get_event_loop()
status_codes = loop.run_until_complete(results)
loop.close()

end_time = datetime.now()

# Our results are now ready, we can print them. Note that calling
# asyn_is_up won't return the status but a task to download the
# status code.
print(f"Task that was run for google: {tasks[0]}")
pprint({url : status for (url, status) in zip(urls, status_codes)})
print(f"Duration: {end_time - start_time}")

Task that was run for google: <coroutine object async_is_up at 0x7fbde7a93c40>
{'https://twitter.com': 200,
 'https://www.bing.com': 200,
 'https://www.google.com': 200}
Duration: 0:00:00.191329

The version using co-routines is faster because it keeps itself busy during blocking operations like fetching http status codes instead of waiting for them to finish.

Item 90: Type Annotations

Starting with version 3.5, Python started to support type hinting. It doesn’t make Python a statically typed language because it is optional and it is not used by Python itself for validation. It however helps documenting expectations with other developers and enable static analysis to complement unit test.

Let’s take a look at an example:

def find_word(text, position):
  """Find the position of a word in a text."""
  pass

def capitalize(word):
  """Capitalize a word."""
  pass

# example usage
text = "roses are red"
capitalized_third_word = capitalize(find_word(text, 2))

Can anything go wrong in this code? What if we added type annotation for it:

from typing import Optional

def find_word(text: str, position: int) -> Optional[str]:
  """Find the position of a word in a text."""
  pass

def capitalize(word: str) -> str:
  """Capitalize a word."""
  pass

# example usage
text = "roses are red"
capitalized_third_word = capitalize(find_word(text, 2))

If you didn’t find what could go wrong with the way we are using the API, let’s ask mypy, one of the many Python type checkers:

$ python3 -m mypy type-checked-python.py
type-checked-python.py:13: error: Argument 1 to "capitalize" has incompatible type "Optional[str]"; expected "str"
Found 1 error in 1 file (checked 1 source file)

The type checker warns us that find_word can return None but capitalize can’t handle it. We can now make our code more robust to make mypy happy:

text = "roses are red"
third_word = find_word(text, 2)
capitalized_third_word = capitalize(third_word) if third_word else None

If we run the type checker again, we get:

python3 -m mypy type-checked-python.py
Success: no issues found in 1 source file

I am ambivalent about type annotations since I haven’t had the chance to use them in a project yet. On one hand, they are a nice complement to unit tests and make documentation easier. On the other, they are optional and I am unsure how they work across code base boundary.

Conclusion

This post is obviously biased toward my needs and opinions. If you already know Python and want a book that goes straight to the point (i.e. no toy projects to motivate the learning, only information), I highly recommend Effective Python. I read it cover to cover and really enjoyed it. The books also makes it easy to cherry pick items if not all of the items are of interest to you.