Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://adamj.eu/tech/2020/08/10/a-guide-to-python-lambda-functions/ below:

A Guide to Python Lambda Functions

This is a cross-post from the Scout APM blog, where I occasionally write. I also maintain the Scout Python integration.

In Python, Lambda functions are rare compared to “normal” functions, and occasionally misunderstood or overused.

In this article we’ll cover what Lambda functions are, how to use them, and how they compare with normal functions and other alternatives. We’ll also touch on the history of lambda functions, such as where the name “lambda” came from and why Python’s creator Guido van Rossum wanted to remove them.

We’ll cover:

• What Lambda Functions Are (in Python)
  • Why They’re Called Lambda Functions
• Examples of Using Lambda Functions in Python
  • Use With list.sort() and sorted()
  • Use With filter()
  • Use With map()
• Alternatives to Lambda Functions
  • Normal Functions
  • List Comprehensions (And Other Types)
  • The operator Module

The lambda keyword allows you to define a lambda function. This is a function that returns a single expression in one line. It’s like a shortcut for using def to create a normal function, which requires at least two lines.

For example, we could define a simple addition function using def like so:

def add_func(a, b):
    return a + b

This takes two lines.

Using lambda we can instead write our function as:

add_lambda = lambda a, b: a + b

This takes only one line.

To convert the syntax, we:

• Replaced def with lambda.
• Removed the parentheses around the arguments.
• Put the function body on the same line as its definition, after the single colon (:).
• Removed the return keyword. Lambda functions always return the result of their one expression.
• Added an explicit assignment of the function to the name add_lambda.

Despite all these syntactical differences, the two versions work identically:

>>> add_func("Hello ", "Scout")
'Hello Scout'
>>> add_lambda("Hello ", "Scout")
'Hello Scout'

Lambda, or λ, is the 11th letter of the Greek alphabet. Due to the use of the Greek alphabet in mathematics, Alonzo Church ended up using it in the 1930’s when describing a concept he called Lambda calculus. This is a formal system describing any possible computation - something like a purely mathematical programming language.

Lambda calculus is so-called because it uses Lambda (λ) to represent functions, which also never have names. The Lisp programming language copied this concept, and Python copied it from Lisp.

The main motivation for using lambda is to create a function and use it as an argument in the same line. These are often done with several built-in functions that take functions as arguments. Let’s look at three examples now.

For our examples, let’s use a list of puppies with their cuteness ratings:

class Puppy:
    def __init__(self, name, cuteness):
        self.name = name
        self.cuteness = cuteness

    def __repr__(self):
        return f"Puppy({self.name!r}, {self.cuteness!r})"


puppies = [Puppy("Doggo", 100), Puppy("Kevin", 200), Puppy("Bock", 50)]

We’ll manipulate the puppies list with some Python built-ins, to which we will pass lambda functions.

The list.sort() method takes with an optional key argument. This is a function to map the list items to values to sort them by. We can use it to sort our puppies by their increasing cuteness ratings, by passing key as a function that extracts a given puppy’s cuteness value.

Using def, we need to define the function separately before we call list.sort():

def get_cuteness(puppy):
    return puppy.cuteness


puppies.sort(key=get_cuteness)

>>> puppies
[Puppy('Bock', 50), Puppy('Doggo', 100), Puppy('Kevin', 200)]

Using lambda, we can define the function inside the call to sort() directly:

puppies.sort(key=lambda puppy: puppy.cuteness)

>>> puppies
[Puppy('Bock', 50), Puppy('Doggo', 100), Puppy('Kevin', 200)]

The lambda version is only one line, whilst the def version is three lines (four if you count the blank line between the function and call to list.sort()).

We can make it even shorter by using a one letter variable name inside the lambda function:

puppies.sort(key=lambda p: p.cuteness)

The sorted() built-in similarly takes a key argument, but it takes with any iterable instead of just lists, so you’ll often see lambda used in conjunction with it. For example:

>>> sorted(puppies, key=lambda p: p.cuteness)
[Puppy('Bock', 50), Puppy('Doggo', 100), Puppy('Kevin', 200)]

The filter() built-in takes a function and an iterable, and returns the items from the iterable for which the function returned true. We can use it to filter our puppies to only the cutest ones, by passing key as a function that returns if a given puppy has enough cuteness.

Using def, we again need to define the function separately, before we call filter():

def is_cute_enough_for_contest(puppy):
    return puppy.cuteness >= 100

>>> list(filter(is_cute_enough_for_contest, puppies))
[Puppy('Doggo', 100), Puppy('Kevin', 200)]

(Note we need to call list() on filter() to see its results, because it is a generator.)

Using lambda, we can again define the function in the same line as its use:

>>> list(filter(lambda p: p.cuteness >= 100, puppies))
[Puppy('Doggo', 100), Puppy('Kevin', 200)]

Again, we’ve saved a few lines.

The map() built-in takes a function and an iterable, and returns a new iterable of the results of applying the function on the items in the passed iterable. We can use it to extract our puppies’ names into a list of strings.

Using def, we again need to define the function separately, before we call filter():

def get_puppy_name(puppy):
    return puppy.name

>>> list(map(get_puppy_name, puppies))
['Doggo', 'Kevin', 'Bock']

(Note we again need to call list() on map() to see its results, because it is also a generator.)

Using lambda, we can once again define the function in the same line as its use, saving some lines:

>>> list(map(lambda p: p.name, puppies))
['Doggo', 'Kevin', 'Bock']

The existence of lambda functions in Python is somewhat controversial. The creator of Python, Guido van Rossum, even advertised his intention to remove it in Python 3.0, along with filter() and reduce(). In his 2005 post The fate of reduce() in Python 3000, he wrote:

About 12 years ago, Python aquired lambda, reduce(), filter() and map()… But, despite of the PR value, I think these features should be cut from Python 3000.

(Python 3000 was the working name for Python 3.0.)

Ultimately Python kept lambda for backwards compatibility, and Guido updated the post with:

lambda, filter and map will stay (the latter two with small changes, returning iterators instead of lists). Only reduce will be removed from the 3.0 standard library. You can import it from functools.

But there are still alternatives to using a lambda function, and they are preferable for many use cases. Let’s look at those now.

The first alternative is to use a normal function. We already compared these with their corresponding lambda functions in our three examples above. Normal functions have a number of advantages that the lambda syntax does not allow.

Normal functions have a name, which allows us to clarify our intention. With a complex lambda function you might find yourself writing a comment to describe what it does. Using a normal function you can embed this informatino in the function’s name itself.

Take our filter() example again. Imagine the filtering we did was because there’s a minimum of cuteness of 100 to enter a contest. We might try embed this in the lambda function version with a comment, which requires us to split filter() across multiple lines:

>>> list(
...     filter(
...         # Minimum cuteness to enter contest
...         lambda p: p.cuteness >= 100,
...         puppies,
...     )
... )
[Puppy('Kevin', 200), Puppy('Doggo', 100)]

But with a normal function, we can put that information in the function name:

def is_cute_enough_for_contest(puppy):
    return puppy.cuteness >= 100

>>> list(filter(is_cute_enough_for_contest, puppies))
[Puppy('Doggo', 100), Puppy('Kevin', 200)]

Note we can give lambda functions names too by assigning them:

is_cute_enough_for_contest = lambda p: p.cuteness >= 100

But if you check this function’s __name__ attribute, you’ll see it’s actually called <lambda>:

>>> is_cute_enough_for_contest.__name__
'<lambda>'

All lambda functions have the name '<lambda>', even after we assign them to variables. This is because Python doesn’t have any name information when creating the function. This will appears in various code inspection tools, including stack traces, and can make debugging a little harder.

Our previous examples all used short functions, so the lambda syntax was readable on a single line. But if our function contained a longer expression, using a lambda function could mean cramming lots of code on one line.

Imagine we wanted to sort our puppies in a more complex way: in reverse order, by the upper-cased first letter of the last part of their names. Using lambda, our call to list.sort() would look like this:

puppies.sort(key=lambda p: p.name.rpartition(" ")[2][0].upper(), reverse=True)

This line contains a lot of different pieces. I count 14 different object names, argument names, keywords, and values, plus a lot of punctuation. That’s a lot to read and understand at once!

We could improve the readability a bit by splitting the code over multiple lines:

puppies.sort(
    key=lambda p: p.name.rpartition(" ")[2][0].upper(),
    reverse=True,
)

But then we have given up some of the benefit of using lambda, as we have the same number of lines of code as if we hadn’t used it. The lambda function is also still quite a lot of steps to understand.

By using a normal function, we can split the expression in two pieces, and assign a name to the intermediate last_name varible:

def upper_last_name_initial(puppy):
    last_name = puppy.name.rpartition(" ")[2]
    return last_name[0].upper()


puppies.sort(key=upper_last_name_initial, reverse=True)

Now we can much more easily follow the calculation, and we’ve again used the function name to clarify our intention.

Python’s decorator syntax allows us to extend the behavior of a function with a wrapper. When declaring a function with lambda, it’s still possible to use decorators, but without the @ syntax which highlights use as a decorator.

For example, if we found that our “is cute enough” check was taking a significant amount of time, we could add caching with the functools.lru_cache decorator from the standard library. Using a normal function, we can add it with the @ syntax:

from functools import lru_cache


@lru_cache
def is_cute_enough_for_contest(puppy):
    return puppy.cuteness >= 100

>>> list(filter(is_cute_enough_for_contest, puppies))
[Puppy('Kevin', 200), Puppy('Doggo', 100)]

When using lambda we have to call the decorator ourselves, without @:

>>> list(filter(lru_cache(lambda p: p.cuteness >= 100), puppies))
[Puppy('Kevin', 200), Puppy('Doggo', 100)]

This works, but it slightly obscures lru_cache being a decorator.

Python’s function annotations allow us to add for type hints. Such hints declare the expected types of variables and we can verify our expectations with a type checker tool such as Mypy. These let us make extra guarantees of our code’s correctness, alongside tests.

Unfortunately, because function annotations use colons (:) as their separator, they are not compatible with lambda. lambda already uses a single colon to separate its arguments from its expression, so there’s nowhere to add annotations.

For example, we could annotate our previous is_cute_enough_for_contest() function like so:

def is_cute_enough_for_contest(puppy: Puppy) -> bool:
    return puppy.cuteness >= 100

>>> list(filter(is_cute_enough_for_contest, puppies))
[Puppy('Kevin', 200), Puppy('Doggo', 100)]

This declares that we expect it to take a Puppy object and return a bool. We could run Mypy to check that is_cute_enough_for_contest() is always called with such types.

If we try to add such annotations to a lambda, we’ll only get a SyntaxError:

>>> list(filter(lambda p: Puppy: -> bool p.cuteness >= 100, puppies))
  File "<stdin>", line 1
    list(filter(lambda p: Puppy: -> bool p.cuteness >= 100, puppies))
                               ^
SyntaxError: invalid syntax

This is because the first colon starts the function body.

One tool for ensuring your tests check all parts of your system is to measure test coverage with coverage.py. This works on a line-by-line basis.

Because lambda functions include their body on the same line as their definition, they will show as fully covered, even if the function is never called. Thus, you might miss bugs in your lambda functions’ bodies, such as mistyping an attribute name.

Normal functions aren’t subject to this problem, because their body starts on a separate line to their declaration. If they don’t get called during tests, coverage will always show them as uncovered.

The second alternative to many of the uses of lambda is to use a comprehension. Many of the built-in functions that lambda functions are typically used with use the passed function to generate a new list of items, so a list comprehension is appropriate. But your use case might mean using a set or dict comprehension, or a generator expression.

For example, any call to filter() can be rewritten with an equivalent list comprehension. Take our cuteness filter() call:

>>> list(filter(is_cute_enough_for_contest, puppies))
[Puppy('Doggo', 100), Puppy('Kevin', 200)]

We can rewrite it using a list comprehension as:

>>> [p for p in puppies if is_cute_enough_for_contest(p)]
[Puppy('Doggo', 100), Puppy('Kevin', 200)]

We can also put the condition inside the comprehension:

>>> [p for p in puppies if p.cuteness >= 100]
[Puppy('Doggo', 100), Puppy('Kevin', 200)]

Using a comprehension without a function call like this is even a little bit faster. This is because the expression uses local variable access, rather passing them into another function with its own namespace.

Similarly, any map() call can be rewritten as a list comprehension. Again, take our previous map() example:

>>> list(map(lambda p: p.name, puppies))
['Doggo', 'Kevin', 'Bock']

We can rewrite this as:

>>> [p.name for p in puppies]
['Doggo', 'Kevin', 'Bock']

This is again simpler.

Comprehensions offer quite flexible syntax, allowing the same things that a for loop would, and so they’re more generally useful than filter() and map().

A third alternative to writing lambda functions is to use the standard library’s operator module. This module contains some predefined functions and function factories, which can replace the most common use cases for lambda functions. Let’s look at both of these separtaely, factories first.

The function factories offered by operator create functions that return a value based on their input. These can replace a lot of common use cases for lambda functions.

Recall the lambda function we used with list.sort():

lambda puppy: puppy.cuteness

We can construct an identical function with the operator.attrgetter function factory. We pass it the name the attribute we want to extract, and it returns a function that does so:

import operator

get_cuteness = operator.attrgetter("cuteness")

>>> get_cuteness(puppies[0])
100

We can use this inline in our list.sort() example:

puppies.sort(key=operator.attrgetter("cuteness"))

The “sort by an attribute” pattern is quite common, so attrgetter is often used to implement it.

Another note: the function that attrgetter returns is implemented in C, so it’s slightly faster than using either a normal or lambda function.

The operator module also offers two other function factories. itemgetter can replace a lambda that gets an item with [], such as lambda puppies: puppies[0]. And methodcaller can replace a lambda that calls a method, such as lambda puppy: puppy.bark().

Another set of functions offered by the operator module are its wrappers of Python’s operators (hence the module’s name). These can replace another bunch of common use cases for lambda functions.

For example, imagine we had a mega-cuteness formula that required us to multiply our puppies’ extracted cuteness values together. We could use functools.reduce() function with a lambda function to do this. reduce() will take pairs of puppies

from functools import reduce

cutenesses = [p.cuteness for p in puppies]

>>> reduce(lambda a, b: a * b, cutenesses)
1000000

Our lambda function, lambda a, b: a * b, is equivalent to operator.mul (short for multiply). So we could also write:

>>> reduce(operator.mul, cutenesses)
1000000

Again because the operator function is implemented in C, it is slightly faster than our handwritten version.

The operator module provides wrapper functions for all of Python’s operators, so there’s no need to write such lambda functions.

I hope this guide to lambda functions has answered many of your questions about them. May your Python be ever more readable,

—Adam

Read my book Boost Your Git DX to Git better.

One summary email a week, no spam, I pinky promise.

Related posts:

• Better Python Decorators with wrapt
• How to Combine Two Python Decorators
• Avoid Hardcoding IDs in Your Tests

Tags: python

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4