Chapter 5: First-Class Functions
functions are just another object
- Higher-Order Functions
- Anonymous Functions
- The Seven Flabors of Callable Objects
- User-defined Callable types
- Function Introspection
- From Positional to Keywork-Only Parameters
- Retrieving Information about Parameters
- Function Annotations
- Packages for Functional Programming
- functools.partial
Funtion in python are a first class objects. Now what is a first class object? A program entity is a first class object if it can be:
- Created at runtime
- Assigned to a variable or element in a data structure
- Passed as argument to a function
- returned as argument to a function
def factorial(n):
"""returns n!"""
return 1 if n < 2 else n*factorial(n-1)
factorial(42)
factorial.__doc__
type(factorial)
dir(factorial)
fact = factorial
fact
fact(5)
map(factorial, range(11))
list(map(factorial, range(5)))
Higher-Order Functions
Having first-class functions enables programming in a functional style. One of the hall-marks of functional programming is the use of Higher-order functions. A function that takes another function as argument or returns a function is called a higher order function. This can be seen in the map example above or the sorted function which has a keyword argumnet key which takes a function.
fruits = ['strawberry', 'fig', 'apple', 'cherry', 'raspberry', 'banana']
sorted(fruits, key=len)
def reverse(word):
return word[::-1]
reverse('testing')
sorted(fruits, key=reverse)
map, filter and reduce are the best-know higher order functions. But there are better alternatives to these and we will see them below.
listcomps and genexps replace the functionalities of map and filter and in most cases they are more readable.
list(map(fact, range(6)))
[fact(n) for n in range(6)]
list(map(fact, filter(lambda n: n%2, range(6))))
[fact(n) for n in range(6) if n%2]
In python3 map and filter return generators so their direct substituion is genexps. genexps are more performant for large lists too, so use that when necessary.
from functools import reduce
from operator import add
reduce(add, range(100))
sum(range(100))
There is also all(iterable) and any(iterable) that also have a reducing behavious and built in.
all([0, 0, 0, 1]), all([1, 1, 1, 1,])
any([0, 0, 0, 0]), any([0, 0, 0, 1])
Anonymous Functions
using the lambda keyword, you can create a anonymous function within a python expression. the lambda function cannot be used to create complex functions with while try etc. but its best use is in the context of arguments lists.
for example lets rewrite the reverse function we saw earlier using lambda.
fruits = ['strawberry', 'fig', 'apple', 'cherry', 'raspberry', 'banana']
sorted(fruits, key=lambda word: word[::-1])
the lambda syntax is just syntatic sugar to create a "callable object" which is exactly what a function is.
The Seven Flabors of Callable Objects
the call operator (ie ()) can be applied to other objects as well. To check if an object is callable use callable() fn. There are 7 callable types definied in the Python Data Model
- User-defined functions - created with def or lambda
- Built-in functions - fn implementation in C. eg
lenortime.strftime - Built-in methods - methods implemented in C like
dict.get - Methods - functions defined in the body of a class
- Classes - creating new instance of a class calls the
__new__and__init__fns - Class instances - if a class defines
__call__method - Generator functions - fns/methods that have
yeildkeyword
callable(map)
import random
class BingoCage:
def __init__(self, items):
self._items = list(items)
random.shuffle(self._items)
def pick(self):
try:
return self._items.pop()
except IndexError:
raise LookupError('pick from empty BingoCage')
def __call__(self):
return self.pick()
bingo = BingoCage(range(3))
bingo.pick()
bingo()
Function Introspection
function, as we mentioned earlier are first-class objects. Now lets look at the factors that make its so.
> > dir(factorial)The __dict__ attribute:Like instances of plain user-defined class, a function uses __dict__ attribute to store user attributes assigned to it.
Looking closer at attributes that are specific to functions. To see the detailed explaination of each attribute ref Table-5.1 pg 148.
class C: pass
obj = C()
def func(): pass
sorted(set(dir(func)) - set(dir(obj)))
def tag(name, *content, cls=None, **attrs):
"""Generates one or more HTML tags"""
if cls is not None:
attrs['class'] = cls
if attrs:
attr_str = ' '.join('%s="%s"' % (attr, value)
for attr, value
in sorted(attrs.items()))
else:
attr_str = ''
if content:
return '\n'.join('<%s%s>%s</%s>' %
(name, attr_str, c, name) for c in content)
else:
return '<%s%s />' % (name, attr_str)
tag('br')
tag('p', 'hello')
tag('p', 'hello', 'world')
tag('p', 'hello', id=33)
my_tag = {'name': 'img', 'title': 'Sunset Boulevard',
'src': 'sunset.jpg', 'cls': 'framed'}
tag(**my_tag)
import bobo
@bobo.query('/')
def hello(person):
return 'Hello %s!' % person
What's special about this is that the decorator @bobo.query() interospects the hello function and understands that it expects one argument. It then parses the response and return the person to the function. All this without the user having to work with the response.
This is achieved using various attributes like __defaults__ and __kwdefaults__, __code__ etc. Lets see another example to see better.
def clip(text, max_len=80):
"""
Return text clipped at the last space before or afte max_len.
"""
end = None
if len(text) > max_len:
space_before = text.rfind(' ', 0, max_len)
if space_before >= 0:
end = space_before
else:
space_after = text.rfind(' ', max_len)
if space_after >= 0:
end = space_after
if end is None:
end = len(text)
return text[:end].rstrip()
clip.__defaults__
clip.__code__
clip.__code__.co_varnames
clip.__code__.co_argcount
clip.__defaults__, clip.__name__, clip.__kwdefaults__
Now this is kind off a messy way to learn about the functions but fortunately there is the inspect module.
from inspect import signature
sig = signature(clip)
sig
str(sig)
for name, param in sig.parameters.items():
print(param.kind, ':', name, '=', param.default)
as you can see this is a much better way to parse the argument info. signature also has a annotation attribute to give info about annotations.
inspect.Signature object also has a bind function that takes any number of arguments and binds them to the parameters. This can be used in frameworks to do validation of arguments before invocations. This is the same machinery the interpreter uses to bind arguments to formal parameters in function calls
sig = signature(tag)
my_tag = {'name': 'img', 'title': 'Sunset Boulevard',
'src': 'sunset.jpg', 'cls': 'framed'}
bound_args = sig.bind(**my_tag)
bound_args
for name, value in bound_args.arguments.items():
print(name, '=', value)
del my_tag['name']
bound_args = sig.bind(**my_tag)
def clip(text:str, max_len:'int>0' = 0) -> str:
"""
Return text clipped at the last space before or afte max_len.
"""
end = None
if len(text) > max_len:
space_before = text.rfind(' ', 0, max_len)
if space_before >= 0:
end = space_before
else:
space_after = text.rfind(' ', max_len)
if space_after >= 0:
end = space_after
if end is None:
end = len(text)
return text[:end].rstrip()
# its not used by the python interpreter
sig = signature(clip)
sig.return_annotation
for param in sig.parameters.values():
note = repr(param.annotation).ljust(13)
print(note, ':', param.name, '=', param.default)
These annotation can be used with tools like mypy to bring the benifits of static type checking into python.
%%timeit
from functools import reduce
def fact(n):
return reduce(lambda a, b: a*b, range(1, n+1))
fact(5)
%%timeit
# operator has these functions for you!
from operator import mul
def fact(n):
return reduce(mul, range(1, n+1))
fact(5)
Another group of one-trick lambdas that operator replaces are functions to pick items from sequences or read attributes from objects: itemgetter and attrgetter actually build custom functions to do that.
metro_data = [
('Tokyo', 'JP', 36.933, (35.689722, 139.691667)),
('Delhi NCR', 'IN', 21.935, (28.613889, 77.208889)),
('Mexico City', 'MX', 20.142, (19.433333, -99.133333)),
('New York-Newark', 'US', 20.104, (40.808611, -74.020386)),
('Sao Paulo', 'BR', 19.649, (-23.547778, -46.635833)),
]
from operator import itemgetter
# sort based of tuple[2]
for city in sorted(metro_data, key=itemgetter(2)):
print(city)
cc_name = itemgetter(1, 0)
for city in metro_data:
print(cc_name(city))
itemgetter supports seqences and mappings (any object with a __getitem__
Simalar to that is attrgetter, which creates a functions to extract ojbect attributes by name. It will also get attributes that require the .(dot)
from collections import namedtuple
LatLong = namedtuple('LatLong', 'lat long')
Metropolis = namedtuple('Metropolis', 'name cc pop coord')
metro_areas = [Metropolis(name, cc, pop, LatLong(lat, long))
for name, cc, pop, (lat, long) in metro_data]
metro_areas[0]
metro_areas[0].coord.lat
from operator import attrgetter
name_lat = attrgetter('name', 'coord.lat')
for city in sorted(metro_areas, key=attrgetter('coord.lat')):
print(name_lat(city))
import operator
[name for name in dir(operator) if not name.startswith('_')]
Similar to itemgetter and attrgetter, methodcaller also returns a function that calls a method by name on the object given as argument.
from operator import methodcaller
s = 'The time has home'
upcase = methodcaller('upper')
upcase(s)
hiphenate = methodcaller('replace', ' ', '-')
hiphenate(s)
Here, in the last case you can see methodcaller froze 2 arguments to the replace function.
from operator import mul
from functools import partial
triple = partial(mul, 3)
triple(7)
[triple(n) for n in range(10)]
partial takes a callable function name and any number of positional or keyword arguments and generates a new function with the args in place. This is effective in situations where you want to pass a callback to an API that expects fewer args that that is there in the function. partial takes a callable as first argument, followed by an arbitrary number of positional and keyword arguments to bind.
partialmethod does the same thing but for methods.