Blog

An occasionally updated list of bugs I find interesting.

• Python 3: Modifying a set as you iterate through it rips open a portal to the Upside Down
• Python 3: List of lists, or list of list?
• Python 3: Default arguments that only work once

Python 3: Modifying a set as you iterate through it rips open a portal to the Upside Down

Why does this happen?

>>> s = {0}
>>> for i in s:
...    s.add(i + 1)
...    s.remove(i)
...
>>> print(s)
{16}

You might expect it to either output {1} or maybe simply loop forever. But it actually loops for a while before eventually exiting the loop and printing the value 16.

That seems pretty weird. Changing an iterable at the same time as iterating through it seems like a pretty bad idea, but if we leave that aside there are a couple of interesting things about Python internals that this demonstrates.

The main thing to remember is that elements in a Python set are stored in a hash table. When you iterate through a set in Python you are really iterating through the indices of the set's internal hash table. So essentially this loop will only continue as long as the next (i+1) element hashes to a hash table index higher than the last. Once this pattern breaks, the iterator will continue on to the end of the hash table and then finish the loop. Clearly the first time this happens with the integers in the range 0...N is the number 16. The number 16 is placed at an index somewhere behind the iterator which carries on to the end of the table and finishes the loop.

Not sure I would actually call this a bug. More like a strange behaviour brought about by doing something with Python that you probably shouldn't. Time to close that portal up and hope the Demogorgon didn't get through.

Python 3: List of lists, or list of list?

Careful how you make a list of lists in Python.

>>> x = [[1]]*4
>>> x
[[1], [1], [1], [1]]
>>> x[0][0]
1
>>> x[0][0] = 2
>>> x
[[2], [2], [2], [2]]
                

Whoops! You've actually made a list of identical references all to the same one list. Avoid this using a list comprehension:

>>> x = [[1] for _ in range(4)]
>>> x
[[1], [1], [1], [1]]
>>> x[0][0]
1
>>> x[0][0] = 2
>>> x
[[2], [1], [1], [1]]
                

Much better.

Python 3: Default arguments that only work once

Have a look at this behaviour.

>>> def func(x, y = []):
...    y.append(x)
...    return y
>>> func(1)
[1]
>>> func(2)
[1,2]
>>> func(3)
[1,2,3]
                

Ok, something strange going on there. Also this:

>>> import datetime
>>> def func(x = datetime.datetime.now()):
...    return x
>>> func()
datetime.datetime(2020, 6, 25, 15, 57, 23, 382974)
>>> func()
datetime.datetime(2020, 6, 25, 15, 57, 23, 382974)
>>> func()
datetime.datetime(2020, 6, 25, 15, 57, 23, 382974)
                

The default value of the current time isn't updating.

Essentially the thing to realise is that default values in Python functions are only evaluated and assigned once. This happens when the function is defined (not each time the function runs). So if you want an argument to have a default value and it is either a mutable type (like a list) or the value of an evaluated expression which might change you should do something like this:

>>> def func(x = None, y = None):
...    if x is None:
...        x = datetime.datetime.now()
...    if y is None:
...        y = []
...    y.append(x)
...    return y
>>> func()
[datetime.datetime(2020, 6, 25, 16, 6, 29, 931275)]
>>> func()
[datetime.datetime(2020, 6, 25, 16, 6, 31, 347211)]
>>> func()
[datetime.datetime(2020, 6, 25, 16, 6, 32, 75136)]
                

\ /                                     \ /                \ /     
''(#)(#)                           (#)(#)''                ''(#)(#)
  /\/\/\                            /\/\/\                   /\/\/\