Search code examples
pythonadditionsyntactic-sugar

How can just using the += "syntactic sugar" cause an infinite loop?

Let a = [-1, -2, -3]. I want to modify the list a so that a == [-1, -2, -3, 1, 2, 3], and I want to use map to achieve that.

I have the following different pieces of code I've written to do this:

  1. a = a + map(abs, a)
  2. a = a + list(map(abs, a))
  3. a += map(abs, a)
  4. a += list(map(abs, a))

Here's the results:

  1. TypeError: can only concatenate list (not "map") to list
  2. Works as expected: a = [-1, -2, -3, 1, 2, 3]
  3. Hangs indefinitely before getting terminated with no error message (potentially ran out of memory?)
  4. Works as expected: a = [-1, -2, -3, 1, 2, 3]

I thought that a += b was just syntactic sugar for a = a + b, but, given how (1) and (3) behave, that's clearly that's not the case.

Why does (1) give an error, while (3) seems to go into an infinite loop, despite one generally just being a syntactic sugar version of the other?

Solution

  • As the examples show, a += b is not equivalent to a = a + b, and this answer explains why.

    Under the hood, a += b calls __iadd__, which, when a is a list, iterates over b, adding each element of b to a. However, a = a + b will call __add__, on (the 2nd) a, which will try to concatenate b to (the 2nd) a essentially all at once, but the method will detect that b is not a list and fail.

    The a += b case causes an infinite loop, because map, when iterated over, only computes and yields one element at a time. Thus, it will yield back the first number, abs(-1) == 1, which will be appended to a so a == [-1, -2, -3, 1]. Then, after the append, it will be asked to yield back the second number, abs(-2) == 2, so a == [-1, -2, -3, 1, 2]. This continues until a == [-1, -2, -3, 1, 2, 3], and then still continues (because there are now more values in a than just the original three values.). So map will return abs(1) == 1 next, and so now a == [-1, -2, -3, 1, 2, 3, 1] and then on the next step, a == [-1, -2, -3, 1, 2, 3, 1, 2], and so on, until it runs out of memory.

    In short, += calls __iadd__ and + calls __add__ and they're implemented differently. The __iadd__ function will run on anything that can be iterated over, but __add__ does a type check to make sure the thing it's concatenating is a list. Thus, += causes an infinite loop while + leads to an error.