Misplaced decref due to PyDict_Next optimization

[jasone]

The PyDict?_Next optimization looks like it might be conflicting with another optimization that collects local temp variables of the same type into a single variable... or something. Cython generates C code for the attached program that has a bogus Py_DECREF call, as indicated in the comments of the program. This can cause a crash (though that's only happening for me in the program that I first noticed the problem for).

[jasone]

I just took a look at what was generated before the PyDict_Next optimization was added, and it looks like there was a temp variable used to store a PyList reference during iteration. That temp variable appears to be what the bogus Py_DECREF call is for.

[scoder]

The problem is the return statement (ReturnStatNode?). In analyse_expressions(), it retrieves the list of active temps:

        self.temps_in_use = env.temps_in_use()

and then frees that in its execution code:

        for entry in self.temps_in_use:
            code.put_var_decref_clear(entry)

Like some other tree transforms, the iter-dict optimisation changes the temp allocation, which leaves some temps unused later in the code, such as the one for the loop iterator in your example. The return statement isn't prepared for that. So the impact of this bug is actually larger than just the iter-dict transform.

[scoder]

Here's a shorter example with the same problem:

def spam(dict d):
    for elm in d:
        return False
    return True

spam(dict(test=2))

[dagss]

From what I can see, these two fixes it:

 http://hg.cython.org/cython-devel/rev/15459e336874  http://hg.cython.org/cython-devel/rev/65990f108724

[dagss]

More temp resolution stuff should be seen in relation to #77.