'with' statement doesn't compile on C++

[scoder]

This crashes due to incorrect temp deallocation:

    with ContextManager(u"value") as x:
        return x

The generated code is:

  __pyx_1 = __Pyx_GetName(__pyx_m, __pyx_kp_ContextManager); if [error goto]
  /* __pyx_t_6 allocated */
  __pyx_t_6 = PyTuple_New(1); if [error goto]
  Py_INCREF(((PyObject *)__pyx_kp_8));
  PyTuple_SET_ITEM(__pyx_t_6, 0, ((PyObject *)__pyx_kp_8));
  /* __pyx_t_7 allocated */
  __pyx_t_7 = PyObject_Call(__pyx_1, ((PyObject *)__pyx_t_6), NULL); if [error goto]
  Py_DECREF(__pyx_1); __pyx_1 = 0;
  Py_DECREF(((PyObject *)__pyx_t_6)); __pyx_t_6 = 0;
  /* __pyx_t_6 released */
  Py_XDECREF(__pyx_t_2);
  __pyx_t_2 = __pyx_t_7;
  __pyx_t_7 = 0;
...
  /*try:*/ {
    {
...
        Py_DECREF(__pyx_t_6); __pyx_t_6 = 0;
        Py_DECREF(__pyx_t_7); __pyx_t_7 = 0;
        Py_DECREF(__pyx_t_8); __pyx_t_8 = 0;
        goto __pyx_L5;

Note how __pyx_t_7 is DECREF-ed on return, as it is not released after retrieving the context manager in the temp variable and assigning it to a new temp.

There's a disabled test case in run/withstat.pyx that tests for this.

[scoder]

The problem described above is fixed in rev 753f53660cc1, however, the test case still crashes due to temp handling.

 http://hg.cython.org/cython-devel/rev/753f53660cc1

The 'return' statement is a clear example of goto being harmful. It cleans up all temps, but since it's surrounded by a try-finally in the case of a 'with' statement, the code in the 'finally' clause accesses temps that the return statement already cleared.

[scoder]

I think the real problem here are long-living temps. They are not made for living longer than one statement or one expression. The 'temps' of a TreeFragment should be held in a kind of specially named, block-local variable, not in a temp.

[robertwb]

Temps can live longer than a statement--for example the temps used in a for loop.

The issue is (I think) that at any state there should be a running list of "possibly used temps" and it needs to be updated when things are decref'ed.

[dagss]

OK I went back to the older strategy of having TreeFragment? use NameNodes?, just with strange names. This will fail if the user uses a variable name on the form __tmpvar_3, so should perhaps be done something about. Follow-up in #197.

[dagss]

There's still an issue with excinfo_temp (~ ParseTreeTransforms?.py:550)

[dagss]

OK done:  http://hg.cython.org/cython-devel/rev/9d0ac0d9659b

Sorry about the quality of these commits; but at least now it is no longer blocking up and one can revisit in #197.

[dagss]

This is very odd: 'with' statement doesn't compile on C++, due to the variables not getting declared. They are declared in the C test run though.

[dagss]

Duh!  http://hg.cython.org/cython-devel/rev/3304bc6fd2c3