Ticket #715 (closed defect: fixed)

Opened 3 years ago

Last modified 3 years ago

Compiler crash while analysing generator expression

Reported by: scoder Owned by: vitja, scoder
Priority: major Milestone: 0.15.1
Component: Type Analysis Keywords:
Cc:

Description (last modified by scoder) (diff)

Compiling test/test_itertools.py in the latest Py3.3, I get this:

Error compiling Cython file:
------------------------------------------------------------
...
            ]:
            target = [tuple([arg[i] if i < len(arg) else None for arg in args])
                      for i in range(max(map(len, args)))]
            self.assertEqual(list(zip_longest(*args)), target)
            self.assertEqual(list(zip_longest(*args, **{})), target)
            target = [tuple((e is None and 'X' or e) for e in t) for t in target]   # Replace None fills with 'X'
                                              ^
------------------------------------------------------------

test/test_itertools.py:597:47: Compiler crash in AnalyseDeclarationsTransform

ModuleNode.body = StatListNode(test_itertools.py:1:0)
StatListNode.stats[24] = StatListNode(test_itertools.py:58:0)
StatListNode.stats[0] = PyClassDefNode(test_itertools.py:58:0,
    name = u'TestBasicOps')
PyClassDefNode.body = StatListNode(test_itertools.py:60:4)
StatListNode.stats[19] = DefNode(test_itertools.py:585:4,
    modifiers = [...]/0,
    name = u'test_ziplongest',
    needs_closure = True,
    num_required_args = 1,
    reqd_kw_flags_cname = '0')
DefNode.body = StatListNode(test_itertools.py:586:8)
StatListNode.stats[0] = ForInStatNode(test_itertools.py:586:8)
ForInStatNode.body = StatListNode(test_itertools.py:593:12)
StatListNode.stats[3] = SingleAssignmentNode(test_itertools.py:597:21)
SingleAssignmentNode.rhs = ComprehensionNode(test_itertools.py:597:21,
    has_local_scope = True,
    result_is_used = True,
    use_managed_ref = True)
ComprehensionNode.loop = ForInStatNode(test_itertools.py:597:65)
ForInStatNode.body = ComprehensionAppendNode(test_itertools.py:597:21)
ComprehensionAppendNode.expr = SimpleCallNode(test_itertools.py:597:27,
    result_is_used = True,
    use_managed_ref = True)
SimpleCallNode.args[0] = GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 5272, in analyse_declarations: GeneratorExpressionNode(test_itertools.py:597:47,
    genexpr_name = u'genexpr15',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    result_is_used = True,
    use_managed_ref = True)
File 'Nodes.py', line 3011, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...]/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2091, in analyse_declarations: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...]/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')
File 'Nodes.py', line 2236, in declare_pyfunction: GeneratorDefNode(test_itertools.py:597:47,
    is_generator = True,
    modifiers = [...]/0,
    name = u'genexpr',
    needs_closure = True,
    no_assignment_synthesis = True,
    reqd_kw_flags_cname = '0')

Compiler crash traceback from this point on:
  File "/.../Cython/Compiler/Nodes.py", line 2236, in declare_pyfunction
    entry = env.declare_pyfunction(name, self.pos, allow_redefine=not self.is_wrapper)
  File "/.../Cython/Compiler/Symtab.py", line 583, in declare_pyfunction
    cname=name, visibility='private')
  File "/.../Cython/Compiler/Symtab.py", line 1508, in declare_var
    cname = '%s%s' % (self.genexp_prefix, self.parent_scope.mangle(Naming.var_prefix, name))
  File "/.../Cython/Compiler/Symtab.py", line 1399, in mangle
    return prefix + name
TypeError: cannot concatenate 'str' and 'NoneType' objects

Change History

Changed 3 years ago by scoder

  • description modified (diff)

Changed 3 years ago by vitja

  • owner changed from somebody to vitja

Changed 3 years ago by vitja

  • status changed from new to closed
  • resolution set to fixed

Changed 3 years ago by scoder

  • status changed from closed to reopened
  • resolution fixed deleted

Changed 3 years ago by scoder

  • owner changed from vitja to vitja, scoder
  • status changed from reopened to new

Additional fix is here:

 https://github.com/cython/cython/commit/f3f5cd7ed6a5d30597075b9e580213033985334c

Generator expressions must create anonymous functions, otherwise, they'd show up in locals().

Changed 3 years ago by scoder

  • status changed from new to closed
  • resolution set to fixed

Changed 3 years ago by vitja

  • milestone changed from wishlist to 0.15.1
Note: See TracTickets for help on using tickets.