NumPy + Cython + Python 3 = compiler crash

[dagss]

From the ML:

I just installed numpy for both python2 and 3 from an up-to-date
checkout of the 1.5.x branch.

I am attempting to cythonize the following code with cython-0.13:

---
cimport numpy as np
import numpy as np

def test():
   cdef np.ndarray[np.float64_t, ndim=1] ret
   ret_arr = np.zeros((20,), dtype=np.float64)
   ret = ret_arr
---

I have this setup.py file:

---
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

import numpy

setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = [
        Extension(
            "test_open", ["test_open.pyx"], include_dirs=[numpy.get_include()]
            )
        ]
    )
---

When I run "python setup.py build_ext --inplace", everything is fine.
When I run "python3 setup.py build_ext --inplace", I get an error:

running build_ext
cythoning test_open.pyx to test_open.c

Error converting Pyrex file to C:
------------------------------------------------------------
...
        # For use in situations where ndarray can't replace PyArrayObject*,
        # like PyArrayObject**.
        pass

    ctypedef class numpy.ndarray [object PyArrayObject]:
        cdef __cythonbufferdefaults__ = {"mode": "strided"}
                                                ^
------------------------------------------------------------

/home/darren/.local/lib/python3.1/site-packages/Cython/Includes/numpy.pxd:173:49:
"mode" is not a buffer option

Error converting Pyrex file to C:
------------------------------------------------------------
...
cimport numpy as np
import numpy as np


def test():
   cdef np.ndarray[np.float64_t, ndim=1] ret
       ^
------------------------------------------------------------

/home/darren/temp/test/test_open.pyx:6:8: 'ndarray' is not a type identifier
building 'test_open' extension
gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
-I/home/darren/.local/lib/python3.1/site-packages/numpy/core/include
-I/usr/include/python3.1 -c test_open.c -o
build/temp.linux-x86_64-3.1/test_open.o
test_open.c:1: error: #error Do not use this file, it is the result of
a failed Cython compilation.
error: command 'gcc' failed with exit status 1

[dagss]

Follow-up from ML:

I think I found the problem. In Cython/Compiler/Buffer.py:

buffer_options = ("dtype", "ndim", "mode", "negative_indices", "cast")

The error is being raised somewhere around line 146, so I added some
print statements:

    ERR_BUF_OPTION_UNKNOWN = '"%s" is not a buffer option: "%s"'

    for name, (value, pos) in dictargs.iteritems():
        if not name in buffer_options:
            print(type(name), type(buffer_options[0]))
            raise CompileError(pos, ERR_BUF_OPTION_UNKNOWN % (name,
buffer_options))
        options[name] = value

    for name, (value, pos) in zip(buffer_options, posargs):
        if not name in buffer_options:
            print(type(name), type(buffer_options[0]))
            raise CompileError(pos, ERR_BUF_OPTION_UNKNOWN % (name,
buffer_options))

The result:

(<class 'Cython.Compiler.StringEncoding.BytesLiteral'>, <class 'str'>)

Error compiling Cython file:
------------------------------------------------------------
...
        # For use in situations where ndarray can't replace PyArrayObject*,
        # like PyArrayObject**.
        pass

    ctypedef class numpy.ndarray [object PyArrayObject]:
        cdef __cythonbufferdefaults__ = {"mode": "strided"}
                                                ^
------------------------------------------------------------

/Users/darren/.local/lib/python3.1/site-packages/Cython/Includes/numpy.pxd:173:49:
"mode" is not a buffer option: "('dtype', 'ndim', 'mode',
'negative_indices', 'cast')"

So the problem appears to be that numpy.pxd is defining
__cythonbufferdefaults__ with instances of
Cython.Compiler.StringEncoding.BytesLiteral, but Buffer.py is defining
valid options as instances of str.

[dalcinl]

Fixed:  https://github.com/cython/cython/commit/e12615215b862bf1a240252dd35a90906794559a