Opened 6 years ago

Closed 5 years ago

#285 closed defect (fixed)

numpy.pxd: No endianness or packing in buffer format string

Reported by: dagss Owned by: dagss
Priority: major Milestone: 0.11.2
Component: Packaging Keywords:
Cc:

Description (last modified by dagss)

numpy.pxd should include endianness and packing information in the buffer format string, so that Cython can reject arrays which are not native endian/native packing.

Attachments (1)

patch1 (5.2 KB) - added by dagss 6 years ago.

Download all attachments as: .zip

Change History (10)

comment:1 Changed 6 years ago by dagss

  • Component changed from Code Generation to Packaging

I have no idea which component this should go under...

comment:2 Changed 6 years ago by dagss

From the ML

Robert Bradshaw wrote:
> On Apr 17, 2009, at 1:45 PM, Kurt Smith wrote:
> 
>> The current numpy_test.pyx file fails for PowerPC macs due to
>> endianness issues in the dtype.  This is a small fix to make it work
>> (and make all tests pass on my machine).  It also adds an explicit
>> big-endian test to the doctest.
> 
> Hmm... This looks like a bug in the code itself, not just the test.

Thanks for bringing this up Kurt.

Using typed buffers in Cython only support the native endianness, so what one should get is an exception on one of the platforms.

It seems that the problem is that numpy.pxd does not include endianness in the formatting string; in fact it looks like this wasn't considered at all.

http://trac.cython.org/cython_trac/ticket/285

When it comes to the testcase, one should have two tests:

a) With native endianness, which should succeed. I think the best way is to modify the result:

x = ...
print(repr(x).replace(">","<")) # then follows little endian result

b) With non-native endianness, which should result in an exception from buffer acquisition.


> 
>> diff -r fc73225aaea1 tests/run/numpy_test.pyx
>> --- a/tests/run/numpy_test.pyx	Fri Apr 17 09:11:16 2009 +0200
>> +++ b/tests/run/numpy_test.pyx	Fri Apr 17 15:43:11 2009 -0500
>> @@ -132,13 +132,20 @@
>>>>> test_recordarray()
>>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>> -            ('a', np.dtype('i,i')),\
>> -            ('b', np.dtype('i,i'))\
>> +            ('a', np.dtype('<i,<i')),\
>> +            ('b', np.dtype('<i,<i'))\
>>          ])))
>>      array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>>            dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
>> '<i4'), ('f1', '<i4')])])
>>
>>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>> +            ('a', np.dtype('>i,>i')),\
>> +            ('b', np.dtype('>i,>i'))\
>> +        ])))
>> +    array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>> +          dtype=[('a', [('f0', '>i4'), ('f1', '>i4')]), ('b', [('f0',
>> '>i4'), ('f1', '>i4')])])
>> +
>> +    >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>>              ('a', np.dtype('i,f')),\
>>              ('b', np.dtype('i,i'))\
>>          ])))
>> @@ -297,6 +304,10 @@
>>      arr[1].b.x = arr[0].a.y + 1
>>      arr[1].b.y = 4
>>      arr[2] = arr[1]
>> +    if not arr.dtype.isnative:
>> +        # required to get the endianness
>> +        # indicator (< or >) correct in output.
>> +        arr.byteswap(True)
>>      return arr
>>
>>  def test_bad_nested_dtypes():
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev@codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
> 
> _______________________________________________
> Cython-dev mailing list
> Cython-dev@codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


-- 
Dag Sverre

comment:3 Changed 6 years ago by dagss

  • Summary changed from numpy.pxd does not include endianness information to numpy.pxd: No endianness in buffer format string

comment:4 Changed 6 years ago by dagss

  • Description modified (diff)
  • Summary changed from numpy.pxd: No endianness in buffer format string to numpy.pxd: No endianness or packing in buffer format string

comment:5 Changed 6 years ago by dagss

patch1 adds this to numpy.pxd, however Cython's buffer support doesn't support these format strings yet so I'm witholding the patch until then.

Changed 6 years ago by dagss

comment:6 Changed 5 years ago by dagss

  • Resolution set to fixed
  • Status changed from new to closed

comment:7 Changed 5 years ago by dagss

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:8 Changed 5 years ago by dagss

  • Owner changed from somebody to dagss
  • Status changed from reopened to new

comment:9 Changed 5 years ago by dagss

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.