Search code examples
pythonpyarrow

Shift a chunkedarray

If I start with

import pyarrow as pa
ca = pa.chunked_array([[1,3, 2], [5, 2, 1]])

I'd like to end up with a pyarrow array with elements

[None, 1, 3, 2, 5, 2]

I don't really mind if the output is an array or a chunkedarray

Solution

  • You can prepend an array with one null element to a slice of the original array, without the last element.

    import pyarrow as pa

ca = pa.chunked_array([[1, 3, 2], [5, 2, 1]])
ca = pa.concat_arrays([pa.nulls(1, ca.type)] + ca[:-1].chunks)