Now that we no longer have a vortex-scalar crate (it has been absorbed by vortex-array, see #6523), we are now able to add an Array variant to ScalarValue to represent scalars like list scalars much more compactly.
By passing around ArrayRef instead of Vec<Option<ScalarValue>> we can likely improve the performance of list-related operations. And in the near future where we would like to have a Tensor type, it is far more ideal to be passing around references directly to the memory rather than making a huge amount of allocations.
This will require us adding a new variant to the protobuf serialization format, as well as passing VortexSession around in a few more places since we now need to validate any Array scalar value variant on deserialization.
Now that we no longer have a
vortex-scalarcrate (it has been absorbed byvortex-array, see #6523), we are now able to add anArrayvariant toScalarValueto represent scalars like list scalars much more compactly.By passing around
ArrayRefinstead ofVec<Option<ScalarValue>>we can likely improve the performance of list-related operations. And in the near future where we would like to have a Tensor type, it is far more ideal to be passing around references directly to the memory rather than making a huge amount of allocations.This will require us adding a new variant to the protobuf serialization format, as well as passing
VortexSessionaround in a few more places since we now need to validate anyArrayscalar value variant on deserialization.