JAX introduces hijax for custom types and operations

Introduction to hijax

JAX's primary data structure is the array, where functions take arrays as inputs and produce them as outputs. However, for more complex data types, the new hijax framework introduces the concept of HiTypes, which allow the creation of distinct types with their own identity and operational constraints.

Defining custom types

To define a new type, users subclass HiType and implement methods like lo_ty, lower_val, and raise_val to explain how the new type relates to standard JAX arrays. This enables better handling of internal invariants and allows for controlled interactions with the data type.

By calling register_hitype, a Python class can be associated with the new type, facilitating its use within the JAX ecosystem.

Operational capabilities with hijax

Hijax types allow developers to define specific operations for their custom types, which can include behaviors for autodiff by implementing tangent rules with VJPHiPrimitive subclasses. This ensures that any derivatives taken with respect to the custom types are handled correctly. Additionally, the framework supports advanced features like batching through vmap and explicit sharding in JAX's computational model.

These capabilities make it easier to work with complex data structures in JAX, promoting extensibility and control over how data is manipulated in computational graphs.

Future and experimental usage

The hijax framework is experimental, with its API subject to change. Users are encouraged to explore its capabilities while being prepared for potential evolutions in the underlying APIs. This experimental nature emphasizes JAX's ongoing development and commitment to enhancing how complex data types are managed in machine learning workflows.