cyreal

Jittable dataset utilities for JAX.

We use a producer and transform model. Sources stream data from datasets and Transforms transform the data stream. The DataLoader class composes sources and transforms into jittable data pipelines.

  • cyreal.datasets contains datasets such as MNIST, CIFAR-10, etc and associated utilities.
  • cyreal.loader contains the DataLoader class for building jittable data pipelines.
  • cyreal.sources contains data sources such as ArraySource and GymnaxSource.
  • cyreal.transforms contains data transforms such as BatchTransform and DevicePutTransform.
  • cyreal.rl contains some RL-specific utilities for the DataLoader.
 1"""Jittable dataset utilities for JAX.
 2
 3We use a producer and transform model. Sources stream data from datasets and Transforms transform the data stream. The DataLoader class composes sources and transforms into jittable data pipelines.
 4
 5
 6- `cyreal.datasets` contains datasets such as MNIST, CIFAR-10, etc and associated utilities.
 7- `cyreal.loader` contains the `DataLoader` class for building jittable data pipelines.
 8- `cyreal.sources` contains data sources such as `ArraySource` and `GymnaxSource`.
 9- `cyreal.transforms` contains data transforms such as `BatchTransform` and `DevicePutTransform`.
10- `cyreal.rl` contains some RL-specific utilities for the DataLoader.
11"""
12
13# from .datasets import (
14#     CIFAR10Dataset,
15#     CIFAR100Dataset,
16#     EMNISTDataset,
17#     FashionMNISTDataset,
18#     KMNISTDataset,
19#     MNISTDataset,
20# )
21# from .loader import (
22#     DataLoader,
23# )
24# from .transforms import (
25#     BatchTransform,
26#     DevicePutTransform,
27#     FlattenTransform,
28#     HostCallbackTransform,
29#     MapTransform,
30#     NormalizeImageTransform,
31#     BufferTransform,
32#     TimeSeriesBatchTransform,
33# )