Added

• LBPP dataset.

Changed

• apache-beam version is pinned at <2.65.0 until related tests are fixed,
  see issue 11055.

Deprecated

Removed

Fixed

• CroissantBuilder now supports Croissant files without patch version (i.e. only
  {major.minor} are provided).
• Various small bug fixes.

Security

Added

• New Beam writer NoShuffleBeamWriter that doesn't shuffle, which speeds up
  dataset generation significantly, but does not have deterministic order
  guarantees. Can be enabled with the flag --nondeterministic_order.
• CroissantBuilder now supports Croissant files that define splits; and new
  feature types: feature dictionaries and multidimensional arrays.
• New datasets.

Changed

Deprecated

Removed

Fixed

• Various small bug fixes.
• Various performance improvements.

Security

Added

• New datasets.

Changed

• CroissantBuilder's API to generate TFDS datasets from Croissant files.

Deprecated

Removed

Fixed

• Versions for existing datasets.

Security

Added

• Full support for Python 3.12.

Added

• Support to download and prepare datasets using the
  Parquet data format.

  builder = tfds.builder('fashion_mnist', file_format='parquet')
  builder.download_and_prepare()
  ds = builder.as_dataset(split='train')
  print(next(iter(ds)))

• tfds.data_source
  is pickable, thus working smoothly with
  PyGrain. Learn more by following the
  tutorial.

• TFDS plays nicely with
  Croissant. Learn more by
  following the
  recipe.

Changed

Deprecated

Removed

Fixed

Security

Added

• A new CroissantBuilder
  which initializes a DatasetBuilder based on a Croissant
  metadata file.
• New conversion options between different bounding boxes formats.
• Better support for HuggingfaceDatasetBuilder.
• A script
  to convert a dataset from one format to another.

Changed

Deprecated

• Python 3.9 support. TFDS now uses Python 3.10

Removed

Fixed

Security

Added

• Segment Anything
  (SA-1B) dataset.

Changed

• Hugging Face datasets accept None values for any features. TFDS has no
  tfds.features.Optional, so None values are converted to default values.
  Those default values used to be 0 and 0.0 for int and float. Now, it's
  -inf as defined by NumPy (e.g., np.iinfo(np.int32).min or
  np.finfo(np.float32).min). This avoids ambiguous values when 0 and 0.0
  exist in the values of the dataset. The roadmap is to implement
  tfds.features.Optional.

Deprecated

• Python 3.8 support. As per
  NEP 29, TFDS now
  uses Python>=3.9.

Removed

Fixed

Security

Added

• [Experimental] A list of freeform text tags can now be attached to a
  BuilderConfig. For example:

  BUILDER_CONFIGS = [
   tfds.core.BuilderConfig(name="foo", tags=["foo", "live"]),
   tfds.core.BuilderConfig(name="bar", tags=["bar", "old"]),
  ]

  The tags are recorded with the dataset metadata and can later be retrieved
  using the info object:

  builder.info.config_tags # ["foo", "live"]

  This feature is experimental and there are no guidelines on tags format.

Changed

Deprecated

Removed

Fixed

• Fixed generated proto files (see issue 4858).

Security

Added

Changed

Deprecated

Removed

Fixed

• The installation on macOS now works (see issues
  4805 and
  4852). The ArrayRecord
  dependency is lazily loaded, so the
  TensorFlow-less path is
  not possible at the moment on macOS. A fix for this will follow soon.

Security