# SparseOperationKit Release Notes #
The release notes for SparseOperationKit.

## What's new in Version 2.1.0 ##
+ The official release of SOK:
    * A new API `sok.incremental_dump` has been added, which allows users to dump newly added keys and values into a numpy array by specifying a time threshold. Currently it only supports `sok.DynamicVariable` that uses HKV as the backend.
    * Fixed the issue of wgrad using too much GPU memory.
    * Fixed an illegal memory access issue in a CUDA kernel during backward propagation.
    * The documentation and examples for SOK (Sparse Operation Kit) have been updated.

## What's new in Version 2.0.0 ##
+ The official release of SOK:
    * Remove the legacy code,`DistributedEmbedding` and `All2AllDenseEmbedding` will be deprecated.
    * `from sparse_operation_kit import experiment as sok` is replaced by `import sparse_operation_kit as sok`.
    * `sok.DynamicVariable` supports Merlin-HKV as its backend.
    * The parallel dump and load are added.

## What's new in Version 1.1.4 ##
+ Add `sok.experiment` module to integrate hugectr 3G embedding:
    * Add `sok.experiment.lookup_sparse`, which support distributed and fused embedding lookup.
    * Add `sok.experiment.DynamicVariable`, whose size can grow dynamically when doing lookup.
    * See API Docs -> Experiment to get other function of `sok.experiment`

## What's new in Version 1.1.3 ##
+ Update pip install instruction and fix some bugs.

## What's new in Version 1.1.2 ##
+ Add TensorFlow Functional API support

## What's new in Version 1.1.1 ##
+ Add Auto-Mixed-Precision training support
+ Add uint32 key dtype support
+ Add TensorFlow initializers support
+ Add DLRM benchmark results

## What's new in Version 1.1.0 ##
+ Supports TensorFlow 1.15.
+ Supports configuring visible devices via `tf.config.set_visible_devices`.
+ Added a dedicated CUDA stream for SOK's Ops.
+ Supports pip installation.
+ Fixed hanging issue in `tf.distribute.MirroredStrategy` when TensorFlow version greater than 2.4.

## What's new in Version 1.0.1 ##
+ Supports Horovod as the synchronized training communication tool.
+ Supports dynamic input in All2AllDenseEmbedding, which means `unique->lookup->gather` pattern can be used.
+ Supports IdentityHashtable, which means no hash-mapping during inserting new keys.
+ Added TF Distributed Embedding totally with TF's ops.

## What's new in Version 1.0.0 ##
+ Implemented a new framework that can be used to easily integrate different embedding algorithms to common DL frameworks.
+ Supports single-node & multi-node synchronized training with TensorFlow.
+ Integrated HugeCTR's DistributedSparseEmbedding algorithm.
+ Integrated All2AllDenseEmbedding algorithm.
+ Added custom Adam optimizer for SOK when TF version <= 2.4.
