Template Class Int8CacheCalibrator

Inheritance Relationships

Base Type

  • private Algorithm

Class Documentation

template<typename Algorithm >
class torch_tensorrt::ptq :: Int8CacheCalibrator : private Algorithm

Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm that only reads from a calibration file.

Template Parameters

  • Algorithm : class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use

Public Functions

Int8CacheCalibrator ( const std::string & cache_file_path )

Construct a new Int 8 Cache Calibrator object.

Parameters

  • cache_file_path :

int getBatchSize ( ) const noexcept override

Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)

Return

int

bool getBatch ( void * bindings [], const char * names [], int nbBindings ) noexcept override

Get the next Batch.

Not used always returns false

Return

false

Parameters

  • bindings : void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input

  • names : const char*[] - Names of bindings

  • nbBindings : int - Number of bindings

const void * readCalibrationCache ( size_t & length ) noexcept override

Read calibration cache.

How to read from the calibration cache, only enabled if use_cache is set

Return

const void* - Pointer to cache data

Parameters

  • length :

void writeCalibrationCache ( const void * cache , size_t length ) noexcept override

Write calibration cache.

Write a the calibration cache provided by TensorRT to a specified file

Parameters

  • cache : const void* - cache data

  • length : size_t - length of cache

operator nvinfer1::IInt8Calibrator* ( )

operator to cast to nvinfer1::IInt8Calibrator*

Convience function to convert to a IInt8Calibrator* to easily be assigned to the ptq_calibrator field in CompileSpec

Return

nvinfer1::IInt8Calibrator*