Skip to content

# Struct Input

- Defined in File torch_tensorrt.h

## Struct Documentation

struct torch_tensorrt :: Input

  A struct to hold an input range (used by TensorRT Optimization profile)

  This struct can either hold a single vector representing an input shape, signifying a static input shape or a set of three input shapes representing the min, optiminal and max input shapes allowed for the engine.

  Public Functions

  Input ( std :: vector < int64_t > shape , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object for static input size from vector, optional arguments allow the user to configure expected input shape tensor format. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)

    Parameters

      - shape – Input tensor shape

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( std :: vector < int64_t > shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object for static input size from vector, optional arguments allow the user to configure expected input shape tensor format.

    Parameters

      - shape – Input tensor shape

      - dtype – Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32)

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( c10 :: ArrayRef < int64_t > shape , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object for static input size from c10::ArrayRef (the type produced by tensor.sizes()), vector, optional arguments allow the user to configure expected input shape tensor format dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)

    Parameters

      - shape – Input tensor shape

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( c10 :: ArrayRef < int64_t > shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object for static input size from c10::ArrayRef (the type produced by tensor.sizes()), vector, optional arguments allow the user to configure expected input shape tensor format.

    Parameters

      - shape – Input tensor shape

      - dtype – Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32)

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( std :: vector < int64_t > min_shape , std :: vector < int64_t > opt_shape , std :: vector < int64_t > max_shape , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)

    Parameters

      - min_shape – Minimum shape for input tensor

      - opt_shape – Target optimization shape for input tensor

      - max_shape – Maximum acceptible shape for input tensor

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( std :: vector < int64_t > min_shape , std :: vector < int64_t > opt_shape , std :: vector < int64_t > max_shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object for a dynamic input size from vectors for minimum shape, optimal shape, and max shape supported sizes optional arguments allow the user to configure expected input shape tensor format.

    Parameters

      - min_shape – Minimum shape for input tensor

      - opt_shape – Target optimization shape for input tensor

      - max_shape – Maximum acceptible shape for input tensor

      - dtype – Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32)

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( c10 :: ArrayRef < int64_t > min_shape , c10 :: ArrayRef < int64_t > opt_shape , c10 :: ArrayRef < int64_t > max_shape , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)

    Parameters

      - min_shape – Minimum shape for input tensor

      - opt_shape – Target optimization shape for input tensor

      - max_shape – Maximum acceptible shape for input tensor

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( c10 :: ArrayRef < int64_t > min_shape , c10 :: ArrayRef < int64_t > opt_shape , c10 :: ArrayRef < int64_t > max_shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )

    Construct a new Input spec object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes.

    Parameters

      - min_shape – Minimum shape for input tensor

      - opt_shape – Target optimization shape for input tensor

      - max_shape – Maximum acceptible shape for input tensor

      - dtype – Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32)

      - format – Expected tensor format for the input (Defaults to contiguous)

  Input ( at :: Tensor tensor )

    Construct a new Input spec object using a torch tensor as an example The tensor’s shape, type and layout inform the spec’s values.

    Note: You cannot set dynamic shape through this method, you must use an alternative constructor

    Parameters

      tensor – Reference tensor to set shape, type and layout

  Public Members

  std :: vector < int64_t > min_shape

    Minimum acceptable input size into the engine.

  std :: vector < int64_t > opt_shape

    Optimal input size into the engine (size optimized for given kernels accept any size in min max range)

  std :: vector < int64_t > max_shape

    Maximum acceptable input size into the engine.

  std :: vector < int64_t > shape

    Input shape to be fed to TensorRT, in the event of a dynamic shape, -1’s will hold the place of variable dimensions

  DataType dtype

    Expected data type for the input.

  TensorFormat format

    Expected tensor format for the input.