Skip to content

# TRTorch

## Ahead-of-time compilation of TorchScript / PyTorch JIT for NVIDIA GPUs

TRTorch is a compiler for PyTorch/TorchScript, targeting NVIDIA GPUs via NVIDIA’s TensorRT Deep Learning Optimizer and Runtime. Unlike PyTorch’s Just-In-Time (JIT) compiler, TRTorch is an Ahead-of-Time (AOT) compiler, meaning that before you deploy your TorchScript code, you go through an explicit compile step to convert a standard TorchScript program into an module targeting a TensorRT engine. TRTorch operates as a PyTorch extention and compiles modules that integrate into the JIT runtime seamlessly. After compilation using the optimized graph should feel no different than running a TorchScript module. You also have access to TensorRT’s suite of configurations at compile time, so you are able to specify operating precision (FP32/FP16/INT8) and other settings for your module.

More Information / System Architecture:

- GTC 2020 Talk

## Getting Started

- Installation

- Getting Started

- Post Training Quantization (PTQ)

- trtorchc

- Using TRTorch Directly From PyTorch

## Python API Documenation

- trtorch

## C++ API Documenation

- Namespace trtorch

## Contributor Documentation

- System Overview

- Writing Converters

- Useful Links for TRTorch Development

## Indices

- Index

- Search Page