# Plugins

This directory contains some tools and samples related to
TensorRT Plugins.

## Listing Registered Plugins

```
$ python list_plugins.py -h
usage: list_plugins.py [-h] [-p [PLUGINS [PLUGINS ...]]]

Script to list registered TensorRT plugins. Can optionally load custom plugin
libraries.

optional arguments:
  -h, --help            show this help message and exit
  -p [PLUGINS [PLUGINS ...]], --plugins [PLUGINS [PLUGINS ...]]
                        Path to a plugin (.so) library file. Accepts multiple
                        arguments.
```

Default `TRT_RELEASE` plugins:
```
$ python list_plugins.py   
2020-03-13 22:12:35 - __main__ - INFO - Registering plugins...
2020-03-13 22:12:35 - __main__ - INFO - Registered Plugin Names:
['RnRes2Br2bBr2c_TRT',
 'RnRes2Br1Br2c_TRT',
 'CgPersistentLSTMPlugin_TRT',
 'SingleStepLSTMPlugin',
 ...
 'SpecialSlice_TRT',
 'InstanceNormalization_TRT']
```

## Registering OSS Plugins 

Load and list multiple plugin libraries:
```
$ python list_plugins.py --plugins TensorRT/build/out/libnvinfer_plugin.so
2020-03-13 22:18:51 - __main__ - INFO - Loading plugin library: /mnt/TensorRT/build/out/libnvinfer_plugin.so
2020-03-13 22:18:51 - __main__ - INFO - Registering plugins...
2020-03-13 22:18:51 - __main__ - INFO - Registered Plugin Names:
['RnRes2Br2bBr2c_TRT',
 'RnRes2Br1Br2c_TRT',
 'CgPersistentLSTMPlugin_TRT',
 'SingleStepLSTMPlugin',
 'CustomEmbLayerNormPluginDynamic',    <------ OSS Plugins added
 'CustomFCPluginDynamic',              <------ OSS Plugins added
 'CustomGeluPluginDynamic',            <------ OSS Plugins added
 'CustomQKVToContextPluginDynamic',    <------ OSS Plugins added
 'CustomSkipLayerNormPluginDynamic',   <------ OSS Plugins added
 ...
 'SpecialSlice_TRT',
 'InstanceNormalization_TRT']
```

## Registering Custom Plugins

Same concept as above for OSS plugins, just giving an example to be extra clear:
```
$ pushd CustomIPluginV2/
$ make
g++ -g -std=c++11 -DNDEBUG -fPIC -MD -MP -I. -I/usr/local/cuda/include -I/usr/src/tensorrt/include -o CustomPlugin.o -c CustomPlugin.cpp
g++ -g -std=c++11 -DNDEBUG -shared -o CustomPlugin.so CustomPlugin.o -L/usr/local/cuda/lib64 -L/usr/src/tensorrt/lib  -lnvinfer -lcudart

$ popd
$ python list_plugins.py --plugins CustomIPluginV2/CustomPlugin.so
2020-03-13 22:16:06 - __main__ - INFO - Loading plugin library: CustomIPluginV2/CustomPlugin.so
Plugin attribute number: 5
2020-03-13 22:16:06 - __main__ - INFO - Registering plugins...
2020-03-13 22:16:06 - __main__ - INFO - Registered Plugin Names:
['RnRes2Br2bBr2c_TRT',
 'RnRes2Br1Br2c_TRT',
 'CgPersistentLSTMPlugin_TRT',
 'SingleStepLSTMPlugin',
 'CustomPlugin',           <------------- Custom plugin registered
 ...
 'SpecialSlice_TRT',
 'InstanceNormalization_TRT']
```
