
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/_rendered_examples/dynamo/cross_runtime_compilation_for_windows.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_tutorials__rendered_examples_dynamo_cross_runtime_compilation_for_windows.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tutorials__rendered_examples_dynamo_cross_runtime_compilation_for_windows.py:


.. _resnet_cross_runtime_compilation_for_windows_example:

cross runtime compilation limitations:
The cross compile and saved model can only be loaded in Windows, it can no longer be loaded in Linux
The cross compile and saved model can only be loaded in the same Compute Capability as the Linux which it was cross compiled
(for example, if the model was cross compiled in Linux with GeForceRTX 4080 which has Compute Capability of 8.9,
It cannot be loaded in Windows with GeForceRTX 3080 which has Compute Capability of 8.6)

Cross runtime compilation for windows example
======================================================

Compile and save the Resnet Model using Torch-TensorRT in Linux:

python examples/dynamo/cross_runtime_compilation_for_windows.py --path trt_resnet.ep

Load the Resnet Model saved in Windows:

python examples/dynamo/cross_runtime_compilation_for_windows.py --path trt_resnet.ep --load True

.. GENERATED FROM PYTHON SOURCE LINES 24-26

Imports and Model Definition
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. GENERATED FROM PYTHON SOURCE LINES 26-53

.. code-block:: python


    import argparse
    import platform

    import torch
    import torch_tensorrt as torchtrt
    import torchvision.models as models

    PARSER = argparse.ArgumentParser(
        description="Cross runtime comilation for windows example: Resnet Model"
    )
    PARSER.add_argument(
        "--load", default=False, type=bool, required=False, help="Load the model in Windows"
    )
    PARSER.add_argument(
        "--path",
        type=str,
        required=True,
        help="Path to the saved model file",
    )

    args = PARSER.parse_args()
    torch.manual_seed(0)
    model = models.resnet18().eval().cuda()
    input = torch.rand((1, 3, 224, 224)).to("cuda")
    inputs = [input]


.. GENERATED FROM PYTHON SOURCE LINES 54-57

According to the argument, it is either cross compile and save resnet model for windows in Linux
or load the saved resnet model in Windows
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. GENERATED FROM PYTHON SOURCE LINES 57-83

.. code-block:: python

    if args.load:
        # load the saved model in Windows
        if platform.system() != "Windows" or platform.machine() != "AMD64":
            raise ValueError(
                "cross runtime compiled model for windows can only be loaded in Windows system"
            )
        loaded_model = torchtrt.load_cross_compiled_exported_program(args.path).module()
        print(f"model has been successfully loaded from ${args.path}")
        # inference
        trt_output = loaded_model(input)
        print(f"inference result: {trt_output}")
    else:
        if platform.system() != "Linux" or platform.architecture()[0] != "64bit":
            raise ValueError(
                "cross runtime compiled model for windows can only be compiled in Linux system"
            )
        compile_spec = {
            "debug": True,
            "min_block_size": 1,
        }
        torchtrt.cross_compile_for_windows(
            model, file_path=args.path, inputs=inputs, **compile_spec
        )
        print(
            f"model has been successfully cross compiled and saved in Linux to {args.path}"
        )


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.000 seconds)


.. _sphx_glr_download_tutorials__rendered_examples_dynamo_cross_runtime_compilation_for_windows.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example




    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: cross_runtime_compilation_for_windows.py <cross_runtime_compilation_for_windows.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: cross_runtime_compilation_for_windows.ipynb <cross_runtime_compilation_for_windows.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
