FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  452.18         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.5677         	|167            	|  428.81         	|  94.833         	|
run_training_batch                 	|  0.37528        	|1000           	|  375.28         	|  82.994         	|
optimizer_step_with_closure_0      	|  0.37258        	|1000           	|  372.58         	|  82.396         	|
training_step_and_backward         	|  0.29332        	|1000           	|  293.32         	|  64.869         	|
model_forward                      	|  0.1703         	|1000           	|  170.3          	|  37.662         	|
training_step                      	|  0.17016        	|1000           	|  170.16         	|  37.631         	|
backward                           	|  0.12169        	|1000           	|  121.69         	|  26.912         	|
get_train_batch                    	|  0.014021       	|1167           	|  16.363         	|  3.6187         	|
fetch_next_train_batch             	|  0.013998       	|1167           	|  16.336         	|  3.6127         	|
zero_grad                          	|  0.0013038      	|1000           	|  1.3038         	|  0.28835        	|
on_train_batch_end                 	|  0.001216       	|1000           	|  1.216          	|  0.26891        	|
training_batch_to_device           	|  0.00031155     	|1000           	|  0.31155        	|  0.0689         	|
on_train_batch_start               	|  9.4851e-05     	|1000           	|  0.094851       	|  0.020977       	|
on_train_epoch_start               	|  0.00027044     	|167            	|  0.045164       	|  0.0099882      	|
on_train_epoch_end                 	|  0.000254       	|167            	|  0.042418       	|  0.0093809      	|
on_train_end                       	|  0.024847       	|1              	|  0.024847       	|  0.0054949      	|
on_pretrain_routine_start          	|  0.022717       	|1              	|  0.022717       	|  0.005024       	|
on_batch_start                     	|  2.0702e-05     	|1000           	|  0.020702       	|  0.0045784      	|
on_before_zero_grad                	|  1.5533e-05     	|1000           	|  0.015533       	|  0.0034352      	|
on_batch_end                       	|  1.4068e-05     	|1000           	|  0.014068       	|  0.0031112      	|
on_after_backward                  	|  1.174e-05      	|1000           	|  0.01174        	|  0.0025964      	|
on_before_optimizer_step           	|  1.122e-05      	|1000           	|  0.01122        	|  0.0024813      	|
on_before_backward                 	|  1.0968e-05     	|1000           	|  0.010968       	|  0.0024256      	|
configure_optimizers               	|  0.0086599      	|1              	|  0.0086599      	|  0.0019152      	|
training_step_end                  	|  6.3952e-06     	|1000           	|  0.0063952      	|  0.0014143      	|
on_epoch_start                     	|  1.0762e-05     	|167            	|  0.0017972      	|  0.00039746     	|
on_epoch_end                       	|  1.0325e-05     	|167            	|  0.0017242      	|  0.00038131     	|
on_train_start                     	|  0.00075408     	|1              	|  0.00075408     	|  0.00016677     	|
on_fit_end                         	|  2.139e-05      	|1              	|  2.139e-05      	|  4.7305e-06     	|
on_before_accelerator_backend_setup	|  1.9917e-05     	|1              	|  1.9917e-05     	|  4.4047e-06     	|
on_fit_start                       	|  1.7874e-05     	|1              	|  1.7874e-05     	|  3.9529e-06     	|
setup                              	|  1.623e-05      	|1              	|  1.623e-05      	|  3.5893e-06     	|
on_configure_sharded_model         	|  1.4066e-05     	|1              	|  1.4066e-05     	|  3.1107e-06     	|
teardown                           	|  1.0179e-05     	|1              	|  1.0179e-05     	|  2.2511e-06     	|
on_pretrain_routine_end            	|  9.878e-06      	|1              	|  9.878e-06      	|  2.1845e-06     	|
prepare_data                       	|  7.894e-06      	|1              	|  7.894e-06      	|  1.7458e-06     	|
configure_callbacks                	|  5.992e-06      	|1              	|  5.992e-06      	|  1.3252e-06     	|
on_train_dataloader                	|  4.799e-06      	|1              	|  4.799e-06      	|  1.0613e-06     	|
configure_sharded_model            	|  4.719e-06      	|1              	|  4.719e-06      	|  1.0436e-06     	|
on_val_dataloader                  	|  4.489e-06      	|1              	|  4.489e-06      	|  9.9275e-07     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1563.5         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.2234         	|167            	|  1540.3         	|  98.514         	|
run_training_batch                 	|  0.18665        	|8000           	|  1493.2         	|  95.501         	|
training_step_and_backward         	|  0.17663        	|8000           	|  1413.0         	|  90.374         	|
model_forward                      	|  0.094002       	|8000           	|  752.02         	|  48.097         	|
training_step                      	|  0.093868       	|8000           	|  750.94         	|  48.028         	|
backward                           	|  0.082445       	|8000           	|  659.56         	|  42.184         	|
optimizer_step_with_closure_0      	|  0.23407        	|1000           	|  234.07         	|  14.971         	|
get_train_batch                    	|  0.0033462      	|8167           	|  27.329         	|  1.7479         	|
fetch_next_train_batch             	|  0.0033288      	|8167           	|  27.186         	|  1.7388         	|
on_train_batch_end                 	|  0.0010824      	|8000           	|  8.6592         	|  0.55382        	|
training_batch_to_device           	|  0.00021558     	|8000           	|  1.7247         	|  0.1103         	|
zero_grad                          	|  0.0013148      	|1000           	|  1.3148         	|  0.084089       	|
on_train_batch_start               	|  9.0251e-05     	|8000           	|  0.722          	|  0.046178       	|
on_batch_start                     	|  1.4857e-05     	|8000           	|  0.11886        	|  0.0076018      	|
on_before_backward                 	|  1.3491e-05     	|8000           	|  0.10793        	|  0.006903       	|
on_batch_end                       	|  1.3187e-05     	|8000           	|  0.1055         	|  0.0067474      	|
on_after_backward                  	|  1.1509e-05     	|8000           	|  0.092071       	|  0.0058886      	|
training_step_end                  	|  6.1909e-06     	|8000           	|  0.049527       	|  0.0031676      	|
on_train_epoch_start               	|  0.00028986     	|167            	|  0.048406       	|  0.0030959      	|
on_train_epoch_end                 	|  0.00026848     	|167            	|  0.044836       	|  0.0028676      	|
on_train_end                       	|  0.025255       	|1              	|  0.025255       	|  0.0016153      	|
on_pretrain_routine_start          	|  0.022949       	|1              	|  0.022949       	|  0.0014678      	|
on_before_zero_grad                	|  1.6457e-05     	|1000           	|  0.016457       	|  0.0010526      	|
on_before_optimizer_step           	|  1.3473e-05     	|1000           	|  0.013473       	|  0.00086168     	|
configure_optimizers               	|  0.0089691      	|1              	|  0.0089691      	|  0.00057364     	|
on_epoch_end                       	|  1.3211e-05     	|167            	|  0.0022062      	|  0.00014111     	|
on_epoch_start                     	|  1.1185e-05     	|167            	|  0.0018678      	|  0.00011946     	|
on_train_start                     	|  0.00076157     	|1              	|  0.00076157     	|  4.8708e-05     	|
on_before_accelerator_backend_setup	|  2.4586e-05     	|1              	|  2.4586e-05     	|  1.5725e-06     	|
on_fit_end                         	|  2.2893e-05     	|1              	|  2.2893e-05     	|  1.4642e-06     	|
on_fit_start                       	|  1.8975e-05     	|1              	|  1.8975e-05     	|  1.2136e-06     	|
setup                              	|  1.4778e-05     	|1              	|  1.4778e-05     	|  9.4517e-07     	|
on_configure_sharded_model         	|  1.3725e-05     	|1              	|  1.3725e-05     	|  8.7781e-07     	|
on_pretrain_routine_end            	|  1.3075e-05     	|1              	|  1.3075e-05     	|  8.3625e-07     	|
teardown                           	|  9.909e-06      	|1              	|  9.909e-06      	|  6.3376e-07     	|
prepare_data                       	|  9.418e-06      	|1              	|  9.418e-06      	|  6.0235e-07     	|
configure_callbacks                	|  7.3831e-06     	|1              	|  7.3831e-06     	|  4.722e-07      	|
configure_sharded_model            	|  4.8792e-06     	|1              	|  4.8792e-06     	|  3.1206e-07     	|
on_val_dataloader                  	|  4.8601e-06     	|1              	|  4.8601e-06     	|  3.1084e-07     	|
on_train_dataloader                	|  4.1469e-06     	|1              	|  4.1469e-06     	|  2.6523e-07     	|
