FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  451.96         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.5707         	|167            	|  429.3          	|  94.987         	|
run_training_batch                 	|  0.37584        	|1000           	|  375.84         	|  83.158         	|
optimizer_step_with_closure_0      	|  0.37316        	|1000           	|  373.16         	|  82.566         	|
training_step_and_backward         	|  0.29375        	|1000           	|  293.75         	|  64.995         	|
model_forward                      	|  0.17045        	|1000           	|  170.45         	|  37.714         	|
training_step                      	|  0.17031        	|1000           	|  170.31         	|  37.682         	|
backward                           	|  0.12196        	|1000           	|  121.96         	|  26.986         	|
get_train_batch                    	|  0.013974       	|1167           	|  16.308         	|  3.6082         	|
fetch_next_train_batch             	|  0.01395        	|1167           	|  16.279         	|  3.602          	|
zero_grad                          	|  0.0013053      	|1000           	|  1.3053         	|  0.2888         	|
on_train_batch_end                 	|  0.0012317      	|1000           	|  1.2317         	|  0.27253        	|
training_batch_to_device           	|  0.00031721     	|1000           	|  0.31721        	|  0.070185       	|
on_train_batch_start               	|  9.5601e-05     	|1000           	|  0.095601       	|  0.021153       	|
on_train_epoch_start               	|  0.00028092     	|167            	|  0.046914       	|  0.01038        	|
on_train_epoch_end                 	|  0.00026192     	|167            	|  0.04374        	|  0.009678       	|
on_train_end                       	|  0.025046       	|1              	|  0.025046       	|  0.0055417      	|
on_pretrain_routine_start          	|  0.022872       	|1              	|  0.022872       	|  0.0050607      	|
on_batch_start                     	|  2.0661e-05     	|1000           	|  0.020661       	|  0.0045714      	|
on_before_zero_grad                	|  1.5715e-05     	|1000           	|  0.015715       	|  0.0034771      	|
on_batch_end                       	|  1.381e-05      	|1000           	|  0.01381        	|  0.0030556      	|
on_after_backward                  	|  1.1848e-05     	|1000           	|  0.011848       	|  0.0026214      	|
on_before_optimizer_step           	|  1.1465e-05     	|1000           	|  0.011465       	|  0.0025367      	|
on_before_backward                 	|  1.0478e-05     	|1000           	|  0.010478       	|  0.0023183      	|
configure_optimizers               	|  0.0081375      	|1              	|  0.0081375      	|  0.0018005      	|
training_step_end                  	|  6.709e-06      	|1000           	|  0.006709       	|  0.0014844      	|
on_epoch_start                     	|  1.0668e-05     	|167            	|  0.0017816      	|  0.0003942      	|
on_epoch_end                       	|  1.0623e-05     	|167            	|  0.0017741      	|  0.00039254     	|
on_train_start                     	|  0.00069795     	|1              	|  0.00069795     	|  0.00015443     	|
on_before_accelerator_backend_setup	|  2.4335e-05     	|1              	|  2.4335e-05     	|  5.3844e-06     	|
on_fit_end                         	|  2.4114e-05     	|1              	|  2.4114e-05     	|  5.3355e-06     	|
on_fit_start                       	|  1.7352e-05     	|1              	|  1.7352e-05     	|  3.8393e-06     	|
setup                              	|  1.6962e-05     	|1              	|  1.6962e-05     	|  3.753e-06      	|
on_configure_sharded_model         	|  1.3505e-05     	|1              	|  1.3505e-05     	|  2.9881e-06     	|
teardown                           	|  1.053e-05      	|1              	|  1.053e-05      	|  2.3299e-06     	|
on_pretrain_routine_end            	|  9.8691e-06     	|1              	|  9.8691e-06     	|  2.1836e-06     	|
prepare_data                       	|  7.975e-06      	|1              	|  7.975e-06      	|  1.7646e-06     	|
configure_callbacks                	|  6.402e-06      	|1              	|  6.402e-06      	|  1.4165e-06     	|
configure_sharded_model            	|  5.089e-06      	|1              	|  5.089e-06      	|  1.126e-06      	|
on_val_dataloader                  	|  4.4679e-06     	|1              	|  4.4679e-06     	|  9.8857e-07     	|
on_train_dataloader                	|  4.3779e-06     	|1              	|  4.3779e-06     	|  9.6866e-07     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1574.2         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.287          	|167            	|  1550.9         	|  98.523         	|
run_training_batch                 	|  0.18796        	|8000           	|  1503.7         	|  95.52          	|
training_step_and_backward         	|  0.17777        	|8000           	|  1422.1         	|  90.341         	|
model_forward                      	|  0.093623       	|8000           	|  748.99         	|  47.579         	|
training_step                      	|  0.093489       	|8000           	|  747.91         	|  47.511         	|
backward                           	|  0.083963       	|8000           	|  671.7          	|  42.67          	|
optimizer_step_with_closure_0      	|  0.23619        	|1000           	|  236.19         	|  15.004         	|
get_train_batch                    	|  0.003342       	|8167           	|  27.294         	|  1.7338         	|
fetch_next_train_batch             	|  0.0033245      	|8167           	|  27.151         	|  1.7248         	|
on_train_batch_end                 	|  0.0010926      	|8000           	|  8.7406         	|  0.55524        	|
training_batch_to_device           	|  0.00022008     	|8000           	|  1.7606         	|  0.11184        	|
zero_grad                          	|  0.0013088      	|1000           	|  1.3088         	|  0.083143       	|
on_train_batch_start               	|  9.2471e-05     	|8000           	|  0.73977        	|  0.046994       	|
on_batch_start                     	|  1.4923e-05     	|8000           	|  0.11938        	|  0.0075837      	|
on_batch_end                       	|  1.3283e-05     	|8000           	|  0.10626        	|  0.0067502      	|
on_before_backward                 	|  1.2831e-05     	|8000           	|  0.10265        	|  0.0065209      	|
on_after_backward                  	|  1.1403e-05     	|8000           	|  0.091227       	|  0.0057952      	|
on_train_epoch_start               	|  0.00028982     	|167            	|  0.0484         	|  0.0030746      	|
training_step_end                  	|  6.0089e-06     	|8000           	|  0.048071       	|  0.0030537      	|
on_train_epoch_end                 	|  0.00026318     	|167            	|  0.043952       	|  0.002792       	|
on_train_end                       	|  0.026089       	|1              	|  0.026089       	|  0.0016573      	|
on_pretrain_routine_start          	|  0.02245        	|1              	|  0.02245        	|  0.0014261      	|
on_before_zero_grad                	|  1.5567e-05     	|1000           	|  0.015567       	|  0.00098889     	|
on_before_optimizer_step           	|  1.2959e-05     	|1000           	|  0.012959       	|  0.00082321     	|
configure_optimizers               	|  0.0085191      	|1              	|  0.0085191      	|  0.00054118     	|
on_epoch_end                       	|  1.0425e-05     	|167            	|  0.001741       	|  0.0001106      	|
on_epoch_start                     	|  1.0317e-05     	|167            	|  0.0017229      	|  0.00010944     	|
on_train_start                     	|  0.00073505     	|1              	|  0.00073505     	|  4.6694e-05     	|
on_fit_end                         	|  2.2823e-05     	|1              	|  2.2823e-05     	|  1.4498e-06     	|
on_before_accelerator_backend_setup	|  2.0669e-05     	|1              	|  2.0669e-05     	|  1.313e-06      	|
on_fit_start                       	|  1.7002e-05     	|1              	|  1.7002e-05     	|  1.0801e-06     	|
on_configure_sharded_model         	|  1.3676e-05     	|1              	|  1.3676e-05     	|  8.6877e-07     	|
setup                              	|  1.3465e-05     	|1              	|  1.3465e-05     	|  8.5537e-07     	|
teardown                           	|  1.2444e-05     	|1              	|  1.2444e-05     	|  7.9051e-07     	|
on_pretrain_routine_end            	|  1.2113e-05     	|1              	|  1.2113e-05     	|  7.6948e-07     	|
prepare_data                       	|  8.1661e-06     	|1              	|  8.1661e-06     	|  5.1875e-07     	|
configure_callbacks                	|  6.3318e-06     	|1              	|  6.3318e-06     	|  4.0223e-07     	|
on_val_dataloader                  	|  4.8899e-06     	|1              	|  4.8899e-06     	|  3.1063e-07     	|
configure_sharded_model            	|  4.3479e-06     	|1              	|  4.3479e-06     	|  2.762e-07      	|
on_train_dataloader                	|  4.207e-06      	|1              	|  4.207e-06      	|  2.6725e-07     	|
