FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  460.24         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.6186         	|167            	|  437.3          	|  95.015         	|
run_training_batch                 	|  0.38384        	|1000           	|  383.84         	|  83.399         	|
optimizer_step_with_closure_0      	|  0.3811         	|1000           	|  381.1          	|  82.806         	|
training_step_and_backward         	|  0.29916        	|1000           	|  299.16         	|  65.0           	|
model_forward                      	|  0.17313        	|1000           	|  173.13         	|  37.617         	|
training_step                      	|  0.17298        	|1000           	|  172.98         	|  37.585         	|
backward                           	|  0.1247         	|1000           	|  124.7          	|  27.094         	|
get_train_batch                    	|  0.013964       	|1167           	|  16.296         	|  3.5408         	|
fetch_next_train_batch             	|  0.013941       	|1167           	|  16.269         	|  3.5349         	|
zero_grad                          	|  0.0013055      	|1000           	|  1.3055         	|  0.28366        	|
on_train_batch_end                 	|  0.0012371      	|1000           	|  1.2371         	|  0.26879        	|
training_batch_to_device           	|  0.00031724     	|1000           	|  0.31724        	|  0.068929       	|
on_train_batch_start               	|  9.5022e-05     	|1000           	|  0.095022       	|  0.020646       	|
on_train_epoch_start               	|  0.00028084     	|167            	|  0.046901       	|  0.010191       	|
on_train_epoch_end                 	|  0.0002622      	|167            	|  0.043788       	|  0.0095141      	|
on_train_end                       	|  0.024889       	|1              	|  0.024889       	|  0.0054079      	|
on_pretrain_routine_start          	|  0.022671       	|1              	|  0.022671       	|  0.004926       	|
on_batch_start                     	|  2.1074e-05     	|1000           	|  0.021074       	|  0.0045789      	|
on_before_zero_grad                	|  1.6072e-05     	|1000           	|  0.016072       	|  0.0034921      	|
on_batch_end                       	|  1.4427e-05     	|1000           	|  0.014427       	|  0.0031346      	|
on_after_backward                  	|  1.21e-05       	|1000           	|  0.0121         	|  0.002629       	|
on_before_optimizer_step           	|  1.1654e-05     	|1000           	|  0.011654       	|  0.0025321      	|
on_before_backward                 	|  1.096e-05      	|1000           	|  0.01096        	|  0.0023814      	|
configure_optimizers               	|  0.0082432      	|1              	|  0.0082432      	|  0.0017911      	|
training_step_end                  	|  6.336e-06      	|1000           	|  0.006336       	|  0.0013767      	|
on_epoch_start                     	|  1.1032e-05     	|167            	|  0.0018424      	|  0.00040031     	|
on_epoch_end                       	|  1.0629e-05     	|167            	|  0.0017751      	|  0.00038569     	|
on_train_start                     	|  0.00073098     	|1              	|  0.00073098     	|  0.00015882     	|
on_fit_end                         	|  2.0178e-05     	|1              	|  2.0178e-05     	|  4.3842e-06     	|
on_before_accelerator_backend_setup	|  1.9866e-05     	|1              	|  1.9866e-05     	|  4.3164e-06     	|
on_fit_start                       	|  1.7072e-05     	|1              	|  1.7072e-05     	|  3.7094e-06     	|
setup                              	|  1.5629e-05     	|1              	|  1.5629e-05     	|  3.3958e-06     	|
on_configure_sharded_model         	|  1.4056e-05     	|1              	|  1.4056e-05     	|  3.054e-06      	|
teardown                           	|  1.068e-05      	|1              	|  1.068e-05      	|  2.3205e-06     	|
on_pretrain_routine_end            	|  1.0089e-05     	|1              	|  1.0089e-05     	|  2.1921e-06     	|
prepare_data                       	|  7.204e-06      	|1              	|  7.204e-06      	|  1.5653e-06     	|
configure_callbacks                	|  6.9831e-06     	|1              	|  6.9831e-06     	|  1.5173e-06     	|
on_val_dataloader                  	|  4.3979e-06     	|1              	|  4.3979e-06     	|  9.5557e-07     	|
on_train_dataloader                	|  4.158e-06      	|1              	|  4.158e-06      	|  9.0344e-07     	|
configure_sharded_model            	|  4.087e-06      	|1              	|  4.087e-06      	|  8.8801e-07     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1564.0         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.2254         	|167            	|  1540.6         	|  98.505         	|
run_training_batch                 	|  0.18668        	|8000           	|  1493.4         	|  95.486         	|
training_step_and_backward         	|  0.17643        	|8000           	|  1411.4         	|  90.244         	|
model_forward                      	|  0.093668       	|8000           	|  749.35         	|  47.911         	|
training_step                      	|  0.093533       	|8000           	|  748.27         	|  47.842         	|
backward                           	|  0.082577       	|8000           	|  660.62         	|  42.238         	|
optimizer_step_with_closure_0      	|  0.23589        	|1000           	|  235.89         	|  15.082         	|
get_train_batch                    	|  0.0033574      	|8167           	|  27.42          	|  1.7532         	|
fetch_next_train_batch             	|  0.0033405      	|8167           	|  27.282         	|  1.7443         	|
on_train_batch_end                 	|  0.0010769      	|8000           	|  8.6152         	|  0.55084        	|
training_batch_to_device           	|  0.00021762     	|8000           	|  1.741          	|  0.11131        	|
zero_grad                          	|  0.001332       	|1000           	|  1.332          	|  0.085166       	|
on_train_batch_start               	|  9.0144e-05     	|8000           	|  0.72115        	|  0.046109       	|
on_batch_start                     	|  1.5852e-05     	|8000           	|  0.12681        	|  0.0081082      	|
on_batch_end                       	|  1.3015e-05     	|8000           	|  0.10412        	|  0.0066572      	|
on_before_backward                 	|  1.2918e-05     	|8000           	|  0.10335        	|  0.0066077      	|
on_after_backward                  	|  1.1666e-05     	|8000           	|  0.093325       	|  0.005967       	|
training_step_end                  	|  6.9889e-06     	|8000           	|  0.055911       	|  0.0035749      	|
on_train_epoch_start               	|  0.00029064     	|167            	|  0.048536       	|  0.0031033      	|
on_train_epoch_end                 	|  0.00025706     	|167            	|  0.042929       	|  0.0027448      	|
on_train_end                       	|  0.025831       	|1              	|  0.025831       	|  0.0016516      	|
on_pretrain_routine_start          	|  0.022997       	|1              	|  0.022997       	|  0.0014704      	|
on_before_zero_grad                	|  1.6033e-05     	|1000           	|  0.016033       	|  0.0010251      	|
on_before_optimizer_step           	|  1.3247e-05     	|1000           	|  0.013247       	|  0.00084699     	|
configure_optimizers               	|  0.0082142      	|1              	|  0.0082142      	|  0.0005252      	|
on_epoch_start                     	|  1.0655e-05     	|167            	|  0.0017793      	|  0.00011376     	|
on_epoch_end                       	|  1.0121e-05     	|167            	|  0.0016903      	|  0.00010807     	|
on_train_start                     	|  0.00071691     	|1              	|  0.00071691     	|  4.5838e-05     	|
on_before_accelerator_backend_setup	|  2.2883e-05     	|1              	|  2.2883e-05     	|  1.4631e-06     	|
on_fit_end                         	|  2.2592e-05     	|1              	|  2.2592e-05     	|  1.4445e-06     	|
on_fit_start                       	|  1.629e-05      	|1              	|  1.629e-05      	|  1.0415e-06     	|
on_configure_sharded_model         	|  1.3946e-05     	|1              	|  1.3946e-05     	|  8.9168e-07     	|
setup                              	|  1.3725e-05     	|1              	|  1.3725e-05     	|  8.7755e-07     	|
on_pretrain_routine_end            	|  1.2363e-05     	|1              	|  1.2363e-05     	|  7.9047e-07     	|
teardown                           	|  1.0028e-05     	|1              	|  1.0028e-05     	|  6.4117e-07     	|
prepare_data                       	|  8.3649e-06     	|1              	|  8.3649e-06     	|  5.3483e-07     	|
configure_callbacks                	|  7.0229e-06     	|1              	|  7.0229e-06     	|  4.4903e-07     	|
on_val_dataloader                  	|  4.719e-06      	|1              	|  4.719e-06      	|  3.0172e-07     	|
configure_sharded_model            	|  4.3281e-06     	|1              	|  4.3281e-06     	|  2.7673e-07     	|
on_train_dataloader                	|  3.997e-06      	|1              	|  3.997e-06      	|  2.5556e-07     	|
