FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1695.0         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  26.538         	|63             	|  1671.9         	|  98.639         	|
run_training_batch                 	|  0.20462        	|8000           	|  1637.0         	|  96.578         	|
training_step_and_backward         	|  0.19355        	|8000           	|  1548.4         	|  91.355         	|
model_forward                      	|  0.10757        	|8000           	|  860.58         	|  50.773         	|
training_step                      	|  0.10744        	|8000           	|  859.52         	|  50.71          	|
backward                           	|  0.085797       	|8000           	|  686.37         	|  40.495         	|
optimizer_step_with_closure_0      	|  0.25234        	|1000           	|  252.34         	|  14.888         	|
get_train_batch                    	|  0.0022517      	|8063           	|  18.155         	|  1.0711         	|
fetch_next_train_batch             	|  0.0022345      	|8063           	|  18.017         	|  1.063          	|
on_train_batch_end                 	|  0.0010871      	|8000           	|  8.6971         	|  0.51311        	|
training_batch_to_device           	|  0.00021491     	|8000           	|  1.7193         	|  0.10144        	|
zero_grad                          	|  0.0013296      	|1000           	|  1.3296         	|  0.078447       	|
on_train_batch_start               	|  8.9883e-05     	|8000           	|  0.71907        	|  0.042424       	|
on_batch_start                     	|  1.4657e-05     	|8000           	|  0.11725        	|  0.0069177      	|
on_before_backward                 	|  1.2999e-05     	|8000           	|  0.10399        	|  0.0061352      	|
on_batch_end                       	|  1.2822e-05     	|8000           	|  0.10258        	|  0.0060519      	|
on_after_backward                  	|  1.1585e-05     	|8000           	|  0.092677       	|  0.0054678      	|
training_step_end                  	|  5.4476e-06     	|8000           	|  0.04358        	|  0.0025712      	|
on_train_end                       	|  0.025394       	|1              	|  0.025394       	|  0.0014982      	|
on_pretrain_routine_start          	|  0.022481       	|1              	|  0.022481       	|  0.0013263      	|
on_train_epoch_start               	|  0.00029306     	|63             	|  0.018463       	|  0.0010893      	|
on_train_epoch_end                 	|  0.0002874      	|63             	|  0.018106       	|  0.0010682      	|
on_before_zero_grad                	|  1.467e-05      	|1000           	|  0.01467        	|  0.0008655      	|
on_before_optimizer_step           	|  1.1312e-05     	|1000           	|  0.011312       	|  0.00066736     	|
configure_optimizers               	|  0.0086026      	|1              	|  0.0086026      	|  0.00050754     	|
on_train_start                     	|  0.00078275     	|1              	|  0.00078275     	|  4.6181e-05     	|
on_epoch_start                     	|  1.035e-05      	|63             	|  0.00065208     	|  3.8471e-05     	|
on_epoch_end                       	|  1.0345e-05     	|63             	|  0.00065175     	|  3.8452e-05     	|
on_before_accelerator_backend_setup	|  2.3815e-05     	|1              	|  2.3815e-05     	|  1.4051e-06     	|
setup                              	|  2.3143e-05     	|1              	|  2.3143e-05     	|  1.3654e-06     	|
on_fit_end                         	|  2.144e-05      	|1              	|  2.144e-05      	|  1.2649e-06     	|
on_configure_sharded_model         	|  1.8254e-05     	|1              	|  1.8254e-05     	|  1.077e-06      	|
on_fit_start                       	|  1.7232e-05     	|1              	|  1.7232e-05     	|  1.0167e-06     	|
prepare_data                       	|  1.063e-05      	|1              	|  1.063e-05      	|  6.2715e-07     	|
teardown                           	|  1.0469e-05     	|1              	|  1.0469e-05     	|  6.1766e-07     	|
on_pretrain_routine_end            	|  1.0259e-05     	|1              	|  1.0259e-05     	|  6.0527e-07     	|
configure_callbacks                	|  5.831e-06      	|1              	|  5.831e-06      	|  3.4402e-07     	|
configure_sharded_model            	|  5.069e-06      	|1              	|  5.069e-06      	|  2.9906e-07     	|
on_val_dataloader                  	|  4.8201e-06     	|1              	|  4.8201e-06     	|  2.8438e-07     	|
on_train_dataloader                	|  4.3879e-06     	|1              	|  4.3879e-06     	|  2.5888e-07     	|
