FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1758.1         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  3.4697         	|500            	|  1734.9         	|  98.677         	|
run_training_batch                 	|  0.20596        	|8000           	|  1647.7         	|  93.721         	|
training_step_and_backward         	|  0.19451        	|8000           	|  1556.1         	|  88.509         	|
model_forward                      	|  0.10945        	|8000           	|  875.62         	|  49.805         	|
training_step                      	|  0.10932        	|8000           	|  874.54         	|  49.743         	|
backward                           	|  0.084873       	|8000           	|  678.98         	|  38.62          	|
optimizer_step_with_closure_0      	|  0.25186        	|1000           	|  251.86         	|  14.326         	|
get_train_batch                    	|  0.0067639      	|8500           	|  57.493         	|  3.2702         	|
fetch_next_train_batch             	|  0.0067447      	|8500           	|  57.33          	|  3.2609         	|
on_train_batch_end                 	|  0.0010913      	|8000           	|  8.7305         	|  0.49659        	|
training_batch_to_device           	|  0.00023576     	|8000           	|  1.8861         	|  0.10728        	|
zero_grad                          	|  0.0013446      	|1000           	|  1.3446         	|  0.076479       	|
on_train_batch_start               	|  9.3646e-05     	|8000           	|  0.74917        	|  0.042612       	|
on_batch_start                     	|  1.7396e-05     	|8000           	|  0.13917        	|  0.0079158      	|
on_train_epoch_start               	|  0.00027616     	|500            	|  0.13808        	|  0.0078539      	|
on_train_epoch_end                 	|  0.00024836     	|500            	|  0.12418        	|  0.0070632      	|
on_before_backward                 	|  1.3335e-05     	|8000           	|  0.10668        	|  0.0060678      	|
on_batch_end                       	|  1.332e-05      	|8000           	|  0.10656        	|  0.0060611      	|
on_after_backward                  	|  1.1523e-05     	|8000           	|  0.092187       	|  0.0052435      	|
training_step_end                  	|  6.0669e-06     	|8000           	|  0.048535       	|  0.0027606      	|
on_train_end                       	|  0.024096       	|1              	|  0.024096       	|  0.0013706      	|
on_pretrain_routine_start          	|  0.022107       	|1              	|  0.022107       	|  0.0012574      	|
on_before_zero_grad                	|  2.0551e-05     	|1000           	|  0.020551       	|  0.0011689      	|
on_before_optimizer_step           	|  1.4858e-05     	|1000           	|  0.014858       	|  0.00084509     	|
configure_optimizers               	|  0.0086379      	|1              	|  0.0086379      	|  0.00049132     	|
on_epoch_end                       	|  1.1078e-05     	|500            	|  0.0055391      	|  0.00031506     	|
on_epoch_start                     	|  1.0973e-05     	|500            	|  0.0054865      	|  0.00031207     	|
on_train_start                     	|  0.00072659     	|1              	|  0.00072659     	|  4.1328e-05     	|
on_configure_sharded_model         	|  2.666e-05      	|1              	|  2.666e-05      	|  1.5164e-06     	|
on_fit_end                         	|  2.3323e-05     	|1              	|  2.3323e-05     	|  1.3266e-06     	|
setup                              	|  2.2362e-05     	|1              	|  2.2362e-05     	|  1.2719e-06     	|
on_before_accelerator_backend_setup	|  2.2342e-05     	|1              	|  2.2342e-05     	|  1.2708e-06     	|
on_fit_start                       	|  1.9707e-05     	|1              	|  1.9707e-05     	|  1.1209e-06     	|
teardown                           	|  1.025e-05      	|1              	|  1.025e-05      	|  5.8301e-07     	|
on_pretrain_routine_end            	|  9.9691e-06     	|1              	|  9.9691e-06     	|  5.6703e-07     	|
prepare_data                       	|  8.225e-06      	|1              	|  8.225e-06      	|  4.6783e-07     	|
configure_callbacks                	|  6.5421e-06     	|1              	|  6.5421e-06     	|  3.7211e-07     	|
on_val_dataloader                  	|  5.611e-06      	|1              	|  5.611e-06      	|  3.1915e-07     	|
configure_sharded_model            	|  5.0899e-06     	|1              	|  5.0899e-06     	|  2.8951e-07     	|
on_train_dataloader                	|  4.207e-06      	|1              	|  4.207e-06      	|  2.3929e-07     	|
