FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  387.59         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  5.7744         	|63             	|  363.79         	|  93.86          	|
run_training_batch                 	|  0.33724        	|1000           	|  337.24         	|  87.01          	|
optimizer_step_with_closure_0      	|  0.33462        	|1000           	|  334.62         	|  86.336         	|
training_step_and_backward         	|  0.25054        	|1000           	|  250.54         	|  64.641         	|
model_forward                      	|  0.15547        	|1000           	|  155.47         	|  40.113         	|
training_step                      	|  0.15533        	|1000           	|  155.33         	|  40.077         	|
backward                           	|  0.093743       	|1000           	|  93.743         	|  24.186         	|
get_train_batch                    	|  0.0080987      	|1063           	|  8.6089         	|  2.2212         	|
fetch_next_train_batch             	|  0.008079       	|1063           	|  8.588          	|  2.2158         	|
zero_grad                          	|  0.001297       	|1000           	|  1.297          	|  0.33464        	|
on_train_batch_end                 	|  0.0011894      	|1000           	|  1.1894         	|  0.30688        	|
training_batch_to_device           	|  0.0002287      	|1000           	|  0.2287         	|  0.059005       	|
on_train_batch_start               	|  9.0328e-05     	|1000           	|  0.090328       	|  0.023305       	|
on_train_end                       	|  0.025295       	|1              	|  0.025295       	|  0.0065262      	|
on_pretrain_routine_start          	|  0.022709       	|1              	|  0.022709       	|  0.0058591      	|
on_train_epoch_end                 	|  0.0002935      	|63             	|  0.01849        	|  0.0047707      	|
on_train_epoch_start               	|  0.00028114     	|63             	|  0.017712       	|  0.0045698      	|
on_batch_start                     	|  1.6163e-05     	|1000           	|  0.016163       	|  0.0041703      	|
on_batch_end                       	|  1.3882e-05     	|1000           	|  0.013882       	|  0.0035817      	|
on_before_zero_grad                	|  1.3251e-05     	|1000           	|  0.013251       	|  0.0034188      	|
on_after_backward                  	|  1.2576e-05     	|1000           	|  0.012576       	|  0.0032448      	|
on_before_optimizer_step           	|  1.0999e-05     	|1000           	|  0.010999       	|  0.0028379      	|
on_before_backward                 	|  1.0427e-05     	|1000           	|  0.010427       	|  0.0026902      	|
configure_optimizers               	|  0.0085046      	|1              	|  0.0085046      	|  0.0021942      	|
training_step_end                  	|  5.7113e-06     	|1000           	|  0.0057113      	|  0.0014735      	|
on_train_start                     	|  0.00074127     	|1              	|  0.00074127     	|  0.00019125     	|
on_epoch_start                     	|  1.0435e-05     	|63             	|  0.00065739     	|  0.00016961     	|
on_epoch_end                       	|  1.0319e-05     	|63             	|  0.0006501      	|  0.00016773     	|
on_fit_end                         	|  2.093e-05      	|1              	|  2.093e-05      	|  5.4001e-06     	|
on_before_accelerator_backend_setup	|  2.0669e-05     	|1              	|  2.0669e-05     	|  5.3328e-06     	|
on_fit_start                       	|  1.8104e-05     	|1              	|  1.8104e-05     	|  4.671e-06      	|
on_configure_sharded_model         	|  1.3705e-05     	|1              	|  1.3705e-05     	|  3.536e-06      	|
setup                              	|  1.3014e-05     	|1              	|  1.3014e-05     	|  3.3577e-06     	|
teardown                           	|  1.0119e-05     	|1              	|  1.0119e-05     	|  2.6108e-06     	|
on_pretrain_routine_end            	|  9.838e-06      	|1              	|  9.838e-06      	|  2.5383e-06     	|
prepare_data                       	|  7.214e-06      	|1              	|  7.214e-06      	|  1.8613e-06     	|
configure_sharded_model            	|  4.8181e-06     	|1              	|  4.8181e-06     	|  1.2431e-06     	|
configure_callbacks                	|  4.679e-06      	|1              	|  4.679e-06      	|  1.2072e-06     	|
on_train_dataloader                	|  4.439e-06      	|1              	|  4.439e-06      	|  1.1453e-06     	|
on_val_dataloader                  	|  4.2981e-06     	|1              	|  4.2981e-06     	|  1.1089e-06     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1554.9         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  24.315         	|63             	|  1531.8         	|  98.513         	|
run_training_batch                 	|  0.18712        	|8000           	|  1497.0         	|  96.271         	|
training_step_and_backward         	|  0.17699        	|8000           	|  1415.9         	|  91.06          	|
model_forward                      	|  0.092263       	|8000           	|  738.1          	|  47.468         	|
training_step                      	|  0.09213        	|8000           	|  737.04         	|  47.4           	|
backward                           	|  0.084545       	|8000           	|  676.36         	|  43.497         	|
optimizer_step_with_closure_0      	|  0.23547        	|1000           	|  235.47         	|  15.143         	|
get_train_batch                    	|  0.0022547      	|8063           	|  18.18          	|  1.1692         	|
fetch_next_train_batch             	|  0.0022381      	|8063           	|  18.046         	|  1.1605         	|
on_train_batch_end                 	|  0.0010714      	|8000           	|  8.5709         	|  0.5512         	|
training_batch_to_device           	|  0.00021226     	|8000           	|  1.6981         	|  0.1092         	|
zero_grad                          	|  0.0013381      	|1000           	|  1.3381         	|  0.086055       	|
on_train_batch_start               	|  9.0133e-05     	|8000           	|  0.72107        	|  0.046372       	|
on_batch_start                     	|  1.4281e-05     	|8000           	|  0.11425        	|  0.0073475      	|
on_batch_end                       	|  1.2861e-05     	|8000           	|  0.10289        	|  0.0066168      	|
on_before_backward                 	|  1.2553e-05     	|8000           	|  0.10043        	|  0.0064585      	|
on_after_backward                  	|  1.1249e-05     	|8000           	|  0.089995       	|  0.0057877      	|
training_step_end                  	|  5.4052e-06     	|8000           	|  0.043242       	|  0.0027809      	|
on_train_end                       	|  0.02553        	|1              	|  0.02553        	|  0.0016419      	|
on_pretrain_routine_start          	|  0.023166       	|1              	|  0.023166       	|  0.0014898      	|
on_train_epoch_start               	|  0.00029749     	|63             	|  0.018742       	|  0.0012053      	|
on_train_epoch_end                 	|  0.00026828     	|63             	|  0.016902       	|  0.001087       	|
on_before_zero_grad                	|  1.4791e-05     	|1000           	|  0.014791       	|  0.00095119     	|
on_before_optimizer_step           	|  1.2213e-05     	|1000           	|  0.012213       	|  0.0007854      	|
configure_optimizers               	|  0.0077136      	|1              	|  0.0077136      	|  0.00049607     	|
on_train_start                     	|  0.00069778     	|1              	|  0.00069778     	|  4.4875e-05     	|
on_epoch_start                     	|  1.0972e-05     	|63             	|  0.00069126     	|  4.4456e-05     	|
on_epoch_end                       	|  9.8547e-06     	|63             	|  0.00062085     	|  3.9927e-05     	|
on_fit_end                         	|  2.0238e-05     	|1              	|  2.0238e-05     	|  1.3015e-06     	|
on_before_accelerator_backend_setup	|  1.9516e-05     	|1              	|  1.9516e-05     	|  1.2551e-06     	|
on_fit_start                       	|  1.595e-05      	|1              	|  1.595e-05      	|  1.0257e-06     	|
on_configure_sharded_model         	|  1.4437e-05     	|1              	|  1.4437e-05     	|  9.2846e-07     	|
setup                              	|  1.2543e-05     	|1              	|  1.2543e-05     	|  8.0664e-07     	|
on_pretrain_routine_end            	|  1.1822e-05     	|1              	|  1.1822e-05     	|  7.6028e-07     	|
teardown                           	|  9.859e-06      	|1              	|  9.859e-06      	|  6.3404e-07     	|
prepare_data                       	|  7.363e-06      	|1              	|  7.363e-06      	|  4.7352e-07     	|
configure_callbacks                	|  6.3919e-06     	|1              	|  6.3919e-06     	|  4.1107e-07     	|
configure_sharded_model            	|  4.679e-06      	|1              	|  4.679e-06      	|  3.0091e-07     	|
on_val_dataloader                  	|  4.458e-06      	|1              	|  4.458e-06      	|  2.867e-07      	|
on_train_dataloader                	|  4.3879e-06     	|1              	|  4.3879e-06     	|  2.8219e-07     	|
