VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  660.34         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  3.6608         	|167            	|  611.36         	|  92.582         	|
run_training_batch                 	|  0.25719        	|1000           	|  257.19         	|  38.948         	|
optimizer_step_with_closure_0      	|  0.25514        	|1000           	|  255.14         	|  38.637         	|
training_step_and_backward         	|  0.18766        	|1000           	|  187.66         	|  28.418         	|
model_forward                      	|  0.1009         	|1000           	|  100.9          	|  15.28          	|
training_step                      	|  0.10072        	|1000           	|  100.72         	|  15.252         	|
backward                           	|  0.085262       	|1000           	|  85.262         	|  12.912         	|
get_train_batch                    	|  0.065656       	|1167           	|  76.621         	|  11.603         	|
fetch_next_train_batch             	|  0.065627       	|1167           	|  76.587         	|  11.598         	|
run_validate_evaluation            	|  16.648         	|1              	|  16.648         	|  2.5212         	|
evaluation_step_and_end            	|  0.10565        	|125            	|  13.207         	|  2.0            	|
validation_step                    	|  0.10558        	|125            	|  13.198         	|  1.9986         	|
on_train_batch_end                 	|  0.0016216      	|1000           	|  1.6216         	|  0.24557        	|
zero_grad                          	|  0.0014685      	|1000           	|  1.4685         	|  0.22239        	|
get_validate_batch                 	|  0.0078388      	|126            	|  0.98769        	|  0.14957        	|
fetch_next_validate_batch          	|  0.0078211      	|126            	|  0.98546        	|  0.14923        	|
training_batch_to_device           	|  0.0004778      	|1000           	|  0.4778         	|  0.072356       	|
on_train_batch_start               	|  9.7945e-05     	|1000           	|  0.097945       	|  0.014832       	|
on_train_epoch_start               	|  0.00032657     	|167            	|  0.054537       	|  0.0082589      	|
on_train_epoch_end                 	|  0.00032216     	|167            	|  0.0538         	|  0.0081473      	|
on_train_end                       	|  0.042646       	|1              	|  0.042646       	|  0.0064581      	|
evaluation_batch_to_device         	|  0.00024148     	|125            	|  0.030185       	|  0.0045712      	|
on_batch_start                     	|  2.7061e-05     	|1000           	|  0.027061       	|  0.004098       	|
on_pretrain_routine_start          	|  0.021307       	|1              	|  0.021307       	|  0.0032266      	|
on_after_backward                  	|  1.7863e-05     	|1000           	|  0.017863       	|  0.0027052      	|
on_before_zero_grad                	|  1.5664e-05     	|1000           	|  0.015664       	|  0.0023721      	|
on_batch_end                       	|  1.5187e-05     	|1000           	|  0.015187       	|  0.0022999      	|
on_before_backward                 	|  1.1762e-05     	|1000           	|  0.011762       	|  0.0017812      	|
on_before_optimizer_step           	|  1.1562e-05     	|1000           	|  0.011562       	|  0.001751       	|
configure_optimizers               	|  0.010047       	|1              	|  0.010047       	|  0.0015215      	|
training_step_end                  	|  6.906e-06      	|1000           	|  0.006906       	|  0.0010458      	|
on_validation_model_eval           	|  0.0046205      	|1              	|  0.0046205      	|  0.00069971     	|
on_validation_batch_end            	|  2.8299e-05     	|125            	|  0.0035373      	|  0.00053568     	|
on_epoch_end                       	|  1.0948e-05     	|168            	|  0.0018393      	|  0.00027854     	|
on_epoch_start                     	|  1.0503e-05     	|168            	|  0.0017646      	|  0.00026722     	|
on_train_start                     	|  0.0017192      	|1              	|  0.0017192      	|  0.00026035     	|
on_validation_batch_start          	|  1.3291e-05     	|125            	|  0.0016614      	|  0.00025159     	|
validation_step_end                	|  6.3044e-06     	|125            	|  0.00078805     	|  0.00011934     	|
on_validation_start                	|  0.00060187     	|1              	|  0.00060187     	|  9.1146e-05     	|
on_validation_end                  	|  0.00014869     	|1              	|  0.00014869     	|  2.2517e-05     	|
on_fit_end                         	|  5.4516e-05     	|1              	|  5.4516e-05     	|  8.2557e-06     	|
setup                              	|  2.6343e-05     	|2              	|  5.2687e-05     	|  7.9787e-06     	|
on_before_accelerator_backend_setup	|  2.4295e-05     	|2              	|  4.8591e-05     	|  7.3584e-06     	|
teardown                           	|  2.4236e-05     	|2              	|  4.8472e-05     	|  7.3404e-06     	|
on_configure_sharded_model         	|  2.2081e-05     	|2              	|  4.4161e-05     	|  6.6877e-06     	|
on_fit_start                       	|  4.3316e-05     	|1              	|  4.3316e-05     	|  6.5596e-06     	|
on_validation_epoch_end            	|  2.8325e-05     	|1              	|  2.8325e-05     	|  4.2895e-06     	|
on_val_dataloader                  	|  1.2037e-05     	|2              	|  2.4075e-05     	|  3.6458e-06     	|
configure_callbacks                	|  9.3868e-06     	|2              	|  1.8774e-05     	|  2.843e-06      	|
configure_sharded_model            	|  7.4003e-06     	|2              	|  1.4801e-05     	|  2.2414e-06     	|
prepare_data                       	|  7.2811e-06     	|2              	|  1.4562e-05     	|  2.2052e-06     	|
on_pretrain_routine_end            	|  1.1528e-05     	|1              	|  1.1528e-05     	|  1.7457e-06     	|
on_validation_epoch_start          	|  1.0183e-05     	|1              	|  1.0183e-05     	|  1.5421e-06     	|
on_train_dataloader                	|  6.387e-06      	|1              	|  6.387e-06      	|  9.6723e-07     	|
