VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1212.3         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.3059         	|500            	|  1153.0         	|  95.103         	|
run_training_batch                 	|  0.26651        	|1000           	|  266.51         	|  21.983         	|
optimizer_step_with_closure_0      	|  0.26288        	|1000           	|  262.88         	|  21.684         	|
training_step_and_backward         	|  0.20198        	|1000           	|  201.98         	|  16.66          	|
get_train_batch                    	|  0.1133         	|1500           	|  169.95         	|  14.019         	|
fetch_next_train_batch             	|  0.11326        	|1500           	|  169.9          	|  14.014         	|
backward                           	|  0.10345        	|1000           	|  103.45         	|  8.5333         	|
model_forward                      	|  0.097156       	|1000           	|  97.156         	|  8.014          	|
training_step                      	|  0.096949       	|1000           	|  96.949         	|  7.9969         	|
run_validate_evaluation            	|  29.434         	|1              	|  29.434         	|  2.4278         	|
evaluation_step_and_end            	|  0.083559       	|313            	|  26.154         	|  2.1573         	|
validation_step                    	|  0.083493       	|313            	|  26.133         	|  2.1556         	|
on_train_batch_end                 	|  0.0019489      	|1000           	|  1.9489         	|  0.16075        	|
zero_grad                          	|  0.0013348      	|1000           	|  1.3348         	|  0.1101         	|
get_validate_batch                 	|  0.0027435      	|314            	|  0.86145        	|  0.071057       	|
fetch_next_validate_batch          	|  0.0027266      	|314            	|  0.85614        	|  0.070619       	|
training_batch_to_device           	|  0.00066324     	|1000           	|  0.66324        	|  0.054708       	|
on_train_epoch_end                 	|  0.00031277     	|500            	|  0.15639        	|  0.012899       	|
on_train_epoch_start               	|  0.00029415     	|500            	|  0.14708        	|  0.012132       	|
on_train_batch_start               	|  0.00012723     	|1000           	|  0.12723        	|  0.010495       	|
evaluation_batch_to_device         	|  0.00020985     	|313            	|  0.065684       	|  0.0054179      	|
on_batch_start                     	|  4.4913e-05     	|1000           	|  0.044913       	|  0.0037046      	|
on_train_end                       	|  0.044155       	|1              	|  0.044155       	|  0.0036421      	|
on_pretrain_routine_start          	|  0.021908       	|1              	|  0.021908       	|  0.0018071      	|
on_before_zero_grad                	|  2.0528e-05     	|1000           	|  0.020528       	|  0.0016932      	|
on_after_backward                  	|  1.8509e-05     	|1000           	|  0.018509       	|  0.0015267      	|
on_batch_end                       	|  1.6722e-05     	|1000           	|  0.016722       	|  0.0013793      	|
on_before_backward                 	|  1.2687e-05     	|1000           	|  0.012687       	|  0.0010465      	|
on_before_optimizer_step           	|  1.2573e-05     	|1000           	|  0.012573       	|  0.0010371      	|
configure_optimizers               	|  0.010009       	|1              	|  0.010009       	|  0.0008256      	|
training_step_end                  	|  7.898e-06      	|1000           	|  0.007898       	|  0.00065147     	|
on_validation_batch_end            	|  2.4357e-05     	|313            	|  0.0076238      	|  0.00062885     	|
on_epoch_end                       	|  1.1025e-05     	|501            	|  0.0055235      	|  0.0004556      	|
on_epoch_start                     	|  1.0748e-05     	|501            	|  0.0053845      	|  0.00044414     	|
on_validation_model_eval           	|  0.0040566      	|1              	|  0.0040566      	|  0.00033461     	|
on_validation_batch_start          	|  1.2481e-05     	|313            	|  0.0039067      	|  0.00032225     	|
validation_step_end                	|  5.822e-06      	|313            	|  0.0018223      	|  0.00015031     	|
on_train_start                     	|  0.0010772      	|1              	|  0.0010772      	|  8.8855e-05     	|
on_validation_start                	|  0.00064829     	|1              	|  0.00064829     	|  5.3474e-05     	|
on_validation_end                  	|  0.00014254     	|1              	|  0.00014254     	|  1.1758e-05     	|
teardown                           	|  3.9448e-05     	|2              	|  7.8896e-05     	|  6.5078e-06     	|
setup                              	|  2.3875e-05     	|2              	|  4.7751e-05     	|  3.9387e-06     	|
on_fit_start                       	|  4.5208e-05     	|1              	|  4.5208e-05     	|  3.729e-06      	|
on_before_accelerator_backend_setup	|  2.2323e-05     	|2              	|  4.4646e-05     	|  3.6826e-06     	|
on_configure_sharded_model         	|  2.1975e-05     	|2              	|  4.3951e-05     	|  3.6253e-06     	|
on_validation_epoch_end            	|  3.4222e-05     	|1              	|  3.4222e-05     	|  2.8228e-06     	|
on_fit_end                         	|  2.9521e-05     	|1              	|  2.9521e-05     	|  2.435e-06      	|
on_val_dataloader                  	|  1.4704e-05     	|2              	|  2.9407e-05     	|  2.4257e-06     	|
prepare_data                       	|  7.7579e-06     	|2              	|  1.5516e-05     	|  1.2798e-06     	|
configure_callbacks                	|  7.5074e-06     	|2              	|  1.5015e-05     	|  1.2385e-06     	|
configure_sharded_model            	|  7.4543e-06     	|2              	|  1.4909e-05     	|  1.2297e-06     	|
on_pretrain_routine_end            	|  1.3895e-05     	|1              	|  1.3895e-05     	|  1.1462e-06     	|
on_validation_epoch_start          	|  9.1996e-06     	|1              	|  9.1996e-06     	|  7.5883e-07     	|
on_train_dataloader                	|  6.1598e-06     	|1              	|  6.1598e-06     	|  5.0809e-07     	|
