VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  578.78         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  8.0586         	|63             	|  507.69         	|  87.718         	|
run_training_batch                 	|  0.37096        	|1000           	|  370.96         	|  64.095         	|
optimizer_step_with_closure_0      	|  0.36826        	|1000           	|  368.26         	|  63.628         	|
training_step_and_backward         	|  0.26495        	|1000           	|  264.95         	|  45.778         	|
model_forward                      	|  0.13677        	|1000           	|  136.77         	|  23.631         	|
training_step                      	|  0.1366         	|1000           	|  136.6          	|  23.601         	|
backward                           	|  0.12675        	|1000           	|  126.75         	|  21.9           	|
get_train_batch                    	|  0.029463       	|1063           	|  31.319         	|  5.4113         	|
fetch_next_train_batch             	|  0.029439       	|1063           	|  31.294         	|  5.4069         	|
run_validate_evaluation            	|  30.95          	|1              	|  30.95          	|  5.3475         	|
evaluation_step_and_end            	|  0.22142        	|125            	|  27.677         	|  4.782          	|
validation_step                    	|  0.22131        	|125            	|  27.664         	|  4.7797         	|
on_train_batch_end                 	|  0.0018412      	|1000           	|  1.8412         	|  0.31813        	|
zero_grad                          	|  0.0013974      	|1000           	|  1.3974         	|  0.24145        	|
get_validate_batch                 	|  0.0066151      	|126            	|  0.8335         	|  0.14401        	|
fetch_next_validate_batch          	|  0.0065892      	|126            	|  0.83024        	|  0.14345        	|
training_batch_to_device           	|  0.00024613     	|1000           	|  0.24613        	|  0.042525       	|
on_train_batch_start               	|  9.5365e-05     	|1000           	|  0.095365       	|  0.016477       	|
on_train_end                       	|  0.045574       	|1              	|  0.045574       	|  0.0078742      	|
evaluation_batch_to_device         	|  0.00027953     	|125            	|  0.034941       	|  0.006037       	|
on_train_epoch_end                 	|  0.00035238     	|63             	|  0.0222         	|  0.0038357      	|
on_after_backward                  	|  2.1618e-05     	|1000           	|  0.021618       	|  0.0037352      	|
on_pretrain_routine_start          	|  0.021567       	|1              	|  0.021567       	|  0.0037263      	|
on_train_epoch_start               	|  0.00033734     	|63             	|  0.021252       	|  0.0036719      	|
on_batch_start                     	|  2.0797e-05     	|1000           	|  0.020797       	|  0.0035933      	|
on_before_zero_grad                	|  1.5806e-05     	|1000           	|  0.015806       	|  0.002731       	|
on_batch_end                       	|  1.5218e-05     	|1000           	|  0.015218       	|  0.0026293      	|
on_before_optimizer_step           	|  1.3123e-05     	|1000           	|  0.013123       	|  0.0022673      	|
on_before_backward                 	|  1.2407e-05     	|1000           	|  0.012407       	|  0.0021436      	|
configure_optimizers               	|  0.01094        	|1              	|  0.01094        	|  0.0018902      	|
training_step_end                  	|  6.4612e-06     	|1000           	|  0.0064612      	|  0.0011164      	|
on_validation_batch_end            	|  4.28e-05       	|125            	|  0.00535        	|  0.00092436     	|
on_validation_model_eval           	|  0.0044106      	|1              	|  0.0044106      	|  0.00076205     	|
on_validation_batch_start          	|  1.6605e-05     	|125            	|  0.0020756      	|  0.00035862     	|
on_train_start                     	|  0.0016689      	|1              	|  0.0016689      	|  0.00028835     	|
validation_step_end                	|  9.7285e-06     	|125            	|  0.0012161      	|  0.00021011     	|
on_epoch_end                       	|  1.122e-05      	|64             	|  0.00071811     	|  0.00012407     	|
on_epoch_start                     	|  1.0878e-05     	|64             	|  0.00069622     	|  0.00012029     	|
on_validation_start                	|  0.00058453     	|1              	|  0.00058453     	|  0.00010099     	|
on_validation_end                  	|  0.00022032     	|1              	|  0.00022032     	|  3.8066e-05     	|
on_before_accelerator_backend_setup	|  2.8769e-05     	|2              	|  5.7539e-05     	|  9.9415e-06     	|
setup                              	|  2.6845e-05     	|2              	|  5.3691e-05     	|  9.2766e-06     	|
on_fit_end                         	|  5.1893e-05     	|1              	|  5.1893e-05     	|  8.966e-06      	|
teardown                           	|  2.4408e-05     	|2              	|  4.8816e-05     	|  8.4344e-06     	|
on_configure_sharded_model         	|  2.4053e-05     	|2              	|  4.8107e-05     	|  8.3118e-06     	|
on_fit_start                       	|  3.9767e-05     	|1              	|  3.9767e-05     	|  6.871e-06      	|
on_validation_epoch_end            	|  3.2321e-05     	|1              	|  3.2321e-05     	|  5.5843e-06     	|
on_val_dataloader                  	|  1.1182e-05     	|2              	|  2.2365e-05     	|  3.8642e-06     	|
prepare_data                       	|  8.7488e-06     	|2              	|  1.7498e-05     	|  3.0232e-06     	|
configure_callbacks                	|  8.394e-06      	|2              	|  1.6788e-05     	|  2.9006e-06     	|
configure_sharded_model            	|  7.8846e-06     	|2              	|  1.5769e-05     	|  2.7246e-06     	|
on_pretrain_routine_end            	|  1.462e-05      	|1              	|  1.462e-05      	|  2.526e-06      	|
on_validation_epoch_start          	|  1.2998e-05     	|1              	|  1.2998e-05     	|  2.2457e-06     	|
on_train_dataloader                	|  5.2117e-06     	|1              	|  5.2117e-06     	|  9.0047e-07     	|
