VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1168.8         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.1995         	|500            	|  1099.7         	|  94.091         	|
run_training_batch                 	|  0.36991        	|1000           	|  369.91         	|  31.648         	|
optimizer_step_with_closure_0      	|  0.36672        	|1000           	|  366.72         	|  31.376         	|
training_step_and_backward         	|  0.26244        	|1000           	|  262.44         	|  22.454         	|
model_forward                      	|  0.13591        	|1000           	|  135.91         	|  11.628         	|
training_step                      	|  0.13569        	|1000           	|  135.69         	|  11.609         	|
get_train_batch                    	|  0.084099       	|1500           	|  126.15         	|  10.793         	|
fetch_next_train_batch             	|  0.08406        	|1500           	|  126.09         	|  10.788         	|
backward                           	|  0.12495        	|1000           	|  124.95         	|  10.691         	|
run_validate_evaluation            	|  30.453         	|1              	|  30.453         	|  2.6055         	|
evaluation_step_and_end            	|  0.22142        	|125            	|  27.677         	|  2.368          	|
validation_step                    	|  0.22129        	|125            	|  27.661         	|  2.3666         	|
on_train_batch_end                 	|  0.0022814      	|1000           	|  2.2814         	|  0.19519        	|
zero_grad                          	|  0.0015078      	|1000           	|  1.5078         	|  0.12901        	|
get_validate_batch                 	|  0.004846       	|126            	|  0.61059        	|  0.052241       	|
training_batch_to_device           	|  0.00060722     	|1000           	|  0.60722        	|  0.051952       	|
fetch_next_validate_batch          	|  0.0048184      	|126            	|  0.60711        	|  0.051943       	|
on_train_epoch_end                 	|  0.00036036     	|500            	|  0.18018        	|  0.015416       	|
on_train_epoch_start               	|  0.00034346     	|500            	|  0.17173        	|  0.014693       	|
on_train_batch_start               	|  0.00012724     	|1000           	|  0.12724        	|  0.010886       	|
on_batch_start                     	|  4.7625e-05     	|1000           	|  0.047625       	|  0.0040746      	|
on_train_end                       	|  0.046263       	|1              	|  0.046263       	|  0.0039582      	|
evaluation_batch_to_device         	|  0.00029346     	|125            	|  0.036682       	|  0.0031385      	|
on_after_backward                  	|  2.4494e-05     	|1000           	|  0.024494       	|  0.0020957      	|
on_pretrain_routine_start          	|  0.023232       	|1              	|  0.023232       	|  0.0019877      	|
on_before_zero_grad                	|  2.0213e-05     	|1000           	|  0.020213       	|  0.0017294      	|
on_before_optimizer_step           	|  1.8524e-05     	|1000           	|  0.018524       	|  0.0015849      	|
on_before_backward                 	|  1.6232e-05     	|1000           	|  0.016232       	|  0.0013888      	|
on_batch_end                       	|  1.5837e-05     	|1000           	|  0.015837       	|  0.001355       	|
configure_optimizers               	|  0.009699       	|1              	|  0.009699       	|  0.00082982     	|
training_step_end                  	|  8.7497e-06     	|1000           	|  0.0087497      	|  0.00074861     	|
on_epoch_end                       	|  1.2208e-05     	|501            	|  0.006116       	|  0.00052327     	|
on_epoch_start                     	|  1.1623e-05     	|501            	|  0.005823       	|  0.00049821     	|
on_validation_batch_end            	|  4.503e-05      	|125            	|  0.0056288      	|  0.00048159     	|
on_validation_model_eval           	|  0.0041845      	|1              	|  0.0041845      	|  0.00035802     	|
on_validation_batch_start          	|  1.8588e-05     	|125            	|  0.0023236      	|  0.0001988      	|
on_train_start                     	|  0.002272       	|1              	|  0.002272       	|  0.00019438     	|
validation_step_end                	|  1.006e-05      	|125            	|  0.0012575      	|  0.00010759     	|
on_validation_start                	|  0.00054873     	|1              	|  0.00054873     	|  4.6948e-05     	|
on_validation_end                  	|  0.00021487     	|1              	|  0.00021487     	|  1.8384e-05     	|
setup                              	|  3.2944e-05     	|2              	|  6.5887e-05     	|  5.6372e-06     	|
on_fit_end                         	|  6.0027e-05     	|1              	|  6.0027e-05     	|  5.1358e-06     	|
on_before_accelerator_backend_setup	|  2.7056e-05     	|2              	|  5.4112e-05     	|  4.6297e-06     	|
teardown                           	|  2.4576e-05     	|2              	|  4.9151e-05     	|  4.2053e-06     	|
on_configure_sharded_model         	|  2.449e-05      	|2              	|  4.898e-05      	|  4.1906e-06     	|
on_fit_start                       	|  4.0207e-05     	|1              	|  4.0207e-05     	|  3.44e-06       	|
on_validation_epoch_end            	|  3.5325e-05     	|1              	|  3.5325e-05     	|  3.0223e-06     	|
on_val_dataloader                  	|  1.1263e-05     	|2              	|  2.2527e-05     	|  1.9273e-06     	|
configure_sharded_model            	|  8.5076e-06     	|2              	|  1.7015e-05     	|  1.4558e-06     	|
configure_callbacks                	|  7.6136e-06     	|2              	|  1.5227e-05     	|  1.3028e-06     	|
on_pretrain_routine_end            	|  1.4754e-05     	|1              	|  1.4754e-05     	|  1.2623e-06     	|
prepare_data                       	|  7.1293e-06     	|2              	|  1.4259e-05     	|  1.2199e-06     	|
on_validation_epoch_start          	|  1.2718e-05     	|1              	|  1.2718e-05     	|  1.0881e-06     	|
on_train_dataloader                	|  6.7819e-06     	|1              	|  6.7819e-06     	|  5.8024e-07     	|
