VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  456.67         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  6.1223         	|63             	|  385.7          	|  84.46          	|
run_training_batch                 	|  0.27774        	|1000           	|  277.74         	|  60.818         	|
optimizer_step_with_closure_0      	|  0.27436        	|1000           	|  274.36         	|  60.079         	|
training_step_and_backward         	|  0.20948        	|1000           	|  209.48         	|  45.871         	|
model_forward                      	|  0.10489        	|1000           	|  104.89         	|  22.968         	|
training_step                      	|  0.10469        	|1000           	|  104.69         	|  22.926         	|
backward                           	|  0.103          	|1000           	|  103.0          	|  22.554         	|
run_validate_evaluation            	|  30.696         	|1              	|  30.696         	|  6.7218         	|
evaluation_step_and_end            	|  0.0864         	|313            	|  27.043         	|  5.9218         	|
validation_step                    	|  0.086275       	|313            	|  27.004         	|  5.9132         	|
get_train_batch                    	|  0.021698       	|1063           	|  23.065         	|  5.0506         	|
fetch_next_train_batch             	|  0.021673       	|1063           	|  23.039         	|  5.0449         	|
on_train_batch_end                 	|  0.0022171      	|1000           	|  2.2171         	|  0.4855         	|
zero_grad                          	|  0.001557       	|1000           	|  1.557          	|  0.34095        	|
get_validate_batch                 	|  0.0033694      	|314            	|  1.058          	|  0.23168        	|
fetch_next_validate_batch          	|  0.0033422      	|314            	|  1.0495         	|  0.22981        	|
training_batch_to_device           	|  0.00025957     	|1000           	|  0.25957        	|  0.05684        	|
on_train_batch_start               	|  9.4805e-05     	|1000           	|  0.094805       	|  0.02076        	|
evaluation_batch_to_device         	|  0.00029783     	|313            	|  0.093221       	|  0.020413       	|
on_train_end                       	|  0.045353       	|1              	|  0.045353       	|  0.0099312      	|
on_pretrain_routine_start          	|  0.025815       	|1              	|  0.025815       	|  0.0056528      	|
on_after_backward                  	|  2.5329e-05     	|1000           	|  0.025329       	|  0.0055464      	|
on_train_epoch_end                 	|  0.00034906     	|63             	|  0.021991       	|  0.0048155      	|
on_train_epoch_start               	|  0.00034749     	|63             	|  0.021892       	|  0.0047938      	|
on_batch_start                     	|  2.0947e-05     	|1000           	|  0.020947       	|  0.0045868      	|
on_before_zero_grad                	|  1.7474e-05     	|1000           	|  0.017474       	|  0.0038263      	|
on_batch_end                       	|  1.5451e-05     	|1000           	|  0.015451       	|  0.0033834      	|
on_before_optimizer_step           	|  1.405e-05      	|1000           	|  0.01405        	|  0.0030765      	|
on_validation_batch_end            	|  4.3452e-05     	|313            	|  0.0136         	|  0.0029782      	|
on_before_backward                 	|  1.2353e-05     	|1000           	|  0.012353       	|  0.0027051      	|
configure_optimizers               	|  0.010738       	|1              	|  0.010738       	|  0.0023514      	|
training_step_end                  	|  6.9572e-06     	|1000           	|  0.0069572      	|  0.0015235      	|
on_validation_batch_start          	|  1.6183e-05     	|313            	|  0.0050654      	|  0.0011092      	|
on_validation_model_eval           	|  0.0046394      	|1              	|  0.0046394      	|  0.0010159      	|
validation_step_end                	|  9.7568e-06     	|313            	|  0.0030539      	|  0.00066873     	|
on_train_start                     	|  0.001706       	|1              	|  0.001706       	|  0.00037357     	|
on_epoch_end                       	|  1.2275e-05     	|64             	|  0.00078563     	|  0.00017203     	|
on_epoch_start                     	|  1.0941e-05     	|64             	|  0.00070026     	|  0.00015334     	|
on_validation_start                	|  0.0006342      	|1              	|  0.0006342      	|  0.00013888     	|
on_validation_end                  	|  0.00017276     	|1              	|  0.00017276     	|  3.783e-05      	|
on_fit_end                         	|  8.4436e-05     	|1              	|  8.4436e-05     	|  1.8489e-05     	|
teardown                           	|  3.899e-05      	|2              	|  7.798e-05      	|  1.7076e-05     	|
setup                              	|  2.8042e-05     	|2              	|  5.6084e-05     	|  1.2281e-05     	|
on_configure_sharded_model         	|  2.5574e-05     	|2              	|  5.1148e-05     	|  1.12e-05       	|
on_validation_epoch_end            	|  4.833e-05      	|1              	|  4.833e-05      	|  1.0583e-05     	|
on_before_accelerator_backend_setup	|  2.3111e-05     	|2              	|  4.6222e-05     	|  1.0121e-05     	|
on_fit_start                       	|  4.5855e-05     	|1              	|  4.5855e-05     	|  1.0041e-05     	|
on_val_dataloader                  	|  1.2446e-05     	|2              	|  2.4892e-05     	|  5.4508e-06     	|
configure_callbacks                	|  1.1639e-05     	|2              	|  2.3277e-05     	|  5.0972e-06     	|
on_pretrain_routine_end            	|  1.391e-05      	|1              	|  1.391e-05      	|  3.046e-06      	|
prepare_data                       	|  6.6441e-06     	|2              	|  1.3288e-05     	|  2.9098e-06     	|
configure_sharded_model            	|  6.1663e-06     	|2              	|  1.2333e-05     	|  2.7005e-06     	|
on_validation_epoch_start          	|  1.0639e-05     	|1              	|  1.0639e-05     	|  2.3298e-06     	|
on_train_dataloader                	|  8.516e-06      	|1              	|  8.516e-06      	|  1.8648e-06     	|
