VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  410.42         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  14.202         	|25             	|  355.06         	|  86.513         	|
run_training_batch                 	|  0.30116        	|1000           	|  301.16         	|  73.38          	|
optimizer_step_with_closure_0      	|  0.29707        	|1000           	|  297.07         	|  72.382         	|
training_step_and_backward         	|  0.22889        	|1000           	|  228.89         	|  55.77          	|
model_forward                      	|  0.1141         	|1000           	|  114.1          	|  27.801         	|
training_step                      	|  0.11389        	|1000           	|  113.89         	|  27.751         	|
backward                           	|  0.11313        	|1000           	|  113.13         	|  27.564         	|
run_validate_evaluation            	|  16.339         	|1              	|  16.339         	|  3.9811         	|
evaluation_step_and_end            	|  0.099249       	|139            	|  13.796         	|  3.3614         	|
validation_step                    	|  0.099151       	|139            	|  13.782         	|  3.358          	|
get_train_batch                    	|  0.01064        	|1025           	|  10.906         	|  2.6574         	|
fetch_next_train_batch             	|  0.010613       	|1025           	|  10.878         	|  2.6506         	|
on_train_batch_end                 	|  0.0027372      	|1000           	|  2.7372         	|  0.66694        	|
zero_grad                          	|  0.001629       	|1000           	|  1.629          	|  0.3969         	|
get_validate_batch                 	|  0.0059055      	|140            	|  0.82677        	|  0.20145        	|
fetch_next_validate_batch          	|  0.005883       	|140            	|  0.82362        	|  0.20068        	|
training_batch_to_device           	|  0.00029828     	|1000           	|  0.29828        	|  0.072678       	|
on_train_batch_start               	|  9.6889e-05     	|1000           	|  0.096889       	|  0.023607       	|
on_train_end                       	|  0.048934       	|1              	|  0.048934       	|  0.011923       	|
evaluation_batch_to_device         	|  0.0003241      	|139            	|  0.04505        	|  0.010977       	|
on_after_backward                  	|  2.9551e-05     	|1000           	|  0.029551       	|  0.0072003      	|
on_pretrain_routine_start          	|  0.023719       	|1              	|  0.023719       	|  0.0057791      	|
on_batch_start                     	|  2.2603e-05     	|1000           	|  0.022603       	|  0.0055073      	|
on_before_zero_grad                	|  1.8687e-05     	|1000           	|  0.018687       	|  0.0045532      	|
on_batch_end                       	|  1.6488e-05     	|1000           	|  0.016488       	|  0.0040174      	|
on_before_optimizer_step           	|  1.3668e-05     	|1000           	|  0.013668       	|  0.0033302      	|
on_before_backward                 	|  1.3329e-05     	|1000           	|  0.013329       	|  0.0032476      	|
on_train_epoch_end                 	|  0.00045691     	|25             	|  0.011423       	|  0.0027832      	|
on_train_epoch_start               	|  0.00041947     	|25             	|  0.010487       	|  0.0025552      	|
configure_optimizers               	|  0.010051       	|1              	|  0.010051       	|  0.0024489      	|
training_step_end                  	|  7.673e-06      	|1000           	|  0.007673       	|  0.0018696      	|
on_validation_model_eval           	|  0.0053375      	|1              	|  0.0053375      	|  0.0013005      	|
on_validation_batch_end            	|  3.5654e-05     	|139            	|  0.0049559      	|  0.0012075      	|
on_validation_batch_start          	|  1.4495e-05     	|139            	|  0.0020148      	|  0.00049092     	|
validation_step_end                	|  7.3868e-06     	|139            	|  0.0010268      	|  0.00025018     	|
on_train_start                     	|  0.00098748     	|1              	|  0.00098748     	|  0.0002406      	|
on_validation_start                	|  0.00070208     	|1              	|  0.00070208     	|  0.00017107     	|
on_validation_end                  	|  0.00034504     	|1              	|  0.00034504     	|  8.407e-05      	|
on_epoch_end                       	|  1.2387e-05     	|26             	|  0.00032207     	|  7.8473e-05     	|
on_epoch_start                     	|  1.2073e-05     	|26             	|  0.00031389     	|  7.6482e-05     	|
teardown                           	|  3.9831e-05     	|2              	|  7.9662e-05     	|  1.941e-05      	|
on_validation_epoch_end            	|  6.7303e-05     	|1              	|  6.7303e-05     	|  1.6399e-05     	|
setup                              	|  2.9657e-05     	|2              	|  5.9314e-05     	|  1.4452e-05     	|
on_configure_sharded_model         	|  2.5546e-05     	|2              	|  5.1092e-05     	|  1.2449e-05     	|
on_before_accelerator_backend_setup	|  2.3509e-05     	|2              	|  4.7019e-05     	|  1.1456e-05     	|
on_fit_start                       	|  4.6078e-05     	|1              	|  4.6078e-05     	|  1.1227e-05     	|
on_val_dataloader                  	|  1.8987e-05     	|2              	|  3.7974e-05     	|  9.2525e-06     	|
on_fit_end                         	|  2.9674e-05     	|1              	|  2.9674e-05     	|  7.2302e-06     	|
configure_sharded_model            	|  1.4435e-05     	|2              	|  2.8869e-05     	|  7.0341e-06     	|
prepare_data                       	|  8.7796e-06     	|2              	|  1.7559e-05     	|  4.2784e-06     	|
on_validation_epoch_start          	|  1.6211e-05     	|1              	|  1.6211e-05     	|  3.9498e-06     	|
on_pretrain_routine_end            	|  1.4342e-05     	|1              	|  1.4342e-05     	|  3.4946e-06     	|
configure_callbacks                	|  6.1942e-06     	|2              	|  1.2388e-05     	|  3.0185e-06     	|
on_train_dataloader                	|  6.916e-06      	|1              	|  6.916e-06      	|  1.6851e-06     	|
