VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1415.2         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.7113         	|500            	|  1355.7         	|  95.794         	|
run_training_batch                 	|  0.38479        	|1000           	|  384.79         	|  27.19          	|
optimizer_step_with_closure_0      	|  0.38056        	|1000           	|  380.56         	|  26.892         	|
training_step_and_backward         	|  0.27727        	|1000           	|  277.27         	|  19.592         	|
get_train_batch                    	|  0.10363        	|1500           	|  155.44         	|  10.984         	|
fetch_next_train_batch             	|  0.10359        	|1500           	|  155.38         	|  10.98          	|
model_forward                      	|  0.14281        	|1000           	|  142.81         	|  10.092         	|
training_step                      	|  0.14257        	|1000           	|  142.57         	|  10.074         	|
backward                           	|  0.13292        	|1000           	|  132.92         	|  9.3925         	|
run_validate_evaluation            	|  30.665         	|1              	|  30.665         	|  2.1669         	|
evaluation_step_and_end            	|  0.2227         	|125            	|  27.837         	|  1.967          	|
validation_step                    	|  0.22258        	|125            	|  27.823         	|  1.966          	|
on_train_batch_end                 	|  0.0023777      	|1000           	|  2.3777         	|  0.16802        	|
zero_grad                          	|  0.0014869      	|1000           	|  1.4869         	|  0.10507        	|
training_batch_to_device           	|  0.00062126     	|1000           	|  0.62126        	|  0.043899       	|
get_validate_batch                 	|  0.0044791      	|126            	|  0.56437        	|  0.03988        	|
fetch_next_validate_batch          	|  0.0044546      	|126            	|  0.56128        	|  0.039662       	|
on_train_epoch_end                 	|  0.0003703      	|500            	|  0.18515        	|  0.013083       	|
on_train_epoch_start               	|  0.00036128     	|500            	|  0.18064        	|  0.012765       	|
on_train_batch_start               	|  0.00011134     	|1000           	|  0.11134        	|  0.0078675      	|
on_batch_start                     	|  4.6195e-05     	|1000           	|  0.046195       	|  0.0032643      	|
on_train_end                       	|  0.043479       	|1              	|  0.043479       	|  0.0030723      	|
evaluation_batch_to_device         	|  0.00027334     	|125            	|  0.034167       	|  0.0024144      	|
on_after_backward                  	|  2.603e-05      	|1000           	|  0.02603        	|  0.0018393      	|
on_before_zero_grad                	|  2.3358e-05     	|1000           	|  0.023358       	|  0.0016505      	|
on_pretrain_routine_start          	|  0.021976       	|1              	|  0.021976       	|  0.0015529      	|
on_before_optimizer_step           	|  1.6336e-05     	|1000           	|  0.016336       	|  0.0011543      	|
on_batch_end                       	|  1.5858e-05     	|1000           	|  0.015858       	|  0.0011206      	|
on_before_backward                 	|  1.4969e-05     	|1000           	|  0.014969       	|  0.0010577      	|
training_step_end                  	|  8.7224e-06     	|1000           	|  0.0087224      	|  0.00061635     	|
configure_optimizers               	|  0.0081376      	|1              	|  0.0081376      	|  0.00057502     	|
on_epoch_start                     	|  1.1813e-05     	|501            	|  0.0059185      	|  0.00041822     	|
on_epoch_end                       	|  1.1625e-05     	|501            	|  0.005824       	|  0.00041154     	|
on_validation_batch_end            	|  4.1399e-05     	|125            	|  0.0051749      	|  0.00036567     	|
on_validation_model_eval           	|  0.0044268      	|1              	|  0.0044268      	|  0.00031281     	|
on_validation_batch_start          	|  1.7074e-05     	|125            	|  0.0021343      	|  0.00015081     	|
on_train_start                     	|  0.001738       	|1              	|  0.001738       	|  0.00012281     	|
validation_step_end                	|  9.0311e-06     	|125            	|  0.0011289      	|  7.977e-05      	|
on_validation_start                	|  0.00069379     	|1              	|  0.00069379     	|  4.9025e-05     	|
on_validation_end                  	|  0.00015147     	|1              	|  0.00015147     	|  1.0703e-05     	|
setup                              	|  2.8548e-05     	|2              	|  5.7096e-05     	|  4.0345e-06     	|
on_before_accelerator_backend_setup	|  2.2713e-05     	|2              	|  4.5426e-05     	|  3.2099e-06     	|
teardown                           	|  2.2097e-05     	|2              	|  4.4193e-05     	|  3.1228e-06     	|
on_configure_sharded_model         	|  2.1753e-05     	|2              	|  4.3506e-05     	|  3.0742e-06     	|
on_fit_end                         	|  3.5962e-05     	|1              	|  3.5962e-05     	|  2.5412e-06     	|
on_val_dataloader                  	|  1.5586e-05     	|2              	|  3.1171e-05     	|  2.2026e-06     	|
on_validation_epoch_end            	|  3.0797e-05     	|1              	|  3.0797e-05     	|  2.1762e-06     	|
on_fit_start                       	|  1.9465e-05     	|1              	|  1.9465e-05     	|  1.3754e-06     	|
on_pretrain_routine_end            	|  1.6604e-05     	|1              	|  1.6604e-05     	|  1.1733e-06     	|
configure_sharded_model            	|  7.8594e-06     	|2              	|  1.5719e-05     	|  1.1107e-06     	|
configure_callbacks                	|  7.35e-06       	|2              	|  1.47e-05       	|  1.0387e-06     	|
on_validation_epoch_start          	|  1.3379e-05     	|1              	|  1.3379e-05     	|  9.4542e-07     	|
prepare_data                       	|  6.3777e-06     	|2              	|  1.2755e-05     	|  9.0133e-07     	|
on_train_dataloader                	|  5.2489e-06     	|1              	|  5.2489e-06     	|  3.709e-07      	|
