VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1681.8         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  73.17          	|21             	|  1536.6         	|  91.364         	|
run_training_batch                 	|  0.18832        	|8000           	|  1506.6         	|  89.581         	|
training_step_and_backward         	|  0.17809        	|8000           	|  1424.7         	|  84.716         	|
model_forward                      	|  0.094072       	|8000           	|  752.58         	|  44.748         	|
training_step                      	|  0.093942       	|8000           	|  751.54         	|  44.686         	|
backward                           	|  0.083843       	|8000           	|  670.74         	|  39.882         	|
optimizer_step_with_closure_0      	|  0.23792        	|1000           	|  237.92         	|  14.147         	|
run_validate_evaluation            	|  101.69         	|1              	|  101.69         	|  6.0465         	|
evaluation_step_and_end            	|  0.099726       	|1000           	|  99.726         	|  5.9297         	|
validation_step                    	|  0.099675       	|1000           	|  99.675         	|  5.9267         	|
get_train_batch                    	|  0.0018222      	|8021           	|  14.616         	|  0.86905        	|
fetch_next_train_batch             	|  0.0018063      	|8021           	|  14.488         	|  0.86146        	|
on_train_batch_end                 	|  0.0010638      	|8000           	|  8.5107         	|  0.50605        	|
training_batch_to_device           	|  0.00020928     	|8000           	|  1.6742         	|  0.09955        	|
get_validate_batch                 	|  0.0014298      	|1001           	|  1.4313         	|  0.085103       	|
fetch_next_validate_batch          	|  0.0014158      	|1001           	|  1.4172         	|  0.084265       	|
zero_grad                          	|  0.001293       	|1000           	|  1.293          	|  0.076879       	|
on_train_batch_start               	|  8.9171e-05     	|8000           	|  0.71337        	|  0.042417       	|
evaluation_batch_to_device         	|  0.0002058      	|1000           	|  0.2058         	|  0.012237       	|
on_batch_start                     	|  1.458e-05      	|8000           	|  0.11664        	|  0.0069353      	|
on_before_backward                 	|  1.299e-05      	|8000           	|  0.10392        	|  0.0061791      	|
on_batch_end                       	|  1.2845e-05     	|8000           	|  0.10276        	|  0.0061099      	|
on_after_backward                  	|  1.1495e-05     	|8000           	|  0.091964       	|  0.0054682      	|
training_step_end                  	|  5.9178e-06     	|8000           	|  0.047342       	|  0.002815       	|
on_train_end                       	|  0.025877       	|1              	|  0.025877       	|  0.0015387      	|
on_pretrain_routine_start          	|  0.022892       	|1              	|  0.022892       	|  0.0013612      	|
on_validation_batch_end            	|  1.7974e-05     	|1000           	|  0.017974       	|  0.0010687      	|
on_before_zero_grad                	|  1.3921e-05     	|1000           	|  0.013921       	|  0.00082774     	|
on_before_optimizer_step           	|  1.1188e-05     	|1000           	|  0.011188       	|  0.00066522     	|
on_validation_batch_start          	|  1.007e-05      	|1000           	|  0.01007        	|  0.00059876     	|
configure_optimizers               	|  0.0085617      	|1              	|  0.0085617      	|  0.00050908     	|
on_train_epoch_start               	|  0.00028379     	|21             	|  0.0059596      	|  0.00035436     	|
on_train_epoch_end                 	|  0.0002678      	|21             	|  0.0056238      	|  0.00033439     	|
validation_step_end                	|  4.8604e-06     	|1000           	|  0.0048604      	|  0.000289       	|
on_validation_model_eval           	|  0.0039129      	|1              	|  0.0039129      	|  0.00023266     	|
on_train_start                     	|  0.00068025     	|1              	|  0.00068025     	|  4.0448e-05     	|
on_validation_start                	|  0.00032006     	|1              	|  0.00032006     	|  1.9031e-05     	|
on_epoch_end                       	|  1.0597e-05     	|22             	|  0.00023314     	|  1.3862e-05     	|
on_epoch_start                     	|  1.0515e-05     	|22             	|  0.00023134     	|  1.3755e-05     	|
on_validation_end                  	|  0.0001081      	|1              	|  0.0001081      	|  6.4276e-06     	|
on_before_accelerator_backend_setup	|  2.0213e-05     	|2              	|  4.0426e-05     	|  2.4037e-06     	|
setup                              	|  1.6421e-05     	|2              	|  3.2842e-05     	|  1.9528e-06     	|
teardown                           	|  1.6104e-05     	|2              	|  3.2208e-05     	|  1.9151e-06     	|
on_configure_sharded_model         	|  1.3661e-05     	|2              	|  2.7322e-05     	|  1.6246e-06     	|
on_fit_end                         	|  2.1149e-05     	|1              	|  2.1149e-05     	|  1.2575e-06     	|
on_validation_epoch_end            	|  1.8384e-05     	|1              	|  1.8384e-05     	|  1.0931e-06     	|
on_fit_start                       	|  1.6191e-05     	|1              	|  1.6191e-05     	|  9.6271e-07     	|
prepare_data                       	|  6.3315e-06     	|2              	|  1.2663e-05     	|  7.5294e-07     	|
on_val_dataloader                  	|  6.1665e-06     	|2              	|  1.2333e-05     	|  7.3332e-07     	|
configure_callbacks                	|  5.946e-06      	|2              	|  1.1892e-05     	|  7.071e-07      	|
on_validation_epoch_start          	|  1.1822e-05     	|1              	|  1.1822e-05     	|  7.0293e-07     	|
on_pretrain_routine_end            	|  9.978e-06      	|1              	|  9.978e-06      	|  5.9329e-07     	|
configure_sharded_model            	|  4.974e-06      	|2              	|  9.9479e-06     	|  5.915e-07      	|
on_train_dataloader                	|  4.458e-06      	|1              	|  4.458e-06      	|  2.6507e-07     	|
