VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1925.8         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  3.4877         	|500            	|  1743.9         	|  90.554         	|
run_training_batch                 	|  0.20712        	|8000           	|  1657.0         	|  86.043         	|
training_step_and_backward         	|  0.19572        	|8000           	|  1565.7         	|  81.305         	|
model_forward                      	|  0.11041        	|8000           	|  883.25         	|  45.865         	|
training_step                      	|  0.11027        	|8000           	|  882.15         	|  45.808         	|
backward                           	|  0.085121       	|8000           	|  680.97         	|  35.361         	|
optimizer_step_with_closure_0      	|  0.25396        	|1000           	|  253.96         	|  13.188         	|
run_validate_evaluation            	|  137.76         	|1              	|  137.76         	|  7.1536         	|
evaluation_step_and_end            	|  0.13576        	|1000           	|  135.76         	|  7.0495         	|
validation_step                    	|  0.13571        	|1000           	|  135.71         	|  7.0469         	|
get_train_batch                    	|  0.0067534      	|8500           	|  57.404         	|  2.9808         	|
fetch_next_train_batch             	|  0.0067339      	|8500           	|  57.238         	|  2.9722         	|
on_train_batch_end                 	|  0.0010925      	|8000           	|  8.7399         	|  0.45383        	|
training_batch_to_device           	|  0.00023679     	|8000           	|  1.8943         	|  0.098367       	|
get_validate_batch                 	|  0.0014602      	|1001           	|  1.4616         	|  0.075897       	|
fetch_next_validate_batch          	|  0.0014457      	|1001           	|  1.4472         	|  0.075148       	|
zero_grad                          	|  0.001373       	|1000           	|  1.373          	|  0.071295       	|
on_train_batch_start               	|  9.1728e-05     	|8000           	|  0.73382        	|  0.038105       	|
evaluation_batch_to_device         	|  0.00021462     	|1000           	|  0.21462        	|  0.011145       	|
on_train_epoch_start               	|  0.00028941     	|500            	|  0.1447         	|  0.0075141      	|
on_batch_start                     	|  1.7973e-05     	|8000           	|  0.14379        	|  0.0074664      	|
on_train_epoch_end                 	|  0.0002614      	|500            	|  0.1307         	|  0.006787       	|
on_before_backward                 	|  1.3403e-05     	|8000           	|  0.10722        	|  0.0055677      	|
on_batch_end                       	|  1.2921e-05     	|8000           	|  0.10337        	|  0.0053675      	|
on_after_backward                  	|  1.1296e-05     	|8000           	|  0.090371       	|  0.0046927      	|
training_step_end                  	|  5.9236e-06     	|8000           	|  0.047389       	|  0.0024608      	|
on_train_end                       	|  0.024261       	|1              	|  0.024261       	|  0.0012598      	|
on_before_zero_grad                	|  2.3427e-05     	|1000           	|  0.023427       	|  0.0012165      	|
on_pretrain_routine_start          	|  0.022958       	|1              	|  0.022958       	|  0.0011922      	|
on_validation_batch_end            	|  1.9227e-05     	|1000           	|  0.019227       	|  0.00099839     	|
on_before_optimizer_step           	|  1.7574e-05     	|1000           	|  0.017574       	|  0.00091258     	|
on_validation_batch_start          	|  9.6959e-06     	|1000           	|  0.0096959      	|  0.00050348     	|
configure_optimizers               	|  0.0083451      	|1              	|  0.0083451      	|  0.00043334     	|
on_epoch_start                     	|  1.1156e-05     	|501            	|  0.005589       	|  0.00029022     	|
on_epoch_end                       	|  1.0352e-05     	|501            	|  0.0051861      	|  0.0002693      	|
validation_step_end                	|  4.7262e-06     	|1000           	|  0.0047262      	|  0.00024542     	|
on_validation_model_eval           	|  0.0043828      	|1              	|  0.0043828      	|  0.00022758     	|
on_train_start                     	|  0.00073028     	|1              	|  0.00073028     	|  3.7922e-05     	|
on_validation_start                	|  0.00033416     	|1              	|  0.00033416     	|  1.7352e-05     	|
on_validation_end                  	|  0.00011508     	|1              	|  0.00011508     	|  5.9756e-06     	|
setup                              	|  2.194e-05      	|2              	|  4.3881e-05     	|  2.2786e-06     	|
on_before_accelerator_backend_setup	|  2.109e-05      	|2              	|  4.218e-05      	|  2.1903e-06     	|
on_configure_sharded_model         	|  1.7863e-05     	|2              	|  3.5727e-05     	|  1.8552e-06     	|
teardown                           	|  1.5234e-05     	|2              	|  3.0467e-05     	|  1.5821e-06     	|
on_fit_end                         	|  2.3715e-05     	|1              	|  2.3715e-05     	|  1.2314e-06     	|
on_validation_epoch_end            	|  1.7603e-05     	|1              	|  1.7603e-05     	|  9.1408e-07     	|
on_fit_start                       	|  1.7373e-05     	|1              	|  1.7373e-05     	|  9.0214e-07     	|
prepare_data                       	|  6.1709e-06     	|2              	|  1.2342e-05     	|  6.4088e-07     	|
on_val_dataloader                  	|  6.0764e-06     	|2              	|  1.2153e-05     	|  6.3106e-07     	|
configure_callbacks                	|  5.5451e-06     	|2              	|  1.109e-05      	|  5.7588e-07     	|
configure_sharded_model            	|  5.5399e-06     	|2              	|  1.108e-05      	|  5.7534e-07     	|
on_validation_epoch_start          	|  9.6881e-06     	|1              	|  9.6881e-06     	|  5.0307e-07     	|
on_pretrain_routine_end            	|  9.4771e-06     	|1              	|  9.4771e-06     	|  4.9212e-07     	|
on_train_dataloader                	|  4.0978e-06     	|1              	|  4.0978e-06     	|  2.1279e-07     	|
