VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  697.3          	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  3.8718         	|167            	|  646.59         	|  92.728         	|
run_training_batch                 	|  0.28038        	|1000           	|  280.38         	|  40.209         	|
optimizer_step_with_closure_0      	|  0.27765        	|1000           	|  277.65         	|  39.817         	|
training_step_and_backward         	|  0.21183        	|1000           	|  211.83         	|  30.379         	|
model_forward                      	|  0.11089        	|1000           	|  110.89         	|  15.903         	|
training_step                      	|  0.11071        	|1000           	|  110.71         	|  15.877         	|
backward                           	|  0.099451       	|1000           	|  99.451         	|  14.262         	|
get_train_batch                    	|  0.066587       	|1167           	|  77.707         	|  11.144         	|
fetch_next_train_batch             	|  0.066555       	|1167           	|  77.67          	|  11.139         	|
run_validate_evaluation            	|  16.673         	|1              	|  16.673         	|  2.3911         	|
evaluation_step_and_end            	|  0.10689        	|125            	|  13.361         	|  1.9161         	|
validation_step                    	|  0.10678        	|125            	|  13.347         	|  1.9141         	|
on_train_end                       	|  1.8921         	|1              	|  1.8921         	|  0.27135        	|
on_train_batch_end                 	|  0.0018703      	|1000           	|  1.8703         	|  0.26822        	|
zero_grad                          	|  0.0014568      	|1000           	|  1.4568         	|  0.20893        	|
get_validate_batch                 	|  0.0065411      	|126            	|  0.82417        	|  0.1182         	|
fetch_next_validate_batch          	|  0.0065215      	|126            	|  0.82171        	|  0.11784        	|
training_batch_to_device           	|  0.00036539     	|1000           	|  0.36539        	|  0.052402       	|
on_train_batch_start               	|  0.00010137     	|1000           	|  0.10137        	|  0.014538       	|
on_train_epoch_end                 	|  0.00034326     	|167            	|  0.057324       	|  0.0082209      	|
on_train_epoch_start               	|  0.00032963     	|167            	|  0.055048       	|  0.0078944      	|
on_batch_start                     	|  2.8908e-05     	|1000           	|  0.028908       	|  0.0041457      	|
evaluation_batch_to_device         	|  0.00023032     	|125            	|  0.02879        	|  0.0041288      	|
on_after_backward                  	|  2.2894e-05     	|1000           	|  0.022894       	|  0.0032832      	|
on_pretrain_routine_start          	|  0.021454       	|1              	|  0.021454       	|  0.0030767      	|
on_before_zero_grad                	|  1.7623e-05     	|1000           	|  0.017623       	|  0.0025273      	|
on_batch_end                       	|  1.534e-05      	|1000           	|  0.01534        	|  0.0021999      	|
on_before_backward                 	|  1.1953e-05     	|1000           	|  0.011953       	|  0.0017141      	|
on_before_optimizer_step           	|  1.1942e-05     	|1000           	|  0.011942       	|  0.0017127      	|
configure_optimizers               	|  0.011623       	|1              	|  0.011623       	|  0.0016669      	|
training_step_end                  	|  7.4556e-06     	|1000           	|  0.0074556      	|  0.0010692      	|
on_validation_model_eval           	|  0.004653       	|1              	|  0.004653       	|  0.0006673      	|
on_validation_batch_end            	|  3.0189e-05     	|125            	|  0.0037737      	|  0.00054118     	|
on_epoch_end                       	|  1.1419e-05     	|168            	|  0.0019184      	|  0.00027512     	|
on_epoch_start                     	|  1.1268e-05     	|168            	|  0.001893       	|  0.00027148     	|
on_train_start                     	|  0.0017118      	|1              	|  0.0017118      	|  0.0002455      	|
on_validation_batch_start          	|  1.3369e-05     	|125            	|  0.0016712      	|  0.00023966     	|
validation_step_end                	|  7.1416e-06     	|125            	|  0.00089271     	|  0.00012802     	|
on_validation_start                	|  0.0004527      	|1              	|  0.0004527      	|  6.4921e-05     	|
on_validation_end                  	|  0.00015473     	|1              	|  0.00015473     	|  2.219e-05      	|
teardown                           	|  4.2011e-05     	|2              	|  8.4022e-05     	|  1.205e-05      	|
on_fit_end                         	|  6.7832e-05     	|1              	|  6.7832e-05     	|  9.7278e-06     	|
setup                              	|  3.2969e-05     	|2              	|  6.5938e-05     	|  9.4562e-06     	|
on_configure_sharded_model         	|  2.948e-05      	|2              	|  5.896e-05      	|  8.4555e-06     	|
on_before_accelerator_backend_setup	|  2.6758e-05     	|2              	|  5.3516e-05     	|  7.6747e-06     	|
on_fit_start                       	|  4.5484e-05     	|1              	|  4.5484e-05     	|  6.5229e-06     	|
on_validation_epoch_end            	|  3.1069e-05     	|1              	|  3.1069e-05     	|  4.4556e-06     	|
configure_callbacks                	|  1.0462e-05     	|2              	|  2.0923e-05     	|  3.0006e-06     	|
configure_sharded_model            	|  9.7472e-06     	|2              	|  1.9494e-05     	|  2.7957e-06     	|
on_val_dataloader                  	|  8.2497e-06     	|2              	|  1.6499e-05     	|  2.3662e-06     	|
prepare_data                       	|  7.9386e-06     	|2              	|  1.5877e-05     	|  2.277e-06      	|
on_pretrain_routine_end            	|  1.2806e-05     	|1              	|  1.2806e-05     	|  1.8365e-06     	|
on_validation_epoch_start          	|  1.0235e-05     	|1              	|  1.0235e-05     	|  1.4678e-06     	|
on_train_dataloader                	|  7.6164e-06     	|1              	|  7.6164e-06     	|  1.0923e-06     	|
