VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1702.2         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.299          	|167            	|  1552.9         	|  91.229         	|
run_training_batch                 	|  0.1881         	|8000           	|  1504.8         	|  88.401         	|
training_step_and_backward         	|  0.17786        	|8000           	|  1422.9         	|  83.59          	|
model_forward                      	|  0.094525       	|8000           	|  756.2          	|  44.423         	|
training_step                      	|  0.094386       	|8000           	|  755.09         	|  44.358         	|
backward                           	|  0.083155       	|8000           	|  665.24         	|  39.08          	|
optimizer_step_with_closure_0      	|  0.2373         	|1000           	|  237.3          	|  13.941         	|
run_validate_evaluation            	|  102.25         	|1              	|  102.25         	|  6.0067         	|
evaluation_step_and_end            	|  0.10024        	|1000           	|  100.24         	|  5.8887         	|
validation_step                    	|  0.10019        	|1000           	|  100.19         	|  5.8855         	|
get_train_batch                    	|  0.0034217      	|8167           	|  27.945         	|  1.6417         	|
fetch_next_train_batch             	|  0.0034044      	|8167           	|  27.804         	|  1.6334         	|
on_train_batch_end                 	|  0.001083       	|8000           	|  8.664          	|  0.50897        	|
training_batch_to_device           	|  0.00021979     	|8000           	|  1.7583         	|  0.10329        	|
get_validate_batch                 	|  0.0014655      	|1001           	|  1.467          	|  0.08618        	|
fetch_next_validate_batch          	|  0.0014513      	|1001           	|  1.4528         	|  0.085344       	|
zero_grad                          	|  0.0013226      	|1000           	|  1.3226         	|  0.077696       	|
on_train_batch_start               	|  9.3689e-05     	|8000           	|  0.74951        	|  0.044031       	|
evaluation_batch_to_device         	|  0.00020967     	|1000           	|  0.20967        	|  0.012317       	|
on_batch_start                     	|  1.5581e-05     	|8000           	|  0.12465        	|  0.0073228      	|
on_batch_end                       	|  1.3362e-05     	|8000           	|  0.10689        	|  0.0062796      	|
on_before_backward                 	|  1.3357e-05     	|8000           	|  0.10686        	|  0.0062776      	|
on_after_backward                  	|  1.1514e-05     	|8000           	|  0.092112       	|  0.0054112      	|
training_step_end                  	|  5.9173e-06     	|8000           	|  0.047338       	|  0.0027809      	|
on_train_epoch_start               	|  0.00028331     	|167            	|  0.047313       	|  0.0027795      	|
on_train_epoch_end                 	|  0.00025532     	|167            	|  0.042639       	|  0.0025049      	|
on_train_end                       	|  0.025529       	|1              	|  0.025529       	|  0.0014998      	|
on_pretrain_routine_start          	|  0.023021       	|1              	|  0.023021       	|  0.0013524      	|
on_validation_batch_end            	|  1.891e-05      	|1000           	|  0.01891        	|  0.0011109      	|
on_before_zero_grad                	|  1.5895e-05     	|1000           	|  0.015895       	|  0.00093376     	|
on_before_optimizer_step           	|  1.2964e-05     	|1000           	|  0.012964       	|  0.0007616      	|
on_validation_batch_start          	|  9.696e-06      	|1000           	|  0.009696       	|  0.0005696      	|
configure_optimizers               	|  0.0082978      	|1              	|  0.0082978      	|  0.00048746     	|
validation_step_end                	|  4.8696e-06     	|1000           	|  0.0048696      	|  0.00028607     	|
on_validation_model_eval           	|  0.0039662      	|1              	|  0.0039662      	|  0.000233       	|
on_epoch_start                     	|  1.082e-05      	|168            	|  0.0018177      	|  0.00010678     	|
on_epoch_end                       	|  9.8254e-06     	|168            	|  0.0016507      	|  9.697e-05      	|
on_train_start                     	|  0.00074131     	|1              	|  0.00074131     	|  4.3549e-05     	|
on_validation_start                	|  0.00034089     	|1              	|  0.00034089     	|  2.0026e-05     	|
on_validation_end                  	|  0.00010571     	|1              	|  0.00010571     	|  6.2099e-06     	|
on_before_accelerator_backend_setup	|  1.7713e-05     	|2              	|  3.5427e-05     	|  2.0812e-06     	|
teardown                           	|  1.5283e-05     	|2              	|  3.0567e-05     	|  1.7957e-06     	|
setup                              	|  1.4938e-05     	|2              	|  2.9876e-05     	|  1.7551e-06     	|
on_configure_sharded_model         	|  1.4397e-05     	|2              	|  2.8794e-05     	|  1.6915e-06     	|
on_fit_end                         	|  2.7451e-05     	|1              	|  2.7451e-05     	|  1.6126e-06     	|
on_validation_epoch_end            	|  1.8735e-05     	|1              	|  1.8735e-05     	|  1.1006e-06     	|
on_fit_start                       	|  1.8204e-05     	|1              	|  1.8204e-05     	|  1.0694e-06     	|
on_val_dataloader                  	|  6.2161e-06     	|2              	|  1.2432e-05     	|  7.3034e-07     	|
on_pretrain_routine_end            	|  1.2243e-05     	|1              	|  1.2243e-05     	|  7.1922e-07     	|
prepare_data                       	|  5.6055e-06     	|2              	|  1.1211e-05     	|  6.586e-07      	|
configure_callbacks                	|  5.5345e-06     	|2              	|  1.1069e-05     	|  6.5026e-07     	|
configure_sharded_model            	|  5.0846e-06     	|2              	|  1.0169e-05     	|  5.9739e-07     	|
on_validation_epoch_start          	|  9.4669e-06     	|1              	|  9.4669e-06     	|  5.5614e-07     	|
on_train_dataloader                	|  4.228e-06      	|1              	|  4.228e-06      	|  2.4838e-07     	|
