VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1674.8         	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  3.2714         	|500            	|  1635.7         	|  97.665         	|
run_training_batch                 	|  0.30021        	|1000           	|  300.21         	|  17.925         	|
optimizer_step_with_closure_0      	|  0.29557        	|1000           	|  295.57         	|  17.648         	|
training_step_and_backward         	|  0.22675        	|1000           	|  226.75         	|  13.539         	|
get_train_batch                    	|  0.13526        	|1500           	|  202.88         	|  12.114         	|
fetch_next_train_batch             	|  0.13521        	|1500           	|  202.82         	|  12.11          	|
model_forward                      	|  0.11632        	|1000           	|  116.32         	|  6.9453         	|
training_step                      	|  0.11606        	|1000           	|  116.06         	|  6.9299         	|
backward                           	|  0.10882        	|1000           	|  108.82         	|  6.4976         	|
run_validate_evaluation            	|  8.0792         	|1              	|  8.0792         	|  0.48239        	|
evaluation_step_and_end            	|  0.10117        	|48             	|  4.8561         	|  0.28995        	|
validation_step                    	|  0.10105        	|48             	|  4.8505         	|  0.28962        	|
on_train_batch_end                 	|  0.0029024      	|1000           	|  2.9024         	|  0.1733         	|
zero_grad                          	|  0.0015575      	|1000           	|  1.5575         	|  0.092993       	|
training_batch_to_device           	|  0.00079509     	|1000           	|  0.79509        	|  0.047474       	|
get_validate_batch                 	|  0.016113       	|49             	|  0.78955        	|  0.047142       	|
fetch_next_validate_batch          	|  0.016089       	|49             	|  0.78838        	|  0.047073       	|
on_train_epoch_end                 	|  0.00044731     	|500            	|  0.22365        	|  0.013354       	|
on_train_epoch_start               	|  0.00044636     	|500            	|  0.22318        	|  0.013326       	|
on_train_batch_start               	|  0.0001189      	|1000           	|  0.1189         	|  0.0070991      	|
on_train_end                       	|  0.064203       	|1              	|  0.064203       	|  0.0038334      	|
on_batch_start                     	|  5.5428e-05     	|1000           	|  0.055428       	|  0.0033095      	|
on_after_backward                  	|  2.8744e-05     	|1000           	|  0.028744       	|  0.0017163      	|
on_before_zero_grad                	|  2.464e-05      	|1000           	|  0.02464        	|  0.0014712      	|
on_pretrain_routine_start          	|  0.024453       	|1              	|  0.024453       	|  0.00146        	|
on_before_backward                 	|  1.7148e-05     	|1000           	|  0.017148       	|  0.0010239      	|
on_before_optimizer_step           	|  1.7014e-05     	|1000           	|  0.017014       	|  0.0010159      	|
on_batch_end                       	|  1.698e-05      	|1000           	|  0.01698        	|  0.0010138      	|
evaluation_batch_to_device         	|  0.00026489     	|48             	|  0.012715       	|  0.00075918     	|
training_step_end                  	|  1.0822e-05     	|1000           	|  0.010822       	|  0.00064618     	|
configure_optimizers               	|  0.0093524      	|1              	|  0.0093524      	|  0.00055842     	|
on_validation_model_eval           	|  0.0061858      	|1              	|  0.0061858      	|  0.00036935     	|
on_epoch_end                       	|  1.2166e-05     	|501            	|  0.0060949      	|  0.00036392     	|
on_epoch_start                     	|  1.1498e-05     	|501            	|  0.0057607      	|  0.00034396     	|
on_validation_batch_end            	|  3.9993e-05     	|48             	|  0.0019196      	|  0.00011462     	|
on_validation_batch_start          	|  1.8405e-05     	|48             	|  0.00088344     	|  5.2749e-05     	|
on_train_start                     	|  0.0007097      	|1              	|  0.0007097      	|  4.2375e-05     	|
on_validation_start                	|  0.00067859     	|1              	|  0.00067859     	|  4.0517e-05     	|
validation_step_end                	|  9.3262e-06     	|48             	|  0.00044766     	|  2.6729e-05     	|
on_validation_end                  	|  0.00017514     	|1              	|  0.00017514     	|  1.0458e-05     	|
on_fit_end                         	|  0.0001558      	|1              	|  0.0001558      	|  9.3025e-06     	|
teardown                           	|  4.1205e-05     	|2              	|  8.2409e-05     	|  4.9205e-06     	|
on_before_accelerator_backend_setup	|  3.4263e-05     	|2              	|  6.8527e-05     	|  4.0916e-06     	|
on_configure_sharded_model         	|  3.0916e-05     	|2              	|  6.1832e-05     	|  3.6919e-06     	|
setup                              	|  2.7782e-05     	|2              	|  5.5565e-05     	|  3.3177e-06     	|
on_val_dataloader                  	|  1.9671e-05     	|2              	|  3.9343e-05     	|  2.3491e-06     	|
on_validation_epoch_end            	|  2.9953e-05     	|1              	|  2.9953e-05     	|  1.7885e-06     	|
configure_callbacks                	|  1.3589e-05     	|2              	|  2.7178e-05     	|  1.6227e-06     	|
on_fit_start                       	|  2.6913e-05     	|1              	|  2.6913e-05     	|  1.607e-06      	|
prepare_data                       	|  1.0177e-05     	|2              	|  2.0355e-05     	|  1.2154e-06     	|
configure_sharded_model            	|  9.2369e-06     	|2              	|  1.8474e-05     	|  1.103e-06      	|
on_validation_epoch_start          	|  1.6961e-05     	|1              	|  1.6961e-05     	|  1.0127e-06     	|
on_pretrain_routine_end            	|  1.3154e-05     	|1              	|  1.3154e-05     	|  7.854e-07      	|
on_train_dataloader                	|  5.899e-06      	|1              	|  5.899e-06      	|  3.5222e-07     	|
