VALIDATE Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
-------------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  654.3          	|  100 %          	|
-------------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  3.6195         	|167            	|  604.45         	|  92.381         	|
run_training_batch                 	|  0.24484        	|1000           	|  244.84         	|  37.419         	|
optimizer_step_with_closure_0      	|  0.24271        	|1000           	|  242.71         	|  37.095         	|
training_step_and_backward         	|  0.18231        	|1000           	|  182.31         	|  27.863         	|
model_forward                      	|  0.094912       	|1000           	|  94.912         	|  14.506         	|
training_step                      	|  0.094735       	|1000           	|  94.735         	|  14.479         	|
backward                           	|  0.086091       	|1000           	|  86.091         	|  13.158         	|
get_train_batch                    	|  0.065571       	|1167           	|  76.522         	|  11.695         	|
fetch_next_train_batch             	|  0.065545       	|1167           	|  76.491         	|  11.69          	|
run_validate_evaluation            	|  16.59          	|1              	|  16.59          	|  2.5355         	|
evaluation_step_and_end            	|  0.1069         	|125            	|  13.362         	|  2.0422         	|
validation_step                    	|  0.10682        	|125            	|  13.352         	|  2.0407         	|
on_train_batch_end                 	|  0.001615       	|1000           	|  1.615          	|  0.24683        	|
zero_grad                          	|  0.0012752      	|1000           	|  1.2752         	|  0.1949         	|
get_validate_batch                 	|  0.0060843      	|126            	|  0.76663        	|  0.11717        	|
fetch_next_validate_batch          	|  0.0060646      	|126            	|  0.76414        	|  0.11679        	|
training_batch_to_device           	|  0.00033952     	|1000           	|  0.33952        	|  0.05189        	|
on_train_batch_start               	|  9.5857e-05     	|1000           	|  0.095857       	|  0.01465        	|
on_train_epoch_start               	|  0.0003233      	|167            	|  0.053992       	|  0.0082518      	|
on_train_epoch_end                 	|  0.00031717     	|167            	|  0.052967       	|  0.0080953      	|
on_train_end                       	|  0.045941       	|1              	|  0.045941       	|  0.0070214      	|
evaluation_batch_to_device         	|  0.00025116     	|125            	|  0.031395       	|  0.0047983      	|
on_batch_start                     	|  2.8276e-05     	|1000           	|  0.028276       	|  0.0043215      	|
on_pretrain_routine_start          	|  0.023678       	|1              	|  0.023678       	|  0.0036188      	|
on_after_backward                  	|  1.8042e-05     	|1000           	|  0.018042       	|  0.0027574      	|
on_before_zero_grad                	|  1.6299e-05     	|1000           	|  0.016299       	|  0.002491       	|
on_batch_end                       	|  1.4325e-05     	|1000           	|  0.014325       	|  0.0021894      	|
on_before_optimizer_step           	|  1.1939e-05     	|1000           	|  0.011939       	|  0.0018246      	|
on_before_backward                 	|  1.1931e-05     	|1000           	|  0.011931       	|  0.0018235      	|
configure_optimizers               	|  0.010942       	|1              	|  0.010942       	|  0.0016724      	|
training_step_end                  	|  6.6202e-06     	|1000           	|  0.0066202      	|  0.0010118      	|
on_validation_model_eval           	|  0.0041072      	|1              	|  0.0041072      	|  0.00062772     	|
on_validation_batch_end            	|  2.8772e-05     	|125            	|  0.0035965      	|  0.00054967     	|
on_train_start                     	|  0.0020631      	|1              	|  0.0020631      	|  0.00031531     	|
on_epoch_end                       	|  1.1714e-05     	|168            	|  0.0019679      	|  0.00030077     	|
on_epoch_start                     	|  1.1069e-05     	|168            	|  0.0018597      	|  0.00028422     	|
on_validation_batch_start          	|  1.3183e-05     	|125            	|  0.0016479      	|  0.00025186     	|
validation_step_end                	|  6.7356e-06     	|125            	|  0.00084195     	|  0.00012868     	|
on_validation_start                	|  0.00047267     	|1              	|  0.00047267     	|  7.2241e-05     	|
on_validation_end                  	|  0.00016152     	|1              	|  0.00016152     	|  2.4686e-05     	|
on_fit_end                         	|  5.853e-05      	|1              	|  5.853e-05      	|  8.9454e-06     	|
setup                              	|  2.9159e-05     	|2              	|  5.8318e-05     	|  8.913e-06      	|
on_before_accelerator_backend_setup	|  2.7986e-05     	|2              	|  5.5972e-05     	|  8.5546e-06     	|
on_fit_start                       	|  4.5739e-05     	|1              	|  4.5739e-05     	|  6.9905e-06     	|
on_configure_sharded_model         	|  2.1596e-05     	|2              	|  4.3193e-05     	|  6.6014e-06     	|
teardown                           	|  2.1327e-05     	|2              	|  4.2655e-05     	|  6.5191e-06     	|
configure_callbacks                	|  1.445e-05      	|2              	|  2.8901e-05     	|  4.4171e-06     	|
on_validation_epoch_end            	|  2.7064e-05     	|1              	|  2.7064e-05     	|  4.1364e-06     	|
prepare_data                       	|  1.1931e-05     	|2              	|  2.3862e-05     	|  3.647e-06      	|
on_val_dataloader                  	|  8.1901e-06     	|2              	|  1.638e-05      	|  2.5035e-06     	|
on_pretrain_routine_end            	|  1.6052e-05     	|1              	|  1.6052e-05     	|  2.4533e-06     	|
configure_sharded_model            	|  6.7325e-06     	|2              	|  1.3465e-05     	|  2.0579e-06     	|
on_validation_epoch_start          	|  1.039e-05      	|1              	|  1.039e-05      	|  1.5879e-06     	|
on_train_dataloader                	|  7.6164e-06     	|1              	|  7.6164e-06     	|  1.164e-06      	|
