FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  458.69         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.6093         	|167            	|  435.76         	|  95.001         	|
run_training_batch                 	|  0.38222        	|1000           	|  382.22         	|  83.33          	|
optimizer_step_with_closure_0      	|  0.37955        	|1000           	|  379.55         	|  82.748         	|
training_step_and_backward         	|  0.29736        	|1000           	|  297.36         	|  64.829         	|
model_forward                      	|  0.17219        	|1000           	|  172.19         	|  37.54          	|
training_step                      	|  0.17205        	|1000           	|  172.05         	|  37.509         	|
backward                           	|  0.12381        	|1000           	|  123.81         	|  26.992         	|
get_train_batch                    	|  0.013984       	|1167           	|  16.319         	|  3.5578         	|
fetch_next_train_batch             	|  0.01396        	|1167           	|  16.291         	|  3.5518         	|
zero_grad                          	|  0.0013341      	|1000           	|  1.3341         	|  0.29085        	|
on_train_batch_end                 	|  0.0012292      	|1000           	|  1.2292         	|  0.26798        	|
training_batch_to_device           	|  0.00031801     	|1000           	|  0.31801        	|  0.069331       	|
on_train_batch_start               	|  9.6658e-05     	|1000           	|  0.096658       	|  0.021073       	|
on_train_epoch_start               	|  0.00027467     	|167            	|  0.045871       	|  0.01           	|
on_train_epoch_end                 	|  0.00026066     	|167            	|  0.04353        	|  0.0094902      	|
on_train_end                       	|  0.024896       	|1              	|  0.024896       	|  0.0054278      	|
on_pretrain_routine_start          	|  0.022574       	|1              	|  0.022574       	|  0.0049216      	|
on_batch_start                     	|  2.1076e-05     	|1000           	|  0.021076       	|  0.0045948      	|
on_before_zero_grad                	|  1.5029e-05     	|1000           	|  0.015029       	|  0.0032764      	|
on_batch_end                       	|  1.3641e-05     	|1000           	|  0.013641       	|  0.0029739      	|
on_after_backward                  	|  1.1426e-05     	|1000           	|  0.011426       	|  0.002491       	|
on_before_optimizer_step           	|  1.0802e-05     	|1000           	|  0.010802       	|  0.0023549      	|
on_before_backward                 	|  1.0429e-05     	|1000           	|  0.010429       	|  0.0022737      	|
configure_optimizers               	|  0.0089034      	|1              	|  0.0089034      	|  0.0019411      	|
training_step_end                  	|  6.3996e-06     	|1000           	|  0.0063996      	|  0.0013952      	|
on_epoch_start                     	|  1.036e-05      	|167            	|  0.00173        	|  0.00037717     	|
on_epoch_end                       	|  1.0191e-05     	|167            	|  0.0017018      	|  0.00037103     	|
on_train_start                     	|  0.00074351     	|1              	|  0.00074351     	|  0.0001621      	|
on_before_accelerator_backend_setup	|  2.134e-05      	|1              	|  2.134e-05      	|  4.6524e-06     	|
on_fit_end                         	|  2.1119e-05     	|1              	|  2.1119e-05     	|  4.6042e-06     	|
on_fit_start                       	|  1.7833e-05     	|1              	|  1.7833e-05     	|  3.8878e-06     	|
setup                              	|  1.641e-05      	|1              	|  1.641e-05      	|  3.5776e-06     	|
on_configure_sharded_model         	|  1.3365e-05     	|1              	|  1.3365e-05     	|  2.9138e-06     	|
on_pretrain_routine_end            	|  1.1542e-05     	|1              	|  1.1542e-05     	|  2.5163e-06     	|
teardown                           	|  1.049e-05      	|1              	|  1.049e-05      	|  2.287e-06      	|
prepare_data                       	|  8.7661e-06     	|1              	|  8.7661e-06     	|  1.9111e-06     	|
configure_callbacks                	|  6.2421e-06     	|1              	|  6.2421e-06     	|  1.3609e-06     	|
on_val_dataloader                  	|  4.899e-06      	|1              	|  4.899e-06      	|  1.0681e-06     	|
configure_sharded_model            	|  4.528e-06      	|1              	|  4.528e-06      	|  9.8716e-07     	|
on_train_dataloader                	|  4.238e-06      	|1              	|  4.238e-06      	|  9.2394e-07     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1553.6         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.1638         	|167            	|  1530.3         	|  98.506         	|
run_training_batch                 	|  0.18539        	|8000           	|  1483.2         	|  95.468         	|
training_step_and_backward         	|  0.17531        	|8000           	|  1402.5         	|  90.278         	|
model_forward                      	|  0.093694       	|8000           	|  749.55         	|  48.247         	|
training_step                      	|  0.093561       	|8000           	|  748.49         	|  48.179         	|
backward                           	|  0.081434       	|8000           	|  651.47         	|  41.934         	|
optimizer_step_with_closure_0      	|  0.23328        	|1000           	|  233.28         	|  15.016         	|
get_train_batch                    	|  0.0033544      	|8167           	|  27.395         	|  1.7634         	|
fetch_next_train_batch             	|  0.0033371      	|8167           	|  27.254         	|  1.7543         	|
on_train_batch_end                 	|  0.0010776      	|8000           	|  8.6206         	|  0.55489        	|
training_batch_to_device           	|  0.000219       	|8000           	|  1.752          	|  0.11277        	|
zero_grad                          	|  0.0013509      	|1000           	|  1.3509         	|  0.086953       	|
on_train_batch_start               	|  9.1538e-05     	|8000           	|  0.7323         	|  0.047137       	|
on_batch_start                     	|  1.5202e-05     	|8000           	|  0.12162        	|  0.0078282      	|
on_batch_end                       	|  1.3345e-05     	|8000           	|  0.10676        	|  0.0068722      	|
on_before_backward                 	|  1.2997e-05     	|8000           	|  0.10398        	|  0.0066927      	|
on_after_backward                  	|  1.1861e-05     	|8000           	|  0.094888       	|  0.0061077      	|
on_train_epoch_start               	|  0.00028349     	|167            	|  0.047342       	|  0.0030473      	|
training_step_end                  	|  5.7423e-06     	|8000           	|  0.045938       	|  0.002957       	|
on_train_epoch_end                 	|  0.00025243     	|167            	|  0.042156       	|  0.0027135      	|
on_train_end                       	|  0.02558        	|1              	|  0.02558        	|  0.0016465      	|
on_pretrain_routine_start          	|  0.022975       	|1              	|  0.022975       	|  0.0014789      	|
on_before_zero_grad                	|  1.6541e-05     	|1000           	|  0.016541       	|  0.0010647      	|
on_before_optimizer_step           	|  1.2645e-05     	|1000           	|  0.012645       	|  0.00081397     	|
configure_optimizers               	|  0.0087276      	|1              	|  0.0087276      	|  0.00056178     	|
on_epoch_end                       	|  1.1261e-05     	|167            	|  0.0018806      	|  0.00012105     	|
on_epoch_start                     	|  1.0786e-05     	|167            	|  0.0018012      	|  0.00011594     	|
on_train_start                     	|  0.00072159     	|1              	|  0.00072159     	|  4.6447e-05     	|
on_fit_end                         	|  2.3404e-05     	|1              	|  2.3404e-05     	|  1.5065e-06     	|
on_before_accelerator_backend_setup	|  1.9366e-05     	|1              	|  1.9366e-05     	|  1.2466e-06     	|
on_fit_start                       	|  1.8204e-05     	|1              	|  1.8204e-05     	|  1.1718e-06     	|
on_configure_sharded_model         	|  1.4176e-05     	|1              	|  1.4176e-05     	|  9.1249e-07     	|
setup                              	|  1.3735e-05     	|1              	|  1.3735e-05     	|  8.8409e-07     	|
on_pretrain_routine_end            	|  1.2955e-05     	|1              	|  1.2955e-05     	|  8.339e-07      	|
teardown                           	|  1.0409e-05     	|1              	|  1.0409e-05     	|  6.7002e-07     	|
prepare_data                       	|  6.7621e-06     	|1              	|  6.7621e-06     	|  4.3526e-07     	|
configure_callbacks                	|  5.0492e-06     	|1              	|  5.0492e-06     	|  3.2501e-07     	|
on_val_dataloader                  	|  4.6489e-06     	|1              	|  4.6489e-06     	|  2.9924e-07     	|
configure_sharded_model            	|  4.479e-06      	|1              	|  4.479e-06      	|  2.883e-07      	|
on_train_dataloader                	|  4.268e-06      	|1              	|  4.268e-06      	|  2.7472e-07     	|
