FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  465.8          	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.6515         	|167            	|  442.8          	|  95.062         	|
run_training_batch                 	|  0.38953        	|1000           	|  389.53         	|  83.626         	|
optimizer_step_with_closure_0      	|  0.38685        	|1000           	|  386.85         	|  83.05          	|
training_step_and_backward         	|  0.30506        	|1000           	|  305.06         	|  65.492         	|
model_forward                      	|  0.17617        	|1000           	|  176.17         	|  37.821         	|
training_step                      	|  0.17603        	|1000           	|  176.03         	|  37.79          	|
backward                           	|  0.12752        	|1000           	|  127.52         	|  27.376         	|
get_train_batch                    	|  0.013895       	|1167           	|  16.215         	|  3.4812         	|
fetch_next_train_batch             	|  0.013872       	|1167           	|  16.188         	|  3.4754         	|
zero_grad                          	|  0.001346       	|1000           	|  1.346          	|  0.28896        	|
on_train_batch_end                 	|  0.0012438      	|1000           	|  1.2438         	|  0.26702        	|
training_batch_to_device           	|  0.00031341     	|1000           	|  0.31341        	|  0.067284       	|
on_train_batch_start               	|  9.4876e-05     	|1000           	|  0.094876       	|  0.020368       	|
on_train_epoch_start               	|  0.00028747     	|167            	|  0.048008       	|  0.010307       	|
on_train_epoch_end                 	|  0.00026259     	|167            	|  0.043852       	|  0.0094143      	|
on_train_end                       	|  0.025164       	|1              	|  0.025164       	|  0.0054023      	|
on_pretrain_routine_start          	|  0.022233       	|1              	|  0.022233       	|  0.0047731      	|
on_batch_start                     	|  2.1003e-05     	|1000           	|  0.021003       	|  0.0045091      	|
on_before_zero_grad                	|  1.553e-05      	|1000           	|  0.01553        	|  0.003334       	|
on_batch_end                       	|  1.3993e-05     	|1000           	|  0.013993       	|  0.0030041      	|
on_after_backward                  	|  1.1859e-05     	|1000           	|  0.011859       	|  0.0025459      	|
on_before_optimizer_step           	|  1.1472e-05     	|1000           	|  0.011472       	|  0.0024629      	|
on_before_backward                 	|  1.0582e-05     	|1000           	|  0.010582       	|  0.0022717      	|
configure_optimizers               	|  0.0081819      	|1              	|  0.0081819      	|  0.0017565      	|
training_step_end                  	|  6.554e-06      	|1000           	|  0.006554       	|  0.001407       	|
on_epoch_end                       	|  1.0574e-05     	|167            	|  0.0017659      	|  0.00037911     	|
on_epoch_start                     	|  1.0538e-05     	|167            	|  0.0017598      	|  0.0003778      	|
on_train_start                     	|  0.0007133      	|1              	|  0.0007133      	|  0.00015314     	|
on_before_accelerator_backend_setup	|  2.2162e-05     	|1              	|  2.2162e-05     	|  4.7578e-06     	|
on_fit_end                         	|  2.1139e-05     	|1              	|  2.1139e-05     	|  4.5382e-06     	|
on_fit_start                       	|  1.7463e-05     	|1              	|  1.7463e-05     	|  3.749e-06      	|
setup                              	|  1.4737e-05     	|1              	|  1.4737e-05     	|  3.1638e-06     	|
on_configure_sharded_model         	|  1.3695e-05     	|1              	|  1.3695e-05     	|  2.9401e-06     	|
teardown                           	|  1.074e-05      	|1              	|  1.074e-05      	|  2.3057e-06     	|
on_pretrain_routine_end            	|  1.007e-05      	|1              	|  1.007e-05      	|  2.1619e-06     	|
prepare_data                       	|  7.4441e-06     	|1              	|  7.4441e-06     	|  1.5981e-06     	|
configure_callbacks                	|  6.7131e-06     	|1              	|  6.7131e-06     	|  1.4412e-06     	|
on_val_dataloader                  	|  4.508e-06      	|1              	|  4.508e-06      	|  9.6779e-07     	|
configure_sharded_model            	|  4.3979e-06     	|1              	|  4.3979e-06     	|  9.4417e-07     	|
on_train_dataloader                	|  4.1979e-06     	|1              	|  4.1979e-06     	|  9.0123e-07     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1576.1         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.2979         	|167            	|  1552.8         	|  98.521         	|
run_training_batch                 	|  0.18817        	|8000           	|  1505.4         	|  95.515         	|
training_step_and_backward         	|  0.17795        	|8000           	|  1423.6         	|  90.328         	|
model_forward                      	|  0.094157       	|8000           	|  753.26         	|  47.793         	|
training_step                      	|  0.094023       	|8000           	|  752.19         	|  47.725         	|
backward                           	|  0.08361        	|8000           	|  668.88         	|  42.44          	|
optimizer_step_with_closure_0      	|  0.23696        	|1000           	|  236.96         	|  15.035         	|
get_train_batch                    	|  0.0033734      	|8167           	|  27.551         	|  1.7481         	|
fetch_next_train_batch             	|  0.0033564      	|8167           	|  27.412         	|  1.7392         	|
on_train_batch_end                 	|  0.0010791      	|8000           	|  8.6332         	|  0.54776        	|
training_batch_to_device           	|  0.00022219     	|8000           	|  1.7775         	|  0.11278        	|
zero_grad                          	|  0.001358       	|1000           	|  1.358          	|  0.086162       	|
on_train_batch_start               	|  9.1172e-05     	|8000           	|  0.72938        	|  0.046278       	|
on_batch_start                     	|  1.5239e-05     	|8000           	|  0.12191        	|  0.007735       	|
on_batch_end                       	|  1.365e-05      	|8000           	|  0.1092         	|  0.0069286      	|
on_before_backward                 	|  1.2903e-05     	|8000           	|  0.10322        	|  0.0065492      	|
on_after_backward                  	|  1.1492e-05     	|8000           	|  0.091937       	|  0.0058333      	|
training_step_end                  	|  5.9443e-06     	|8000           	|  0.047555       	|  0.0030173      	|
on_train_epoch_start               	|  0.00027689     	|167            	|  0.04624        	|  0.0029339      	|
on_train_epoch_end                 	|  0.00025478     	|167            	|  0.042549       	|  0.0026997      	|
on_train_end                       	|  0.025504       	|1              	|  0.025504       	|  0.0016182      	|
on_pretrain_routine_start          	|  0.023185       	|1              	|  0.023185       	|  0.0014711      	|
on_before_zero_grad                	|  1.6372e-05     	|1000           	|  0.016372       	|  0.0010388      	|
on_before_optimizer_step           	|  1.2783e-05     	|1000           	|  0.012783       	|  0.00081108     	|
configure_optimizers               	|  0.0083324      	|1              	|  0.0083324      	|  0.00052868     	|
on_epoch_start                     	|  1.0818e-05     	|167            	|  0.0018065      	|  0.00011462     	|
on_epoch_end                       	|  1.0238e-05     	|167            	|  0.0017098      	|  0.00010849     	|
on_train_start                     	|  0.00073182     	|1              	|  0.00073182     	|  4.6433e-05     	|
on_fit_end                         	|  2.5708e-05     	|1              	|  2.5708e-05     	|  1.6311e-06     	|
on_before_accelerator_backend_setup	|  2.1821e-05     	|1              	|  2.1821e-05     	|  1.3845e-06     	|
on_fit_start                       	|  1.7142e-05     	|1              	|  1.7142e-05     	|  1.0876e-06     	|
on_configure_sharded_model         	|  1.3715e-05     	|1              	|  1.3715e-05     	|  8.702e-07      	|
setup                              	|  1.3546e-05     	|1              	|  1.3546e-05     	|  8.5947e-07     	|
teardown                           	|  1.1081e-05     	|1              	|  1.1081e-05     	|  7.0308e-07     	|
on_pretrain_routine_end            	|  9.9791e-06     	|1              	|  9.9791e-06     	|  6.3316e-07     	|
prepare_data                       	|  8.326e-06      	|1              	|  8.326e-06      	|  5.2828e-07     	|
configure_callbacks                	|  7.113e-06      	|1              	|  7.113e-06      	|  4.5131e-07     	|
configure_sharded_model            	|  4.5188e-06     	|1              	|  4.5188e-06     	|  2.8671e-07     	|
on_val_dataloader                  	|  4.448e-06      	|1              	|  4.448e-06      	|  2.8222e-07     	|
on_train_dataloader                	|  4.278e-06      	|1              	|  4.278e-06      	|  2.7144e-07     	|
