FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  461.17         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  2.6239         	|167            	|  438.19         	|  95.017         	|
run_training_batch                 	|  0.38511        	|1000           	|  385.11         	|  83.508         	|
optimizer_step_with_closure_0      	|  0.38242        	|1000           	|  382.42         	|  82.923         	|
training_step_and_backward         	|  0.30094        	|1000           	|  300.94         	|  65.255         	|
model_forward                      	|  0.17402        	|1000           	|  174.02         	|  37.735         	|
training_step                      	|  0.17388        	|1000           	|  173.88         	|  37.704         	|
backward                           	|  0.12557        	|1000           	|  125.57         	|  27.229         	|
get_train_batch                    	|  0.013937       	|1167           	|  16.264         	|  3.5267         	|
fetch_next_train_batch             	|  0.013914       	|1167           	|  16.238         	|  3.521          	|
zero_grad                          	|  0.0013157      	|1000           	|  1.3157         	|  0.28529        	|
on_train_batch_end                 	|  0.0012188      	|1000           	|  1.2188         	|  0.26428        	|
training_batch_to_device           	|  0.00032178     	|1000           	|  0.32178        	|  0.069774       	|
on_train_batch_start               	|  9.5575e-05     	|1000           	|  0.095575       	|  0.020724       	|
on_train_epoch_end                 	|  0.00027395     	|167            	|  0.045749       	|  0.0099202      	|
on_train_epoch_start               	|  0.00027008     	|167            	|  0.045104       	|  0.0097803      	|
on_train_end                       	|  0.025268       	|1              	|  0.025268       	|  0.0054791      	|
on_pretrain_routine_start          	|  0.022465       	|1              	|  0.022465       	|  0.0048714      	|
on_batch_start                     	|  2.0735e-05     	|1000           	|  0.020735       	|  0.0044962      	|
on_before_zero_grad                	|  1.5172e-05     	|1000           	|  0.015172       	|  0.00329        	|
on_batch_end                       	|  1.4166e-05     	|1000           	|  0.014166       	|  0.0030717      	|
on_before_optimizer_step           	|  1.1255e-05     	|1000           	|  0.011255       	|  0.0024406      	|
on_after_backward                  	|  1.1229e-05     	|1000           	|  0.011229       	|  0.0024348      	|
on_before_backward                 	|  1.0963e-05     	|1000           	|  0.010963       	|  0.0023772      	|
configure_optimizers               	|  0.0082471      	|1              	|  0.0082471      	|  0.0017883      	|
training_step_end                  	|  6.0048e-06     	|1000           	|  0.0060048      	|  0.0013021      	|
on_epoch_start                     	|  1.0581e-05     	|167            	|  0.001767       	|  0.00038316     	|
on_epoch_end                       	|  1.0284e-05     	|167            	|  0.0017174      	|  0.00037239     	|
on_train_start                     	|  0.00074311     	|1              	|  0.00074311     	|  0.00016114     	|
on_fit_end                         	|  2.2552e-05     	|1              	|  2.2552e-05     	|  4.8902e-06     	|
on_before_accelerator_backend_setup	|  1.9897e-05     	|1              	|  1.9897e-05     	|  4.3145e-06     	|
on_fit_start                       	|  1.6431e-05     	|1              	|  1.6431e-05     	|  3.5629e-06     	|
setup                              	|  1.5719e-05     	|1              	|  1.5719e-05     	|  3.4085e-06     	|
on_configure_sharded_model         	|  1.3996e-05     	|1              	|  1.3996e-05     	|  3.0349e-06     	|
on_pretrain_routine_end            	|  1.0119e-05     	|1              	|  1.0119e-05     	|  2.1942e-06     	|
teardown                           	|  1.0049e-05     	|1              	|  1.0049e-05     	|  2.179e-06      	|
prepare_data                       	|  7.694e-06      	|1              	|  7.694e-06      	|  1.6684e-06     	|
configure_callbacks                	|  6.6231e-06     	|1              	|  6.6231e-06     	|  1.4362e-06     	|
on_val_dataloader                  	|  4.6691e-06     	|1              	|  4.6691e-06     	|  1.0124e-06     	|
configure_sharded_model            	|  4.5691e-06     	|1              	|  4.5691e-06     	|  9.9076e-07     	|
on_train_dataloader                	|  4.3481e-06     	|1              	|  4.3481e-06     	|  9.4284e-07     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1582.1         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  9.3336         	|167            	|  1558.7         	|  98.523         	|
run_training_batch                 	|  0.18895        	|8000           	|  1511.6         	|  95.546         	|
training_step_and_backward         	|  0.17873        	|8000           	|  1429.9         	|  90.378         	|
model_forward                      	|  0.094157       	|8000           	|  753.25         	|  47.611         	|
training_step                      	|  0.094022       	|8000           	|  752.17         	|  47.543         	|
backward                           	|  0.084386       	|8000           	|  675.08         	|  42.671         	|
optimizer_step_with_closure_0      	|  0.23776        	|1000           	|  237.76         	|  15.028         	|
get_train_batch                    	|  0.0033411      	|8167           	|  27.286         	|  1.7247         	|
fetch_next_train_batch             	|  0.0033234      	|8167           	|  27.142         	|  1.7156         	|
on_train_batch_end                 	|  0.0010753      	|8000           	|  8.602          	|  0.54371        	|
training_batch_to_device           	|  0.00021679     	|8000           	|  1.7343         	|  0.10962        	|
zero_grad                          	|  0.0013721      	|1000           	|  1.3721         	|  0.086729       	|
on_train_batch_start               	|  9.0307e-05     	|8000           	|  0.72246        	|  0.045665       	|
on_batch_start                     	|  1.5786e-05     	|8000           	|  0.12629        	|  0.0079822      	|
on_batch_end                       	|  1.3709e-05     	|8000           	|  0.10967        	|  0.0069319      	|
on_before_backward                 	|  1.334e-05      	|8000           	|  0.10672        	|  0.0067455      	|
on_after_backward                  	|  1.1923e-05     	|8000           	|  0.095385       	|  0.0060291      	|
on_train_epoch_start               	|  0.00028605     	|167            	|  0.04777        	|  0.0030194      	|
training_step_end                  	|  5.8314e-06     	|8000           	|  0.046651       	|  0.0029487      	|
on_train_epoch_end                 	|  0.00026759     	|167            	|  0.044687       	|  0.0028246      	|
on_train_end                       	|  0.025865       	|1              	|  0.025865       	|  0.0016349      	|
on_pretrain_routine_start          	|  0.022435       	|1              	|  0.022435       	|  0.001418       	|
on_before_zero_grad                	|  1.6514e-05     	|1000           	|  0.016514       	|  0.0010438      	|
on_before_optimizer_step           	|  1.3725e-05     	|1000           	|  0.013725       	|  0.0008675      	|
configure_optimizers               	|  0.0081189      	|1              	|  0.0081189      	|  0.00051317     	|
on_epoch_start                     	|  1.1002e-05     	|167            	|  0.0018373      	|  0.00011613     	|
on_epoch_end                       	|  1.0771e-05     	|167            	|  0.0017988      	|  0.0001137      	|
on_train_start                     	|  0.00074223     	|1              	|  0.00074223     	|  4.6915e-05     	|
on_before_accelerator_backend_setup	|  2.4345e-05     	|1              	|  2.4345e-05     	|  1.5388e-06     	|
on_fit_end                         	|  2.2452e-05     	|1              	|  2.2452e-05     	|  1.4191e-06     	|
on_fit_start                       	|  1.8575e-05     	|1              	|  1.8575e-05     	|  1.1741e-06     	|
on_configure_sharded_model         	|  1.4827e-05     	|1              	|  1.4827e-05     	|  9.3717e-07     	|
setup                              	|  1.4086e-05     	|1              	|  1.4086e-05     	|  8.9034e-07     	|
on_pretrain_routine_end            	|  1.3365e-05     	|1              	|  1.3365e-05     	|  8.4477e-07     	|
teardown                           	|  1.0981e-05     	|1              	|  1.0981e-05     	|  6.9408e-07     	|
prepare_data                       	|  8.8259e-06     	|1              	|  8.8259e-06     	|  5.5786e-07     	|
configure_callbacks                	|  6.4219e-06     	|1              	|  6.4219e-06     	|  4.0592e-07     	|
on_val_dataloader                  	|  4.638e-06      	|1              	|  4.638e-06      	|  2.9316e-07     	|
configure_sharded_model            	|  4.278e-06      	|1              	|  4.278e-06      	|  2.704e-07      	|
on_train_dataloader                	|  4.0971e-06     	|1              	|  4.0971e-06     	|  2.5897e-07     	|
