FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  398.32         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  5.9431         	|63             	|  374.41         	|  93.998         	|
run_training_batch                 	|  0.34767        	|1000           	|  347.67         	|  87.285         	|
optimizer_step_with_closure_0      	|  0.34503        	|1000           	|  345.03         	|  86.622         	|
training_step_and_backward         	|  0.25887        	|1000           	|  258.87         	|  64.992         	|
model_forward                      	|  0.16027        	|1000           	|  160.27         	|  40.236         	|
training_step                      	|  0.16013        	|1000           	|  160.13         	|  40.202         	|
backward                           	|  0.097261       	|1000           	|  97.261         	|  24.418         	|
get_train_batch                    	|  0.0080992      	|1063           	|  8.6094         	|  2.1614         	|
fetch_next_train_batch             	|  0.0080802      	|1063           	|  8.5893         	|  2.1564         	|
zero_grad                          	|  0.0013206      	|1000           	|  1.3206         	|  0.33155        	|
on_train_batch_end                 	|  0.0012148      	|1000           	|  1.2148         	|  0.30499        	|
training_batch_to_device           	|  0.00022627     	|1000           	|  0.22627        	|  0.056807       	|
on_train_batch_start               	|  9.1406e-05     	|1000           	|  0.091406       	|  0.022948       	|
on_train_end                       	|  0.025662       	|1              	|  0.025662       	|  0.0064426      	|
on_pretrain_routine_start          	|  0.022795       	|1              	|  0.022795       	|  0.0057229      	|
on_train_epoch_start               	|  0.00029629     	|63             	|  0.018666       	|  0.0046863      	|
on_train_epoch_end                 	|  0.00028201     	|63             	|  0.017767       	|  0.0044604      	|
on_batch_start                     	|  1.6257e-05     	|1000           	|  0.016257       	|  0.0040813      	|
on_before_zero_grad                	|  1.4376e-05     	|1000           	|  0.014376       	|  0.0036091      	|
on_batch_end                       	|  1.3149e-05     	|1000           	|  0.013149       	|  0.0033011      	|
on_after_backward                  	|  1.1734e-05     	|1000           	|  0.011734       	|  0.0029459      	|
on_before_optimizer_step           	|  1.0407e-05     	|1000           	|  0.010407       	|  0.0026126      	|
on_before_backward                 	|  1.0133e-05     	|1000           	|  0.010133       	|  0.002544       	|
configure_optimizers               	|  0.0086511      	|1              	|  0.0086511      	|  0.0021719      	|
training_step_end                  	|  5.99e-06       	|1000           	|  0.00599        	|  0.0015038      	|
on_train_start                     	|  0.00076707     	|1              	|  0.00076707     	|  0.00019258     	|
on_epoch_start                     	|  1.0266e-05     	|63             	|  0.00064674     	|  0.00016237     	|
on_epoch_end                       	|  9.8952e-06     	|63             	|  0.0006234      	|  0.00015651     	|
on_fit_end                         	|  2.1861e-05     	|1              	|  2.1861e-05     	|  5.4883e-06     	|
on_before_accelerator_backend_setup	|  2.0809e-05     	|1              	|  2.0809e-05     	|  5.2242e-06     	|
on_fit_start                       	|  1.7182e-05     	|1              	|  1.7182e-05     	|  4.3136e-06     	|
on_configure_sharded_model         	|  1.3475e-05     	|1              	|  1.3475e-05     	|  3.383e-06      	|
setup                              	|  1.3144e-05     	|1              	|  1.3144e-05     	|  3.2999e-06     	|
on_pretrain_routine_end            	|  9.939e-06      	|1              	|  9.939e-06      	|  2.4952e-06     	|
teardown                           	|  9.849e-06      	|1              	|  9.849e-06      	|  2.4726e-06     	|
prepare_data                       	|  7.5051e-06     	|1              	|  7.5051e-06     	|  1.8842e-06     	|
configure_callbacks                	|  4.7991e-06     	|1              	|  4.7991e-06     	|  1.2048e-06     	|
configure_sharded_model            	|  4.7591e-06     	|1              	|  4.7591e-06     	|  1.1948e-06     	|
on_train_dataloader                	|  4.539e-06      	|1              	|  4.539e-06      	|  1.1395e-06     	|
on_val_dataloader                  	|  4.438e-06      	|1              	|  4.438e-06      	|  1.1142e-06     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1542.9         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  24.123         	|63             	|  1519.7         	|  98.5           	|
run_training_batch                 	|  0.18556        	|8000           	|  1484.5         	|  96.215         	|
training_step_and_backward         	|  0.17545        	|8000           	|  1403.6         	|  90.974         	|
model_forward                      	|  0.092764       	|8000           	|  742.11         	|  48.099         	|
training_step                      	|  0.092633       	|8000           	|  741.07         	|  48.031         	|
backward                           	|  0.082511       	|8000           	|  660.09         	|  42.783         	|
optimizer_step_with_closure_0      	|  0.23328        	|1000           	|  233.28         	|  15.12          	|
get_train_batch                    	|  0.0022773      	|8063           	|  18.362         	|  1.1901         	|
fetch_next_train_batch             	|  0.0022606      	|8063           	|  18.227         	|  1.1814         	|
on_train_batch_end                 	|  0.0010813      	|8000           	|  8.6502         	|  0.56064        	|
training_batch_to_device           	|  0.00021532     	|8000           	|  1.7226         	|  0.11164        	|
zero_grad                          	|  0.0012879      	|1000           	|  1.2879         	|  0.083474       	|
on_train_batch_start               	|  9.1231e-05     	|8000           	|  0.72984        	|  0.047304       	|
on_batch_start                     	|  1.5441e-05     	|8000           	|  0.12353        	|  0.0080065      	|
on_batch_end                       	|  1.3119e-05     	|8000           	|  0.10495        	|  0.0068021      	|
on_before_backward                 	|  1.3016e-05     	|8000           	|  0.10412        	|  0.0067487      	|
on_after_backward                  	|  1.1561e-05     	|8000           	|  0.092489       	|  0.0059945      	|
training_step_end                  	|  5.9023e-06     	|8000           	|  0.047218       	|  0.0030604      	|
on_train_end                       	|  0.025262       	|1              	|  0.025262       	|  0.0016373      	|
on_pretrain_routine_start          	|  0.022707       	|1              	|  0.022707       	|  0.0014717      	|
on_train_epoch_start               	|  0.00032303     	|63             	|  0.020351       	|  0.001319       	|
on_train_epoch_end                 	|  0.00028086     	|63             	|  0.017694       	|  0.0011468      	|
on_before_zero_grad                	|  1.4534e-05     	|1000           	|  0.014534       	|  0.00094202     	|
on_before_optimizer_step           	|  1.2189e-05     	|1000           	|  0.012189       	|  0.00079002     	|
configure_optimizers               	|  0.0088455      	|1              	|  0.0088455      	|  0.0005733      	|
on_train_start                     	|  0.00074918     	|1              	|  0.00074918     	|  4.8557e-05     	|
on_epoch_start                     	|  1.0761e-05     	|63             	|  0.00067794     	|  4.3939e-05     	|
on_epoch_end                       	|  1.0536e-05     	|63             	|  0.00066377     	|  4.3021e-05     	|
on_fit_end                         	|  2.2902e-05     	|1              	|  2.2902e-05     	|  1.4843e-06     	|
on_before_accelerator_backend_setup	|  2.2192e-05     	|1              	|  2.2192e-05     	|  1.4383e-06     	|
on_fit_start                       	|  1.9327e-05     	|1              	|  1.9327e-05     	|  1.2526e-06     	|
on_configure_sharded_model         	|  1.55e-05       	|1              	|  1.55e-05       	|  1.0046e-06     	|
setup                              	|  1.3325e-05     	|1              	|  1.3325e-05     	|  8.6363e-07     	|
on_pretrain_routine_end            	|  1.1321e-05     	|1              	|  1.1321e-05     	|  7.3376e-07     	|
teardown                           	|  1.043e-05      	|1              	|  1.043e-05      	|  6.7601e-07     	|
prepare_data                       	|  8.967e-06      	|1              	|  8.967e-06      	|  5.8118e-07     	|
configure_callbacks                	|  6.763e-06      	|1              	|  6.763e-06      	|  4.3833e-07     	|
on_val_dataloader                  	|  4.899e-06      	|1              	|  4.899e-06      	|  3.1752e-07     	|
configure_sharded_model            	|  4.7889e-06     	|1              	|  4.7889e-06     	|  3.1038e-07     	|
on_train_dataloader                	|  4.1679e-06     	|1              	|  4.1679e-06     	|  2.7014e-07     	|
