FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  383.86         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  5.7144         	|63             	|  360.01         	|  93.785         	|
run_training_batch                 	|  0.33336        	|1000           	|  333.36         	|  86.843         	|
optimizer_step_with_closure_0      	|  0.33075        	|1000           	|  330.75         	|  86.162         	|
training_step_and_backward         	|  0.24674        	|1000           	|  246.74         	|  64.279         	|
model_forward                      	|  0.15334        	|1000           	|  153.34         	|  39.946         	|
training_step                      	|  0.1532         	|1000           	|  153.2          	|  39.91          	|
backward                           	|  0.09207        	|1000           	|  92.07          	|  23.985         	|
get_train_batch                    	|  0.0080959      	|1063           	|  8.6059         	|  2.2419         	|
fetch_next_train_batch             	|  0.0080773      	|1063           	|  8.5862         	|  2.2368         	|
zero_grad                          	|  0.0013104      	|1000           	|  1.3104         	|  0.34138        	|
on_train_batch_end                 	|  0.0012001      	|1000           	|  1.2001         	|  0.31263        	|
training_batch_to_device           	|  0.00022788     	|1000           	|  0.22788        	|  0.059365       	|
on_train_batch_start               	|  9.4495e-05     	|1000           	|  0.094495       	|  0.024617       	|
on_train_end                       	|  0.026374       	|1              	|  0.026374       	|  0.0068706      	|
on_pretrain_routine_start          	|  0.022913       	|1              	|  0.022913       	|  0.0059689      	|
on_train_epoch_end                 	|  0.00029568     	|63             	|  0.018628       	|  0.0048527      	|
on_train_epoch_start               	|  0.0002897      	|63             	|  0.018251       	|  0.0047546      	|
on_batch_start                     	|  1.6451e-05     	|1000           	|  0.016451       	|  0.0042856      	|
on_before_zero_grad                	|  1.4245e-05     	|1000           	|  0.014245       	|  0.0037109      	|
on_batch_end                       	|  1.2962e-05     	|1000           	|  0.012962       	|  0.0033766      	|
on_after_backward                  	|  1.2135e-05     	|1000           	|  0.012135       	|  0.0031613      	|
on_before_optimizer_step           	|  1.1167e-05     	|1000           	|  0.011167       	|  0.0029092      	|
on_before_backward                 	|  1.0412e-05     	|1000           	|  0.010412       	|  0.0027124      	|
configure_optimizers               	|  0.008418       	|1              	|  0.008418       	|  0.002193       	|
training_step_end                  	|  6.037e-06      	|1000           	|  0.006037       	|  0.0015727      	|
on_train_start                     	|  0.0007313      	|1              	|  0.0007313      	|  0.00019051     	|
on_epoch_end                       	|  1.1144e-05     	|63             	|  0.0007021      	|  0.0001829      	|
on_epoch_start                     	|  1.053e-05      	|63             	|  0.00066341     	|  0.00017282     	|
on_fit_end                         	|  2.3063e-05     	|1              	|  2.3063e-05     	|  6.0081e-06     	|
on_before_accelerator_backend_setup	|  2.135e-05      	|1              	|  2.135e-05      	|  5.5619e-06     	|
on_fit_start                       	|  1.7403e-05     	|1              	|  1.7403e-05     	|  4.5336e-06     	|
on_configure_sharded_model         	|  1.3906e-05     	|1              	|  1.3906e-05     	|  3.6226e-06     	|
setup                              	|  1.3385e-05     	|1              	|  1.3385e-05     	|  3.4869e-06     	|
teardown                           	|  1.3205e-05     	|1              	|  1.3205e-05     	|  3.44e-06       	|
on_pretrain_routine_end            	|  9.9989e-06     	|1              	|  9.9989e-06     	|  2.6048e-06     	|
prepare_data                       	|  9.0069e-06     	|1              	|  9.0069e-06     	|  2.3464e-06     	|
configure_callbacks                	|  5.4401e-06     	|1              	|  5.4401e-06     	|  1.4172e-06     	|
on_val_dataloader                  	|  4.7691e-06     	|1              	|  4.7691e-06     	|  1.2424e-06     	|
on_train_dataloader                	|  4.448e-06      	|1              	|  4.448e-06      	|  1.1587e-06     	|
configure_sharded_model            	|  4.328e-06      	|1              	|  4.328e-06      	|  1.1275e-06     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1532.2         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  23.955         	|63             	|  1509.1         	|  98.495         	|
run_training_batch                 	|  0.18428        	|8000           	|  1474.2         	|  96.215         	|
training_step_and_backward         	|  0.17424        	|8000           	|  1394.0         	|  90.978         	|
model_forward                      	|  0.092477       	|8000           	|  739.82         	|  48.285         	|
training_step                      	|  0.092346       	|8000           	|  738.77         	|  48.216         	|
backward                           	|  0.081588       	|8000           	|  652.7          	|  42.599         	|
optimizer_step_with_closure_0      	|  0.23291        	|1000           	|  232.91         	|  15.201         	|
get_train_batch                    	|  0.0022593      	|8063           	|  18.217         	|  1.1889         	|
fetch_next_train_batch             	|  0.0022429      	|8063           	|  18.085         	|  1.1803         	|
on_train_batch_end                 	|  0.001074       	|8000           	|  8.592          	|  0.56076        	|
training_batch_to_device           	|  0.00021579     	|8000           	|  1.7263         	|  0.11267        	|
zero_grad                          	|  0.0012937      	|1000           	|  1.2937         	|  0.084432       	|
on_train_batch_start               	|  9.1469e-05     	|8000           	|  0.73175        	|  0.047758       	|
on_batch_start                     	|  1.4716e-05     	|8000           	|  0.11773        	|  0.0076837      	|
on_batch_end                       	|  1.3561e-05     	|8000           	|  0.10849        	|  0.0070808      	|
on_before_backward                 	|  1.2314e-05     	|8000           	|  0.098513       	|  0.0064295      	|
on_after_backward                  	|  1.1116e-05     	|8000           	|  0.088927       	|  0.0058039      	|
training_step_end                  	|  5.8301e-06     	|8000           	|  0.046641       	|  0.0030441      	|
on_train_end                       	|  0.025431       	|1              	|  0.025431       	|  0.0016598      	|
on_pretrain_routine_start          	|  0.022629       	|1              	|  0.022629       	|  0.0014769      	|
on_train_epoch_start               	|  0.00029635     	|63             	|  0.01867        	|  0.0012185      	|
on_train_epoch_end                 	|  0.00027705     	|63             	|  0.017454       	|  0.0011391      	|
on_before_zero_grad                	|  1.4136e-05     	|1000           	|  0.014136       	|  0.00092258     	|
on_before_optimizer_step           	|  1.153e-05      	|1000           	|  0.01153        	|  0.00075252     	|
configure_optimizers               	|  0.0086945      	|1              	|  0.0086945      	|  0.00056746     	|
on_train_start                     	|  0.00072006     	|1              	|  0.00072006     	|  4.6995e-05     	|
on_epoch_start                     	|  1.0384e-05     	|63             	|  0.00065421     	|  4.2698e-05     	|
on_epoch_end                       	|  1.0324e-05     	|63             	|  0.00065042     	|  4.245e-05      	|
on_fit_end                         	|  2.4336e-05     	|1              	|  2.4336e-05     	|  1.5883e-06     	|
on_before_accelerator_backend_setup	|  1.8325e-05     	|1              	|  1.8325e-05     	|  1.196e-06      	|
on_fit_start                       	|  1.7222e-05     	|1              	|  1.7222e-05     	|  1.124e-06      	|
on_configure_sharded_model         	|  1.3294e-05     	|1              	|  1.3294e-05     	|  8.6765e-07     	|
on_pretrain_routine_end            	|  1.2303e-05     	|1              	|  1.2303e-05     	|  8.0296e-07     	|
setup                              	|  1.1773e-05     	|1              	|  1.1773e-05     	|  7.6838e-07     	|
teardown                           	|  1.1471e-05     	|1              	|  1.1471e-05     	|  7.4867e-07     	|
prepare_data                       	|  6.242e-06      	|1              	|  6.242e-06      	|  4.0739e-07     	|
configure_callbacks                	|  5.9619e-06     	|1              	|  5.9619e-06     	|  3.8911e-07     	|
on_val_dataloader                  	|  4.739e-06      	|1              	|  4.739e-06      	|  3.093e-07      	|
configure_sharded_model            	|  4.027e-06      	|1              	|  4.027e-06      	|  2.6283e-07     	|
on_train_dataloader                	|  3.9071e-06     	|1              	|  3.9071e-06     	|  2.55e-07       	|
