FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  386.44         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  5.7566         	|63             	|  362.67         	|  93.847         	|
run_training_batch                 	|  0.33605        	|1000           	|  336.05         	|  86.958         	|
optimizer_step_with_closure_0      	|  0.3334         	|1000           	|  333.4          	|  86.274         	|
training_step_and_backward         	|  0.24809        	|1000           	|  248.09         	|  64.199         	|
model_forward                      	|  0.15394        	|1000           	|  153.94         	|  39.834         	|
training_step                      	|  0.1538         	|1000           	|  153.8          	|  39.799         	|
backward                           	|  0.092818       	|1000           	|  92.818         	|  24.018         	|
get_train_batch                    	|  0.0081021      	|1063           	|  8.6126         	|  2.2287         	|
fetch_next_train_batch             	|  0.0080834      	|1063           	|  8.5927         	|  2.2235         	|
zero_grad                          	|  0.0013156      	|1000           	|  1.3156         	|  0.34044        	|
on_train_batch_end                 	|  0.0011856      	|1000           	|  1.1856         	|  0.30679        	|
training_batch_to_device           	|  0.00022493     	|1000           	|  0.22493        	|  0.058205       	|
on_train_batch_start               	|  9.0521e-05     	|1000           	|  0.090521       	|  0.023424       	|
on_train_end                       	|  0.025347       	|1              	|  0.025347       	|  0.0065591      	|
on_pretrain_routine_start          	|  0.022163       	|1              	|  0.022163       	|  0.005735       	|
on_train_epoch_start               	|  0.00028755     	|63             	|  0.018116       	|  0.0046878      	|
on_train_epoch_end                 	|  0.00026395     	|63             	|  0.016629       	|  0.004303       	|
on_batch_start                     	|  1.6142e-05     	|1000           	|  0.016142       	|  0.004177       	|
on_before_zero_grad                	|  1.3743e-05     	|1000           	|  0.013743       	|  0.0035563      	|
on_batch_end                       	|  1.2963e-05     	|1000           	|  0.012963       	|  0.0033543      	|
on_after_backward                  	|  1.1693e-05     	|1000           	|  0.011693       	|  0.0030258      	|
on_before_backward                 	|  1.0806e-05     	|1000           	|  0.010806       	|  0.0027962      	|
on_before_optimizer_step           	|  1.0231e-05     	|1000           	|  0.010231       	|  0.0026475      	|
configure_optimizers               	|  0.0084156      	|1              	|  0.0084156      	|  0.0021777      	|
training_step_end                  	|  5.9549e-06     	|1000           	|  0.0059549      	|  0.0015409      	|
on_train_start                     	|  0.00072885     	|1              	|  0.00072885     	|  0.00018861     	|
on_epoch_start                     	|  1.0801e-05     	|63             	|  0.00068044     	|  0.00017608     	|
on_epoch_end                       	|  1.0544e-05     	|63             	|  0.00066427     	|  0.00017189     	|
on_fit_end                         	|  2.1871e-05     	|1              	|  2.1871e-05     	|  5.6595e-06     	|
on_before_accelerator_backend_setup	|  2.0458e-05     	|1              	|  2.0458e-05     	|  5.2939e-06     	|
on_fit_start                       	|  1.7001e-05     	|1              	|  1.7001e-05     	|  4.3994e-06     	|
setup                              	|  1.3586e-05     	|1              	|  1.3586e-05     	|  3.5156e-06     	|
on_configure_sharded_model         	|  1.3245e-05     	|1              	|  1.3245e-05     	|  3.4274e-06     	|
teardown                           	|  1.1351e-05     	|1              	|  1.1351e-05     	|  2.9373e-06     	|
on_pretrain_routine_end            	|  1.0009e-05     	|1              	|  1.0009e-05     	|  2.59e-06       	|
prepare_data                       	|  7.1441e-06     	|1              	|  7.1441e-06     	|  1.8487e-06     	|
configure_callbacks                	|  5.1101e-06     	|1              	|  5.1101e-06     	|  1.3223e-06     	|
configure_sharded_model            	|  4.9589e-06     	|1              	|  4.9589e-06     	|  1.2832e-06     	|
on_val_dataloader                  	|  4.5981e-06     	|1              	|  4.5981e-06     	|  1.1898e-06     	|
on_train_dataloader                	|  4.428e-06      	|1              	|  4.428e-06      	|  1.1458e-06     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1547.7         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  24.203         	|63             	|  1524.8         	|  98.518         	|
run_training_batch                 	|  0.18624        	|8000           	|  1490.0         	|  96.266         	|
training_step_and_backward         	|  0.17611        	|8000           	|  1408.9         	|  91.028         	|
model_forward                      	|  0.092448       	|8000           	|  739.58         	|  47.785         	|
training_step                      	|  0.092315       	|8000           	|  738.52         	|  47.716         	|
backward                           	|  0.083478       	|8000           	|  667.83         	|  43.149         	|
optimizer_step_with_closure_0      	|  0.23506        	|1000           	|  235.06         	|  15.187         	|
get_train_batch                    	|  0.00225        	|8063           	|  18.142         	|  1.1722         	|
fetch_next_train_batch             	|  0.0022336      	|8063           	|  18.009         	|  1.1636         	|
on_train_batch_end                 	|  0.0010769      	|8000           	|  8.6148         	|  0.55661        	|
training_batch_to_device           	|  0.00021116     	|8000           	|  1.6893         	|  0.10914        	|
zero_grad                          	|  0.0013212      	|1000           	|  1.3212         	|  0.085364       	|
on_train_batch_start               	|  8.9789e-05     	|8000           	|  0.71831        	|  0.04641        	|
on_batch_start                     	|  1.4789e-05     	|8000           	|  0.11831        	|  0.007644       	|
on_before_backward                 	|  1.3232e-05     	|8000           	|  0.10585        	|  0.0068392      	|
on_batch_end                       	|  1.3097e-05     	|8000           	|  0.10477        	|  0.0067695      	|
on_after_backward                  	|  1.1591e-05     	|8000           	|  0.092731       	|  0.0059914      	|
training_step_end                  	|  5.5463e-06     	|8000           	|  0.04437        	|  0.0028668      	|
on_train_end                       	|  0.025312       	|1              	|  0.025312       	|  0.0016354      	|
on_pretrain_routine_start          	|  0.023          	|1              	|  0.023          	|  0.001486       	|
on_train_epoch_start               	|  0.00030214     	|63             	|  0.019035       	|  0.0012299      	|
on_train_epoch_end                 	|  0.00029056     	|63             	|  0.018305       	|  0.0011827      	|
on_before_zero_grad                	|  1.4482e-05     	|1000           	|  0.014482       	|  0.00093567     	|
on_before_optimizer_step           	|  1.1448e-05     	|1000           	|  0.011448       	|  0.00073964     	|
configure_optimizers               	|  0.0082794      	|1              	|  0.0082794      	|  0.00053493     	|
on_train_start                     	|  0.00072419     	|1              	|  0.00072419     	|  4.679e-05      	|
on_epoch_start                     	|  1.0385e-05     	|63             	|  0.00065427     	|  4.2273e-05     	|
on_epoch_end                       	|  1.0189e-05     	|63             	|  0.00064192     	|  4.1475e-05     	|
on_fit_end                         	|  2.1119e-05     	|1              	|  2.1119e-05     	|  1.3645e-06     	|
on_before_accelerator_backend_setup	|  1.9857e-05     	|1              	|  1.9857e-05     	|  1.283e-06      	|
on_fit_start                       	|  1.7303e-05     	|1              	|  1.7303e-05     	|  1.118e-06      	|
on_configure_sharded_model         	|  1.4187e-05     	|1              	|  1.4187e-05     	|  9.1663e-07     	|
setup                              	|  1.2693e-05     	|1              	|  1.2693e-05     	|  8.201e-07      	|
on_pretrain_routine_end            	|  1.2123e-05     	|1              	|  1.2123e-05     	|  7.8327e-07     	|
teardown                           	|  9.818e-06      	|1              	|  9.818e-06      	|  6.3434e-07     	|
prepare_data                       	|  7.7949e-06     	|1              	|  7.7949e-06     	|  5.0363e-07     	|
configure_callbacks                	|  5.6801e-06     	|1              	|  5.6801e-06     	|  3.67e-07       	|
on_val_dataloader                  	|  4.709e-06      	|1              	|  4.709e-06      	|  3.0425e-07     	|
configure_sharded_model            	|  4.438e-06      	|1              	|  4.438e-06      	|  2.8674e-07     	|
on_train_dataloader                	|  4.0571e-06     	|1              	|  4.0571e-06     	|  2.6213e-07     	|
