FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  401.92         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  6.0024         	|63             	|  378.15         	|  94.086         	|
run_training_batch                 	|  0.35147        	|1000           	|  351.47         	|  87.448         	|
optimizer_step_with_closure_0      	|  0.34876        	|1000           	|  348.76         	|  86.773         	|
training_step_and_backward         	|  0.26094        	|1000           	|  260.94         	|  64.923         	|
model_forward                      	|  0.16133        	|1000           	|  161.33         	|  40.141         	|
training_step                      	|  0.16119        	|1000           	|  161.19         	|  40.105         	|
backward                           	|  0.098261       	|1000           	|  98.261         	|  24.448         	|
get_train_batch                    	|  0.0081114      	|1063           	|  8.6224         	|  2.1453         	|
fetch_next_train_batch             	|  0.008092       	|1063           	|  8.6018         	|  2.1402         	|
zero_grad                          	|  0.0013182      	|1000           	|  1.3182         	|  0.32798        	|
on_train_batch_end                 	|  0.0012186      	|1000           	|  1.2186         	|  0.30319        	|
training_batch_to_device           	|  0.00022591     	|1000           	|  0.22591        	|  0.056208       	|
on_train_batch_start               	|  9.189e-05      	|1000           	|  0.09189        	|  0.022863       	|
on_train_end                       	|  0.025843       	|1              	|  0.025843       	|  0.00643        	|
on_pretrain_routine_start          	|  0.022508       	|1              	|  0.022508       	|  0.0056001      	|
on_train_epoch_end                 	|  0.00030129     	|63             	|  0.018982       	|  0.0047227      	|
on_train_epoch_start               	|  0.00029851     	|63             	|  0.018806       	|  0.0046791      	|
on_batch_start                     	|  1.6641e-05     	|1000           	|  0.016641       	|  0.0041405      	|
on_before_zero_grad                	|  1.3814e-05     	|1000           	|  0.013814       	|  0.003437       	|
on_batch_end                       	|  1.3759e-05     	|1000           	|  0.013759       	|  0.0034234      	|
on_after_backward                  	|  1.1532e-05     	|1000           	|  0.011532       	|  0.0028692      	|
on_before_optimizer_step           	|  1.1051e-05     	|1000           	|  0.011051       	|  0.0027496      	|
on_before_backward                 	|  1.0744e-05     	|1000           	|  0.010744       	|  0.0026733      	|
configure_optimizers               	|  0.0082025      	|1              	|  0.0082025      	|  0.0020408      	|
training_step_end                  	|  6.1139e-06     	|1000           	|  0.0061139      	|  0.0015212      	|
on_train_start                     	|  0.00076064     	|1              	|  0.00076064     	|  0.00018925     	|
on_epoch_end                       	|  1.0914e-05     	|63             	|  0.00068757     	|  0.00017107     	|
on_epoch_start                     	|  1.0597e-05     	|63             	|  0.00066762     	|  0.00016611     	|
on_fit_end                         	|  2.1881e-05     	|1              	|  2.1881e-05     	|  5.4442e-06     	|
on_before_accelerator_backend_setup	|  1.8886e-05     	|1              	|  1.8886e-05     	|  4.699e-06      	|
on_fit_start                       	|  1.7212e-05     	|1              	|  1.7212e-05     	|  4.2825e-06     	|
on_configure_sharded_model         	|  1.3344e-05     	|1              	|  1.3344e-05     	|  3.3201e-06     	|
setup                              	|  1.2373e-05     	|1              	|  1.2373e-05     	|  3.0785e-06     	|
teardown                           	|  1.043e-05      	|1              	|  1.043e-05      	|  2.5951e-06     	|
on_pretrain_routine_end            	|  1.0109e-05     	|1              	|  1.0109e-05     	|  2.5152e-06     	|
prepare_data                       	|  7.204e-06      	|1              	|  7.204e-06      	|  1.7924e-06     	|
configure_sharded_model            	|  4.929e-06      	|1              	|  4.929e-06      	|  1.2264e-06     	|
configure_callbacks                	|  4.809e-06      	|1              	|  4.809e-06      	|  1.1965e-06     	|
on_train_dataloader                	|  4.7591e-06     	|1              	|  4.7591e-06     	|  1.1841e-06     	|
on_val_dataloader                  	|  4.589e-06      	|1              	|  4.589e-06      	|  1.1418e-06     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1560.9         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  24.412         	|63             	|  1538.0         	|  98.534         	|
run_training_batch                 	|  0.18786        	|8000           	|  1502.9         	|  96.286         	|
training_step_and_backward         	|  0.17772        	|8000           	|  1421.8         	|  91.088         	|
model_forward                      	|  0.092764       	|8000           	|  742.11         	|  47.545         	|
training_step                      	|  0.092632       	|8000           	|  741.06         	|  47.478         	|
backward                           	|  0.084773       	|8000           	|  678.19         	|  43.45          	|
optimizer_step_with_closure_0      	|  0.2359         	|1000           	|  235.9          	|  15.113         	|
get_train_batch                    	|  0.0022783      	|8063           	|  18.37          	|  1.1769         	|
fetch_next_train_batch             	|  0.0022619      	|8063           	|  18.238         	|  1.1684         	|
on_train_batch_end                 	|  0.0010712      	|8000           	|  8.5698         	|  0.54905        	|
training_batch_to_device           	|  0.00021355     	|8000           	|  1.7084         	|  0.10945        	|
zero_grad                          	|  0.0013076      	|1000           	|  1.3076         	|  0.083775       	|
on_train_batch_start               	|  9.0699e-05     	|8000           	|  0.72559        	|  0.046487       	|
on_batch_start                     	|  1.4881e-05     	|8000           	|  0.11905        	|  0.007627       	|
on_batch_end                       	|  1.3401e-05     	|8000           	|  0.10721        	|  0.0068685      	|
on_before_backward                 	|  1.2548e-05     	|8000           	|  0.10039        	|  0.0064315      	|
on_after_backward                  	|  1.1244e-05     	|8000           	|  0.08995        	|  0.0057629      	|
training_step_end                  	|  5.7312e-06     	|8000           	|  0.04585        	|  0.0029375      	|
on_train_end                       	|  0.025464       	|1              	|  0.025464       	|  0.0016314      	|
on_pretrain_routine_start          	|  0.022695       	|1              	|  0.022695       	|  0.001454       	|
on_train_epoch_start               	|  0.00029003     	|63             	|  0.018272       	|  0.0011706      	|
on_train_epoch_end                 	|  0.0002834      	|63             	|  0.017854       	|  0.0011439      	|
on_before_zero_grad                	|  1.4164e-05     	|1000           	|  0.014164       	|  0.00090742     	|
on_before_optimizer_step           	|  1.1548e-05     	|1000           	|  0.011548       	|  0.00073983     	|
configure_optimizers               	|  0.0084428      	|1              	|  0.0084428      	|  0.00054091     	|
on_train_start                     	|  0.00069035     	|1              	|  0.00069035     	|  4.4229e-05     	|
on_epoch_start                     	|  1.0785e-05     	|63             	|  0.00067946     	|  4.3531e-05     	|
on_epoch_end                       	|  1.0288e-05     	|63             	|  0.00064813     	|  4.1524e-05     	|
on_fit_end                         	|  1.9717e-05     	|1              	|  1.9717e-05     	|  1.2632e-06     	|
on_before_accelerator_backend_setup	|  1.8445e-05     	|1              	|  1.8445e-05     	|  1.1817e-06     	|
on_fit_start                       	|  1.616e-05      	|1              	|  1.616e-05      	|  1.0353e-06     	|
on_configure_sharded_model         	|  1.3946e-05     	|1              	|  1.3946e-05     	|  8.9349e-07     	|
setup                              	|  1.2533e-05     	|1              	|  1.2533e-05     	|  8.0296e-07     	|
teardown                           	|  9.5479e-06     	|1              	|  9.5479e-06     	|  6.1171e-07     	|
on_pretrain_routine_end            	|  9.347e-06      	|1              	|  9.347e-06      	|  5.9884e-07     	|
configure_callbacks                	|  6.4629e-06     	|1              	|  6.4629e-06     	|  4.1406e-07     	|
prepare_data                       	|  6.211e-06      	|1              	|  6.211e-06      	|  3.9792e-07     	|
on_val_dataloader                  	|  4.8291e-06     	|1              	|  4.8291e-06     	|  3.0939e-07     	|
configure_sharded_model            	|  4.1679e-06     	|1              	|  4.1679e-06     	|  2.6703e-07     	|
on_train_dataloader                	|  3.987e-06      	|1              	|  3.987e-06      	|  2.5544e-07     	|
