FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  387.61         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  5.777          	|63             	|  363.95         	|  93.896         	|
run_training_batch                 	|  0.33717        	|1000           	|  337.17         	|  86.986         	|
optimizer_step_with_closure_0      	|  0.33454        	|1000           	|  334.54         	|  86.308         	|
training_step_and_backward         	|  0.24946        	|1000           	|  249.46         	|  64.358         	|
model_forward                      	|  0.15487        	|1000           	|  154.87         	|  39.954         	|
training_step                      	|  0.15472        	|1000           	|  154.72         	|  39.917         	|
backward                           	|  0.093297       	|1000           	|  93.297         	|  24.07          	|
get_train_batch                    	|  0.0082033      	|1063           	|  8.7201         	|  2.2497         	|
fetch_next_train_batch             	|  0.0081838      	|1063           	|  8.6994         	|  2.2444         	|
zero_grad                          	|  0.00127        	|1000           	|  1.27           	|  0.32765        	|
on_train_batch_end                 	|  0.0011996      	|1000           	|  1.1996         	|  0.30949        	|
training_batch_to_device           	|  0.00022513     	|1000           	|  0.22513        	|  0.058082       	|
on_train_batch_start               	|  9.1643e-05     	|1000           	|  0.091643       	|  0.023643       	|
on_train_end                       	|  0.026123       	|1              	|  0.026123       	|  0.0067395      	|
on_pretrain_routine_start          	|  0.022693       	|1              	|  0.022693       	|  0.0058547      	|
on_train_epoch_start               	|  0.00029721     	|63             	|  0.018724       	|  0.0048306      	|
on_train_epoch_end                 	|  0.00028676     	|63             	|  0.018066       	|  0.0046608      	|
on_batch_start                     	|  1.6244e-05     	|1000           	|  0.016244       	|  0.0041908      	|
on_before_zero_grad                	|  1.3898e-05     	|1000           	|  0.013898       	|  0.0035855      	|
on_batch_end                       	|  1.3328e-05     	|1000           	|  0.013328       	|  0.0034384      	|
on_after_backward                  	|  1.1651e-05     	|1000           	|  0.011651       	|  0.0030059      	|
on_before_backward                 	|  1.0469e-05     	|1000           	|  0.010469       	|  0.0027009      	|
on_before_optimizer_step           	|  1.0155e-05     	|1000           	|  0.010155       	|  0.00262        	|
configure_optimizers               	|  0.0081125      	|1              	|  0.0081125      	|  0.0020929      	|
training_step_end                  	|  5.8876e-06     	|1000           	|  0.0058876      	|  0.0015189      	|
on_train_start                     	|  0.0007294      	|1              	|  0.0007294      	|  0.00018818     	|
on_epoch_start                     	|  1.0614e-05     	|63             	|  0.00066871     	|  0.00017252     	|
on_epoch_end                       	|  1.0397e-05     	|63             	|  0.00065502     	|  0.00016899     	|
on_fit_end                         	|  2.123e-05      	|1              	|  2.123e-05      	|  5.4772e-06     	|
on_before_accelerator_backend_setup	|  1.9897e-05     	|1              	|  1.9897e-05     	|  5.1332e-06     	|
on_fit_start                       	|  1.654e-05      	|1              	|  1.654e-05      	|  4.2671e-06     	|
setup                              	|  1.3746e-05     	|1              	|  1.3746e-05     	|  3.5463e-06     	|
on_configure_sharded_model         	|  1.3375e-05     	|1              	|  1.3375e-05     	|  3.4506e-06     	|
teardown                           	|  1.0479e-05     	|1              	|  1.0479e-05     	|  2.7035e-06     	|
on_pretrain_routine_end            	|  9.608e-06      	|1              	|  9.608e-06      	|  2.4788e-06     	|
prepare_data                       	|  8.345e-06      	|1              	|  8.345e-06      	|  2.1529e-06     	|
configure_sharded_model            	|  4.9099e-06     	|1              	|  4.9099e-06     	|  1.2667e-06     	|
configure_callbacks                	|  4.7081e-06     	|1              	|  4.7081e-06     	|  1.2146e-06     	|
on_val_dataloader                  	|  4.528e-06      	|1              	|  4.528e-06      	|  1.1682e-06     	|
on_train_dataloader                	|  4.178e-06      	|1              	|  4.178e-06      	|  1.0779e-06     	|
FIT Profiler Report

Action                             	|  Mean duration (s)	|Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------
Total                              	|  -              	|_              	|  1535.2         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------
run_training_epoch                 	|  24.003         	|63             	|  1512.2         	|  98.498         	|
run_training_batch                 	|  0.18466        	|8000           	|  1477.3         	|  96.225         	|
training_step_and_backward         	|  0.17465        	|8000           	|  1397.2         	|  91.01          	|
model_forward                      	|  0.092141       	|8000           	|  737.13         	|  48.014         	|
training_step                      	|  0.092009       	|8000           	|  736.07         	|  47.945         	|
backward                           	|  0.082331       	|8000           	|  658.65         	|  42.902         	|
optimizer_step_with_closure_0      	|  0.23268        	|1000           	|  232.68         	|  15.156         	|
get_train_batch                    	|  0.0022591      	|8063           	|  18.215         	|  1.1865         	|
fetch_next_train_batch             	|  0.0022426      	|8063           	|  18.082         	|  1.1778         	|
on_train_batch_end                 	|  0.0010629      	|8000           	|  8.5031         	|  0.55386        	|
training_batch_to_device           	|  0.00021367     	|8000           	|  1.7093         	|  0.11134        	|
zero_grad                          	|  0.0013144      	|1000           	|  1.3144         	|  0.085615       	|
on_train_batch_start               	|  8.9798e-05     	|8000           	|  0.71839        	|  0.046793       	|
on_batch_start                     	|  1.4924e-05     	|8000           	|  0.11939        	|  0.0077767      	|
on_before_backward                 	|  1.3704e-05     	|8000           	|  0.10963        	|  0.0071409      	|
on_batch_end                       	|  1.3142e-05     	|8000           	|  0.10513        	|  0.0068479      	|
on_after_backward                  	|  1.185e-05      	|8000           	|  0.094799       	|  0.0061748      	|
training_step_end                  	|  5.9685e-06     	|8000           	|  0.047748       	|  0.0031101      	|
on_train_end                       	|  0.025522       	|1              	|  0.025522       	|  0.0016624      	|
on_pretrain_routine_start          	|  0.0225         	|1              	|  0.0225         	|  0.0014656      	|
on_train_epoch_end                 	|  0.00029233     	|63             	|  0.018417       	|  0.0011996      	|
on_train_epoch_start               	|  0.00028665     	|63             	|  0.018059       	|  0.0011763      	|
on_before_zero_grad                	|  1.4816e-05     	|1000           	|  0.014816       	|  0.00096507     	|
on_before_optimizer_step           	|  1.1725e-05     	|1000           	|  0.011725       	|  0.00076371     	|
configure_optimizers               	|  0.0088531      	|1              	|  0.0088531      	|  0.00057665     	|
on_train_start                     	|  0.00072377     	|1              	|  0.00072377     	|  4.7143e-05     	|
on_epoch_start                     	|  1.055e-05      	|63             	|  0.00066462     	|  4.3291e-05     	|
on_epoch_end                       	|  1.0225e-05     	|63             	|  0.00064418     	|  4.1959e-05     	|
on_before_accelerator_backend_setup	|  2.3103e-05     	|1              	|  2.3103e-05     	|  1.5048e-06     	|
on_fit_end                         	|  2.2082e-05     	|1              	|  2.2082e-05     	|  1.4383e-06     	|
on_fit_start                       	|  1.8395e-05     	|1              	|  1.8395e-05     	|  1.1982e-06     	|
setup                              	|  1.3846e-05     	|1              	|  1.3846e-05     	|  9.0187e-07     	|
on_configure_sharded_model         	|  1.3816e-05     	|1              	|  1.3816e-05     	|  8.9992e-07     	|
on_pretrain_routine_end            	|  1.2233e-05     	|1              	|  1.2233e-05     	|  7.9681e-07     	|
teardown                           	|  9.7081e-06     	|1              	|  9.7081e-06     	|  6.3235e-07     	|
prepare_data                       	|  9.2669e-06     	|1              	|  9.2669e-06     	|  6.0361e-07     	|
configure_callbacks                	|  6.1209e-06     	|1              	|  6.1209e-06     	|  3.9869e-07     	|
configure_sharded_model            	|  4.608e-06      	|1              	|  4.608e-06      	|  3.0014e-07     	|
on_train_dataloader                	|  4.478e-06      	|1              	|  4.478e-06      	|  2.9168e-07     	|
on_val_dataloader                  	|  4.3181e-06     	|1              	|  4.3181e-06     	|  2.8126e-07     	|
