Python – set `torch.backends.cudnn.benchmark = True` or not

python, pytorch

I am using pytorch and I wonder if I should use torch.backends.cudnn.benchmark = True. I find on google that I should use it when computation graph does not change. What is computation graph in pytorch?

Best Solution

If your model does not change and your input sizes remain the same - then you may benefit from setting torch.backends.cudnn.benchmark = True.
However, if your model changes: for instance, if you have layers that are only "activated" when certain conditions are met, or you have layers inside a loop that can be iterated a different number of times, then setting torch.backends.cudnn.benchmark = True might stall your execution.