What does mean «train_config» → «gradient_clipping_by_norm» in TensorFlow?

dmitrii_fediuk (Dmitrii Fediuk) May 19, 2019, 1:29am 1

github.com

tensorflow/models/blob/v1.13.0/research/object_detection/samples/configs/faster_rcnn_resnet101_voc07.config#L105-L105


gradient_clipping_by_norm: 10.0

dmitrii_fediuk (Dmitrii Fediuk) May 19, 2019, 1:30am 2

github.com

tensorflow/models/blob/v1.13.0/research/object_detection/protos/train.proto#L28-L29


      
          // If greater than 0, clips gradients by this value.
          optional float gradient_clipping_by_norm = 6 [default=0.0];

This is necessary to avoid exploding gradients.
We set the value of 10 through experimentation but it can be adjusted.

blog.algorithmia.com/deep-dive-into-object-detection-with-open-images-using-tensorflow