I have the following problem with the function meta_backward:
X = X+g.mul(a[0].mul(-lr)+a[1]).sum() why should i use -lr?(https://github.com/alinlab/L2T-ww/blob/master/train/meta_optimizers.py#L134)
a_new = (a[0].mul(1-lrwd).add_(wd, a[1]).add_(p.grad.data),
a[1].mul(momentum).add_(-lrmomentum, a[0])) what does this function mean?(https://github.com/alinlab/L2T-ww/blob/master/train/meta_optimizers.py#L142-L143)
I would be very appreciated if you can help me:)
I have the following problem with the function meta_backward:
X = X+g.mul(a[0].mul(-lr)+a[1]).sum() why should i use -lr?(https://github.com/alinlab/L2T-ww/blob/master/train/meta_optimizers.py#L134)
a_new = (a[0].mul(1-lrwd).add_(wd, a[1]).add_(p.grad.data),
a[1].mul(momentum).add_(-lrmomentum, a[0])) what does this function mean?(https://github.com/alinlab/L2T-ww/blob/master/train/meta_optimizers.py#L142-L143)
I would be very appreciated if you can help me:)