Understanding pytorch’s autograd with grad_fn and next_functionsAs we know, the gradient is automatically calculated in pytorch. The key is the property of grad_fn of the final loss function and the…Sep 13, 20212Sep 13, 20212
[Deep learning pipeline] Dataset managementAs there is lots of engineering when we work on deep learning projects, I will present some tips I learned in practice for data/job/model…Mar 26, 20211Mar 26, 20211
Gradient backpropagation with torch.distributed.all_gatherThis blog presents some tips on how to use torch.distributed.all_gather and make sure the gradient is correctly calculated for deep…Feb 7, 20212Feb 7, 20212
A simple introduction on Sinkhorn distancesThe Sinkhorn distance [1] was proposed in 2013, and the work was accepted in NIPS conference. The goal is to improve the optimization…Dec 19, 2020Dec 19, 2020
