Equivariance is growing in popularity in machine learning, so these tricks will be helpful if one wants to study, or publish, in that area (I'm thinking about it). I'd recommend for any ML-related folks to look into this area and save this thread whenever they're trying to implement an equivariant neural network.