I read on Wikipedia:
AdaGrad (for adaptive gradient algorithm) is a modified stochastic gradient descent algorithm with per-parameter learning rate, first published in 2011. Informally, this increases the learning rate for sparser parameters and decreases the learning rate for ones that are less sparse.
What do they mean by parameter sparsity? I read that AdaGrad averages / accumulates gradients over time. Does parameter sparsity refer to how often (dense?) these gradients are accumulated? or something else?