I originally posted this on the Mathematics Stack Exchange site, but I think it fits more on the OR-site.
I'm reading a paper where the goal is to determine the weights of a weighted arithmetic mean to create a sales forecast for a future event (e.g. a promotion). The weights weigh a vector of historical sales of historical events, in the following equation denoted as the vector y.
$$\hat{y}_k = \frac{w^Ty}{w^T1} \tag{1} $$ where w, y and 1 are all vectors of size $N$.
Now every event can be described by some numerical features of size $m$ (or categorical features transformed to numerical ones). And ofcourse, the event we are trying to forecasts has this vector of features as well.
What they are trying to do is to estimate the weights by looking how similar each historical event is compared to the event we are trying to forecast (by looking at these features). Then a more similar event will have more weight and the volume of that event will thus have more impact on the value of our forecast.
To measure the similarity, they use the Euclidean distance on the vector of features that describes each event. In the paper, they normalize each feature! This will give us an Euclidean distance matrix A of size $N \times m$: $$A^{(e)} = (X - 1x^e)\space\odot\space(X-1x^e) $$ With the superscript (e) to indicate that the result has been calculated wrt the $e^{th}$ event, matrix X which contains the numerical features for every historical event and $x^e$ the vector of features is of the event we are trying to forecast. But every numerical feature has its relative importance, so we also want to find a vector v that scales the matrix A so that you eventually get a vector of size $N$. This final vector can we call the scaled Euclidean distance of every historical event compared to the event of which its volume we are trying to forecast.
So the more similar -the lower the scaled Euclidean distance distance is for an event compared to the event we are trying to forecast- the more weight we want to give it. So the weights are inversely proportional to the scaled Euclidean distance which is obvious.
And here comes the question:
In the paper they take the inverse square root of the Euclidean distance matrix to define the relationship between the weights and the similarity:
$$\text{weight}_j = \frac{1}{\sqrt{a^iv}}\tag{2}$$ where $a^i$ is the $i$th row of the matrix A and $j$ represents each of the historical events.
But why do they use the square root? Why not just:
$$\mathrm{weight}_j= \frac{1}{a^iv}$$ or $$\mathrm{weight}_j= \frac{1}{(a^iv)^2}$$
Why do we use the inversed square root and not just the inverse or the inversed square to define the proportionality ?
I can think of some reasons but it's not clear if these are the right ones:
- A is a matrix of squared distances and thus maybe they use the square root to convert back to unit distance. But in optimization theory, often squared distances are preferred, so that's kinda weird. This could be due to the fact that features are normalized (interval 0 and 1).
- They want to use the squared distance but the relationship between the weights and the squared distance is assumed to be non-linear (else it would just be the inverse) and twice the squared distance between two events means the impact on the forecast will be lower than two (else it would be the inverse square root).
- Numerical stability (because of the normalized features and the squared distance matrix that's being built).
- The square root has some mathematical properties in optimization that these other two don't have (feel free to elaborate on this).
So the answer I'm looking for depends on the reason behind using the square root,
or it necessary to be mathematically consistent; then I want to know: how so?
or it is just a model assumption and best practice: then I want to know: why is it best practice to use the square root?
IMPORTANT
To be complete: the paper continues with using Eq. (2) and substituting it to the Eq. (1) to get an optimization problem which is used to find the vector v: $$\hat{v} =\operatorname{argmin}_{v^*}\left[ \left\lVert y_k - \frac{\sum_{i=1}^n \frac{1}{\sqrt{a^iv}}y_i}{\sum_{i=1}^n \frac{1}{\sqrt{a^iv}}}\right\rVert^2 \right]$$ Once v is found, we calculate the weights with Eq. (2):
$$\text{weight}_j = \frac{1}{\sqrt{a^iv}}$$ and using this in the Eq. (1):
$$\hat{y}_k = \frac{w^Ty}{w^T1}$$ to eventually get our forecast.
Reference of the paper: https://oa.upm.es/67130/1/INVE_MEM_2019_334744.pdf
