Minimum operations of given type to make all elements of a matrix equal

Ques:
Given an integer K and a matrix of N rows and M columns, the task is to find the minimum number of operations required to make all the elements of the matrix equal. In a single operation, K can be added to or subtracted from any element of the matrix. Print -1 if it is impossible to do so.

Input 1:
    A = [   [0, 2, 8]
            [8, 2, 0]
            [0, 2, 8]   ]
    K = 2
Output 1:
    12

Input 2:
    A = [   [5, 17, 100, 11]
            [0, 0,  2,   8]    ]
    K = 3
Output 2:
    -1

Solution:
Since we are only allowed to add or subtract K from any element, we can easily infer that mod of all the elements with K should be equal because x % K = (x + K) % K = (x – K) % K.
If that is not the case simply print -1. Else, sort all the elements of the matrix in non-deceasing order and find the median of the sorted elements. The minimum number of steps would occur if we convert all the elements equal to the median. Calculate these steps and print the result.

Can anyone give reason for this ques that why all the elements are to be converted to the median ?