Denote the numbers on the board by $a_1, a_2, \dots, a_{p-1}$.
Without loss of generality we may assume that $a_1 < a_2 < \dots < a_{p-1}$. Then the numbers $a_2 - a_1 < a_3 - a_1 < \dots < a_{p-1} - a_1$ are also written on the blackboard. There are $p-2$ of them and they are all distinct. This is only possible when
$$\begin{aligned} a_{p-2} &= a_{p-1} - a_1 \\ &\vdots \\ a_2 &= a_3 - a_1 \\ a_1 &= a_2 - a_1 \end{aligned}$$
Thus, $a_2 = 2a_1, a_3 = 3a_1, \dots, a_{p-2} = (p-2)a_1$ and $a_{p-1} = (p-1)a_1$. Since $p$ is a prime and it appears on the blackboard we must have $a_1 = p$. We conclude that all the numbers on the blackboard are divisible by $p$.