Let $a_1, a_2, \dots, a_n$ denote the numbers on the blackboard and let $X = \{1, 2, \dots, n\}$ be the index set.
The remainder corresponding to an index subset $A \subseteq X$ is the remainder of the sum $\sum_{k \in A} a_k$ modulo $n$ and we denote it by $\sigma(A)$. Here we assume $\sigma(\emptyset) = 0$. For $0 \le k \le n-1$, let $S_k = |\{A \subseteq X \mid \sigma(A) \equiv k \pmod{n}\}|$ denote the number of index subsets that give the remainder $k$. From the assumption, we have $S_1 = S_2 = \dots = S_{p-1} = S_{p+1} = \dots = S_{n-1}$.

Now set $N = \sigma(X) = a_1 + a_2 + \dots + a_n \pmod{n}$ and consider the sum
$$T = \sum_{A \subseteq X} \sigma(A) \pmod{n}.$$
Since each $k \in X$ belongs to exactly $2^{n-1}$ subsets, we have $T \equiv 2^{n-1}N \pmod{n}$. On the other hand, we have
$$T \equiv \sum_{k=0}^{n-1} k S_k \equiv 0S_0 + pS_p + \left(\frac{(n-1)n}{2} - p\right) S_1 \pmod{n}.$$
Therefore, we get $2^{n-1}N \equiv p(S_p + 2(p-1)S_1) \pmod{n}$ and hence $N \equiv 0 \pmod{p}$ since $p$ is odd.