If $p=2$, our equation becomes $a(a-1) = 2b^2$, whose smallest solution in $\mathbb{N}$ is $a=9$.

Now let $p \ge 3$. Since $a$ and $a^{p-1}-1$ are coprime and $a(a^{p-1}-1) = pb^2$, either $a$ or $a^{p-1}-1$ must be a square, and it is obviously not the latter; hence $a$ is a square. Assume that $a=4$. Then
$$\frac{4^{p-1}-1}{p} = \frac{(2^{p-1}-1)(2^{p-1}+1)}{p}$$
is a square, so either $2^{p-1}-1$ or $2^{p-1}+1$ is a square, but the former is $3$ (mod $4$), so the latter is the square: $2^{p-1}+1 = c^2$. Then $(c+1)(c-1) = 2^{p-1}$, so both $c-1$ and $c+1$ are powers of $2$ and they must be $2$ and $4$, but then $p=4$, a contradiction. In conclusion, $a=9$ is the answer. $\square$