Let $p$ be a prime satisfying the condition of the problem. By assumption, there is a positive integer $A$ such that $3^{p-1}-1=p A^{2}$.

It is clear that $p=2$ is a solution. Now, we consider $p>2$. Put $p-1=2k$, one has $(3^{k}-1)(3^{k}+1)=p A^{2}$. Since $(3^{k}-1,3^{k}+1)=2$, it follows that there are positive integers $B$ and $C$ such that either
$$3^{k}-1=2p B^{2}, \quad 3^{k}+1=2 C^{2}$$
or
$$3^{k}-1=B^{2}, \quad 3^{k}+1=2p C^{2} .$$
But, the first case cannot hold since $2 C^{2}=3^{k}+1 \equiv 1 \pmod{3}$, we get $C^{2} \equiv 2 \pmod{3}$ which is impossible.

For the second case, if $k$ is odd then $4 \mid 3^{k}+1=2p C^{2}$, hence $2 \mid C$ (since $p$ is odd). This turns out to be that $3^{k}+1=2p C^{2}$ is divisible by $8$ which contradicts to $3^{k}+1 \equiv 4 \pmod{8}$ (since $k$ is odd). Thus, $k$ must be even. Put $k=2m$, then
$$2 B^{2}=3^{k}-1=(3^{m}-1)(3^{m}+1) .$$
Again, since $(3^{m}-1,3^{m}+1)=2$, there are positive integers $D, E$ such that either
$$3^{m}-1=E^{2}, \quad 3^{m}+1=2 D^{2}$$
or
$$3^{m}-1=2 E^{2}, \quad 3^{m}+1=D^{2} .$$
As above, the equality $3^{m}+1=2 D^{2}$ leads to a contradiction. Hence, $3^{m}+ 1=D^{2}$, that is
$$3^{m}=D^{2}-1=(D-1)(D+1) .$$
Therefore, there are non-negative integers $t>s$ such that $D-1=3^{s}$ and $D+1=3^{t}$. This gives,
$$2=(D+1)-(D-1)=3^{t}-3^{s}=3^{s}(3^{t-s}-1) .$$
This happens if and only if $3^{s}=1$ and $3^{t-s}-1=2$, i.e. $s=0$ and $t=1$, we find that $p=5$ (satisfied).

In conclusion, $p=2 ; 5$. $\square$