First observe that $n = k^2 - 1$ for some positive integer $k$. As $n$ is not a perfect square, $\tau(n) = k$ must be even, ergo $n$ is odd.
We now establish a bound on $\tau(n)$. Recall that, since $n$ is not a perfect square, there exists a bijective correspondence between factors of $n$ greater than $\lfloor\sqrt{n}\rfloor = k-1$ and factors of $n$ at most $\lfloor\sqrt{n}\rfloor$. Furthermore, since $n$ is odd, all divisors of $n$ must also be odd. Therefore
$$k = \tau(n) \le 2 \cdot |\{1, 3, \dots, k-1\}| = 2 \cdot \frac{k}{2} = k.$$
Hence equality must hold, and so every odd number between 1 and $k-1$ must divide $n = k^2 - 1$.
In particular, $k-3$ must divide $k^2-1$; but
$$\frac{k^2 - 1}{k - 3} = k + 3 + \frac{8}{k - 3}.$$
Therefore, $k-3$ is either $-1$ or $1$, so $k$ equals either $2$ or $4$ and $n \in \{3, 15\}$, both of these work. $\square$