Notice that $n^2 + 2 \ge 3$, $n^2 + 2 > 2n - 1$ and also that numbers $n^2 + 2$ and $2n - 1$ have the same prime factors.
Let $p$ be a prime factor of $n^2 + 2$ and $2n - 1$.
$$p \mid n^2 + 2 \quad \text{and} \quad p \mid 2n - 1 \implies p \mid (n^2 + 2) + (2n - 1) = (n + 1)^2 \implies p \mid n + 1.$$
Therefore, $p \mid 2(n + 1) - (2n - 1) = 3$ and we conclude that $p = 3$.
Hence there are positive integers $k$ and $m$, $k > m$, such that $n^2 + 2 = 3^k$ and $2n - 1 = 3^m$. We get
$$4 \cdot 3^k = 4n^2 + 8 = (3^m + 1)^2 + 8 = 3^{2m} + 2 \cdot 3^m + 9,$$
from which it follows that $3^m$ divides 9, i.e. $m \le 2$.

We have two possibilities:

1) If $m = 1$, then $2n - 1 = 3$, and consequently $n = 2$ and $3^k = 6$, which is impossible.

2) If $m = 2$, then $2n - 1 = 9$, and consequently $n = 5$ and $k = 3$.

Therefore, $n = 5$ is the only positive integer with the given property.