We claim that those integers are $n=2$ and $n=3$.

Suppose $n$ is a composite integer, $n=a b$, $1<a \leq b$. If $a<b$, then $n=a b$ divides $b!$ and $(b+1)!$. Since $b<b+1<n-1$, it follows that $b!$ and $(b+1)!$ yield equal remainders (both $0$) at division by $n$. If $2<a=b$ then $(2 a)!$ and $(2 a+1)!$ are divisible by $n=a^{2}$. Since $2 a+1<a^{2}-1=n-1$, it follows that $n$ does not satisfy the desired property.

It remains to settle the case of prime $n$. The value $n=2$ checks; let then $n \geq 3$. From Wilson's theorem we have
$$(n-1)!\equiv-1 \quad(\bmod n)$$
and then
$$(n-2)!\equiv 1 \quad(\bmod n) .$$
Since $1!=1$ yields remainder $1$ at division by $n$, it follows $n-2=1$, that is $n=3$, which checks.