We claim only $n = 3$ and $n = 4$ are the only two solutions. First, it is clear that both solutions work.
Next, we claim that $n < 5$ . For $n \geq 5$ , let $x$ be the smallest $x$ such that $x+1$ is not a factor of $n!$ . Let the smallest factor larger than $x$ be $x+k$ .
Now we consider $\frac{n!}{x-1}$ , $\frac{n!}{x}$ and $\frac{n!}{x+k}$ . Since $\frac{n!}{x-1} > \frac{n!}{x} > \frac{n!}{x+k}$ , if $n$ were to satisfy the conditions, then $\frac{n!}{x-1}-\frac{n!}{x} \geq \frac{n!}{x} - \frac{n!}{x+k}$ . However, note that this is not true for $x \geq 5$ and $k > 1$ .
Note that the inequality is equivalent to showing $\frac{1}{x(x-1)} \geq \frac{k}{x(x+k)}$ , which simplifies to $x+k \geq kx-k$ , or $\frac{x}{x-2} \geq k \geq 2$ . This implies $x \geq 2x-4$ , $x \leq 4$ , a contradiction, since the set of numbers $\{1, 2, 3, 4, 5\}$ are all factors of $n!$ , and the value of $x$ must exist. Hence, no solutions for $n \geq 5$ .