For $n = 6$, one can take $a_1 = a_2 = a_3 = a_4 = 1$ and $a_5 = a_6 = -1$; then the quadratic from the problem becomes $x^2 - 8x + 7$ and has two integer roots: $1$ and $7$. It remains to show that this is the smallest possible value of $n$.

Suppose the numbers $a_1, a_2, \dots, a_n$ satisfy the problem's condition; then the discriminant of the quadratic, divided by $4$, must be a perfect square. It equals
$$d = (a_1 + a_2 + \dots + a_n)^4 - (a_1^4 + a_2^4 + \dots + a_n^4 + 1).$$
Then the number $d$ is odd and is a square, so it leaves remainder $1$ when divided by $8$.

Rewrite the above equality as
$$d + 1 + a_1^4 + a_2^4 + \dots + a_n^4 = (a_1 + a_2 + \dots + a_n)^4$$
and consider it modulo $8$. It is easy to check that the fourth powers of integers give only remainders $0$ and $1$ modulo $8$, so the right side of the equation gives remainder $0$ or $1$. The left side is congruent to $1 + 1 + k$, where $k$ is the number of odd numbers among the $a_i$. Therefore, $n \ge k \ge 6$.