The smallest perfect square, apart from $1$, is $2^2 = 4$. So let $a_1, a_2, \dots, a_k$ be the numbers on the list modulo $4$. We cannot have $a_i = 0$; also, there is at most one $a_i$ equal to $2$ and we cannot have $a_i = 1$ and $a_j = 3$ simultaneously.

We claim that among any four distinct numbers $a_1, a_2, a_3, a_4$ fulfilling the above properties there are three of them whose sum is a multiple of $4$. Indeed, there are two equal numbers, say $a_1, a_2$. We cannot have $a_1 = a_2 = 2$, so either $a_1 = a_2 = 1$ or $a_1 = a_2 = 3$. We can suppose wlog $a_1 = a_2 = 1$ (otherwise, reverse the signs of all four numbers modulo $4$). But since we also cannot have $a_j = 3$ and $a_3 = a_4 = 1$, one of $a_3, a_4$, say $a_3$, is $2$. But then $a_1 + a_2 + a_3 = 1 + 1 + 2 = 4$.

So the quantity of numbers is at most $3$. $5$, $13$ and $17$ is an example of a list with three numbers.