, 2020

Since the expressions $4a^2 + kb + c$, $4b^2 + kc + a$ and $4c^2 + ka + b$ are invariant under cyclic permutations of $a$, $b$ and $c$, we can assume that $c = \max(a, b, c)$. Then for $k \le 3$
$$(2c)^2 < 4c^2 + ka + b \le 4c^2 + 3c + c < 4c^2 + 4c + 1 = (2c + 1)^2.$$
Hence $4c^2 + ka + b$ is not a perfect square if $k \le 3$.

For $k = 4$, one can find the triple $(a, b, c) = (120, 85, 141)$, considering the expressions modulus $20$.
For the alternative version we will find all triples $(a, b, c)$ which satisfy the problem conditions for $k = 4$.
As at the beginning, we say that $c = \max(a, b, c)$ and
$$(2c)^2 < 4c^2 + 4a + b < 4c^2 + 5c < 4c^2 + 8c + 4 = (2c + 2)^2,$$
hence $4c^2 + 4a + b = (2c + 1)^2$, i.e. $4a + b = 4c + 1$.
Suppose $b > a$, then
$$4b^2 + 4b + 1 \le 4b^2 + 4c - 4 + 1 < 4b^2 + 4c + a = 4b^2 + 4a + b - 1 + a < 4b^2 + 8b + 4.$$
Whence $(2b+1)^2 < 4b^2 + 4c + a < (2b+2)^2$ – a contradiction. Therefore $b < a < c$. Note that $(2b+1)^2 < 4b^2 + 4c + a$ is true since we didn't use the inequality $b > 1$.
Since $4a^2 + 4b + c > (2a)^2$, we get $4a^2 + 4b + c \ge (2a + 1)^2$, which is equivalent to $4b + c \ge 4a + 1$. Using the fact that $c = a + (b - 1)/4$, we obtain $3a \le 4\frac{1}{4}b - 1\frac{1}{4}$. Consider the expression $4b^2 + 4c + a$ one more time:
$$\begin{aligned} 4b^2 + 4c + a &= 4b^2 + 5a + b - 1 \le 4b^2 + \frac{5}{3} \cdot (4\frac{1}{4}b - 1\frac{1}{4}) + b - 1 = \\ &= 4b^2 + 8\frac{1}{12}b - 3\frac{1}{12} < 4b^2 + 12b + 9 = (2b + 3)^2. \end{aligned}$$

Therefore $(2b+1)^2 < 4b^2 + 4c + a < (2b+3)^2$, whence $4b^2 + 4c + a = (2b+2)^2$, i.e. $8b+4 = 4c+a$.
Note that $8a+4 = 2(4c+1-b)+4 = 8c+6-2b > 4b+c$, hence
$$(2a)^2 < 4a^2 + 4b + c < 4a^2 + 8a + 4 = (2a + 2)^2,$$
which yields $4a^2 + 4b + c = (2a + 1)^2$, i.e. $4b + c = 4a + 1$. Thus the triple $(a, b, c)$ satisfy the system
$$\begin{cases} 4a + b = 4c + 1; \\ 8b + 4 = 4c + a; \\ 4b + c = 4a + 1. \end{cases}$$
The solution of this system is $(120, 85, 141)$.