, 2012

Solution 1. Multiplying $b_n = 2a_{n-1} + 4b_{n-1}$ by $t \in \mathbb{R}$ and adding $a_n = 9a_{n-1} - 2b_{n-1}$ we have
$$a_n + t b_n = (9 + 2t)a_{n-1} + (-2 + 4t)b_{n-1}.$$
Selecting $t$ such that $9 + 2t = (-2 + 4t)/t$, that is, $t = -1/2$ or $t = -2$; this becomes
$$a_n + t b_n = (9 + 2t)(a_{n-1} + t b_{n-1})$$
and by induction we have
$$a_n + t b_n = (9 + 2t)^n (a_0 + t b_0) = (9 + 2t)^n (1 + t).$$
The last equality becomes
$$2b_n - a_n = 5^n \text{ and } 2a_n - b_n = 8^n$$
for $t = -2$ and $t = -1/2$, respectively. Adding these we obtain $c_n = a_n + b_n = 8^n + 5^n$.
Assume that for some indices $k < r < m$ we have $c_r^2 = c_k c_m$. Then
$$(8^r + 5^r)^2 = (8^k + 5^k)(8^m + 5^m)$$
leads to a contradiction as $8^m + 5^m$ has at least one prime factor that does not divide $8^r + 5^r$ for any $r < m$ by Zsigmondy Theorem.

Solution 2. It can be easily verified that $c_0 = 2$, $c_1 = 13$ and $c_n = 13c_{n-1} - 40c_{n-2}$ for $n \ge 2$. Since the roots of $\lambda^2 - 13\lambda + 40 = 0$ are 5 and 8, we obtain $c_n = 5^n + 8^n$ for all $n \ge 0$.
We will now show that
$$c_k c_{2r-k-1} < c_r^2 < c_k c_{2r-k},$$
which, together with the monotonicity of $(c_n)_{n=1}^\infty$, will imply that no such $(k, r, m)$ exists.
The right hand side inequality is equivalent to
$$2 \cdot 5^r \cdot 8^r < 5^k 8^{2r-k} + 5^{2r-k} 8^k,$$
which follows from the AM-GM inequality. The left hand side inequality follows from
$$c_r^2 > 8^{2r} > 2 \cdot 8^k \cdot 2 \cdot 8^{2r-k-1} \ge c_k c_{2r-k-1}$$
as $8^n < c_n \le 2 \cdot 8^n$ for all $n \ge 0$.