Solution:
By cancellation of terms we see that
$$a_{n+1}-x a_{n}=y^{n+1} \quad \text{ and } \quad a_{n+1}-y a_{n}=x^{n+1}$$
In particular, $a_{n+2}-x a_{n+1}=y^{n+2}=y \cdot y^{n+1}=y\left(a_{n+1}-x a_{n}\right)$, which we can write as
$$a_{n+2}=(x+y) a_{n+1}-x y a_{n} .$$
We let $s=x+y$ and $t=x y$. Then the four given integral values of $a_{n}$ yield a pair of linear equations in $s$ and $t$ (or, to be precise, $s$ and $-t$ ):
$$\begin{aligned} & a_{m+2}=s a_{m+1}-t a_{m} \\ & a_{m+3}=s a_{m+2}-t a_{m+1} \end{aligned}$$
If the determinant $a_{m+1}^{2}-a_{m+2} a_{m}$ is nonzero, these equations have a unique solution. In fact,
$$\begin{aligned} a_{m+1}^{2}-a_{m+2} a_{m} & =a_{m+1}\left(a_{m+1}-x a_{m}\right)-a_{m}\left(a_{m+2}-x a_{m+1}\right) \\ & =a_{m+1} y^{m+1}-a_{m} y^{m+2}=y^{m+1}\left(a_{m+1}-y a_{m}\right)=x^{m+1} y^{m+1}=t^{m+1} \end{aligned}$$
So we distinguish two cases.

Case 1. $t=0$. Then without loss of generality $y=0$, so $a_{n}=x^{n}$. The conclusion follows from the following lemma:

Lemma 1. If $x$ is a real number and $n$ a nonnegative integer such that $x^{n}$ and $x^{n+1}$ are integers, then $x$ is an integer.

Proof. Note that $x=x^{n+1} / x^{n}$ is rational (if $x=0$, the conclusion is immediate). Write $x=p / q$ in lowest terms with $q>0$; then $x^{n}=p^{n} / q^{n}$ is also in lowest terms, hence $q=1$.

Case 2. $t \neq 0$. Then $t^{m+1}$ is an integer, and likewise $t^{m+2}=a_{m+2}^{2}-a_{m+3} a_{m+1}$ is an integer. So by Lemma 1 again, $t$ is an integer. Now $s$ is rational by Cramer's rule applied to (7). If we can prove $s$ is an integer, then since $a_{0}=1$ and $a_{1}=s$, we will be done by 6 .

Write $s=u / v$ in lowest terms and assume that $v>1$. Every $a_{n}$ is rational; write $a_{n}=u_{n} / v_{n}$ in lowest terms. We claim that $v_{n}=v^{n}$ for every $n \geq 0$. The cases $n=0$ and $n=1$ are clear. We now induct, using (6):
$$\frac{u_{n+2}}{v_{n+2}}=\frac{u}{v} \cdot \frac{u_{n+1}}{v^{n+1}}-t \cdot \frac{u_{n}}{v^{n}}=\frac{u u_{n+1}-t v^{2} u_{n}}{v^{n+2}}$$
Since $u$ and $u_{n+1}$ are coprime to $v$, the last fraction is reduced. So $v_{n+2}=v^{n+2}$ as desired. Hence $a_{n} \notin \mathbb{Z}$ for $n \geq 1$, which is a contradiction.