First note that $a=0$ satisfies the problem condition (for example, the equation is satisfied by the function $f(x) \equiv 0$).

Now suppose $a \neq 0$.

Lemma. $f(y)=f(z)$ if and only if $[y]=[z]$.

Suppose $f(y)=f(z)$ for some $y, z$. Then the given equation implies $f(x)+a[y]=f(x-f(y))=f(x-f(z))=f(x)+a[z]$ whence $[y]=[z]$. Conversely, if $[y]=[z]$ then $f(x-f(y))=f(x)+a[y]=f(x)+a[z]=f(x-f(z))$. It follows from previous observation that $[x-f(y)]=[x-f(z)]$ for all $x$. Set $x=\frac{f(y)+f(z)}{2}$, then $\left[\frac{f(y)-f(z)}{2}\right]=\left[-\frac{f(y)-f(z)}{2}\right]$, so $f(y)=f(z)$. The lemma is proved.

Now we claim that $f(m) \in \mathbb{Z}$ for any $m \in \mathbb{Z}$. Setting $y=m$ in the given equation we obtain $f(x-f(m))=f(x)+a m$ for any $m \in \mathbb{Z}, x \in \mathbb{R}$. Suppose that $f(m) \notin \mathbb{Z}$ for some $m \in \mathbb{Z}$. Choose $t \in(0,1)$ such that $[f(m)]=[f(m)+t]$. Then for $x=0$ we have $f(-f(m))=f(0)+a m$ and for $x=-t$ we have $f(-t-f(m))=f(-t)+a m$. Using the lemma we have $f(-f(m))=f(-t-f(m))$, so $f(0)=f(-t)=f(-1)$, which contradicts the lemma.

From now on we will use in the given equation $f(x-f(y))=f(x)+a y$ (1) only integer numbers $x, y$. Setting $y=1$ in (1) we obtain that $a \in \mathbb{Z}$. Further, for $y=0$ we have $f(x-f(0))=f(x)$ and therefore $x-f(0)=x$ (by lemma), whence $f(0)=0$. Now set $x=f(y)$, then $f(f(y))=-a y$ (2); replacing $y$ by $f(y)$ in (1) we get $f(x+a y)=f(x)+a f(y)$ (3). Denoting $f(1)$ by $n$ and setting $y=1$ in (3) we obtain $f(x+a)=f(x)+a n$ (4). Applying (4) to $x=0$ we get $f(a)=a n$. From (4) we easily conclude that $f(k a)=k a n$ for any $k \in \mathbb{Z}$; in particular $f(a n)=a n^{2}$. Now setting $y=a$ in (2) gives $-a^{2}=f(f(a))=a n^{2}$ as stated.

It remains to note that if $a=-n^{2}$ then the function $f(x)=n[x]$ satisfies the given condition: $n[x-n[y]]=n[x]-n^{2}[y]$, which is obvious.