Problem 1513

Put $x = y = 0$; we get $f(0)^2 = 4f(0)^2$ and hence $f(0) = 0$.

$$f(y)(f(y) - f(-y)) = 0.$$

We may conclude that either $f(y) = 0$ or $f(y) = f(-y)$ for each $y \in \mathbb{R}$. Replacing $y$ by $-y$, we may also conclude that $f(-y)(f(-y) - f(y)) = 0$. If $f(y) = 0$ and $f(-y) \neq 0$ for some $y$, then we must have $f(-y) = f(y) = 0$, a contradiction. Hence either $f(y) = f(-y) = 0$ or $f(y) = f(-y)$ for each $y$. This forces $f$ is an even function.

Taking $y = 1$ in (1), we get
$$f(x+1)f(x-1) = (f(x)+f(1))^2 - 4x^2f(1).$$

Replacing $y$ by $x$ and $x$ by $1$, you also get
$$f(1+x)f(1-x) = (f(1)+f(x))^2 - 4f(x).$$

Comparing these two using the even nature of $f$, we get $f(x) = cx^2$, where $c = f(1)$. Putting $x = y = 1$ in (1), you get $4c^2 - 4c = 0$. Hence $c = 0$ or $1$. We get $f(x) = 0$ for all $x$ or $f(x) = x^2$ for all $x$.