, 2023

Let $x = y = 0$ in the problem, we have $f(0) = f(0) + f(0)^2$ so $f(0) = 0$.

Continue to replace $(x, y) = (1, -1)$ into the problem, we have $f(-1) = f(1) \cdot f(-1)$ deduce $f(1) = 1$ or $f(-1) = 0$. However, if $f(-1) = 0$, substituting $y = -1$ in the problem, then $f(-x) = 0$ for all real numbers $x$, not satisfied. Thus, $f(1) = 1$.

Continue to replace $x = y = -1$, we have $f(1) = f(0) + f(-1)^2$ inferred $f(-1) = 1$ or $f(-1) = -1$. But if $f(-1) = 1$, substitute $y = -\frac{1}{2}$ then $f(\frac{x}{2}) = -f(x)f(-\frac{1}{2})$, $\forall x \in \mathbb{R}$; continue to substitute $x = -1$ then $f(-\frac{1}{2}) = 0$ so $f(\frac{x}{2}) = 0$, $\forall x \in \mathbb{R}$, also not satisfied. Hence, $f(-1) = -1$.

Finally, substitute $y \to -1$ in the given, then $f(-x) = -f(x)$ for all real numbers $x$, that is, $f$ is an odd function.

Change $y$ by $-y$, $y - 1$ and using the property $f$ as an odd function, then
$$f(x)f(y) = f(2xy - x) - f(xy - x) = f(xy) + f(x)f(y - 1) - f(xy - x). \quad (*)$$
Thus for all positive integers $y$ then
$$\begin{aligned} f(xy) - f(x)f(y) &= f(x(y - 1)) - f(x)f(y - 1) \\ &= f(x(y - 2)) - f(x)f(y - 2) \\ &= \cdots = f(0) - f(x)f(0) = 0. \end{aligned}$$
From this it follows that $f(xy) = f(x)f(y)$ for all $x \in \mathbb{R}, y \in \mathbb{Z}^+$. In $(*)$, let $(x, y) = (1, 2)$ then $f(3) - f(1) = f(2)^2$. Let $y = 1$ in the given equation, then
$$f(2x + 1) = f(x + 1) + f(x), \forall x \in \mathbb{R}.$$
Continue to replace $x = 1$ then $f(2) = f(3) - f(1) = f(2)^2$ so $f(2) = 2$ or $f(2) = 0$. Notice that if $f(2) = 0$, then due to the multiplication, we have $f(2x) = f(2)f(x) = 0$, which is also not satisfied. Therefore, $f(2) = 2$ and $f(2x) = 2f(x)$, $\forall x \in \mathbb{R}$ and $f(2x + 1) = f(x) + f(x + 1)$.

Here, by induction, we can prove $f(n) = n$ for all positive integers $n$. Replace $y \to \frac{y}{x}$ in the given, then
$$f(2y + x) = f(x + y) + f(x)f\left(\frac{y}{x}\right), \forall x \neq 0, y \in \mathbb{R}. \quad (1)$$
Change the role $x, y$ of the above expression then
$$f(2x + y) = f(x + y) + f(y)f\left(\frac{x}{y}\right), \forall y \neq 0, x \in \mathbb{R}. \quad (2)$$
Replace $x \to 2x$ in (1) then
$$f(2x+2y) = f(2x+y)+f(2x)f\left(\frac{y}{2x}\right) = f(x+y)+f(y)f\left(\frac{x}{y}\right)+f(2x)f\left(\frac{y}{2x}\right), \forall x,y \neq 0.$$
On the other hand, $f(2x) = 2f(x)$ so $f(x+y) = f(x)f\left(\frac{y}{x}\right) + f(y)f\left(\frac{x}{y}\right)$, $\forall x, y \neq 0$. Adding the sides of (1) and (2), combined with this last equality, we get
$$f(2x+y)+f(2y+x) = 2f(x+y)+f(x+y)+f(y)f\left(\frac{x}{y}\right)+f(x)f\left(\frac{y}{x}\right) = 3f(x+y), \forall x,y \neq 0.$$
To handle the condition $x, y \neq 0$, replace $(x, y) \to (\frac{2b-a}{3}, \frac{2a-b}{3})$ with $a, b \in \mathbb{R}$ then
$$f(a) + f(b) = f(a + b); \forall a \neq 2b \text{ và } b \neq 2a.$$
For every pair of numbers $a, b \in \mathbb{R}$, we choose $c$ such that $c > 4|a| + 4|b|$ then make sure $c \neq 2a, c \neq 2b$. We have
$$f(c) + f(a + b) = f(a + b + c) = f(c + a) + f(b) = f(c) + f(a) + f(b).$$
From here we infer that $f$ is an additive function on $\mathbb{R}$ so $f(xy) = f(2xy + x) - f(xy + x) = f(x)f(y)$, $\forall x, y \in \mathbb{R}$. Therefore, $f$ adds, multiplies, and is non-constant, so $f(x) = x$. It is easy to check that they satisfy the given condition. So $f(x) = x$ with $x \in \mathbb{R}$. $\square$