Solution:
The answer is $f(x) = kx$, $f(x) = a \sin(c x)$, $f(x) = a \sinh(c x)$, where $a, c \in \mathbb{R}$. The given functional equation is
$$f(x)^2 - f(y)^2 = f(x+y) f(x-y).$$
Observe that $x = y = 0$ gives $f(0) = 0$.
Since $f$ is smooth, we may differentiate with respect to $x$ and obtain
$$2 f'(x) f(x) = f'(x+y) f(x-y) + f(x+y) f'(x-y).$$
Now differentiate this with respect to $y$ to obtain
$$\begin{aligned} 0 &= \left[f''(x+y) f(x-y) - f'(x+y) f'(x-y)\right] \\ &\quad + \left[f'(x+y) f'(x-y) - f(x+y) f''(x-y)\right] \\ &= f''(x+y) f(x-y) - f(x+y) f''(x-y) \end{aligned}$$
From this we conclude the key relation: for any real numbers $X$ and $Y$:
$$f''(X) f(Y) = f(X) f''(Y).$$
Assume $f$ isn't identically zero. Then we deduce there's a constant $k$ such that
$$f''(x) = k f(x)$$
for all $x$.
This is a standard differential equation with cases on $k$.
- If $k = 0$, the solution set is $f(x) = a x + b$. Then $f(0) = 0 \Longrightarrow b = 0$, and we can check $f(x) = a x$ works.
- If $k < 0$, the solution set is $f(x) = a \sin(-k x) + b \cos(-k x)$. Again $f(0) = 0 \Longrightarrow b = 0$, and we can check $f(x) = a \sin(-k x)$ works.
- If $k > 0$, the solution set is $f(x) = a \sinh(-k x) + b \cosh(-k x)$. Again $f(0) = 0 \Longrightarrow b = 0$, and we can check $f(x) = a \sinh(k x)$ works.