Let $f(x)$ satisfies the problem condition:
$$\left\{ \begin{array}{l} \{f(x)\} \sin^2 x + \{x\} \cos f(x) \cos x = f(x), \\ f(f(x)) = f(x), \end{array} \right. \qquad (1) \qquad (2)$$
for all real $x$.
Replacing $x$ by $f(x)$ in (1) we obtain:
$$\{f(f(x))\} \sin^2 f(x) + \{f(x)\} \cos^2 f(x) = f(f(x)).$$
From (2) it follows that $\{f(x)\} = f(x)$. So $f: [0, 1] \to [0, 1)$.
From (1) and the equality $\{f(x)\} = f(x)$ it follows that:
$$\{x\} \cos f(x) \cos x = f(x)(1 - \sin^2 x) = f(x) \cos^2 x. \quad (3)$$
We have $\cos x > 0$ and $\{x\} = x$ for $x \in [0, 1]$. Therefore from (3) we have $x \cos f(x) = f(x) \cos x$ for $x \in [0, 1]$. Dividing the last equality by $\cos x \cos f(x)$ we obtain:
$$\frac{x}{\cos x} = \frac{f(x)}{\cos f(x)}. \qquad (4)$$
Consider the function $g(x) = x/\cos(x)$, $g: [0, 1] \to \mathbb{R}$. It is increasing since $x$ increases and $\cos x$ decreases. Since in view of (4) $g(x) = g(f(x))$ for $x \in [0, 1]$, we obtain $f(x) = x$.