Problem 1703

Solution:

First, observe that both $x=c$ and $x=1$ are fixed points of $f$, and thus fixed points of $f \circ f$. Indeed, a fixed point of $f$ is a value of $x$ such that $f(x)-x=0$, and we have $f(x)-x = x^{2}-(c+1)x+c = (x-1)(x-c)$. Moreover, $(f \circ f)(x)-x$ is a quartic polynomial in $x$, and its roots are the fixed points of $f \circ f$. Thus, it follows that $(f \circ f)(x)-x = (x-1)(x-c)g(x)$ for some quadratic polynomial $g$. Explicitly solving gives us $(f \circ f)(x)-x = (x-c)(x-1)\left(x^{2}-(c-1)x+1\right)$.

Thus, we need only $x^{2}-(c-1)x+1$ to have two distinct roots, neither of which is equal to $c$ or $1$. This means that $(c-1)^{2}-4>0$, i.e. $(c+1)(c-3)>0$. Thus, we want $c<-1$ or $c>3$. We check now that $1$ and $c$ are not roots: we get if $x=1$, $x^{2}-(c-1)x+1=3-c \neq 0$ (since $c \neq 3$) and if $x=c$, $x^{2}-(c-1)x+1=c+1 \neq 0$ since $c \neq -1$.

Thus, all $c \in (-\infty,-1) \cup (3,+\infty)$ work.