Solution:
a. If we can write xn in the form xn=qnpn with pn and qn coprime integers, then xn+1 can also be written in the same form. Indeed, if we define the integer k as k=−⌊1−xn1⌋ (where ⌊x⌋ denotes the greatest integer less than or equal to x), we have
xn+1=1−xn1+k=pn(1+k)pn−qn=qn+1pn+1
with pn+1 and qn+1 coprime.
In particular qn+1≤pn and the inequality is strict whenever pn is not coprime with (1+k)pn−qn.
Now, since for n≥1 we have 0≤xn<1, necessarily qn>pn≥qn+1. Therefore the sequence of positive integers qm is strictly decreasing, hence it cannot be infinite.
Note. The fraction pn(1+k)pn−qn is in fact already reduced to lowest terms: indeed MCD(pn,(1+k)pn−qn)=MCD(pn,qn)=1, where the first equality follows from the well-known property (a+kb,b)=(a,b).
b. The sequence stops precisely when a certain element xn is equal to 0. Given the formula that defines the sequence, xn={1−xn−11}, this tells us that 1−xn−11 is an integer, say m, and hence that xn−1=1−m1 is a rational number. We can now proceed backwards: if a certain term xh is rational, then from the formula xh={1−xh−11} we obtain that there exists an integer mh such that xh=1−xh−11−mh, from which xh−1=1−mh−xh1 is also a rational number. Proceeding in this way we obtain (by induction) that x0 is also rational, as required.
Alternative solution.
Proving that if x0>0 has a finite sequence, then it is a rational number (that is, x0=qp for suitable integers p,q) is equivalent to proving that if x0>0 is not rational, then its sequence is infinite.
We show that if xn is not rational, then neither is xn+1; indeed, if there existed p,q∈N such that qp=xn+1={1−xn1}, we would have xn=(1+k)q−pq (with k as in the first part of the proof of the exercise), that is, xn would be rational, contrary to the hypothesis.
Therefore, by induction, if x0>0 is not rational, no term of the sequence is rational, so in particular 0 cannot appear in the sequence.