We observe that a child that has $k - 1$ or $k + 1$ pieces of candy at the start of Round $k$ will receive an additional piece because of $\text{gcd}(k, k \pm 1) = 1$ and be in the same situation in the next round. Furthermore, in each round, the number of the round will increase by 1 and the number of pieces of each child will increase by 0 or by 1. Therefore, for each child, the difference of pieces and the round number is positive or zero at the start and after each round will either remain equal or drop by 1. Since we have already seen that the difference is stable at $-1$, it cannot drop below $-1$.
Therefore, it remains to show that the difference $+1$ and $-1$ are the only ones that can stay constant forever which will prove that for each child the number of pieces of candy will eventually drop to $k - 1$ or $k + 1$.

If the difference is 0, the child has $k$ pieces of candy. For $k = 1$, the child receives a piece, but receives nothing in the following round, so the difference drops to $-1$ after two steps. For $k > 1$, the child immediately receives nothing and the difference drops to $-1$.
If the difference $d$ is bigger than 1, then there must occur a round with a number divisible by $d$ after at most $d$ steps. Either the difference already drops before this round or this $d$ will be a common divisor of round number and candy piece number, therefore the difference will drop by 1 after at most $d$ steps.
This proves that after sufficiently long time all children will have $k-1$ or $k+1$ pieces of candy at the start of Round $k$ and all of them will receive one additional piece during each round forever after.