(a) Let $I$ be the reflection of $O$ with respect to $P$. Since $O'$ is the midpoint of $OH$, it follows that $O'P \parallel IH$. Moreover, we have $PO = PO'$, thus $IO = IH$.
Let $S, G$ be midpoints of $AI$ and $AH$, respectively. We have
$$SP = \frac{1}{2}AO = \frac{1}{2}R = O'D$$
and $SP \parallel AO \parallel O'D$, thus $O'SPD$ is a parallelogram. It follows that $DP \parallel O'S$.
Furthermore, we have $SP = \frac{1}{2}R = O'D'$, therefore $SD'PO'$ is an isosceles trapezoid which leads to $O'P = SD'$ and
$$IH = 2O'P = 2SD' = IA'.$$
Thus $IA' = IH = IO$ which implies $A'$ lies on the circle $(I, IO)$. Similarly, $B'$ and $C'$ also lie on $(I, IO)$. This leads to the conclusion of (a).

(b) Let $R$ be the radius of the circle $(O)$. It is obvious that $GD = R$. Consider the homothetic transformation with center $A$ and ratio $\frac{1}{2}$ which sends $B, C, A', X, H, M$ and the perpendicular bisector of $BC$ to $F, E, D', U, G, M'$ and the perpendicular bisector $EF$, respectively. Then $\frac{MB}{MC} = \frac{M'F}{M'E}$ and $U$ is the reflection of $D'$ with respect to $EF$. Thus
$$\frac{MB}{MC} = \frac{M'F}{M'E} = \frac{GF}{GE} \cdot \frac{UF}{UE} = \frac{\sqrt{R^2 - DF^2}}{\sqrt{R^2 - DE^2}} \cdot \frac{D'E}{D'F}.$$
Similarly, we can calculate $\frac{NC}{NA}$ and $\frac{KA}{KB}$.

Since $DD'$, $EE'$, $FF'$ are concurrent, it follows
$$\frac{D'F}{D'E} \cdot \frac{F'E}{F'D} \cdot \frac{E'D}{E'F} = 1.$$
Thus $\frac{MB}{MC} \cdot \frac{NC}{NA} \cdot \frac{KA}{KB} = 1$, which implies $M, N, K$ are collinear. This is the desired result.