(a) Write $BC = a$, $CA = b$ and $AB = c$. Let $\angle PTQ = 2x$, $\angle KTL = 2y$ and $\angle MTN = 2z$. Since $TP = TQ$, we get $\angle TQP = \angle TPQ = 90^\circ - x$. Thus $\angle 30^\circ + x$. Similarly, $\angle BNP = 30^\circ + z$. It follows that $z + x = 120^\circ - B$. Likewise $x + y = 120^\circ - C$ and $y + z = 120^\circ - A$. Solving these, we get $x = A - 30^\circ$, $y = B - 30^\circ$ and $z = C - 30^\circ$. As each angle $A, B, C$ is greater than $30^\circ$, we see that $x, y, z$ are all positive. Further, $\angle BPN = 30^\circ + x = A$ and $\angle BNP = 30^\circ + z = C$. So the triangle $PBN$ is similar to $ABC$. So are $QKC$ and $ALM$. If we take $BN = ka$, $NP = kb$ and $PB = kc$, then $QC = kb^2/c$ (Note $PN = KQ$). Thus
$$\begin{align*} a = BP + PQ + QC = kc + 2kb \sin x + \frac{kb^2}{c} &= \frac{k}{c}(c^2 + b^2 + 2bc \sin(A - 30^\circ)) \\ &= \frac{k}{c}(b^2 + c^2 - 2bc \cos(A + 60^\circ)) \\ &= \frac{k}{c}f^2, \end{align*}$$
where $f^2 = (a^2 + b^2 + c^2 + 4\sqrt{3}F)/2$. This shows that $k = ac/f^2$; $PN = kb = abc/f^2$, the radius of the circle as desired.

(b) Using cosine rule,
$$\begin{align*} BT^2 &= BP^2 + PT^2 - 2BP \cdot PT \cdot \cos \angle BPT \\ &= k^2(c^2 + b^2 - 2bc \cos(90^\circ + x)) \\ &= k^2(b^2 + c^2 - 2bc \cos(A + 60^\circ)) \\ &= k^2 f^2. \end{align*}$$
So $BT = kf = ac/f$. This shows that $BT \cdot AC = abc/f$. The symmetry of right side shows that
$$BT \cdot AC = AT \cdot BC = CT \cdot AB.$$