a) If the triangle $OAB$ is oriented counterclockwise, then $m(\angle AOB) = \arg\frac{b}{a}$, otherwise $m(\angle AOB) = \arg\frac{a}{b}$. Then
$$\sin(\angle AOB) = \left| \frac{|a|}{2|b|} \left( \frac{b}{a} - \frac{\bar{b}}{\bar{a}} \right) \right| = \frac{|\overline{ab} - \overline{ab}|}{2|a||b|}$$
and since $[OAB] = \frac{1}{2} OA \cdot OB \cdot \sin(\angle AOB)$, we obtain the conclusion.

b) Let $\varepsilon = \cos \frac{2\pi}{3} + i \sin \frac{2\pi}{3}$. We may assume that the complex coordinates of the points $A$, $B$ and $C$ are $1$, $\varepsilon$, and $\varepsilon^2$, respectively. Let $P$ be a point with complex coordinate $p$, in the interior of the circle. Then $(p-1) + \varepsilon(p-\varepsilon) + \varepsilon^2(p-\varepsilon^2) = 0$. Consider the points $D(p-1)$, $E(\varepsilon(p-\varepsilon))$ and $F(\varepsilon^2(p-\varepsilon^2))$. Observe that, $OD = PA$, $OE = PB$ and $OF = PC$.
Let $J$ be a point such that $ODJF$ is a parallelogram. We obtain that the complex coordinates of points $J$ and $F$ are opposite numbers. Then $OJ = PC$, and the side lengths of $ODJ$ are $PA$, $PB$, and $PC$. But
$$\begin{aligned} [ODJ] &= [ODE] = \frac{1}{4} |(\bar{p}-1) \cdot \varepsilon \cdot (p-\varepsilon) - (p-1) \cdot \bar{\varepsilon} \cdot (\bar{p}-\bar{\varepsilon})| \\ &= \frac{1}{4} ((\varepsilon - \varepsilon^2) |p|^2 - (\varepsilon - \varepsilon^2)) = \frac{\sqrt{3}}{4} |p|^2 - 1 \end{aligned}$$
Since $|p| < 1$, we obtain $[ODE] = S(P) = \frac{\sqrt{3}}{4} (1 - |p|^2)$, from which the conclusion follows easily.