Solution:

The answer is (E). Let $AT$ and $AS$ be the two segments drawn, with $T$ on side $BC$, $S$ on side $CD$; the angle at $A$ is right and we have divided it into three equal angles, so $B\widehat{A}T = S\widehat{A}D = 30^\circ$. The right triangles $ABT$ and $ASD$ are therefore two halves of an equilateral triangle whose height is the side of the square. Calling $l$ the length of $AB$, the area of the remaining quadrilateral is thus by difference $l^2 - \frac{1}{2} l \frac{2}{\sqrt{3}} l = l^2\left(1-\frac{1}{\sqrt{3}}\right)$. The area of $ABT$ instead equals $\frac{1}{2\sqrt{3}} l^2$ (half that of the equilateral triangle). Their ratio is $2\sqrt{3}\left(1-\frac{1}{\sqrt{3}}\right) = 2(\sqrt{3}-1)$.