Since the graph of $y=ax+2$ is symmetric to the graph of $y=3x-b$ about the line $y=x$,

then for any point $P(x, ax+2)$ on the graph of $y=ax+2$, its symmetric point $Q(ax+2, x)$ about $y=x$ must lie on the graph of $y=3x-b$.

Therefore, for all real numbers $x$, we have $x=3(ax+2)-b$.

Thus, we get the system of equations:

$$\begin{align*} 1 &= 3a \\ 0 &= 6-b \end{align*}$$

This leads to:

$$\begin{align*} a &= \frac{1}{3} \\ b &= 6 \end{align*}$$

Therefore, the correct answer is $\boxed{\text{A}}$.