Solution:

The common difference should be a positive divisor of $39-3=36=2^{2} \cdot 3^{2}$, which has $(2+1)(2+1)$ factors. If the common difference is $1$, then any one of $1, 2$, or $3$ may be the first term. If the common difference is $2$, then the first term may be either $1$ or $3$ only. For the other $7$ possible common differences, the first term must be $3$. Thus, there are $12$ such arithmetic progressions in all.