First Solution: We will show that a dissection satisfying the requirements of the problem is possible if and only if $k \neq 1$.

We first show by contradiction that such a dissection is not possible when $k=1$. Assume that we have such a dissection. The common boundary of the two dissecting polygons must be a single broken line connecting two points on the boundary of the square (otherwise either the square is subdivided in more than two pieces or one of the polygons is inside the other). The two dissecting polygons must have the same number of vertices. They share all the vertices on the common boundary, so they have to use the same number of corners of the square as their own vertices. Therefore, the common boundary must connect two opposite sides of the square (otherwise one of the polygons will contain at least three corners of the square, while the other at most two). However, this means that each of the dissecting polygons must use an entire side of the square as one of its sides, and thus each polygon has a side of length $1$. A side of longest length in one of the polygons is either a side on the common boundary or, if all those sides have length less than $1$, it is a side of the square. But this is also true of the other polygon, which means that the longest side length in the two polygons is the same. This is impossible since they are similar but not congruent, so we have a contradiction.

We now construct a dissection satisfying the requirements of the problem when $k \neq 1$. Notice that we may assume that $k > 1$, because a $1 \times k$ rectangle is similar to a $1 \times \frac{1}{k}$ rectangle.

We first construct a dissection of an appropriately chosen rectangle (denoted by $ABCD$ below) into two similar incongruent polygons. The construction depends on two parameters ($n$ and $r$ below). By appropriate choice of these parameters we show that the constructed rectangle can be made similar to a $1 \times k$ rectangle, for any $k > 1$. The construction follows.

Let $r > 1$ be a real number. For any positive integer $n$, consider the following sequence of $2n + 2$ points:
$$\begin{aligned} A_0 &= (0, 0), \quad A_1 = (1, 0), \quad A_2 = (1, r), \quad A_3 = (1 + r^2, r), \\ A_4 &= (1 + r^2, r + r^3), \quad A_5 = (1 + r^2 + r^4, r + r^3), \end{aligned}$$
and so on, until
$$A_{2n+1} = (1 + r^2 + r^4 + \dots + r^{2n},\ r + r^3 + r^5 + \dots + r^{2n-1}).$$
Define a rectangle $ABCD$ by $A = A_0, C = A_{2n+1}$,
$$B = (1 + r^2 + \dots + r^{2n}, 0), \quad \text{and} \quad D = (0, r + r^3 + \dots + r^{2n-1}).$$

The sides of the $(2n + 2)$-gon $A_1A_2\dots A_{2n+1}B$ have lengths $r, r^2, r^3, \dots, r^{2n}, r + r^3 + r^5 + \dots + r^{2n-1}, r^2 + r^4 + r^6 + \dots + r^{2n}$, and the sides of the $(2n+2)$-gon $A_0A_1A_2\dots A_{2n}D$ have lengths $1, r, r^2, \dots, r^{2n-1}, 1 + r^2 + r^4 + \dots + r^{2n-2}, r + r^3 + r^5 + \dots + r^{2n-1}$, respectively. These two polygons dissect the rectangle $ABCD$ and, apart from orientation, it is clear that they are similar but incongruent, with coefficient of similarity $r > 1$. The rectangle $ABCD$ and its dissection are thus constructed.

The rectangle $ABCD$ is similar to a rectangle of size $1 \times f_n(r)$, where
$$f_n(r) = \frac{1 + r^2 + \dots + r^{2n}}{r + r^3 + \dots + r^{2n-1}}.$$
It remains to show that $f_n(r)$ can have any value $k > 1$ for appropriate choices of $n$ and $r$. Choose $n$ sufficiently large so that $1 + \frac{1}{n} < k$. Since
$$f_n(1) = 1 + \frac{1}{n} < k < k \frac{1 + k^2 + k^4 + \dots + k^{2n}}{k^2 + k^4 + \dots + k^{2n}} = f_n(k)$$
and $f_n(r)$ is a continuous function for positive $r$, there exists an $r$ such that $1 < r < k$ and $f_n(r) = k$, so we are done.

Second Solution: (By Oleg Golberg) We present another proof of the fact that $k = 1$ is impossible. Assume for the sake of contradiction that we have a dissection of a unit square into two polygons that are similar but not congruent. As in the first solution, the dissection must be accomplished via a single path connecting opposite sides of the square. Without loss of generality, suppose that the endpoints of the path are $K \in BC$ and $L \in AD$, where $K$ is a corner of the square if and only if $L$ is the opposite corner. Also, without loss of generality, assume that the right-hand part is strictly smaller than the left-hand part (they are given to be similar but not congruent).

Now, the right-hand part is supposed to be similar to the left-hand part, so let the function $F$ map the right-hand polygon to the left-hand polygon according to the similarity. Observe that the right-hand polygon has the property that if one draws the two perpendiculars to $CD$ at $C$ and $D$, then these lines completely bound the right-hand polygon. Therefore, after applying $F$, the same property must hold for $F(CD)$; this must be some side of the left-hand polygon, and its perpendiculars at $F(C)$ and $F(D)$ must bound the entire left-hand polygon. In particular, they must bound $A$ and $B$. However, there are only a few ways this can be done:

(a) $F(\{C, D\}) = \{A, B\}$. Yet the lengths of $CD$ and $AB$ are equal, so this violates the fact that the similarity is not a congruence.

(b) $F(\{C, D\}) = \{K, L\}$, and $KL \parallel AB$. This has the same problem as the first case.

(c) $F(\{C, D\}) = \{B, K\}$. Since the right-hand polygon is strictly smaller than the left-hand one, this forces $BK > 1 \Rightarrow BC > 1$, contradicting the fact that $ABCD$ is a square.

(d) $F(\{C, D\}) = \{A, L\}$. This has a similar problem to the previous case.

Therefore, all cases yield contradictions, so we have proven $k \neq 1$.