Yes, it can be done with a single traversal. For easier explanation, assume you have an array of integers, and those integers are either 1, 2 or 3. While you read the input, count how many 1's, 2's and 3's you have. Now you know what portion of the array should have 1's in the end, what portion should have 2's and what portion should have 3's. Let G₁ be the portion of the array that should have 1's at the end, and similarly, G₂ and G₃ the same for 2's and 3's.

• Step 1: Do a traversal of the array and let s_xy be the amount of x's that are in G_y. A swap affects at most two elements, so you can't correct more than that with a single swap. Swap 1's in G₂ with 2's in G₁, 1's in G₃ with 3's in G₁ and 2's in G₃ with 3's in G₂. You'll be correcting two elements with every one of those swaps, so it's optimum. That's min(s₁₂, s₂₁) + min(s₁₃, s₃₁) + min(s₂₃, s₃₂).
• Step 2: Suppose you're not finished yet, then it means that there's some group G_x that contains an element y (y! = x), which also means that group G_y contains an unwanted element z (z! = y), but this also implies that z! = x because otherwise it's a swap we should have done in step 1, and finally group G_z contains an unwanted element which must be x. In other words, the remaining unordered elements form cycles, and whatever the direction of the cycles, there must either be a 1 in G₂ or a 2 in G₁ and you need two swaps to correct three elements. So the remaining number of swaps is 2*(max(s12, s21)- min(s12, s21)).