On startup, left handed mouse controls are ignored, and the mouse works like right handed. When I go to Settings -> Input Devices -> Mouse it shows 'Left handed' as selected, so at that stage I have to select 'Right handed', click Apply, and then select 'Left handed' again. Only then it starts behaving as expected.
*** This bug has been marked as a duplicate of bug 395401 ***