X and Y motors calibration problem

Folks,
I am having some trouble getting the motors to the right position. I am using version 2 model.

Here are some pre conditions.
1. I tensioned the belt just so there is no slack in the belt.
2. the belt is almost inline with the pulleys on both sides.

Whenever I change a direction of motors, I see this 1.5mm less of what is commanded.
for the same direction after a command, it follows correctly.
In other words,
command:0 to 20mm, 20mm to 40mm, 40 to 30mm
actual:0 to 18.5mm, 18.5mm to 38.5mm, 38.5 to 30mm.

Hence I loosened the belt a bit more and the offset is down to maybe 1mm but still carrying the same behavior.

Has anyone seen this problem?
Any help or feedback is appreciated, thanks.