Think of negative and positive like this; when you <em>add</em>, you <em>move to the right</em> on the number line. When you <em>subtract</em>, you <em>move to the left</em>.
With this, you have a positive number and you are <em>taking away four</em>. This means you <em>move four spaces to the left</em>.