The force of earth's gravitational field is always directed downwards (towards the center of the earth. When the ball is thrown up, it is going against the earth's gravitational field and so, the earth's gravitational force pulls it back down, accelerating it downwards.
