If an object is lying on the ground, and is nowhere near a "hole" or a cliff, then for the purposes of this system, the object would have a height of zero, therefore would have no gravitational potential energy. And therefore, no work could be performed on the object by gravity.
