Point of reference is a point in a space that can be used to tell where other things are positioned. It is also known as the reference point. A simple way to put this is the starting point. As you can see in your example, Jack started from his home and is moving in directions depending on his house's location.
