If you carry the sled level all the way to your house, you're not moving it against gravity.
Although you're huffing and puffing and sweating by the time you reach your house, still, according to the Physics definition of work, you have done NO work on the sled all the way home.