This question needs to be looked from two perspectives.
The first perspective is that of the characters from the book, The Giver, themselves. From their perspective it is possible as we can see from the novel when they almost showed no emotion to the news that Gabe will be released. This is due to the circumstances of the book where everyone is thought to be detached and they do not even understand the concept of love. Only Jonas gets attached to Gabe.
Second is our own perspective and it is nigh impossible for a person not to get attached to a baby for whom it is caring at least a little bit. The emotional attachment develops quickly for almost all of the people. So the contract in the real world probably would not work at all.
