We take the imprint others have left on us and overlay them on the people that follow. If I knew someone aggressive and they hurt me, I would therefore be wary of any other aggressive persons I come across on the chance they may hurt me as well. If I met someone kind and they said they were kind because of their religion, I would naturally be inclined to think most others of the given religion are also as kind.
