The dreaded black spot is never a good thing to get. It represents the crew no longer has belief or use in the person it is given to. When Pew delivers one to Billy Bones, it contains a demand for him to appear before his crewmates. While Billy Bones is filled with terror, Silver is more rebellious and mocks whoever dare to cut into a Bible. He mocks the hypocrisy of a scourge using a black spot to threaten him.
remote means having very little connection with or relationship to. so the thing is remote meaning its never had any actual connection to anything making it fearful.