The colour we see is a measure of the wavelengths of light nor absorbed by the cloth. We would call the colour of the cloth blue if it absorbed longer wavelengths of yellow/orange/red. Sunlight contains all the visible wavelengths. Whatever is not absorbed is reflected.
