(You think you know about music, but then when you try to describe these intrinsic things, it becomes difficult! Ok, let me give this a stab!)
Saxophone : The sax has a floaty, more round timbre, as compared to something more direct like a clarinet. Its distinct reediness and tendency towards a rich vibrato give it a jazzy effect when composers use it in music, even in a non-jazz song like Ravel's Bolero.
Tuba : The tuba is the giant of the brass section, and it has a heavy, almost blatty timbre. It's not known for its agility, just for its reliable bassline, and incorporating tuba can give the music a polka-like effect or just a solid undertone in the low decibel region.
Banjo : The banjo is the earthy cousin to the guitar. Its timbre is best suited to folk music or peppy country songs because of the unique strum of each note.
Harp : The harp has an attack and sustain very similar to the piano, and its tambre is similar to a cross between a piano and a string pizzicato. That's how I like to think about it. Use of it in music gives it a dreamy quality, especially because of the massive scales harpists can pull off.
Xylophone: The timbre of the xylophone, as pitched percussion, carries through a section. If composers mean to include it, they had better be ready to have it dominate, because its clear attack and maintained tone will stand out.