I believe that writers are definitely justified in challenging the artistic status quo, because that's what true artists do. But to answer the question of why they do it, there are more answers. Think of Emily Dickinson, for example. She always strongly stood by her own freedoms and decisions to go against the current, and she's one of the most famous of American writers because of it. Aside from the fact she wanted to, going against the norm for writers often gives them more attention than if they wrote what was "expected" at the time. When studying famous American writers, we are often told to study things that they did differently than most, some, mostly the less notable today, only had minor differences, like they made their stories from different tenses, etc. But the most notorious used themes that may have been taboo and writing styles even more diverse. There is always the counter culture and most writers that we study belonged to it, sick of the large amount of similar, traditional stories that lacked element, or simply wanted to stand out. Another reason could be that writers wanted to spread the written word to all different kinds of things that have yet to be written about, different characters that haven't yet been discovered. And there are the related reasons like how writers didn't even know they were writing for the public, only time tells, like with Ann Frank. She wasn't afraid to put opinions down on paper because it was her own personal journal but it had become a famous piece of literature because of the opinions. I think writers break from tradition because the traditions are often not realistic and these artists are the only ones who will tell the truth, and that is why they do it, and that is why they are so important.
