Creator Backlash: Joel got sick of "Piano Man" for a time and refused to sing it in concert. He got over it, though the audience tends to save him the trouble of singing it when he plays it nowadays. And reportedly, he's not any too fond of "Just the Way You Are," either, because it's a love song to someone he ended up divorcing. Joel also retired "Uptown Girl" (another love song to an ex-wife) from his stage show for a long time, but he eventually reintroduced it.
Old Shame: Before launching his solo career, Joel was part of a hard rock duo called Attila. They released exactly one album, which was critically savaged and which Joel to this day is thoroughly embarrassed by.
What Could Have Been: According to Joel's account, he first wrote "Only The Good Die Young" as a Reggae song. When he brought it into the studio and first played it, his drummer Liberty DeVitto threw his sticks at him, yelled "I fucking hate reggae!" and said "the closest you ever got to Jamaica is when you change trains in Queens!". DeVitto then demanded that the song be changed to a shuffle beat instead of a reggae beat, resulting in the version we know today. Joel also mentions that initially the single stiffed on the charts, and only began selling tons of copies after it was banned for being "anti-Catholic", and claims that he wrote a letter to the Archbishop of St. Louis saying "Thank you very much, please ban my next album as well."