wtrayah,
I think one reason he is "the bad man" is because in the Quaker community, any kind of violence is considered wrong. Duke's character was a known gun-slinger - he is the last guy those parents would want their daughter falling in love with. And of course, he did hang out with the "wrong" crowd, too.