Almost sounds like maybe Mann had some preconceived notion that Murphy, a war hero with no acting training or experience, had no business stepping into movies as a star. As for the Murphy and Stewart not passing for brothers, you can find families everywhere who will have children that do not bear a resemblance to each other. My own is an example. I've got three sons, one is 5' 8", the other two are both near 6 ft and none of the three look enough like brothers.
