Most people who think seriously about HBD tend to view it from either a Darwinian frame of reference or from the frame of what I'll call folk animal husbandry. Folk animal husbandry tends to talk about things like 'the apple not falling far from the tree'---an expression, which, if you think about it, implies pretty strongly a strong central tendency based on genetics with some variation due to randomness and environment, especially when the modifier, the wind wasn't blowing too hard when that apple fell from the tree, is applied. In addition the expressions about 'good stock' (sometimes 'good pioneer stock', with the stipulation that 'the cowards never came, the weak died along the way' applied) abound in the language.
For most practical HBD purposes, these frames are equivalent. Honestly, creationists are more likely to agree to the practical application of HBD than are evolutionists, probably due to memetic entanglements.
Most creationists, for instance, will not dispute the claim that the races are partially inbred families writ large. The ones who know their Old Testament will even point out the particular pedigrees involved going back to Noah. Most also won't dispute the claim that different families have different tendencies towards large or small endowments in various attributes. The ones who are lower case o orthodox won't even balk when one points out that said endowments are not fair in any human sense of the word---there's no point-based character generation going on here, God is not a Champions or Hero system gamemaster. Those with a good practical command of Scripture will quote 'Hath not the potter power over the clay, to make one vessel unto honor and another unto dishonor?' It's also not unlikely that the Parable of the Talents will be shared with you. The creationist feels no need to pretend equality in any human sense, because he is confident that human beings are of equal (or at least inestimable) value to God.
But whichever frame one chooses, one has two huge problems to deal with---two huge brute facts that challenge our frame.
The first is alluded to by a commentator on the 'Final Judgment of Darwin' on homosexuality. The evidence that the trait is partially influenced by genetics is reasonably strong, but the depression in TFR that it causes is extremely strong. Using either frame of reference, one would predict that it would be rapidly extinguished from the population, even leaving aside the impact of various 'social diseases', pogroms, or the like.
The second one is the massive differences that exist between women in terms of fertility and ability to safely carry a child to term. Presumably, if Darwin or animal husbandry optimize ANYTHING, they optimize the ability to produce offspring. That's about as fundamental as it gets. Yet we have women like, say, my wife, mother, or great great grandmother who have had no significant difficulties whatsoever bringing a fair number of descendants into the world. On the other side, we have women like two of my sisters in law who have had a great deal of such difficulty, one of whom would not survive a pre-modern childbirth. Similar differences exist in terms of ability to conceive in the first place. Given that even the youngest of Young Earth Creationists believe that humanity is around 300 generations old, that's plenty of time for natural/artificial selection and/or animal husbandry to optimize this pretty key capability and to largely fix whatever genetic variants promote such throughout the population. Compare, for instance, lactose tolerance, which took very little time to become near universal in populations where cattle were common.
Both of these problems point to the conclusion that we don't understand this portion of reality anywhere near as well as perhaps we think we do. This isn't to say that we know nothing, or that what we know is not useful (look to the radically increased yields we've been able to squeeze out of plants, for instance, even before modern 'genetic engineering' or, for instance, the incredible amount of intellectual talent the first wave of psychometrics was able to mine out of unexpected sources). But it does tend to indicate that we should try to avoid straying too far from the actual data.