So you say you want smaller government? The only way to get that is to have a voting population that actually favors such, and the only segment of the population that actually is in favor of such is easy to identify. If you've ever been to a gathering of libertarians, you know the demographic.
White males, that's pretty much it. Anything that decreases or dilutes the influence of white males is bad for conservatism or libertarianism. Full Stop.
There's no prize for being the last group to retribalize.