Predicting exactly what would happen, if, for instance, we were to legalize all drugs more or less overnight by removing authority for their regulation from the FDA is a difficult matter. Accordingly, I suggest a first step.
1) Remove marijuana from the list of controlled substances. Basically, legalize it at the federal level. Let states do what they want insofar as state or local regulation with it, in much the way alchohol is handled post Prohibition (there are still dry counties in the US if memory serves).
2) Concurrent with step 1, make users of MJ an 'anti-protected class'. This is to say, they're totally fair game for any public or private discrimination, and membership in the MJ users class removes any other protected class memberships you might happen to possess. Thus, if you're a pot-smoking elderly black handicapped lesbian Moslem, you'll be treated for all legal purposes like a white 30 year old Christian heterosexual able-bodied male---which is to say under almost all circumstances, you can't sue for discrimination or the like. This is intended to insure that the non-using population has means of expressing their disapproval of the using population without necessity of recourse to the hammer of the state. If they want to break out their old 'we don't serve your kind here' from very cold storage...so much the better. Enforcement of these provisions would likely require a user registry and probably an MJ purchaser's card or maybe just a stamp on your driver's license. MJ prices would fall to those commensurate with other leaf crops and the violence inherent in the distribution and sales as well. It could be treated much like cigarettes or alchohol.
If 2) results in a greatly enhanced ability of the non-using and the using population to coexist peacefully without either side feeling the need to launch governmental crusades, this solution could be advanced to regular smoking as well, and perhaps to a few other relatively soft drugs. My gut is that legal MJ would suck a lot of the oxygen out of the harder drug market---prohibition has always tended to drive a move towards more concentrated drugs.