Keep in mind that even if they do start a game as a Pure Mortal (complete with +2) and sprout a power mid-session, buying off the Pure Mortal bonus would count toward the price for the mid-session upgrade. For example, such a character who buys Inhuman Strength mid-session would lose four refresh, and would therefore have to trade in four Fate points (or gain four debt, or a combination thereof), as well. This is significant enough that I don't think that there need be further limitations on when such a purchase.
That said, I still favor the concept the characters who are Pure Mortal are mundane until they aren't Pure Mortals any more. So, for example, until they buy off that Pure Mortal bonus, they could not invoke their High Concept for bonuses related to their supernatural heritage. That is my opinion; others differ.