I am writing on behalf of someone else. But the question will be easier to pose if I put myself in his shoes and speak in the first person.
I will soon graduate with a Bachelor's in economics and mathematics in a Third World country. I have, however, also studied physics. I actually have more physics than mathematics courses on my transcript. I want to study for a PhD in mathematical physics in the U.S. I would need support (e.g. TAship).
Where I live, subject GREs are only offered once a year, and one is not permitted to take two different ones on the same day. It is too far and expensive for me to travel to another country to take a second subject GRE.
Suppose, for the purpose of this question:
I decide to take the Physics GRE, and apply to physics departments
I get admission to a physics department, and a TAship
I take a combination of both math and physics courses and do well
After several semesters I realize I'd rather that my home department were math, instead of physics
How hard would it be to apply to switch over to a math department (either in the same university, or a different one), assuming I had taken sufficient math courses, and done very well in them?
Note: The similar question is somewhat helpful, but I would like to see a specific focus on the math/physics interface; also, please consider the specific option of pausing at completion of a masters, with a switch of department at that point.