I'll add some points from a German perspective and the perspective of someone who defended her thesis in another university than where most of the research was actually done.
I suspect that a large part of the conflict about the supervision may be due to OP and professor working with quite different concepts of PhD thesis work which may be arising from a recent paradigm shift about what exactly PhD research should be.
Long version:
Background: when I started my PhD also in my field (chemistry) the typical set up (A) was that PhD research is "your own private fun" (as in: you're not paid for this - but you may use university infrastructure and project consumables for your research). My professor made sure, though, that all PhD students were hired as TAs or had some scholarship. Nevertheless it was made very clear that wages were paid for teaching, not for research.
This has changed considerably in my field, where the norm is now (B) that PhD students are employed to do research for a particular project.
(BTW: my old professor retired, I went working abroad and years later handed in and defended my thesis at another university. Old professor was one of the reviewers of my thesis, though.)
There are important differences/trade-offs: B of course gets you money for work that isn't paid in A. On the other hand, A gave far more freedom to choose the subject and arrange time and place youself (e.g. PhDs that were collected research of a decade alongside full time industrial work and external PhD students). Employed PhD students B are often quite tightly bound by that employment contract (project and job description have to be adhered to, there are restrictions on canceling these contracts also for the employee side).
The whole paradigm has also changed in that under A PhD students were seen and treated as fully responsible professionals (from the very beginning of the PhD - which they legally are in Germany) organizing their own life on their own behalf, whereas B students are often treated as not yet quite fully qualified professionals, and the PhD studies take more the shape of a university program.
The professor giving only general advise at infrequent meetings may very well be an expression that they expect you work independently under paradigm A. To reiterate it: paradigm A thinks of the PhD "student"* as a fully trained top professional (only top professionals should do a PhD) from the very beginning. The PhD "student" does the thesis to proove these [already existing] skills of doing research without supervision (a research work done under supervision would qualify only as Diplom/Master thesis).
Whereas paradigm B thinks of the PhD as a training program where the student learns doing research.
Wrt. expected time line of the PhD, A theses took as long as they took (professor decides whether research content is sufficient for promotion = PhD), whereas B PhDs are supposed to be done within not too much more than the 3 years of the research project employment contract (this is also meant to guard students against exploitation).
* PhD "student": aren't actually called students in German (student = Student, PhD student = Doktorand) and are typically not required to sign up with the university as student.
In your situation, not having such a research employment contract is a huge advantage as there is nothing that binds you to your old institute/professor legally.
(Side note: B students have the right to demand cancellation of their employment contract if they do have a better offer.)
Nevertheless, I'd suggest treating your situation in analogy.
I know it sounds not very ethical, I said nothing to my supervisor until I'm officially accepted by the new university
This is not unethical, but expected and correct behaviour.
You can safely rely on the academic community in Germany accepting reasons that would be considered suffient under employment law for temporal contract to demand cancellation.
obvious that our field of research doesn't match
A better match of field is a good reason. How much that counts would depend on how much better the match is (but then the default assumption is that noone moves if there isn't anything gained by that move), and how long it took you to decide that the match wasn't good.
If my guess is right that there is a considerable gap between his and your concepts of good supervision of a PhD thesis and the expected level of independenc in your work, then:
he may be considering any closer level of supervision bad in the sense that the need for closer supervision implies bad mark (possibly to the point where he may consider the work not up to the independence standard of a PhD thesis) when judging your research. So close supervision means depriving you of your chance to demonstrate the ability of unsupervised independent work.
A better match also in these expectations is needed for a successful PhD. This doesn't imply any direction: if you are already up to do your research practically without supervision it will be good for you if you can show that in your PhD. If not, then your only chance to surviving a PhD is one with (intially) closer supervision.
He reassured me that he had sufficient knowledge on my topic, but it turns out that he doesn't. [...long snip...] He never admits he doesn't master my topic
Of course, it would be better if he were able to tell you that he isn't really into the particular field, but it's really you not he who's expected to be the expert on the topic of your thesis. He is actually only required to be able to judge and testify that you did do good research, and it would be totally legitimate (A) if he expects/hopes to learn from you about this topic.
My old professor once expressed this roughly as "You are a fully qualified professional. You are able and are expected to be able to decide and judge on your own what to do." and another told me "I'm not able to have in-depth scientific discussions with you on your [particular specialization]. It is exactly because I perceived a lack of this expertise in my group that I hired you."
The important conclusion is: openly discuss these paradigms with your new professor to make sure you understand each other.
6 months into the PhD would translate in paradigm B to "at the end of probation period" and that's a totally acceptable time frame to realize that the match wasn't that good.
any financial advantage is a very good reason.
For the employees that could be more total pay, better hourly wage, more total hours, longer duration of fixed term contract, maybe a scholarship. As you are not an employee, lower cost of living in the new university town or better opportunities for part-time jobs would be acceptable as well. Extra bonus if you find a part time job there that is related to your profession.
Keep in mind: all you really need to get your PhD is:
- your research (which doesn't need to be done in formal association with any university) and
- a professor/institute that is willing to read your thesis and who then considers that it fulfils the criteria for promotion/successful thesis so they accept you as PhD candidate. (Of course, in practice this is easier if you are all the time inside the academic system and supervised, but few professors will reject you if you arrive with a good thesis or research proposal that matches their field)
This closely matches A, but is equally valid for B. Check the regulations of your new university early, though: I know of some faculties that demand a certain number of TA hours.
Your PhD research is your own work (and in your case there are no difficulties due to the old university owning work done as an employee), you can take your already existing work with you wherever you go.
Some more points:
academia is a small world, and even smaller within Germany.
A professor retiring doesn't mean they vanish from academia. Just university cannot demand that they do any particular work for them any more. But in most Länder, the professor retains the rights to teach and also to lecture and to take exams, so they can still act as supervisor and/or reviewer for a thesis.
The other consequence is that your academic community probably will know also their side of what happend about you leaving (you may not be that important now academically speaking, but the next occasion is when your new faculty is looking for an external reviewer for your thesis).
But if you treat this change of university honestly and professionally it doesn't matter that much how your old professor reacts. He may be temporarily annoyed when you tell him, but if he reacts unprofessionally, your academic community will realize it. Just as they will realize if you are dishonest and/or unprofessional.
I hope I did guess correctly and Old Professor is not particularly but but just acts according to PhD paradigm A.
If so, acknowledging and sincerely thanking him (don't if you cannot sincerely) may smooth the situation considerably. If you can express that you weighted and appreciate the chance of showing you're able to do the research entirely on your own (in his group) vs. joining the other group where having colleagues working on much closer topics will hopefully give your specialization a boost due to more in-depth scientific discussion with your peers, more closely related seminars, etc..
After all, the immediately important point for you is how much bad feelings you carry around about these 6 months.
I see no career prospect here, as he'll retire in three years, and then his department will close
If you want to go for an academic carreer, the perspective shouldn't be a postdoc at the same institute where you did your PhD. The expectation is a postdoc somewhere else, possibly abroad. (Although there's less stress on this if you have longer research stays abroad and/or changed groups/universities before).
