No, there is nothing inherently wrong with hitbodedut.
Hitbodedut (התבודדות) is a particular type of meditation which is part of the path of learning prophecy. It comes from the root of 'badad' (בדד) which is to be alone or solitary. This type of meditation is closely associated with the idea that the Avot, Avraham, Yitzchok and Yaacov tended their flocks in relative isolation. David HaMelech too, when he was young, was involved in the same type of activity. It is also in keeping with the same practice outlined in Sefer Mesillat Yesharim. One of the classics discussing this subject at length is by Rabbeinu Avraham ben HaRambam called "HaMaspik l'Ovdei HaShem". The final chapter of what remains of this text is called "Perek al HaHitbodedut" and deals with this subject exclusively.
The primary occupation during this part of the activity is making yichudim, which are a special type of kavannah. It is not considered "bitul Torah" and is actually a more complete way of praying or performing any of the mitzvot together with their blessings. The expression often quoted from the Sages about this is that 'prayer without kavannah is like a body without a soul.'
Another part of this 'hitbodedut' can also involve a personal kind of spiritual accounting, called 'cheshbon hanefesh'. This is often described as what a diligent person does when trying to improve upon their activities. It is often compared to good business practice, like reviewing where one profits, where one has deficiencies and experiences losses and how to maximize the return on ones investment. It is a very small part of the process of hitbodedut and should never leave one feeling sad. This is because the conclusion of this part of hitbodedut is in knowing that HaShem is absolutely forgiving and will help all who truly reach out for assistance.
There are some teachings that discuss the idea of reflecting upon what one sees as the influence of their 'yetzer hara' in order to overcome it. Depending upon the individual, that can result in deleterious effects. That is why, for that specific type of activity, it is often recommended to have a 'mashpiah' or spiritual mentor who can help an individual to stay on a good path. The "Yetzer HaRah" can bring one to have doubts, or sadness or simply to be distracted. It has been compared to the idea of someone coming up and talking to you while you are in the middle of saying the Amidah prayer. The best course of action is often to ignore it and stay focused on the task at hand. For specifics on this subject, see Tanya, Section 1, Chapters 28 and 29.
Like with anything in the Torah, it is a "Tree of Life" to those who take hold of it. Hitbodedut is a part of that tree.
There are countless seforim which deal with the subject of hitbodedut in Hebrew along with all kabbalistic siddurim. If you are looking for a couple of excellent English sources on the subject, the three books by Rabbi Aryeh Kaplan are a good place to start.
1) Meditation and the Bible,
2) Meditation and Kabbalah, and
3) Jewish Meditation