I sympathise with your predicament, and I certainly would not suggest that you need to "suck it up". In fact, I would probably go even further --- your persistence to the contrary notwithstanding, the vast majority of math presentations at university ---particularly those delivered by students--- are barely watchable, and they are quite often a waste of their audience's valuable time.* If you find that you are bored and/or overwhelmed in the first few slides, this suggests to me that the presenter is doing a bad job motivating interest in their topic, and keeping audience context in mind. That is entirely consistent with my experience viewing mathematical presentations in university seminars; by and large, the quality of these presentations is abysmally low, and fundamental presentation problems are ubiquitous.
One of the most common problems I have observed with mathematical presentations is that they often tend to cram in too much material, and then overwhelm the audience by going too fast through specialist material that is unfamiliar to the audience. It is common to observe "flashers" who put up walls of definitions, equations, derivations, etc., and then rush over them to try to make their talk fit into the allocated time. Often the presenter makes no effort to motivate interest in their topic, and just assumes that the mere presence of an audience vitiates any obligation to be interesting. In these cases, the audience gets lost after the first few slides, or bored, and then simply tunes out. That is not the fault of the audience member --- it happens because the presentation is a steaming pile of shit. If you attended a rock concert, or a comedy night, the presenters would know that it is their job to make you interested, and make sure the audience can follow along with the material. Mathematics presenters often have not learned this lesson.
So, my advice: if you are compelled to go and see crappy presentations that fail to motivate your interest, and lose you within the first few slides, use that boredom as motivation to improve your own presentations, and make sure you are not subjecting others to the same boredom and bewilderment that you are being forced to endure. Scrutinise each presentation with a critical eye and note the things that the presenter is doing wrong. (Here is a related answer with some suggestions for how to give a good presentation.) Did the presenter motivate your interest in the topic? Is he/she enthusiastic about the material? Has the presenter provided a "hook" that caught your interest? Did the presenter explain the material to you nice and slowly so that you could follow along? Did the presenter limit their definitions and notation to what you can reasonable remember as you follow along, or did they overwhelm you with definitions and notation? Did the presenter rush through something you didn't understand? Did the presenter flash up a "wall of algebra" for a microsecond and then move on to the next slide?
One final note on this issue is your mention of the fact that your own specialty area is quite narrow, and you have trouble following along with presentations on foreign areas. That is very common, and again, it is generally the fault of the presenter. For that reason, when you come to give your own presentation, make sure you put yourself in the shoes of someone who has no knowledge of your specialty topic, and no interest in it. Realise that it is your job to motivate their interest in what you are saying --- start strongly and make them (internally) say "wow!" and make sure your presentation gets them paying attention. Simplify your problem and obey the crow epistemology (people can remember three things). Take your audience on a nice slow journey through your topic, that is interesting, and so simple that they think your work is trivial. Good luck!
...except to the extent that they are practice for the students. Often we hold seminars of student presentations to give students practice delivering presentations, and asking questions of speakers. We expect them to be bad at it, which is why they are students.
