What is the relation between a person capability and his caste?
There is some relation. Usually a Brahmin is associated with knowledge and learning and a Kshatriya with valor and virility. So, a Kshatriya will have naturally more power to tolerate pain as they will get often wounded during wars.
Manu Smriti 2.155. The seniority of Brahmanas is from (sacred)
knowledge, that of Kshatriyas from valour, that of Vaisyas from wealth
in grain (and other goods), but that of Sudras alone from age.
So, that's how Parashurama had guessed although I don't think that the method is full proof (i.e it may not work all the time).
Regarding your other query, Kshatriyas obviously have every right to get trained in warfare, as participating in wars is one of their prime duties, but Parashurama had vowed to train only Brahmins and not the Kshatriyas. Parashurama's enmity with the Kshatriyas is quite well known.
And another reason why Parasharuma was terribly upset with Karna is that Karna had lied to his Guru about his caste. Lying to the Guru is of course a sin and apart from that pretending to be someone from a higher caste is also a serious sin as the following verse says:
Manu Smriti 11.55. Falsely attributing to oneself high birth,
giving information to the king (regarding a crime), and falsely
accusing one’s teacher, (are offences) equal to slaying a
Brahmana.
Karna, in order to acquire the training, pretended to be a Brahmin in front of his Guru fully knowing that he is not one.