The following is only speculation from me watching the movie and I don't have any other sources. There might be more materials or information somewhere, so if anyone knows more, feel free to post. It's been a while since I've seen it, so feel free to correct me, if I remember anything wrong.
To me this actually made sense, but you'll have to consider the whole picture to understand.
- Kaiju blood (and probably body liquids in general) his highly toxic.
- Humankind is on the edge of extinction.
- The last human survivors are in densely populated, fortified cities.
- While hard, it seems to be perfectly possible to beat a Kaiju to death.
So why not shoot or hack and slice Kaiju? Simple: You don't want a mess.
- Using any kind of ranged weaponry you'd typically apply as much force as possible in a relatively small area to penetrate your target.
- Same applies to slicing or impaling melee weapons.
- First of all, this impact energy would also be applied to any body parts separated in that hit (think of the cannon balls depicted in The Patriot). You don't want Kaiju parts to fly all over the place.
- In a similar vein, the more open wounds you add to the Kaiju, the more blood and other body liquids will be spilled all over the place, without necessarily killing it right away.
- They could have made blades (like the sword) immediately cauterize open wounds to not spill blood – and to look even more awesome on screen–, but then again this would defeat it's actual purpose (especially considering Kaiju seem to be fine continue fighting with body parts missing).
As such one of the possibly best options to kill a Kaiju is to use blunt force (like fists) to try to beat it to death causing internal wounds rather than external ones.
Gipsy Danger kills the first Kaiju (of the movie) in the open sea. Considering this happens early on and they're not fighting in a densely populated area, the plasma gun seems to be a good and valid choice.
But what about the sword scene? Pretty simple: They're at very high altitude (and pretty much out of other options, the guns aren't working), so using the sword is their best option. Most (if not all) of the blood would be gone and not reach the ground.
As such I wouldn't ask whether actual weapons would be part of all Jaegers, it's probably been the case at some point. The bigger question is, why don't they continue using them. But you'll also have to remember that there once were a lot more Jaegers around. These are just the remaining forces and they probably ran out of replacement parts and supplies long ago (or otherwise they could probably rebuild more as well).