I've used wood from popsicle sticks and wooden coffee stirrers for model making. Matchsticks might be more fragile, but what I used would be a starting point.
My usual tool is a razor saw, like Chris H suggested, used in a tiny aluminum miter box. The saw I use is similar to this:
Amazon
It has a very thin blade with very fine teeth that cut on pull. Very fine teeth will minimize tearing, so the wood will hold together better. The miter box looks like this, but mine is a different brand:
Amazon
My miter box is sized to use with razor saws (important so the blade doesn't drift around in the slot, and it will be in the slot for the length of the cutting stroke).
For something as tiny as matchsticks, I would do them in batches. Make a block of aligned matchsticks. Gluing one end, like Chris H mentioned, will hold the block together (only the far end is glued to stabilize the pieces; as you cut the other end, you have loose cubes). The bottom of the miter box usually has some ridges, so you would want to line it with cardboard or something to level it for this. If you can, find a way to clamp the block in the miter box. These miter boxes usually have a lip so you can press it against the edge of the worktop to keep it stable while you cut.
Don't try to speed up the process with more pressure on the saw. It should make a clean cut with minimal loss of wood.
If you want to use a Dremel tool, a carbide blade wouldn't be the best for this. You can get circular saw-tooth disks. They come in different diameters and different teeth.
Amazon
The ones pictured are a little thick. I use ones that look like they're cut from thin sheet metal. They make a clean cut in small soft wood and don't remove much (couldn't find a picture). A large diameter disk might have too much drag if your rotary tool doesn't have a strong motor. For something as small and fragile as matchsticks, use one with a lot of fine teeth.
Another type of disk that would work for this is resin cutting disks:
Amazon