There are, to my knowledge, three incidences of Portkeys that were created in areas that would guarantee that only magical people would use them.
Firstly, the Triwizard Cup from The Goblet of Fire, which certainly was not inconspicuous.
Next the old kettle in Dumbledore's office from The Order of the Phoenix. Though this is described as an "old kettle" and seemingly falls into the "rubbish" category, the book describes it as, "...the old kettle lying innocently on his desk." It seems that the kettle went unnoticed until Dumbledore brought their attention to it, otherwise sitting innocently on his desk. So not exactly "rubbish" but certainly inconspicuous.
Finally, the Portkey at the Ministry of Magic later in The Order of the Phoenix which was the golden wizard's head from the statue which Dumbledore had used to protect Harry from Voldemort. Certainly not "rubbish" though admittedly its value was diminished at being removed from the statue. (Additionally a wizard's head would not be very inconspicuous to Muggles.)
So in conclusion, two of the three Portkeys from canon that would be guaranteed not to be picked up by Muggles were magical in nature. So Portkey's aren't "always" made out of rubbish, and there is an indication that when the chance of a Muggle stumbling upon them is remote they can and do utilize less "inconspicuous" objects.
Note: The "guarantee" that these objects wouldn't be stumbled upon by Muggles comes from the magical protection which Hogwarts provides against Muggles stumbling onto the grounds (for the first and second example), and from the fact that the last example was created within the Ministry of Magic itself.
Firstly, the Triwizard Cup from The Goblet of Fire, which certainly was not inconspicuous.– Mithical Jan 17 '16 at 09:08