Seems unsafe to even try anything like that. I recall our hospital had a whole bunch of things they said not to do with pacifiers that pretty much all revolved around the notion of "don't do like your grandmother did." (if you look at old pacifiers, you'll see they have 4 holes, two on each side... those were for tying ribbon behind the baby's head to keep the pacifier in place.)
Two thoughts though:
try one with a different shape. We found that one shape always stayed in longer than the others. Different kids seem to want different shapes.
get a pacifier tether or clip to keep it handy when it does fall out. They're just a short length of ribbon with either a velcro closure, or a loop that you can thread it through on one end and a clip of some sort on the other. These were an amazing help at keeping pacifiers clean and close at hand to pop back in. And as torbengb points out in the comments, make sure it's not long enough to be a safety hazard, either from choking or strangulation.