You will have a hard time using a needle to dispense accurately for 0.5mm pitch pads. But you might have better success with a different paste. I used to use this Kester solder paste but I found this ChipQuik flows better (NOTE: these both have lead). Or maybe your paste is older; it will get hard to dispense as it ages, I found.
If I'm manually applying paste for parts like that I usually just apply a straight bead along all the pads and clean it up afterward if necessary. It is possible to get the bead size (pressure and speed) just right that there are no solder shorts after reflow, but it takes some trial and error.
As for the stencil, I don't know how you have it set up, but the stencil should just lay flat on the PCB. Sometimes I need to add dummy PCB material around the PCB I'm pasting (if it's not a full panel) to keep the stencil level. Of course your stencil should not be warped or dented when you start. I'm assuming a stainless stencil. I tried the cheap plastic ones, but they come in a roll and don't always come out completely flat. I stick with SS now and they are shipped flat between two thick boards so they are good.
Even if there is some gap, once you start applying the paste with the squeegee, it should flatten the stencil out before the paste gets to the openings. I find the issue is more with removing the stencil afterward. I don't use a stencil holder, I just align the stencil and tape it to the panel. If you are not using panels, I suggest taping the PCB to a board along with other spacer PCBs to provide even elevation, and then tape the stencil to the spacers. The tape is not ideal as it adds a small gap, so put it in an area as far as possible from SM components. At least that's what I do, but I haven't tried using something like the stencil holder you use.
I find it's difficult to apply all the paste in one pass of the squeegee so it's very important that the stencil doesn't move so you can pass over it again. If the stencil is not laying fairly flat to start with, and it lifts off after one pass, this could be causing your smearing problem. Do whatever you have to do to get the stencil laying as flat as possible on the PCB before applying the paste and it should be OK.
All that being said, I still have some issues using stencils on 0.5mm pitch parts. Sometimes they come out perfectly but other times not. I find it's more about the paste thickness than smearing though. Even with the same thickness stencil, the evenness of the paste application is influenced by squeegee technique. Like with using a dispenser, I think you will have to go through some trial and error to get your process tuned right. It can be done.