OK I think i have isolated the issue. I've been to a store today and compared with a note 3 that was on display.
I used the "show pointer location" debug in the developer options for analysis.
On my note 10.1 2014 even though the spen is docked it still registers pressure somehow. Shown as "prs" in the top bar that appears when you enable pointer location. The pressure using just my finger as input fluctuates between 0.00 and about 0.15 even without spen! Where it should be at a steady 1.00 without spen.
That is exactly how it works on the note 3 (and all other android devices without pressure support like my sgs3). If you use your finger it sticks to 1.00 and when you use the pen it scales between 0.00 and 1.00.
Now on our notes normal touch input with finger only registers as very low pressure (~0.15) and I'm pretty sure that causes the shaking.
And what I conclude from that is that it should be a simple software fix in the driver to correct that. Finger = 1.00 prs. Any way to let samsung know about this? Or maybe even a dev could try to fix that in a custom rom?
Phone: Samsung Galaxy S III | Stock rooted
Tablet: Galaxy Note 10.1 2014 Edition Wifi | Stock rooted