I've been doing a bit more digging.
I have a main router and then two access points on a gigabit network backbone and currently all the Wifi is running at B, G, and N @ 2.4Ghz on Channel 11.
Both the Desire and One are showing a connection speed of 72Mbs, so theoretically the two phones are operating at the same speed however the download speeds are radically different.
To transfer a 13.5Mb file took:-
HTC Desire HD - 32 seconds
HTC One 12mins 58 seconds.
There has got to be a problem with it somewhere.
Any suggestions appreciated.
One on EE, Android 4.1.2, Software 126.96.36.199
Desire HD on Orange, Android 2.3.5, Software 3.12.405.1