I own both a 32 GB HP Touchpad (which I'm now selling) and a 32GB Verizon 3G Motorola Xoom I just bought refurbished from Daily Steals a couple weeks ago. If I were you, I would go ahead and get yourself a Xoom. Here's my thoughts on the two devices compared:
Screen
The colors on the HP Touchpad are beautiful, and are vastly superior to the Xoom (which honestly looks washed out in comparison), though the Xoom has the high screen resolution (and is slightly larger), which is important to me. Responsiveness I haven't found to be very different--the CM9 build I was running on my Touchpad was exceptionally smooth for the most part, so I'd say that the Xoom isn't a major improvement, but your results may vary depending on your experience with the ROMs you've been running on your Touchpad. Touchpad/Xoom draw, or perhaps a slight advantage to the Xoom, but it will depend on what you want.
Battery Life
Battery Life on the Xoom is superior, no doubt. Xoom win.
Hardware/Build
I personally liked the hardware on the Touchpad just fine--I felt it had good build quality, and it was nice and light. The Xoom is also very well built, however, and I prefer the metal case. I will say the Xoom's volume buttons are a bit more difficult to press than the Touchpad's, but this isn't a big deal to me. The Xoom is also a bit heavier, but not enough to be a problem. Xoom win (only slightly, IMO).
Accessories (Case)
Yes, this matters to me. The OEM case I had for the Touchpad was perfect. It conformed to the shape of the tablet perfectly, and covered the entire device without limiting button functionality. The screen cover also doubled as a nice stand for the tablet, which I used extensively. I bought that case for $20. I have not found a case that is anywhere near as good for the Xoom, though I do like the one I finally found (Bear Motion Leather Case
http://www.amazon.com/gp/product/B004ZAZMKE/ref=oh_details_o02_s00_i00). Touchpad win here (again, IMO. You may find a case that works better for your needs)
Speakers
The speakers on the Touchpad sounded much better, and their placement I found to be more convenient (at the bottom of the tablet). The Xoom's speakers are decent, though their placement on the back of the device is not entirely ideal. Touchpad win.
Usability
This is the biggest reason I am ditching my Touchpad for the Xoom. The Touchpad's camera does not work on any Android build, while the Xoom has two you can choose from (both front facing and rear facing). My Touchpad also experienced Force Closes on many apps randomly, and I would often have to restart my device. Additionally, I ran into many apps on the Touchpad that were not compatible, and I could not even install. The Xoom has a big win here, because, as a native Android device, I don't have any of these issues.
Anyhow, these are just my two cents, as somebody who just moved from the Touchpad to the Xoom. I still really like the Touchpad, and it was a great, cheap way to get into Android tablets, but the Xoom was a good step up. If I could tolerate a 7" screen, I would have grabbed the Nexus 7, but I really prefer the larger screen size.
I think you'd do well to get yourself a Xoom. I love mine, and don't regret the move one bit.