harddrives are about your best bet for longer term storage. and yes, you can format a drive from an older computer and use it in a new one (or in the old one, for that matter). if you use it in the old one, you'll obviously still need an operating system instaleld in order to access it. but, if you want to put it in a new one, windows is very accomadating about installing and removing drives. i move drives in and out of a system fairly regularly.
you might also consider getting a usb external drive. this is probably about the best way of backing up data. usb has 'hot swap' capability. that basically means you can have the computer running, but not the external drive and then turn the drive on and windows will mount the drive. then, when you're done, turn it off and windows will unmount it automatically. the drive stays off till you need it again, saving wear and tear on the device.
other methods of backup are hot swap drives. these almost got popular a few years back. havent seen any for a while now, though. usb probably put them out of business.
you can also back up to cd or dvd, but these dont have as long of a shelf life as harddrives with only one exception i've ever seen. there was a japanese company that was making archival cd's that claimed a shelf life of something like 15 years. sorry, i no longer remember the company name or have the link. there were also fairly expensive.
cheap cd's and dvd's may surprise you as to how short their shelf life is and maybe even some name brands. a couple years ago some were reporting a reliable life span of only about 18 months. hopefully, this has changed, but you shld probably always research your brands before buying.
you might also consider backing up online. there are plenty of services out there that can and will do this for a fee. this is fairly reliable, since the service is making backups of your backup and can simply restore your files if their own servers go down.
RAID is another method for protecting your data. this involves multiple drives on the same computer. there are several versions of RAID, but at least one version is specifically a redundant data back-up mode. if one drive goes down, the data is already on the other drive.
and if you really want to keep your data 'forever', i've heard of a cd type that basically writes on a metal alloy type disk. this is about as permanent as you can get