To boot from a USB hard drive your computer's BIOS must support that feature. Go into your systems BIOS and find the section for boot devices or boot order and see if you have the boot from USB option. Then you would have to install Linux on the external drive and make it bootable.
If not and you want to be able to play around with Linux you can run a version called Knoppix. It lets you boot from the Knoppix CD and run Linux on your system without actually installing it on your hard drive. When you are done you can boot right back into Windows. The only downside is that any changes you make are lost since its not saved once you shut down.
You can also install Ubuntu Linux on your Windows computer and have it dual boot to either OS and if you want to get rid of Linux its easy to uninstall.