There are a few answers. Becuase you're recording a podcast, you will only need stereo/mono functionality.
Many people in your boat use a free program called Audacity. It's open source and work very well.
If you'd like to spend a few more bucks, Steinberg make an audio editing suite called Sequel. It's a fantastic and concise audio editing and recording app.
You might also want to check out Sony's Sound Forge Studio. It a stereo recorder/editor. I've used it for many years and love the program.
In addition, you'll need to get a mic and preamp that connect to your PC via USB. If it's just you and no other mic's are needed then you might want to go the USB mic route, forgoing the preamp. If you decide to go with a preamp, try ones from M-Audio - they're inexpensive and work great. In fact, M-Audio makes a podcasting package with included software I think. Check out Guitar Center for more info.
When it come time to upload your podcast, you'll want to do some research on RSS feed readers, check out feedburner, as well as possibly getting a site of your own going to host them there. Wordpress offers some great add-ons for podcasters looking at creating sites with audio.