Before you jump onto the web and start spending lots of money on expensive computer parts, there are two important questions you should answer which will guide your purchases:

1. What are you going to use your new computer for?
2. Will parts be available to use from your old computer--or do you want to reuse parts from your old computer?

(Often, you will either want to hand your old computer down to someone else, in which case you must keep it functional, or it may be so old that you don't want to use any parts from it, because they will slow down your new machine too much.)
View: The Wikibook PC Building Guide

This guide breaks it down. It goes into great depth explaining all the components, their function and so on. It takes OEM vs Retail, salvaging old components, etc.

If you are looking for decent information for setting up a BYO PC, this guide will help you immensely.