CPU stands for "Central Processing Unit" and is essentially the brains of your computer.  The CPU is usually located right on the motherboard and has a metal fins with a big fan on it.  The CPU fan is probably the loudest thing in your computer (unless there is another problem!).

The CPU is the single biggest source of heat and if not cooled properly can "fry" leading to erradic and unpredictable errors like blue screens or random reboots.  Make sure you clean the CPU frequently.