Just to expand on what others have already posted:
1. Socket Type - This is exactly what the name infers, just the connection type. Generally speaking you should find the CPU you want, then find a motherboard with a matching socket type. A cpu of one socket type will not fit in a motherboard of a different socket type. These two MUST MATCH.
2. How many cores they have (Single Core, Duel Core, Quad Core)
Cores is how many cpu's they fit on a sigle dye (chip). Cpu's are so small today (45 nanometers) that they can fit multiple cpu's on one chip. This means that you can do more computations with the same chip (b/c on that chip there are more cpu's). So when you have a dual core you actually have two processors in the same chip, and with quad you have four processors on the same chip. This is not to be confused with multiple processor computers like servers, which actually physically have multiple cpu chips on the same board. Mathmatically speaking a 1 ghz dual core chip will MORE THAN double the output of a 1 ghz single core chip. I'm not a math major but the reason is because of efficiency. Two processors on the same chip (so they communicate extremely fast b/c they are in the same dye) will compute more than twice as fast as one. So a 1 ghz double core processor will more than double the output of a 1 ghz single core. I should note that your OS/software must be able to multithread (send more than one process to the cpu) in order to take advantage of multiple cores.
3. Front Side Bus vs. Hyper Transport.
I think Soul explained this one well. The faster the better in general, because it is the speed to which the processor/ram/gpu are communicating.
4. 64 bit vs. 32 bit
This one confuses a lot of people. Yes there is the instant advantage of being able to hold more than 4 gig of ram on a system running a 64 bit OS. But the advantages DO NOT stop there. Overall your system will perform much faster (almost double) running at 64 bit. So identical systems, one running 32 and the other running 64 bit, compared the 64 bit would almost run twice as fast (assuming the OS/Software can run 64 bit). Think of it this way, if you had 128 bits of data to move from one room to the other. And you had a bucket to carry them in, with a 32bit bucket you would have to move back and forth 4 times in order to move all of that data. Now with 64 bits your bucket is twice as big, allowing you to complete the task in 2 trips. So the advantages are not just in RAM capacity, your software, os, and mb will be able to transfer data (commands) at a much faster rate, resulting in better overall performance.
5. Chip architecture (45nm Harpertown, 65nm Brisbane, 65nm Allendale, 90nm Italy, and so on).
45 nano meters is the size. These are just the sizes of the processors, and the name is just a code name for the processor. The name really means nothing, just a way to tell different processors apart instead of saying "ya i have a 45nm Harpertown with 1 mb L2 cache and 512 kb L1 cache" you can just say you have an Intel Harpertown.
6. Caches (L1, L2, L3)
This is where data (commands) are stored ON THE CHIP ITSELF. So the data can be acccess at incredible speeds. So the CPU doesn't have to look off the chip for the code to say move the mouse around. With larger caches (in generally the larger the better) the cpu can store more commands on the cpu ship itself, meaning that the process of moving your mouse, opening a program, etc (depending on what it stores in the cache) it can just grab the code from the cache, instead of moving accross the bus speed of your system to get the code from RAM.
Replacement policy is important, but cache will definately help the speed of your processing. It also depends on the manufacturer (i know more variables). Intel chips use a north bridge to communicate between ram and the gpu, but amd chips have a pipeline straight to the ram. So in the gaming community your commands (hard core processing like games) are not stored in the cache, so cache really doesn't effect gaming performance much. Which is why amd is more popular, because the cpu can communicate with the ram directly.
No i don't believe there is a magical formula, there are WAY TOO MANY variables involved. Really i would suggest reviewing what you will use the computer for, then build accordingly. Another thing to consider is power consumption, the cpu's today are actually trying to use less energy then before, so keep that in mind too.