Old programs on the first computers' CPUs were written in machine code or assembly code (which is just an easier way of writing machine code). Machine code is incredibly difficult to handle, since it's just binary numbers which represent memory addresses, CPU registers, operators, etc. Assembly code made things easier but still much harder to read/write compared to today's high-level languages.
If you want to go really old school, all instructions were handled by punch cards, just paper cards with holes punched in them, each hole is one way to represent binary by allowing an electric rod to make an electric connection where there was a hole (representing 1) and the lack of a hole would break the connection (representing 0). This was later replaced by transistors and logic gates on a CPU. Of course once the mircoprocessor came around the amount of transistors, and therefore computing power, increased exponentially.
Keyboards just send electric impulses to the computer which has a listener which interprets which key is pressed and stores it's binary value which can then be processed by the CPU to start a list of instructions.