This has just dawned on me that if you're writing an operating system then what are you writing it on? I ask this as I am reading a microprocessor fundamentals book from 1980 and this question popped in to my head:
How was the first microprocessor chip programmed?
The answer may be obvious but its bugging me.
I will take your question literally and discuss mostly microprocessors, not computers in general.
All computers have some sort of machine code. An instruction consists of an opcode and one or more operands. For example, the ADD instruction for the Intel 4004 (the very first microprocessor) was encoded as 1000RRRR where 1000 is the opcode for ADD and RRRR represented a register number.
The very first computer programs were written by hand, hand-encoding the 1's and 0's to create a program in machine language. This is then programmed into the chip. The first microprocessors used ROM (Read-Only Memory); this was later replaced by EPROM (Erasable Programmable ROM, which was erased with UV light); now programs are usually programmed into EEPROM ("Electrically...-EPROM", which can be erased on-chip), or specifically Flash memory.
Most microprocessors can now run programs out of RAM (this is pretty much standard for everything but microcontrollers), but there has to be a way of loading the program into RAM in the first place. As Joby Taffey pointed out in his answer, this was done with toggle switches for the Altair 8080, which was powered by an Intel 8080 (which followed the 4004 and 8008). In your PC, there is a bit of ROM called the BIOS which is used to start up the computer, and load the OS into RAM.
Machine language gets tedious real fast, so assembler programs were developed that take a mnemonic assembler language and translate it, usually one line of assembly code per instruction, into machine code. So instead of 10000001, one would write ADD R1.
But the very first assembler had to be written in machine code. Then it could be rewritten in its own assembler code, and the machine-language version used to assemble it the first time. After that, the program could assemble itself. This is called bootstrapping and is done with compilers too -- they are typically first written in assembler (or another high-level language), and then rewritten in their own language and compiled with the original compiler until the compiler can compile itself.
Since the first microprocessor was developed long after mainframes and minicomputers were around, and the 4004 wasn't really suited to running an assembler anyway, Intel probably wrote a cross-assembler that ran on one of its large computers, and translated the assembly code for the 4004 into a binary image that could be programmed into the ROM's. Once again, this is a common technique used to port compilers to a new platform (called cross-compiling).
Originally, programs were written on paper, and then transposed to whatever input method the computer had available. This went from knobs, switches, and jumper wires on the first computers, to punch cards, to keypads/boards.
The illustration below shows what present-day real programmers use:
Well I was around when micros first came out, we wrote cross assemblers and compilers on mainframes and minis, then we bootstrapped them onto 8-bit hardware, people didn't really bother building compilers/assemblers on micros until they had enough local storage to make it useful
One early form of read-only memory was a grid where the the address would select a row (pulling it low), and the columns represented data. Each column would have a pull-up, and each intersection would have a diode soldered in to represent a "zero", or no diode to represent a "one" [bit levels could be inverted if desired, in cases where that would reduce the number of diodes required]. Any desired pattern of bits could be "programmed" by soldering in the proper diodes.
This design was brought into the integrated-circuit world in the form of a fuse-PROM. Essentially a fuse PROM was a circuit just like the above, built onto a chip, except that all diodes were populated, and every diode had a weak fuse in series with it. As shipped, a PROM would read "0" at all locations. One could, however, selectively remove diodes by selecting the appropriate address and driving the appropriate data lines "hard" high [note: I think each bit may have had its own transistor, rather than a diode, but the principle is the same].
Fuse-programmed PROMs were succeeded by semiconductor memories, which store bits by implanting charges on internal capacitors. Not only are such memories more compact than fuse-PROMs, but if they are housed in UV-transparent packages they may be erased and reused. Note that even so-called "one-time programmable" memories almost always use this same design, but are simply housed in UV-opaque packages.
diode-matrix control store of the 1950 MIT Whirlwind Computer
transistor-matrix control store of 2005 the MT15 CPU
I'm not sure that anyone has addressed the OP's real question, which seems to be "How does the computer know how to get started?", so I'll take a crack at it.
Start with an analogy. "How does a transistor know how to get started?" Of course, it doesn't, it just works in accordance with physics, and the engineer builds the circuit so that it starts in a known state.
Computers know how to get started because they are circuits (very complicated, of course) and they are built to start in a particular state. In a modern machine that state might be one in which the main bus remaps part of the address space to user a ROM on the motherboard as low memory (the ROM having been burned with a BIOS). In a first generation machine the initial state might have been CPU halted, the program counter at zero, and memory toggle sub-system active.
An operating system is a computer program which runs directly on the processor. It can be written in any language which can be compiled or assembled down to machine instructions. Assembly and C are common choices.
Code is loaded into processors a single instruction at a time from a store - a ROM or RAM.
The Z3, the first programmable processor was built out of electromechanical relays and read instructions from punched film.
The Altair 8800, the first personal computer, was programmed by toggling switches.
To learn how a processor is constructed from digital logic, see https://electronics.stackexchange.com/questions/5516/a-fun-book-to-learn-computer-architecture-for-not-exactly-beginners
For a history, see http://en.wikipedia.org/wiki/History_of_computing_hardware
Here's a bit about really old computers:
As far as I know, back in the days, you had a big computer front panel which was used to program them. Basically, each memory cell in those computers was connected to switches. You'd power the memory and then use switches to set data in each cell. This way, you'd input program bit by bit into computer's memory. Then you'd set starting point of the program and start execution.
As technology progressed, so called bootloader programs were developed. They are short programs whose whole point is to start another program from computer's memory. The problem with entering programs bit by bit (and the later more than one bit using octal and later hexadecimal numbers) was that it was very slow and there was big chance that operator would make a mistake while loading the program. So short programs would be used to load larger programs stored in computer's memory. As time progressed, portion of code which had to be entered by hand decreased and in modern computers we often have bootloaders written in some sort of ROM or even flash memory.
I few years ago came across a microcontroller programmer which had a 4 digit seven segment display, and a hex keypad. I'm sure this didn't have a microcontroller on board and allowed you to enter machine code directly a byte at a time.
So this would allow you to (painfully) compile a list of Op codes manually one by one and enter them into the chip.
I have no idea if you can still get them, but I'm guessing that something similar was used back when microcontrollers were relatively new.
The first microprocessors would have been programmed with the help of existing computers which were not based on microprocessors. Before the first CPU was produced on one chip, there already existed advanced computer architectures which were built from discrete components rather than microprocessors. Take a look at, oh, say, IBM 360.
