Which critical processor information is stored when a program is interrupted by a hardware interrupt source ? Where does the CPU keep the information ?
What gets stored is the current core "State" of the processor. What actually gets stored depends a bit on the processor.
Typically, however as a minimum the current program counter, (the address of the next instruction to be executed after the interrupt is processed), and any and all core registers like the accumulators and flags such as carry, zero etc. They are stored in memory on something called a stack.
The stack is a section of memory that is a last in first out buffer. A pointer, unimaginatively called, the "stack pointer" is used to keep track of the current "end" of the used buffer.
The items mentioned above are "stacked" in a predefined order. As interrupts or subroutines are called the stack is appended with these states and the pointer moved forward.
As returns happen the last state is read back from the end of the stack and the "stack pointer" adjusted back appropriately.
Not all processors handle everything automatically though. Some, like the PIC, require you to stack and return everything except the program counter in code, and in the right order. You really need to check the documentation for the specific device you are using.
Stacks always have a maximum size allocated to them. If you branch out too many times the stack will overflow.
Which is what this forum website is named after.