When we connect a light bulb to a voltage source (230 v, 125v, 12v, ...) the light bulb offers a resistance that, applying ohms law, gives the amount of current that will go across the bulb. The bulb is designed to support this amount of current. Thus, everything ok. Same for the buzzer.
The push button is nothing else than a near than infinite resistance (when not pushed) or zero resistance (when pushed). So, when pushed, as it offers near than zero resistance, the amount of current across it could be very high, enough high to destroy to push button or the arduino. For this reason, a resistor is add.
A led connected to a source of more than 2v offers a very small resistance, being the amount of current so big that, before to reach the maximum, the led burns. For this reason, an external resistor is add, that increase the resistence, limiting the amount of current across it and across the led to a value that both can support.
In particular, the led is not a lineal component, so it doesn't follows the ohms law. We can say, in a educative way, that it offers an infinite resistance to the first 2v and none to the remainder voltage. The exact relation between the voltage applied and the current is show in this draw (from here), it depends also of the led color.
The real number are: if you use a 5v source and you want connect a red led (1.9v and 20 mA for an optimal performance), the resistor must be:
R=(5v-1.9v)/0.02A=155 ohms.
For these reason we have always a box with a lot of 220 ohms resistors :-) Hope you enjoy these experiences in electronics.