They are not always electrically neutral.
An n-type semiconductor has an excess of 'free' electrons -- electrons that can move about freely in the semiconductor (very similar to electrons in a metal). These electrons are 'donated' by immobile donor impurities doped in to the semiconductor.
If you imagine starting from that state, then the result is still neutral. However since the electrons can move, they have a tendency to diffuse away from regions of high concentration. If you connect another material (e.g. p-type) to the n-type (forming a pn junction), electrons will diffuse from the high concentration region to the low-concentration region. This won't continue forever (unless you have a power source connected), because in leaving the n-type region, they leave a + charge behind. This creates a restoring electric field, and at some point this restoring field will balance the diffusion process and an equilibrium will be obtained. The specifics of this depend on the materials, the doping and temperature, as well as any external voltage applied between the 2 materials forming the p-n junction.
Since (starting from neutral), electrons (negative charge) have left the n-type region, it will become net positively charged, and the p-type negatively charged. In a similar way holes ('anti-electrons') from the p-type diffuse over to the n-type, further charging it positively.
A similar behaviour would occur if you connected a heavily doped n-type to a lightly-doped (in fact it occurs any time there is a concentration (or temperature) gradient).
The material as a whole isn't charged (just polarized), but if you connected it to another conductor (e.g. a wire), charge would move between the cloud of free electrons in the wire to the semiconductor, putting a net negative charge on it. Although it is small, it could in principle be detected by observing electrostatic forces. It cannot be measured by (e.g.) connecting a voltmeter to the semiconductor and the metal because charges would also flow into the leads of the voltmeter, exactly canceling and leaving no net voltage. If in fact there was a temperature difference, you would be able to measure a voltage -- this is the Seebeck (thermocouple) effect.