Ungrounded devices float at some potential. This potential may be set by AC line filtering capacitors or just stray capacitance in the mains transformer or safety capacitor that is from the primary to secondary side.
Connecting two floating devices together makes their potential difference equal but there will be a surge of current at the moment of contact to charge/discharge the capacitances. This is fine if connectors have grounds contacting first and they stay connected while data connections mate. But if there is a break in the ground connection due to bad connection, all that potential difference between data pins may damage chips.
This can be witnessed even at home; try taking for example a TV and some source device that are both ungrounded. Touching their metal cases can cause a tingling or stinging sensation (leakage current) and sometimes a small spark can be seen when connecting the equipment cases with wire. This is the reason it reads in the manuals that equipment must be connected to other equipment while devices are unpowered.
A mobile phone is not connected to other equipment than the charger so having a floating device is not a problem or a hazard by itself.
Laptops have grounded power supplies so they are grounded.
1) Having a grounded/earthed supply means your prototype is always at ground/earth potential so it is safe to connect to other grounded/earthed equipment like PC JTAG adapter or oscilloscope. But there is also a downside for having ground-referenced power supply as it could also create a ground loop. Imagine that power supply is driving 2A into load and the ground wire between power supply and load disconnects. If load is connected to other equipment like grounded PC or grounded oscilloscope, the 2A would keep flowing via earth wiring, oscilloscope ground lead or PC USB cable. This is why I keep my lab power supply ground potential floating when I don't need earth ground referenced supply.
2) Oscilloscopes are grounded for many reason, one of them is safety. Imagine accidetally connecting one scope ground lead to mains live voltage. All metal parts and other scope channels grounds and equipment they are connected to would then become live.