Usually not, but sometimes.
Most devices that run off of two AA cells or similar are just powered directly from the two cells in series. The components inside the device are chosen to be able to operate over the valid voltage range of the two cells. For example, there are many microcontrollers, especially those intended for low power applications, that can run from 1.8 to 3.5 V.
Using the battery voltage directly simplifies these devices. The microcontroller can be directly connected to the battery. When nothing is happening, the micro is in sleep mode drawing very little current. But, it can be woken up quickly when something happens, like a button is pressed. The sleep current of modern micros is well below the self-discharge current of typical batteries, so there is effectively no difference in battery life between everything being truly off and the micro sleeping when there is no activity.
Some devices, typically ones that handle more power, have a switching power supply that makes a fixed internal voltage from the varying battery voltage. Those devices will draw more current as the battery runs low, since they are effectively drawing a constant power, not a constant current. However, going into low power mode when nothing is happening gets more complicated. This is usually not cost-effective in cheap high-volume devices, like TV remotes, for example.
For other devices, the current goes down significantly with voltage. Something that just connects the battery to a LED thru a resistor will actually draw lower current even more quickly than linearly with battery voltage.
In the end you don't know, but again, most devices will draw somewhat lower current with lower voltage. Replace the batteries with a variable power supply and ammeter in series and see.