After reading your comments, I'm going to make a bit different answer to this question.
What exactly is a current source? It's nothing, or to put it a bit better it's just a mathematical model. The one you're describing does not exist, just as a voltage source doesn't exist.
I think that the main problem here is cased by this statement: for example a battery which has a constant potential difference across its ends irrespective of the changes in the circuit it is connected to
which is incorrect. That it the behavior of ideal battery which is real as ideal current source and just like ideal current source it doesn't exist. The output (and internal state) of every real battery is affected by the circuit it is connected to.
So why do we have voltage and current sources? Well the idea is that engineer's job is to basically construct a device which does something pretty well and as it turns out for that complete understanding of how each component used in the device isn't needed. That is why we have things such as ideal current and voltage sources.
Let's get back to the battery example once again. Here's a simple experiment I did with a lithium polymer battery I have: First I completely charged the battery. Since it's a two cell battery, its voltage was 8.4 V when fully charged, even though its nominal voltage is 7.4 V. Then I connected a \$ 100 \mbox{ } k\Omega\$ resistor to the battery. It's voltage remained 8.4 V and from that I could perhaps conclude that the battery is indeed ideal voltage source since I connected the load to it, but its voltage did not change. Then I took an electrical motor I have and connected it to the battery and measured battery's voltage again. This time, it was 8.2 V. Clearly the motor affected the battery and it is no longer an ideal voltage source, even though it's the same battery as before. So I disconnected the motor and connected the resistor again and again the voltage at the battery was 8.4 V.
So what's happening here? Is the battery an ideal voltage source or not? Well we know it isn't because cause I said so at the beginning of the answer, but here I'll explain why it sometimes seems that it is and sometimes it seems that it isn't. As I said, the voltage source is a mathematical model. When the external circuit doesn't make a big impact on the operation of the battery, I can use it an when the external circuit does make a big impact on the battery, I can't use it. So we're using a simple model to represent behavior of a real circuit. Another model would be to use an ideal voltage source with a resistor in series at its output. When I connect and external load to that circuit, some voltage will be dropped at the internal resistor and the external resistor will see lower voltage at the output. This allows me to once again use the ideal voltage source to represent the battery and since I'm using the internal resistor together with the ideal voltage source, the output will more closely represent the behavior of a real battery. Should I want more precision, I could decide to use a more complex model and get more accurate results.
An important point of electrical engineering is to learn when to use right model to represent an extremely complex real-life circuit component (and even the humble resistor, when analyzed in detail, is a masterpiece of modern science). But to be able to do that, we start with simple circuits so that we can get to know how the simplest mathematical models actually work.
When we start analyses of more complicated circuit components, like transistor or diode for example, we'll break them down into a simple circuit consisting of things such as resistors and ideal current and voltage sources. This will allow us to simplify the behavior of more complex component and avoid analyzing in detail how it works, if the simple model is sufficient for our needs.
Completely same story works for current sources, but I decided not to tell it here since, as you can see from the other answers, circuits which can be modeled as ideal current sources are too complicated for you to understand at this point.
So to sum this up: There are no real-life objects that can be used to represent ideal voltage and current sources, but there are some objects that can be (in some cases quite closely) represented with ideal voltage and current sources. The best thing you can do now is to memorize correctly the definitions of ideal voltage and current sources and not to confuse them with real objects. This way you won't be surprised if a battery doesn't provide its nominal voltage or if a circuit labeled ideal current source starts smoking at one point although it should be completely immune to external changes in the circuit.
As a side-note consider what happens to ideal voltage source when its outputs are shorted and what happens to ideal current source when its outputs are open? And what happens when you short a battery an why all batteries have the warning not to short the output pins?