From what i know, HDD is built on magnetic recording technology. My question is, is it possible to create a storage device from scratch (USB, HDD, etc.)? I'm just eager to try out building some electronic devices...
Yes, you can do it-- but it's hard and it won't store very much. I think the thing that makes it hard is that you need to know a lot of very specialized areas to make it works. Things like: software, signal processing, electronics, electro-magnetism, metalworking, motors/gears/etc, and materials science (somewhat like chemistry). While not impossible, it is rare to find someone who is proficient in all of those areas.
If you want to make this easier, I would recommend starting with a standard cassette tape player/recorder. Rip out all of the electronics and keep the motors, gears, mechanical stuff, and the read/write/erase heads. Then add back in your own electronics. This still offers a lot of challenges, but the odds of success go way up. Then, if you get that working you can take the knowledge you gained and go on to a hard drive or something.
If you go the cassette tape route, let me mention that if you Google that you'll find a lot of pages that do something similar, but without modifying the player much. They do that by modulating the data into something that resembles audio and can be stored as audio. That's not what I'm recommending. If you rip out the guts of the player/recorder then you can have direct control of the motors and heads, which opens up lots of possibilities.
Hard drives will be harder, mostly because you'd have to figure out how to make the hard drive platters. Meaning, you have to make the magnetic recording medium and somehow spread it out evenly and smoothly on the glass or aluminum platter "base". Making the heads isn't easy either.
I should point out, however, that a clean room is not required. I remember playing with a "removable hard drive" on a DEC PDP-8 computer. Instead of removing the whole drive, you only removed the platters. The platters were about 12 inches across and contained in something like a piece of Tupperware that you'd carry a cake in. About 6 or so platters per carrier. Before putting the platters into the drive you would have to remove them from the Tupperware. It was big, and didn't store a lot, but no clean room either. Don't get me wrong, modern drives do need a clean room. But a DIYer has little or no hope of building a modern drive in his home so it's not really an issue.
Another form of storage that could be interesting is an optical fiber "drive". Light travels approximately 6 inches per nanosecond in a fiber optic. So if you have a fiber that is 100 feet long and you're transmitting stuff at 1 gbps then you're really storing 200 bits of data in that fiber. Make the fiber several kilometers long and you could store a barely useful amount of data. Get a fiber transmitter and receiver set up so whatever is received will be retransmitted and your data will just recirculate endlessly. Some extra stuff will then allow you to read/write the data.
Probably the most useful, and least satisfying, thing to build would be something like a USB thumb drive. Basically you buy the flash chip, and the controller chip, connect them together and you're done. To make it a little harder, replace the controller chip with a microcontroller and write lots of software. It's not super interesting, in my opinion. I don't think it offers the same sense of accomplishment that the other approaches offer-- even though the performance and capacity would be the highest this way.
A HDD is not a very good idea for a DIY project. You need lots of special parts which aren't available for DIY, like voice coil, the platters and the magnetic head. You'd also need clean room conditions. And of course it's all about high precision mechanics.
Also, if you would succeed in constructing one it would probably cost 10 to 100 times more than what you pay for a commercial product.
If you are really really bored. You could delve into organic storage. Slow but huge capacity.
If your goal is to build something to be interesting, rather than practical, there are variety of DIY-compatible ways one could store information electronically. While it's extremely doubtful that one could achieve anything resembling cost-effective performance, it's entirely possible that one might be able to, with modern technology, achieve a level of performance for some techniques which would be significantly above what could have been achieved a few years ago with similar techniques.
For example, it might be interesting to play with acoustic delay lines. Generally, their performance has been limited by the fact that signals will spread out a certain amount as they travel down the lines; if one tries to push the bandwidth too high, bits may blur into each other by the time they reach the far end. Back in the days when delay lines were used for storage, that would have been an absolute limiting factor. With today's DSP's, however, it may be possible to reconstruct waves which would have been unreadably blurry a few decades ago.
I'm not sure how many bits one could store in something like a spring reverb, but it might be interesting to play with one and find out.
There's always drum memory. A soup can wrapped in magnetic wire or sticky tape covered in rust may work. Then add a small A/C motor and gear train to move the drum one word at a time, allowing very precise control. And finally one or more read heads, consisting of a c-shaped ferromagnet wrapped in wire. Ferrite was generally used for these types of heads, but maybe steel or iron will work, too.
And if all else fails, there's always the paper drum: paper with holes in it wrapped around the drum. Apply a charge the the drum, and the other charge to the "read heads", and it makes for a simple ROM.
You can make magnetic tape from sticky tape and rust.
Axeman beat me to it by suggesting magnetic core memory. I'd add that if you're looking for permanent storage (ROM) then you could investigate 'core rope memory'. This could be useful for 'code shadowing' on a very small project, where the permanently coded ROM containing the code instructions is loaded into RAM during boot-time.
Both magnetic core and core rope are conceptually similar, although they function differently. the ferrite rings in the magnetic core memory function by easily switching their polarity (north-south). This switching is done with a current carrying wire pair through the middle of the ferrite ring. The polarity indicates the binary memory state, and a sensor wire then reads the state. Core ropes function more as tiny transformers: a data address wire is powered, and every core tied to that address will energize. Functionally 8 cores could be wired to each address, and by energizing the individual addresses, the 8-bit binary value 'stored' at that address can be read.
These technologies were used in the Apollo project. Although they have low storage space per volume, the point is they answer your original question; they are possible to build entirely from scratch. I've seen groups dedicated to making their own (I've thought about making one myself as a demonstration/teaching aide.) and even someone making a module to display numbers on a 7-segment display using 7 cores and just wrapping them in the correct order; each number is then shown by powering each 'address' from 0-9. http://hackaday.com/2013/10/09/making-a-core-rope-read-only-memory/
If you are more interested in simply building a functional memory for a bit of electronics practice, then there are options; the STM32F4 family of microcontrollers can be programmed as a 'USB-on-the-go' device. You could then get some SPI flash memory chips (a few Kb up to several Mb) and use the STM32 both as the USB device, and the driver to store/read from the memory chip. STMicro produce an 'F4 discovery' board, which comes with a suitable USB port wired on for USB-OTG. Once you start looking into the SPI protocol, you can see that each chip uses the same 3 data transfer wires, and a separate dedicated chip select wire: building your own 16MB USB memory stick using four 2MB SPI chips and switching which 'bank' is used in software would be an excellent learning tool, if a bit on the advanced side.
A similar project could be done with an Arduino or Picaxe microcontroller (much much easier to program than the STM32, but not as powerful). A simple Arduino project which takes data from a serial port and stores it to an SPI memory shouldn't take more than a few days to get working.
I came up with a variant of remove 5s to contact the delay line independently, using the effect of infrared quenched phosphorescence in ZnS glow in the dark (GITD) material. My research suggested that a single platter with 16 UV SMD LEDs and 16 photodiodes tuned to green emission (doable) and a single infrared quencher 300 degrees away in direction of rotation with analogue ICs to do the data refresh could potentially store just shy of 500MB if the goal was to store the data for a minute at a time and continuously refresh from external storage (ie scope, etc) For something like encryption keys it would be ideal as the original could be on paper, paper then destroyed etc.