Embedded Development - Taking the Next Step

Question

I'm going to start by telling you what I know. Then I'm going to tell you that I want to get to this magical land of knowing everything about embedded systems development. Then I'm going to ask you what my next steps should be to get there. This answer is rather informative, but I'm trying to get a little more detailed:

What I Know

Let's see, I'm fair with C and C++. Obviously, I want to get better with those languages but I think at this point the best way for me to improve is to just keep using them and continually try to improve my code as I write it. I don't think it would be very beneficial to dedicate any learning exercises to just learning C anymore.

I'm fairly comfortable with designing simple circuits. I see a chip with an open collector output and I know I need a pull up etc. I'm fairly confident that given an IC and its datasheet, I can either figure out how to interface with it or at least ask the right questions to figure out how to interface it.

I'm very good at math and logical thinking. There are few algorithms/design concepts that throw me for a loop. This is definitely my strongest area.

What I've Done

Until now, all of my experience has been with 8-bit microcontrollers. My college course utilized a Motorola HC08, an Intel 8051, and a Zilog EZ8. My professor had built a little dev board for all three of them with a 7-seg display and some other stuff. We programmed them in assembly so I'm somewhat familiar with assembly and I took a basic computer architecture course so I have a decent idea about the hardware. However, we worked on a windows environment and all 3 MCUs had their own IDE and tool chain already setup so I never really learned how my code went from assembly to running on the MCU.

My next step was learning on my own. I got very familiar with the Arduino environment by interfacing with sensors/memories/displays that both had pre-written libraries and some that did not. Next I built my own Arduino board with an ICSP on it and connections to another MCU so that I could flash the bootloader onto bare ATmega328s. Again, however, the IDE and tool-chain was already setup and I never got an understanding of how I went from C/Arduino to code actually running on the MCU.

What I Want to Know

So from the previously linked answer, I think the most interesting/beneficial things for me would be the bullet about learning the tools (compiler and linker), and learning different styles of software architecture (going from interrupt based control loops to schedulers and RTOSes). That is my opinion on what would be most useful... but since I don't know it yet it's hard to say whether that's correct or not so feel free to suggest other bullet points from that answer if you think they would be better and please provide an explanation as to why you think that.

---

So taking my current knowledge as a starting point and the above description of what I want to know as a goal, I'm looking for very specific suggestions on where to go next. I'm looking for exact project suggestions, websites/blog entries to read, chapters in books, etc.

Also, are there any other holes in my knowledge base that you think I should fill before moving on to the above mentioned topics?

Answer 1

So from the previously linked answer, I think the most interesting/beneficial things for me would be the bullet about learning the tools (compiler and linker), and learning different styles of software architecture (going from interrupt based control loops to schedulers and RTOSes)

Porting a small operating system to a new device could help you to understand schedulers and RTOSs. FreeRTOS is popular and well documented. eCos is another.

Writing a bootloader is a good way to get to grips with a linker as you'll want to divide up memory and flash into regions.

Another tip is to pick a completely new architecture or chip and build yourself a development board. Forcing yourself to start right from the beginning and look everything up in the datasheet is a good way to learn.

Explore Protothreads. Try writing the same programs in both a threaded and state machine style. Once you're done with Protothreads, write a real thread scheduler.

Answer 2

The MicroC OS II book is probably something to invest in. You should also create projects to learn the various interfaces i2c, spi, mdio, etc. In particular how to bit bang each one. From time to time the hardware will support the bus (need to learn that on a vendor by vendor basis) but often for various reasons you wont be able to use i2c/spi hardware and have to bit bang.

The avr/arduino is fine, you should learn ARM, thumb and thumb2, and the msp430 and some older (non-mips) pic. Look at the bootloader code for the arduino, and figure out how to make a loader program, erase the flash on it and take over the board/chip. Get an lpc based arm micro, same deal look at the serial port programming protocol. Get a sam7s or something with an arm7 that has a traditional jtag, get an olimex wiggler or jtag-tiny (I recommend the latter). Get comfortable with openocd. ARM's swd is more painful that normal jtag but in this market will prevail (for cortex-m based products). In short, learn the various ways that vendors provide for in circuit programming. You will brick boards from time to time and will want to be comfortable with unbricking them. Along these lines, write some code to parse intel hex, srec, and elf files, you may someday need to write a loader and need to support one or more of these popular formats.

You mentioned tools. You cant go wrong learning gcc and binutils, learn how to cross compile, at least for the supported platforms (usually involves --target=msp430 --prefix=/something for example). Supported platforms for the mainline gcc and binutils is a moving target so avrgcc and mspgcc and the like are basically done for you. You need to learn to write linker scripts, how to write your C code so that for example fixed tables show up in the rom not the ram. Also get a feel for disassembling the binaries, you need to insure that the tables are in the right place, insure that code is where you think it is, that the vector tables and boot/startup code are where the processor needs it to be to boot. Also doesnt hurt to find out what the compiler optimizations do and what C code looks like when compiled to assembler/machine code. if possible dont limit yourself to gcc/gnu. llvm is a strong player, it has the potential for passing gcc by as a better tool. You may have already used sdcc. try the eval versions of kiel, iar, etc. You will find quickly that there is a lot of grey area in the C/C++ standards and each compiler interprets those differently, also dramatic differences in the quality of the code produced from the same high level source. If you stick with this profession there will be times when you are forced to use a not so great compiler and have to work around its warts and weaknesses. In the desktop business you can often get away with refusing to use non-standards compliant tools. In the microcontroller world, sometimes you get what you get and that is it. Sometimes you get vendors that modify/enhance the C language to meet their hardware features or supposedly make your life easier (rabbit semi and xmos come to mind). (xmos is a very attractive platform for many reasons, I consider it advanced, but from the sounds of your experience you are likely ready, the tools are a free download, really good simulator, important to learn to study .vcd/waveforms of your code executing).

chibios is another one to look at.

Creating sucessful bootloaders is an important skill. The bootloader, or at least the beginning part of wants to be rock solid, you dont want to deliver a product that is easily bricked. A simple boot with a way to re-load the application portion of the flash without compromising the entry part of the bootloader is key.

The stellaris eval boards are loaded with peripherals, although they provide libraries it is worth learning, esp since how they tell you it works and how it actually works differ and you have to examine their code and other resources to find out.

Being an avr fan, if still out there, I recommend getting an avr butterfly. Learn a little serial programming, solder on a connector and reprogram it. A few peripherals on there to learn to program.

maybe get a formerly dallas semi, now maxim I think, one wire device. Like their temperature sensors. Even more painful than i2c and mdio with their bidirectional data bus, this one wire thing, is one wire (and ground). Power, master to dependent and dependent to master all on one wire.

When I was where you are now, I found decoding infrared remote control protocols fun. The ir reciver modules are easy to come by, radio shack actually had a good one. Basically the opposite of bit banging, you want to measure the time between state changes, using that timing detect and or decode the protocol. A universal receiver is not necessary, one protocol at a time is fine. Likewise being able to then bitbang commands to an ir led, in particular if you bit bang the carrier frequency.

Talking to an sd card via spi is probably a good idea as well. Definitely learn how to erase and program i2c and/or spi flash parts, you will come across these often for serial numbers and mac addresses and the like.

I recommend learning the basic ethernet protocols as well. Be able to parse and create arp, and udp packets (from scratch) (and icmp/ping as well). Pretty easy to create an udp stack if you cheat a little, not following the the actual arp rules, if someone sends you something send the reponse back to the sending mac/ip. Or perhaps go so far as to watch the arp packets for other folks go buy and keep track of the mac/ip addresses around you. tcp takes a lot more work, doable, but better to just read about it first than to try to implement it.

Good luck, and most important, have fun.

Answer 3

That's a big question. Unfortunately there is no one way to learn embedded microcontroller development, just like with learning most everything else. I'll assume that since you're here asking, you don't just want to get a specific project done but want to really understand what is going on so you can get any project done yourself later. That's a good start.

There is no substitute for doing a few projects in assembler to really understand things. Part of the job of high level languages is to hide some of the complexity, but that also obscures things you want to understand. Arduino is yet another layer up from that. It may be fine for someone that doesn't want to know how things work to get something done, but it's really bad if the objective is to learn the details. So ditch the arduino, forget you have a compiler, roll up your sleaves and get ready to really dive in.

The first place to start with any microcontroller is with its datasheet. The various micrcontroller lines have different details, but the concepts are the same. Pick one and stick with it until you've done a few projects and feel comfortable with it. The TI MSP430, Atmel, and Microchip PICs are all mainstream microcontroller lines. I'm a PIC guy so I'll talk about that. It's not wrong to go with one of the others, but I think there is more accessible community support for PICs out there, and the PIC line is very broad so it will be easier to go to both larger and smaller micros from where you start.

If you start with the PIC line, I recommend the 18F2520. It's quite a capable microcontroller, comes in a easy-to-screw-around-with 28 pin package, has a decent amount of program memory and RAM, and a decent set of common peripherals. Get a small handful and use it to start with even for small projects that could have been done with something less. The point is to get to know it well. Eventually you'll want to do something that requires more than that PIC can do. Hopefully by that time you'll be pretty good at the basic stuff and can easily substitute a different model. Actually, all 18F PICs are the same except for the amount of pins, program memory, RAM, and the peripheral mix. After a few projects with the 18F2520, you'll have no problem jumping right into any PIC 18.

If you end up doing this professionally, eventually you'll run accross a requirements that dictate something smaller or cheaper. That's when you look at the 16F or possibly even 12F or 10F families. Things will look largely familiar, but a few things will be a little more difficult. This should be no problem to handle if you're starting with a solid understanding of a 18F. Nowadays, the 16F and lower families are really only for volume production or perhaps when physical space and sometimes power are very important. There is little advantage in them compared to a 18F for a hobbyist.

Going up, you can look at the 24, 30, and 33 families. These are basically all the same except for some minor wrinkles like what voltage they run at, top speed, and whether they have the DSP math ALU or not. They are more complicated to program in assembler, but also strangely easier. There are more instruction to keep in mind, but its therefore also easier to do a lot of things. This is also a good point to start getting back to C if that's where you want to go. The C compiler for these parts is actually quite good, unlike the C compilers for the PIC 18 and below. It's also easier to mix C and assembly in the same project on these parts. Don't forget assembler altogether for these though. It will still be useful to know when you have to write a bootloader, deal with multitasking which performs unnatural acts on the stack, write really tight code to read a bunch of A/Ds and compute the PWM duty cycle for the next pulse of your 100KHz PFC power supply, etc.

One nice thing about the Microchip line is that a single programmer/debugger works accross the whole lot. If you're serious about doing this professionally, get the Real Ice. To save a little money, you can get the ICD3, which is just a slightly stripped down Real Ice. The one thing it's not is a ICE (dumb marketing strikes again), but it is a good in-circuit debugger. MPLAB, their IDE debugger and simulator, also works accross the whole product line.

Again, for professionals or those wanting to become professionals at this, go to the annual Microchip Masters conference. Yes it's in Arizona in the summer, but you're there to learn, so forget about the 115 degF in the shade. (Actually I like exploring Arizona in the summer, so I go out a week early with my little tent and bump around the hinterlands. It's amazing how few people are out in the vast national forests only a short distance from Phoenix.) Masters is also a good place to pick up development tools. They usually sell them for something like 45% off at Masters. (I'm a platinum level Microchip design partner, so I get them 45% off all the time, which is why I'm not up on the exact Masters discount).

If you start exploring microcontrollers by doing PIC projects, check out my web site where I get into more details of how I usually structure PIC projects and give you a bunch of free tools to do some fancy things. If you understand good software design and cleanliness, you'll get aggrevated at MPASM. Do a project or two to understand what it does and doesn't, then look at my wrappers and other tools that let you do a lot more while sticking to good software design practises. Assembler doesn't have to be a mess just because most of it is. If you're really into it, you might stop by the class I'm teaching at Masters this summer on this subject.

Answer 4

Here's another idea. Implement your own background tasking system that allows you to create both timed tasks and demand tasks that run only when timed tasks are not running. It's not a true RTOS, but acts more like a cooperative scheduler. Convert a previous project to use the new tasking system.

This kind of system worked really well on a products we used to use on a 8051. It was originally written in assembly, but later on we converted it to C to help with porting it to other architectures. It was really slick where the heartbeat of the this system was on a 5 ms tick and the timed tasks ran on 5 ms increments. We had a file that we used to name all our tasks (function pointers) with their time values and those that were on demand. That file was then converted to either assembly or C depending on how we implemented it and compiled into the code.

If you get it working pretty well, then you can tackle writing your own simple RTOS for something a little heftier.

Answer 5

To directly address your question of "What I Want to Know":

I've found that calling the compiler directly via command line, becoming familiar with its options, and then writing your own Makefiles to do all of your builds has been extremely beneficial to me in learning the build process - which sounds like something that you want to learn. This basically separates the tool chain from the IDE and allows you to learn the tool chain more than the IDE. This is an on-going thing that I'm trying to improve on as well.

I noticed that you've used arduino in the past, which is great because now I can recommend using avr-gcc as your compiler from now on. Give it a try, it's available on all platforms (Linux, WinAVR for windows, Mac) and the documentation on the avr-gcc tool chain and avrdude (programmer) is great, and there should be plenty of example Makefiles out there for you to learn from. A fair amount of this information is transferable to other hardware as well, for example arm-gcc.

Answer 6

One thing you haven't mentioned is communications.

It seems that one hole you could plug would be to learn the various standard communications protocols used in industry - things like:

• Profibus
• EIA-485
• Modbus

etc.

Answer 7

So the question is "How to learn, when every toolchain is a blackbox ?"

I suggest to find a off-the shelf very old experimentors, debugging board with any common CPU. Something like 2 feet wide contraption with CPU, LEDS, switches and "Execute one single step" button. Manually create 5-10 instructions long loop program using machine code binary instructions from datasheet. Put it onto pluggable huge ROM chip. Insert ROM, hit power on/reset and debug it step by step.

Answer 8

How good of an understanding do you have of registers, operation and such on an 8-bit micro? It might be a good idea to do a little assembly. This has the benefit of teaching you exactly what is going on. This can help solve weird bugs with higher languages.

AVRs have nice simple assembler and registers. It is a good platform to get your feed wet on. There are also some good tutorials for that platform out there.

This will give you the bottom line of what the micro is doing. Then the next step of how the compiler and linker take C/Arduino to machine code will be easier to grasp.

Answer 9

Realtime Mantra contains several articles about embedded software development.