Understanding programmers for microcontrollers

Question

I'm confused about the concept of a programmer for microcontrollers. Hoping someone can clarify.

When I use an Arduino, I can use another Arduino to program it via the SPI pins. I understand this to mean that the SPI pins behave as programming pins using a certain programming protocol when certain conditions that put the microcontroller into programming mode are met. So, the programmer is nothing more than hardware that knows how to speak the programming protocol.

When I read about ARM processors, often I read that I need a Segger JLink. Otherwise I cannot program it. Development kits often have a Segger chip on the board. So, this is a programmer for ARM chips. If so, why can't an Arduino, or some other general purpose hardware, also speak the programming protocol for ARM?

Also, what is so special about programmers that it has to be hardware and cannot just be software as long as the microcontroller pins can be hooked up properly and the software can control those pins? If this really is the case, why are Segger products so expensive? What is their secret sauce that seems to give them some sort of monopoly?

If it's important to this discussion, the specific microcontroller I'm thinking about is the nRF51822 with an ARM Cortex M0.

Answer 1

There is no magic sauce, it's all just clocks and data lines going up and down in voltage in the right succession. Sometimes the programming voltages are a bit higher than the normal interface voltages, but that's not hard to reproduce. As you noticed, your arduino can do that too, if you know how the protocol works. And that's where the problem lies.

On one hand the more complex the microcontroller is, the more complex the protocol to program it will be. You will have modes to initialize different forms of memories, set configuration parameters that control fundamental behaviour of the controller (e.g. Atmel "Fuses" which can shut off SPI or disable the internal clock source oscillator). The protocol how to do this is very low level and usually not of interest to the developer who wants to use the controller for his own product. So naturally it is the creator of the microcontroller providing an abstraction layer between controller and PC, which makes it easier to program the controller.

The other aspect is monopoly in order to profit from selling the hardware and software needed to program a certain controller. Most companies aren't interested in making it easy to program their devices with third-party or DIY programmers. Not because they're evil, but simply because it's their price policy to make the controller cheap but the periphery necessary to develop with it expensive, which results in less per-device cost. So some companies might even go the extra mile and add security mechanisms and tamper-proof protocols. A simple example would be a secret key generation algorithm and a key verification algorithm, on the programmer and the controller, respectively. When the programming begins, they perform a handshake with challenge-response-authentication. So even if you sniff the protocol and replay the exact bits on the lines, it won't work because the controller will randomly provide a different challenge at the handshake, and your parrot-programmer will produce a wrong response, telling the controller that it's not talking to the original programmer.

Answer 2

There are four methods to get code into a microcontroller. In order of accessibility:

1. Bootloader.
   This is what the arduino uses. There is code running on the microcontroller that is capable of receiving and programming the internal flash over some interface. In this case UART.

2. In System Programming (ISP)
   You use additional hardware to communicate with a peripheral inside the microcontroller to program the internal flash. Or accessing more such are core registers and sram, then it becomes debugging. Of even more by streaming live data to the IDE, as ARM CoreSight can do.
   The Arduino based on ATMega chips has Atmel ISP while ARM uses the debug core in their processor over SWD. It does not have any of the other fancy debug features.

3. Off board programming
   More common with older parts. You have to take the part off the board, put it in a socket and use specialty hardware to program is.

4. Mask ROM
   During manufacturing code is put in the chip using the various lithography steps fabricating the chip. This is permanent and cannot be altered.

When I read about ARM processors, often I read that I need a Segger JLink. Otherwise I cannot program it.

Some chip vendors include ROM bootloaders that can be accessed by an interface when keeping some combination of pins in the correct state on power-up. An example of this would be the LPC11U24 with a USB Mass Storage bootloader. Drag&Drop the .bin file.

Also, what is so special about programmers that it has to be hardware and cannot just be software as long as the microcontroller pins can be hooked up properly and the software can control those pins?

In production, this code would only be used once. During the lifetime it would just occupy expensive flash or ROM real-estate.

If this really is the case, why are Segger products so expensive?

Aside from hardware, they also need to develop software and drivers to be able to support all of the chip vendors and IDE's. You can buy fake JLinks for $10 in china... the hardware is not expensive.

---

Do not buy fake Jlinks

Answer 3

A STMicro ST-Link/V2 can be picked up for ~$22 from Mouser. A ST-Link/V3MINI is only $9.75 from Mouser.

And then you have the MBed compatible STMicro Nucleo series which all have an on board ST-Link/V2-1, which can also be converted to a OEM JLink. The ones on the 64 and 144 pin models can be detached and used stand alone. They range in price from $10(which include some 64 pin parts) to $30. Note, all of the ST-Link/V2-1 allow drag and drop programming, when you plug them into the USB they appear as mass storage.

But the way to do it even cheaper is to buy a 5 pack of blue pills(AKA STM32F103 minimum development board) from Amazon for $16(Prime), and convert 1 of those 5 into a Black Magic Probe. You then have good Cortex-M probe/debugger(JTAG and SWD) and 4 boards to play with. That is what I did for two of my nephews who showed an interest in getting into embedded development.

So no, you do not need a JLink to do ARM development, nor do you have to spend a lot money. If you just want to dabble a little bit, get a NUCLEO-F303RE which mounts a STM32F303RET6, it costs $10.33 at Mouser. This gets you at STM32 with 512KB flash, 64/16KB SRAM, 4 5MSPS ADCs, etc.

Yes, I know it's an older question.

Answer 4

It is not that black and white. Among the AVR family they have more than one serial protocol one is more of a pain than the other. The AVRs are nice in that you have the chip in reset so if your program does something stupid like repurpose the programming pins (yep, been there done that more than once) and the chip is designed that it is out of reset you might not be able to reprogram it, but there are sometimes other solutions.

It is all about the chip vendor and what features they want to put in there at what cost. An 8 pin part with multiple pins dedicated to programming would be a really bad idea, even dual purposing might be bad as the user might not be able to design their board to actually use those pins and make it field programmable. If you have enough pins though you could go so far as to dedicate some if you dont want to mux them. It is up to the chip vendor/designer to decide what protocol or what set they want to use. As already mentioned sometimes they want to make it so you have to buy their tools because they keep the protocol secret and proprietary. They may provide multiple solutions to make it easier on the user.

Some, a lot these days, allow for self programming, the mcu can program its own flash from code running on the mcu. So that opens the door to bootloaders. You can always make your own with your own protocol in a case like this so you dont have to rely on the vendor. But a number of them have one or two layers of bootloaders, one they place and protect you from using, I guess the AVR one is hardware the other is software, the bootloader the arduino folks place and use with a uart. It is not uncommon, esp with the ARM based ones, to have a serial, usb and maybe spi or other on the built in, placed by the chip vendor, bootloader. Strap an input pin on reset or set of pins and the bootloader takes over and doesnt boot the app but waits for you to field program. NXP, ST, Atmel, are vendors that do this.

Some, true, are jtag only, now with the reduced pin SWD jtag, they often offer that as a solution along with the others, and some are jtag only. But because we dont use parallel flash parts anymore that all programmed with the same protocol and you just needed an abstraction to read and write addresses, the jtag tools often dont/wont mess with learning the protocols of every possible chip out there, so either you use a special tool from them or you do it some other way (for example, my preference, write a program and download it and have it use the self programming, some of the solutions, (stlink, etc) actually download a program or design, and then the tool talks to the thing that is downloaded to support the task)

this is on you as the user though, to add this to the things to look for when choosing the mcu for your project. How do we program it, are you going to mass program these things, program per board, let ICT take care of it. But before you get to mass production, what is your solution for the software developers, do they end up bricking boards because there is no solution on the prototype boards to reprogram? Or do you expose something, etc. If the chip doesnt give any good solutions, then that just adds to the cost of the development. Certainly work with your software developers before designing the board, if nothing else a couple of uart pins and a strap are often enough to make a bootloader if one is not already present. Or a strap and jtag, they can have their application stop very early, if the strap is set, and then take over with jtag to either debug, download programs, or reprogram the chip.

In the old days you just removed the prom and erased it then stuck it in a programmer, or left it to erase and took the next erased one and programmed it put it back in the socket in the board. That or pulled the mcu itself out and put it in the eraser and/or programmer.

Most now have a field programming solution or you can make your own field programming solution (in software) if there are enough pins. If not make a socketed board for the developer prototypes, and a fixture for reprogramming, or just throw the parts away until the developers get it right.

Answer 5

As DerManu says, there's nothing magical about it -- just a lot of implementation details. People buy emulators because building and programming their own is a pain. Take a look some key items from the J-Link feature list:

• Up to 3 MByte/s download speed
• Intelligence in the emulator firmware [handling for weird edge cases]
• Compatible with all popular tool chains
• Support for multi core debugging
• Supports an unlimited number of software breakpoints in flash memory
• Setting breakpoints in external flash memory of Cortex-M systems is possible with J-Link's Unlimited Flash Breakpoints technology only!
• Cross platform support (Microsoft Windows, Linux, Mac OS X)
• Supports concurrent access to CPU by multiple applications
• Remote Server included. Allows using J-Link remotely via TCP/IP
• RDI / RDDI interface DLL available
• Software comes with free GDBServer, allowing usage of J-Link with all GDB-based debug solutions
• Production flash programming software (J-Flash) available
• Debugger independent flash download (internal flash, CFI flash, SPIFI flash)
• Supports CPU/MCU internal trace buffer (ETB, MTB, ...)
• Supports ETM tracing (J-Trace Cortex-M, J-Trace ARM)
• Can be connected to the host PC via USB or Ethernet (J-Link PRO)
• Wide target voltage range: 1.2V - 3.3V, 5V tolerant
• All JTAG signals can be monitored, target voltage can be measured
• Target power supply: J-Link can supply up to 300 mA to target with overload protection
• An SDK is available (Allows customized use of J-Link)
• Various target adapters and optical isolation adapters available
• Supports multiple target interfaces (JTAG, SWD, FINE, SPD, ...)

That's a lot of work! The low-end model is under $400 and the high-end model is about $1000, which is really not bad for a JTAG emulator with drivers, IDE support, remote debug hosting, etc. Remember that these have a narrow target market -- Segger is bragging about selling over 400,000 units! And the market is mostly other companies, who won't bat an eye at spending $1000 -- one good oscilloscope costs 5-10 times that much. The investment in software development for a given microcontroller is much, much larger than the investment for programming tools.

You could certainly try making a programmer using an Arduino -- it might be a fun project. But it will definitely be a project, and not a quick hack.