I've been tasked with defining a format for (asynchronous) binary
RS232 data sent from one device to another.
Just a clarification that may or may not help. RS-232 is a very detailed and specific set of related elements, which include electrical specifications as well as functional ones (and more.) Part of the standard does include the use of asynchronous methods for communication. But just using an asynchronous method, by itself and even if using the exact asynchronous method described in the RS-232 standard, isn't necessarily RS-232. In order to be RS-232, it must meet the entire specification, including the electrical and interface parts of the standard.
When you write RS-232 in your question, I will take this to mean all of what that entails. Including the electrical specifications. I'll also assume that your situation doesn't exceed those specifications.
Now, to each question in turn:
What determines 5 vs 8 bits for the dataset? Is that just my decision,
based on the number of error words I'd like to include in each
If you control both ends of the wire, then you can choose any format that your peripherals (or bit-banging software) on both ends support. So it technically would be fine to use 5-bits, 6-bits, etc. But unless you have a good reason otherwise, you are better off in most cases selecting an 8 bit format.
The earliest code I've encountered myself was called ITA2, or as I knew it then, simply "Baudot." This was a 5-bit code standardized across the world in the 1920's and used up through the early 1960's, at least. My dad had a 5-bit Teletype Model 32 in the basement of our home, in fact, to receive messages. The modems of that day supported 50 bits per second.
When the US created the ASCII standard (7-bit at the time), which supported lower case characters as well as some specialized control characters, it gradually took over (in the US, anyway.) There were other standards, such as EBCDIC, used on IBM computers of the day. But ASCII had largely taken over by the mid 1970's (perhaps just a little earlier.) A parity bit was soon often added as an 8th bit to aid in detecting errors. The Teletype ASR-33 supported this addition and widely took over for the KSR-35 (which was far more expensive than the ASR-33.)
By the 1980's, almost everything was operating on an 8-bit data format for RS-232 asynchronous communications. Either the 8th bit was retained as "parity" or else the 8th bit was used for actual data values. Either way, it was almost entirely performed as 8-bit communications by the beginning of the 1980's.
You have to decide how you want to handle it, though. If you will be emitting only human-readable output, in ASCII for example, then you might be wise to use the 8-bit format but to only use the lower 7-bits for the ASCII text. You can choose to set the upper, 8th, bit as a parity bit (many peripherals support that.) But that is your call. You just need to be consistent in your approach.
If you do not control both ends of the wire, then you will have to find out what is accepted and transmitted by the end you do not control and follow the guidelines you work out from reading about it. I can't speculate about that, if so.
In defining a "format," what parameters do I need to define? From
reading online, seems the following: 8 bits of data, x Baud Rate,
checksum algorithm, parity type, other headers (?)
First off, baud doesn't mean what you think it means. And writing on the web and in books, sadly, doesn't often make the distinctions. But the concept of "one baud" and "one bit per second" aren't the same. (Baud \$\ne\$ BPS.) See the difference between baud and bps on stack overflow for some discussion about it. Just one more thing to learn about. (You may also like seeing Russel's answer here about 4-QAM, because it includes some nice animation for probably the simplest QAM method plus some added links to review. It may also help a lot in getting across the differences between baud and bps.)
When defining a format, you should first research standard formats that may already exist and might be useful to you. For example, have you looked up IEEE 488.2-1992? It's a standard developed for an entirely different communications format (IEEE 488/GPIB.) But it might offer some useful ideas. If time allows, it is better to first do a survey of formats others have developed and tested, before inventing one of your own. You may find everything you need in one of them and it can save you difficulties later on due to your own oversights.
Of course, you are free to develop your own. You might want to worry about subjects such as: (1) communication errors and recovery from them; (2) ability for humans to read the format or else generate it, by hand; (3) the ability for older software to handle newer formats by reading what they can and ignoring what they cannot understand; (4) the ability for newer software to understand older formats and to make reasoned guesses where added elements aren't present in the data flow; etc. None of us can tell you what's more important, or less. And for each of these topics and more, there are entire books' worth to read.
Do your research. Then choose and make your choices work. That's the best summary here.
What Baud rate do you recommend if ultimately we want to see each
label at 1Hz update rates? The devices are connected via cables and
located near each other in an aircraft. I have very limited info about
the two devices.
Once you have made the decision about the format you will use, you can work out the worst case length of one such message. With that, and the worst case frequency of such messages (or similar), you can work out the bit rate you require as a minimum. It would probably be wise to settle on something that is at least twice as fast, if not still faster, as you feel you need to start. Give yourself some margin to work with.
Also, you must work within the peripheral's abilities (or, if you don't control both ends of the wire, then the limits of the device you don't control.) Typically, there are clock dividers involved and these usually only work with integers. And RS-232 specifies a worst-case bit timing error of about 2%, if I recall. It will turn out that certain specific bit rates work better than others, for a given MCU clock feeding the peripheral section. You can't just pick any random rate. You need to play well with the peripheral and its clocking.
Is it typical to send the "name" of the label before the data is sent?
Or is it more typical to reserve 1-2 data bits for a label number?
This presumes something not in evidence: "typical." So far as I'm aware, there isn't a typical situation if you look at RS-232 as a whole. However, I did provide one possible standard you could consider (IEEE 488.2-1992.) In that case, they do use names.
Again, all I can do is recommend that you perform your due diligence and do the research. Focus as much as possible on your business area, because there actually may be an expected standard. If so, you should seriously consider it. If not, you are free to make your own choice. (And again, this depends on if you control both ends of the wire.)
How are "other headers" sent? as the next data transmission?
You get to call the shots here. Do your research. See what others do. Try to find circumstances as close as possible to your own situation. Check out your business area. Etc.
It's not possible to provide you with a comprehensive list here that covers everything everyone has ever done on all devices in the world over all time. Obviously. One cannot answer "how are other headers sent" without having a comprehensive view. And that's certainly not me.
All I can suggest is that you go do your research and then think about these questions in that context. If you have a more specific question here, ask it. But provide full context of it that shows evidence that you've taken the time to do some study and ask a question that is narrow enough that we have a chance of providing a good answer without having to write a complete textbook on it.
For the engineering data (say, altitude), where/how do you send
information like units?
Once again, I'd suggest that you thoroughly read IEEE 488.2-1992. It covers the transmission of physical units along with numerical data values, and their interpretation. For example, see the section headed as
7.7.3 <SUFFIX PROGRAM DATA>.
But it is only one choice of many. You will need to see what others have done.
In one system I developed, using IEEE 488.2 as a standard reference, my instrumentation could handle units in a variety of ways, converting them as needed to units I used internally (if possible.) There are a lot of ways of specifying units. You can choose to be very limited in what you accept, or adaptable. Again, it's your call. But there is at least one standard you can examine for ideas here, too.