If I test an embedded software (and the product based on this software) today and it fulfills all the customer requirements successfully at this point in time then I can safely say that this software is reliable.

But if the customer requirements change in the future then what software metric should I measure today that can tell me how easy or difficult it will be to do the required change in this software in future?

  • \$\begingroup\$ People I know would vote for true software-objects, written by 10+ year practitioner of that craft, to be the underpinnings of quality software. In another mindset, the ACM had in 1989 a paper on meeting software development schedules; the key was having at least one person on the team with prior experience in developing similarly-behaving software, no matter what language was used. \$\endgroup\$ – analogsystemsrf Jun 9 '19 at 11:41
  • 5
    \$\begingroup\$ I can't answer your question, but I do not agree with the assumption you make in the first sentence. It's not enough to verify that the software does what it is supposed to do, you must also prove that it will not do anything that it is not supposed to do, under any conditions. Saying that software is functional does not at all mean that it is reliable. \$\endgroup\$ – Elliot Alderson Jun 9 '19 at 13:30
  • 1
    \$\begingroup\$ @ElliotAlderson That proof is in general infeasable as it would require exploring the entire state space of the system and even with only 64 bits of state that is going to take a while. There are software systems amenable to a formal methods approach of course but that also does not scale particularly well, and gets hard when dealing with multiple interrupt sources and realtime constraints. It is one thing to say "A occurs if conditions B & C become true", that is testable, "A ONLY occurs when B and C are true" is much harder to verify in a real system. You cannot test quality into a product. \$\endgroup\$ – Dan Mills Jun 9 '19 at 15:04
  • 1
    \$\begingroup\$ @DanMills Amen to that. I was taken by the phrase "I can safely say that this software is reliable". \$\endgroup\$ – Elliot Alderson Jun 9 '19 at 19:15
  • 1
    \$\begingroup\$ @ElliotAlderson Indeed, I know people who come out with things like "I can safely say that this software is reliable", they tend to have never worked on software.... There is a reason I drive a classic car with a cable linking the pedal to the carb butterfly valve, and that has quite a lot to do with knowing what REALLY goes on in an automotive software shop. \$\endgroup\$ – Dan Mills Jun 9 '19 at 19:26

There are no metrics that completely capture this (or much else), but cyclomatic complexity seems likely to correlate somewhat with difficulty in making changes (Some shit however is just complicated).

One thing that DOES (on average) help is having a development team that has subject matter expertise in how the software is likely to be employed. If you are writing a tool for say petroleum geology, then for gods sake have at least one petroleum geologist on the team, same thing (With the appropriate expert) if you are writing a broadcast control system or an engine management unit.
What having such expertise on the team buys you is an eye on where the product may want to go in the future, it is not perfect but if your team can guess 50% of the time about what features are likely to be asked for in the future at initial design time it can make a huge difference.

IMHO this is so key that I would take a real subject matter expert who is only a so-so programmer over some code god who only knows code about 95% of the time. This also really helps to fill in the blanks in the spec (all specifications have blanks, if you think you have a complete spec you are kidding yourself).

Remember always that you cannot test quality into a product, it has to be put in at design time.

  • \$\begingroup\$ The opposite is also possible, have some domain expert/so-so programmer do the work, then have them stumble over programming basics or writing beginner-level bugs. That's a complete waste of expertise. You need a domain expert and a programming expert both, in order to produce a quality product. Have the domain expert write the spec, have them tell the programming expert how the program should behave, and have the domain expert test the prototypes. This doesn't mean that the programming expert has to write all of the code, but they need to be the one doing the program design and code reviews. \$\endgroup\$ – Lundin Jun 11 '19 at 8:01
  • \$\begingroup\$ Better idea, set the two of them up to do pair programming.... It has been my experience that no spec ever seems to fully capture the intent of the users sufficiently, and it is shocking how often the spec will miss basics. Have the programmers spend some time doing whatever job the software is intended to help with is sometimes a few weeks at the start of a project very well spent. Architecture is key, but requirements (written and unwritten) should drive architecture, and that is much easier if the architect has the big picture about the use of the thing (This also applies to buildings). \$\endgroup\$ – Dan Mills Jun 15 '19 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.