Wow. I didn't actually read all of that EESE exchange! Too much to have to process other than to focus on what little you asked. So I'll focus on that.
In von Neumann architectures, broadly speaking, a program has the following appearance:
Section Description Access Non-Volatile Size
-----------------------------------------------------------------------
Code Execute Yes Fixed/static
Constants Read Yes Fixed/static
Initialized Data Read/Write Yes Fixed/static
Uninitialized Data Read/Write No Fixed/static
Heap Read/Write No Variable, up
Stack Read/Write No Variable, down
The code section can be execute-only, as it often is with the x86 family. But (obviously) the operating system must write it, somehow, before starting the program. If this is an embedded, stand-alone target without an operating system involved, it is usually stored in non-volatile memory or else it is fetched from secondary non-volatile memory (which is very often done without any extra software using the built-in ROM boot loader found in the ADSP-21xx DSP from Analog Devices.) During execution, it can be non-volatile (if there's no need to modify it at run-time) or it can be volatile (if modifications are required -- such as what may happen with late binding.)
As you can see, it might be possible for a compiler to arrange for all of code, constants, and initialized data to reside in non-volatile memory somewhere. There's no need to store any of the last three sections mentioned in the table above in non-volatile memory, though.
The above table is almost correct. However, note that the initialized data section does require read-write access. Since almost all non-volatile memory isn't easily writable, except for FeRAM (or FRAM), this is almost always stored in RAM that is transferred out of non-volatile storage (primary or secondary) into the RAM just prior to starting the code (other than the start-up code that you usually don't see or write.) This means the above table probably should be:
Section Description Access Non-Volatile Size
-----------------------------------------------------------------------
Code Execute Yes Fixed/static
Constants Read Yes Fixed/static
Initialized Data Copy Read Yes Fixed/static
Initialized Data Read/Write No Fixed/static
Uninitialized Data Read/Write No Fixed/static
Heap Read/Write No Variable, up
Stack Read/Write No Variable, down
The above table now highlights the fact that the initialized data section must reside in two places -- non-volatile memory (such as flash) and volatile memory (SRAM) and that there needs to be start-up code that transfers data from the initialized data copy section to the initialized data section. But still remember that some processors, such as certain MSP430's that are built with FRAM, can keep the initialized data in their non-volatile memory. So either one of the tables might be correct.
The heap section isn't static at run time. Normally, this section is
set up having zero size to start and then grows and shrinks during
execution. This is the area used by routines like malloc(), for
example. A simple design for the heap has it growing upwards and away
from the last memory location required by all the static data areas
and towards the stack.
The stack section also isn't static at run time. Normally, this
section also starts out with zero size and grows and shrinks during
execution. This is usually where function parameters and local,
"auto" variables reside. It's also used by the C compiler for
temporary storage, spilling registers, etc. And it usually grows
downward and away from the last possible memory location for the
program and towards the growing end of the heap section. In this
fashion, there is a single, invisible area of read-write memory, a "no
man's land" so to speak, between the heap and the stack, which each
section grows "into" like a candle burning at both ends. If, during
execution, the heap grows into the stack (or visa versa) then the
program will probably fail to operate correctly.
In Harvard architectures, there are at least two different memory systems -- one for code access and one for data access. Usually, there are specialized instructions which permit reading from the code memory system, treating it as data. Here, the details get a little more complex than above.
When the system starts up, the code, constants,
and initialized data must be available. Since different memory systems are involved, and since all of the data must be accessed via the data memory system, the startup code usually copies out values required for the initialized data section and for the constants section into data SRAM where it can be treated as regular data.
So the Harvard table might look like this:
Section Description Access Non-Volatile Size
-----------------------------------------------------------------------
Code Execute Yes Fixed/static
Constants Copy Read Yes Fixed/static
Initialized Data Copy Read Yes Fixed/static
Constants Read/Write No Fixed/static
Initialized Data Read/Write No Fixed/static
Uninitialized Data Read/Write No Fixed/static
Heap Read/Write No Variable, up
Stack Read/Write No Variable, down
Here, the start-up code must transfer the constants copy section to the constants section and then also transfer the initialized data copy section to the initialized data section, before letting the program continue further.
To directly answer your question about being able to execute code directly from ROM with von Neumann, yes you can. In fact, it's often done that way. You just burn the code into flash. That works for both types, just fine.
(And multiprocessing and threads complicate the above tables further.)