Is it possible to perform JPEG compression on a Cortex-M4 MCU? Are there existing libraries that can do this and can accept a 10-bit RAW RGB?
Since the M4 core (and pretty much any other CPU used in a microncontroller) is turing complete, the answer to your first question is yes, conditional on the other details of the MCU.
In particular, while the CPU itself is certainly capable of the instructions required to do JPEG compression, it must have enough RAM to implement whatever algorithm is chosen, so your question should include additional details of the specific MCU. A very quick Google search indicates that entry-level M4 MCUs include something like 64 KiB of RAM, which is almost certainly sufficient to perform at least a basic JPEG compression, since, wiht the right options, JPEG is generally a multi-pass streaming compressor (i.e., works its way though the data without needed random access). Exceptions include the DCT step, but that's only an 8x8 (perhaps up to 16x16 depending on chroma reduction settings) block, so only requires a small amount of RAM.
For more details, you need to dig into the JPEG standard, and decide what subset you want to support. Since you own the encoder, you can just generate JPEGs using the settings that work with your limitations. This is much different to writing a decoder for general JPEGs, since then you must support whatever options and modes the encoder chooses.
A good primer on memory use can be found in the mozjpeg readme - it is brief, but at least includes key info, including the fact that progressive mode is generally out for your use case since it requires a whole-image buffer.
About your second question, it is unlikely that you'll find an existing library targeting the Cortex-M specifically that accepts RAW. You may have to combine a library that supports RAW decoding with one that supports JPEG encoding, and ensure that the interface between them allows streaming. Something like mozjpeg may be a good start on the JPEG side. You may need to adjust how temporary results are stored depending on your exact configuration.
If you can't find a JPEG library already targeting this MCU, you may have to compile one of the existing free ones yourself. This one looks promising since it explicitly targets MCUs and low memory use, but the linked author's page is 404, which is less promising.