Actually you can do it and this sort of thing is often done. In particular, C compilers are often written in C itself. Even weirder, the first compiler for C on a new architecture might be written in C through an iterative process called bootstrapping. So, since LaTex is actually a compiler, you can do the same thing.
You work on such things in a series of stages, with only the very first stage needing to be done in another language (or no language other than absolute machine language).
First write a simple thing that will only compile somewhat more complex things, not the full language. Write a somewhat more complex thing that only uses features supported by the simple thing. Then, compile the new thing with the simple thing. Replace the simple thing with the new thing as your basic compiler. Then repeat. Write the V2 new thing using only features of the V1 simple thing and compile it with V1. Replace V1 with V2 and repeat as necessary until Vn is what you want.
In practice two or three iterations might be needed, but usually not more.
In fact, the latest versions of most languages are written this way by writing a compiler for them in the language of the previous version of the language.
There is usually a final stage in such bootstrapping in which a compiler for level n is written in the level n language itself, which only previously had a compiler written in level n-1 language. The compiler then compiles itself, and replaces the earlier version. Once this newest thing is compiled the result replaces the version that compiled it.
I don't know when this was first used, but I'll guess it was during the development of C.
For your application it is much simpler if you already have a LaTex system, write your OS to be compatible with that. Then compile it. Then replace your original OS with this new version. Presumably this new version has its own LaTex compiler.
Alternatively, if you have a Turing Complete language (maybe C) on the new machine that doesn't have a LaTex system, write a compiler for LaTex in C on that new machine and compile it there. You now have a running LaTex compiler on that machine. So, write your LaTex compiler in LaTex and compile it with that first (C) version and then replace the first version with the new one. Now you have a Turning Complete language that you can use to write other programs.
You can perform any computational process on any Turing Machine, but it doesn't need to be easy.
And, you should probably read the following pdf if you have any interest in compilers: https://cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf
