BrickOS was a small OS for the Lego RCX. It is an Open Source project and allows fairly advanced programming of the brick with a full C and C++ programming environment using the gcc/g++ toolchain.

How advanced?

Normally RCX code is actually interpreted with a firmware system into code to run on the Hitachi HC-8 microcontroller - see RCX Specifications. This however can be slow, and even NQC relies on this. There is an accelerated firmware from Dick Swan to help this, but it is still not as fast as running native code.

BrickOS actually allows you to compile binaries to run on the Lego RCX so you get a level of speed and control not available from the other microcontrollers.

This may come at the cost of increased complexity, but if you are already a seasoned C/C++ coder, you may find this more comfortable than trying to learn one of the other syntaxes or programming systems for the RCX.

If you are a Java programmer, you could consider LeJOS instead.