BrickOS was a small OS for the Lego RCX. It is an Open Source project and allows fairly advanced programming of the brick with a full C and C++ programming environment using the gcc/g++ toolchain.

How advanced?

Normally RCX code is actually interpreted with a firmware system into code to run on the Hitachi HC-8 microcontroller - see RCX Specifications. This however can be slow, and even NQC relies on this. There is an accelerated firmware from Dick Swan to help this - but it is still not as fast as running real native code.

BrickOS actually allows you to compile binaries to run on the Lego RCX so you get a level of speed and control not available from the other microcontrollers.

This may come at the cost of increased complexity, but if you are already a seasoned C/C++ coder, you may find this more comfortable than trying to learn one of the other syntaxes or programming systems for the RCX.

If you are a Java programmer, you could consider LeJOS instead.