Don't fix it if it isn't broke. NASA has lots of operational experience with VxWorks. They know all its warts and how to avoid potential issues. They also know the power consumption of the kernel. Compared to the cost of the overall mission the cost of VxWorks is negligible, yet the risk of replacing it with another kernel is quite high. It just doesn't make any sense.
Plus timeline: the Curiosity software was being written starting in 2005(?) and completed in 2008.[1] WindRiver released its first version of linux in 2005... and that was its "Platform for Network Equipment", not a suitable candidate for Curiosity's RTOS.[2] It wasn't until 2007 that they acquired FSMLabs and thus acquired an embedded real time linux (RTLinux). [ibid]
