Sounds like a rad idea - but alas… no. You can’t do that.
Castable Refractory fully hardens under extreme heat / temperatures. And since the firebrick & insulation layer receive the majority of the heat, a hearth slab made out of Castable refractory would never reach full cure / hardness and may become brittle.
How would you insert that all-to-important insulation layer under the top 2" of Castable Refractory?
On a standard firebrick hearth pizza oven base - if a firebrick cracks, you just pull that firebrick out and replace it with a new firebrick. What do you do if your Castable Refractory hearth slab gets a crack???