Imagine that you built your oven on a sand surface (with no drainage gravel) and a rain came the day before you planned on using your wood-fired oven. Not only is the oven going to get wet (hopefully you have it covered), but also the ground around the oven is going to get wet too. And since you built your oven base on sand - all that sand is holding the water.
And water wicks - which means the water on the ground is going to wick into your base slab… then up the masonry walls (which is where it usually stops going up). While it didn’t reach the oven / hearth slab, it does keep the oven base slab damp… and this is where your firewood.
Now, it isn’t life or death that you build your oven base slab on drainage gravel - but IT IS highly recommended!