The crust, which is Earths outer solid layer of rock, is around 20km thick if I remembered correctly, and the ocean is nowhere near that, so there would be rock under the ocean.
If you find a better answer, then use it, because I'm not completely sure, but that's my best guess based on the possible answers
