If you work out any linux books, it will always talk in one way that it sets up
pgd and then pmd and then pte (depending on two or three level paging scheme)

but, it depends on which virtual address is mapped to which physical address.

mostly the drivers will be a part of rootfs or could be loaded as a module.
so at linking/loading time virtual address resolution is done.

only thing for kernel to setup/update the page table entries is reamined, which will be done just at the boot time or just after module is loaded.

I think, from now onwards you have control over that physical page.

again may be we need to check UART is memory mapped or IO mapped, depends on processor.

but concept of accessing it will not change anyway using page table entries.

let me know if anybody has different views