We present a lightweight, position-aware network designed for real-time novel view synthesis from a single input image. Unlike existing MPI or Image-Based methods which utlizie an explicit warping operator, we directly query the final views from the network itself. Our network inference is real-time making it suitable for fine-tuning on domain specific applications for live feeds.
Flowers Dataset
Stanford Dataset
The visual comparison against TMPI, AdaPI, SinMPI on Blender Dataset.
The visual comparison against SinMPI, TMPI, AdaMPI on Blender Dataset.
The below example shows the NVS results when using different positional embedding methods.
There's a lot of excellent work that was introduced around the same time as ours.
NViST: In the Wild New View Synthesis from a Single Image with Transformers introduces an idea similar to as our embedding scheme.
We compare the FPS rate on different resolutions against other methods when rendering end-to-end.
@article{gond2024real,
title={Real-Time Position-Aware View Synthesis from Single-View Input},
author={Gond, Manu and Zerman, Emin and Knorr, Sebastian and Sj{\"o}str{\"o}m, M{\aa}rten},
journal={arXiv preprint arXiv:2412.14005},
year={2024}
}