Non-sequential Autoregressive Shape Priors for
3D Completion, Reconstruction and Generation

Paritosh Mittal^{* 1}

Yen-Chi Cheng^{* 1}

Maneesh Singh²

Shubham Tulsiani¹

¹Carnegie Mellon University

²Verisk Analytics

(* indicates equal contribution)

Overview. Our approach combines a non-sequential autoregressive prior for 3D shapes with task-specific conditionals
to generate multiple plausible and high-quality shapes consistent with input conditioning. We show the efficacy of our approach across diverse tasks such as
(Left) shape completion, (Middle) single-view reconstruction and (Right) language-guided generation.

Input

Multimodal Shape Completion

Input

Multimodal Shape Completion

Shape Completion. Given the partial inputs, the proposed approach is able to generate diverse plausible 3D shapes consistent with the partial input.
For example in row-5, a table like structure is reconstructed as an aeroplane. Red cuboid denotes the missing region.

Input Image

Ground Truth

Ours

ResNet2Voxel

ResNet2SDF

Pix2Vox

Single-view Reconstruction. Given an image as input, we show the single-view reconstruction results with the proposed method and how it compares against other competing methods.

Input Image

Multimodal Shape Completion

Input Image

Multimodal Shape Completion

Single-view Reconstruction. We present more results from the proposed method.


thin legs, thin arms	Stool, has a square floor mount	No holes in arms?

no arm rest	kitchen chair	tall thinest legs

cup shaped	lawn chair, two slats	Most ornate, rounded back with design

Language-guided Generation. Bold: Text Description. GIF: Three random samples generated by our approach.


Ours	JE	T2S
	curved looking one with four legs

Ours	JE	T2S
	tall narrow back

Ours	JE	T2S
	Single leg on square base

Ours	JE	T2S
	Wide seat with armrest

Qualitative Comparison with Baselines Bold: Text Description. GIF: Three random samples generated by our approach. Left to Right: Columns 1-3 are the generations from our approach. Columns 4-6 are generations from JE and columns 7-9 are from Text2Shape