PAct: Part-Decomposed Single-View Articulated Object Generation

PAct turns a single input image into a part-decomposed, articulated 3D object ready for physics-based simulation.

Simulated environment containing articulated 3D furniture and appliances generated by PAct — cabinets, desks, tables, storage units and a washing machine with doors and drawers in different open states — with robotic arms interacting with the generated assets.

Given a single image, PAct generates an articulated 3D object by predicting a part-decomposed structure, synthesizing high-fidelity part geometry and appearance, and estimating articulation parameters — for downstream applications such as building simulation environments for embodied intelligence.

Abstract

Articulated 3D assets are central to interactive 3D applications, including embodied AI, robotics, and VR/AR, where functional part decomposition and valid kinematic motion are essential. Yet producing high-fidelity articulated assets remains difficult to scale, as it requires reliable part decomposition and kinematic rigging. Existing approaches largely fall into two paradigms: optimization-based reconstruction or score-distillation methods, which can be accurate but often require tens of minutes to hours per instance, and feedforward methods that rely on template or part retrieval, producing plausible results that may not match the specific structure and appearance of the input observation.

We present PAct, a feedforward, part-centric framework that converts a single-view input into an articulated 3D asset. Built on a pretrained 3D-native generative model, it uses part-decomposed latents and explicit part signals for controllable part decomposition, while alternating global and within-part attention to preserve both part-specific structure and cross-part coherence. It further predicts articulation parameters using multi-step denoising features, effectively leveraging the pretrained 3D generative prior for kinematic estimation.

Experiments on common articulated-object categories demonstrate that PAct consistently improves part geometry, articulation plausibility, and input-image consistency over optimization-based and feedforward baselines, while substantially reducing inference time.

Highlights

Instead of retrieving parts from a database or optimizing one instance at a time, PAct generates the parts and regresses their kinematics in a single feedforward pass — about 15 seconds per object.

Part-decomposed prior

A pretrained 3D-native generative model (TRELLIS) is adapted to emit a set of movable parts rather than one fused shape, so the strong static-3D prior is reused at the part level.

Local + global attention

The denoising transformer alternates within-part attention and global attention by depth, refining each part while still reasoning over the whole object.

Articulation for free

Joint type, axis, pivot and motion range are regressed from multi-step denoising features already computed during sampling — a lightweight MLP head, negligible overhead.

Controllable decomposition

A 2D part mask conditions the generation, resolving the ambiguity of “which parts move”. Works with ground-truth masks or fully automatic VLM+SAM2 masks.

Instance faithful

Because geometry and appearance are synthesized rather than retrieved, results stay close to the observed instance — the highest CLIP similarity among all compared methods.

Simulation ready

Outputs GLB meshes, Gaussian splats and articulation metadata, and exports URDF descriptions that load directly into physics engines.

Problem Setup

Given a single RGB image, PAct reconstructs an articulated object O = {P₁, …, Pₖ}, where every part Pᵢ = (Gᵢ, Aᵢ) carries its own geometry and appearance Gᵢ together with articulation parameters

Joint type τᵢ — revolute, prismatic, or fixed.
Semantic label sᵢ — e.g. base, door, drawer.
Joint origin oᵢ and joint axis uᵢ in world coordinates (the axis is re-normalized for stability).
Motion range ρᵢ = [ρ^min, ρ^max], and the parent qᵢ in the articulation tree.

Fine-grained sub-components such as handles and knobs are connected by fixed joints and add no degrees of freedom, so PAct collapses them into their parent part — keeping the kinematic DoFs while removing redundant nodes. Following prior work, the remaining structure is a depth-1 tree: base is the root and every movable part attaches directly to it, which covers the articulated objects most commonly used in embodied environments.

Method

Two stages. Stage 1 predicts which parts exist and where; Stage 2 fills in detail and appearance and reads out how the parts move.

PAct pipeline: a Part-Aware Flow Model turns the input image and part mask into a part-decomposed sparse structure; a sparse transformer denoises structured noise into reconstructed parts while an articulation module aggregates multi-step features into joint parameters.

Stage 1 predicts a part-decomposed sparse structure from a single image using a Part-Aware Flow Model. Stage 2 refines it into detailed 3D part representations via sparse-transformer denoising, while an articulation module aggregates multi-step features to estimate joint parameters for each part. The predicted joint parameters together with the reconstructed part geometries form the final articulated object.

1

Part-Decomposed Structure Generation

Stage 1 architecture: flattened part latents plus a shared part embedding pass through repeated blocks of global attention, cross attention, local (within-part) attention and cross attention, conditioned on a frozen image encoder.

Part latent tokens. The object is a set of part-level latents zᵢ; a learnable part-identity embedding E[i] is added to every token of part i so parts stay distinguishable.
Repurposed attention. TRELLIS was pretrained on single coherent objects, so a subset of its attention layers is converted to within-part attention while the rest stay global — in practice even-indexed layers are global, odd-indexed layers are local.
Mask-based part conditioning. A 2D part mask is embedded with the same embedding E, downsampled and added to DINOv2 image features, then injected through cross-attention. The mask acts as a coarse localization cue, not a pixel-accurate constraint; indices are randomly permuted during training for robustness and controllability.
Output. Part-wise sparse voxels — coarse geometry plus segmentation, at 64³ resolution.

2

Detail Synthesis & Articulation Readout

Stage 2: part latents are flattened into one token sequence and denoised over T steps; features from the last transformer block are averaged over timesteps, pooled per part, and mapped by an MLP to that part's articulation parameters.

Structure-conditioned generation. The Stage 1 sparse voxels are used directly as the part-wise condition — no separate bounding-box predictor. Tokens from all parts are concatenated into one global object sequence (with a second, distinct part embedding) so denoising reasons over the whole object; the frozen TRELLIS decoder then reconstructs each part.
Multi-step feature aggregation. Inspired by Diffusion Hyperfeatures, last-block token features are averaged over the final S = 20 denoising steps. Early features carry coarse layout, late features carry fine geometry; alternating local/global attention mixes shape cues with cross-part relations.
Articulation head. Per part, mean- and max-pooled descriptors are concatenated and mapped by a 6-layer MLP (width 512) to the full parameter set — type, label, origin, axis, range.
Training. L = L_fm + λ L_art: flow matching over part latents plus an ℓ₂ loss on the articulation vector, trained at the object level.

Implementation details

Stage 1 (Part-Aware Flow Model) initialized from the TRELLIS Stage 1 checkpoint; the Stage 2 sparse transformer initialized from OmniPart's Stage 2 checkpoint.
Two stages trained separately: lr 1×10⁻⁴, 40K steps, 4× NVIDIA A800 (80 GB), effective batch size 32.
Inference: CFG scale 7 and 25 denoising steps per stage; articulation features aggregated over the last 20 steps. End-to-end runtime ≈ 15 s on a single A800.
Part masks come from ground-truth annotations, or fully automatically from a VLM-guided prompting pipeline refined with SAM2.
Trained on PartNet-Mobility with the SINGAPO splits (20 rendered views per training object, two held-out views per test object) and evaluated on PartNet-Mobility and ACD.

Qualitative Comparisons

Retrieval-driven baselines produce category-plausible objects that drift from the observed instance; PAct generates the parts, so shape, texture and articulation stay tied to the input.

Grid comparing ground truth, SINGAPO, Articulate-Anything and PAct on eight objects from PartNet-Mobility and ACD, each shown in a resting and an articulated state.

Qualitative comparison on PartNet-Mobility and ACD against ground truth (GT), SINGAPO and Articulate-Anything. All methods are rendered as textured meshes with the same Blender pipeline. SINGAPO retrieves parts without conditioning on image appearance and drifts from the target instance (first row); Articulate-Anything relies on vision-language reasoning and can mislabel parts — in the seventh row a white cabinet is read as a “door”, corrupting the downstream structure.

vs. optimization-based FreeArt3D

Two objects compared against FreeArt3D, showing textured meshes and per-part geometry rendered in distinct colors; FreeArt3D shows fragmented part boundaries while PAct's parts are clean and coherent.

FreeArt3D optimizes each instance from multiple images captured under different articulation states. Rendering each part in a distinct color exposes noisy, fragmented decompositions; PAct produces cleaner part geometry and more coherent boundaries from a single image — and in seconds rather than minutes.

Quantitative Results

Method	RS-d_gIoU ↓	AS-d_gIoU ↓	RS-d_cDist ↓	AS-d_cDist ↓	RS-d_CD ↓	AS-d_CD ↓	AOR ↓	CLIP ↑
SINGAPO	0.5119	0.5166	0.1177	0.1505	0.0040	0.0202	0.0097	0.8779
DIPO^†	0.6221	0.6310	0.0564	0.1119	0.0035	0.0188	0.0573	0.8774
ArtFormer	1.3165	1.3212	0.3542	0.5099	0.0504	0.1849	0.0191	0.7877
PhysX-3D	0.7610	0.7637	0.3233	0.4562	0.0115	0.1204	0.2125	0.7972
PhysX-Anything	0.9596	0.9661	0.2538	0.4034	0.0097	0.1106	0.0056	0.8859
Articulate-Anything	0.6865	0.6969	0.1386	0.3414	0.0107	0.1230	0.0056	0.8795
PAct (VLM+SAM mask)	0.2485	0.2736	0.0707	0.1191	0.0019	0.0176	0.0233	0.8996
PAct (GT mask)	0.1395	0.1695	0.0202	0.0653	0.0016	0.0139	0.0011	0.9053

Method	RS-d_gIoU ↓	AS-d_gIoU ↓	RS-d_cDist ↓	AS-d_cDist ↓	RS-d_CD ↓	AS-d_CD ↓	AOR ↓	CLIP ↑
SINGAPO	0.7272	0.7306	0.1585	0.1944	0.0090	0.0256	0.0115	0.8376
DIPO^†	0.9927	0.9940	0.1379	0.1747	0.0081	0.0198	0.0200	0.8517
ArtFormer	1.3489	1.3876	0.3456	0.4414	0.0396	0.0825	0.0147	0.7690
PhysX-3D	0.6449	0.6464	0.2591	0.3396	0.0209	0.0677	0.0960	0.8132
PhysX-Anything	1.1673	1.1709	0.2965	0.3950	0.0162	0.0481	0.0099	0.8456
Articulate-Anything	1.2677	1.2693	0.5632	0.5781	0.2583	0.2803	0.1236	0.8648
PAct (VLM+SAM mask)	0.5316	0.5412	0.1249	0.1656	0.0065	0.0218	0.0248	0.8901
PAct (GT mask)	0.4182	0.4289	0.0830	0.1142	0.0060	0.0194	0.0107	0.8862

RS- / AS- denote the resting and articulated state. d_gIoU is generalized IoU between part boxes, d_cDist the distance between part centers, d_CD the Chamfer distance between meshes, AOR the Average Overlapping Ratio (inter-part collision), and CLIP the image similarity to the input. Models are trained on PartNet-Mobility, so ACD is an unseen benchmark. The GT-mask row is an oracle evaluation of the part-aware generation module; VLM+SAM is the practical automatic-mask setting. ^†DIPO uses images of two articulation states, giving it motion cues unavailable to our single-image method.

Runtime per object

~15sPAct

~8sSINGAPO

~96sPhysX-3D

~155sArtFormer

~3minArticulate-Anything

~10minFreeArt3D

~20minPhysX-Anything

All measurements on a single NVIDIA A800 GPU.

Ablation study — what each design choice buys

Configuration	RS-d_gIoU ↓	AS-d_gIoU ↓	RS-d_cDist ↓	AS-d_cDist ↓	RS-d_CD ↓	AS-d_CD ↓	AOR ↓	CLIP ↑
S = 1 denoising step	0.1734	0.1999	0.0300	0.0966	0.0017	0.0220	0.0088	0.8903
S = 15 denoising steps	0.1737	0.2002	0.0320	0.0762	0.0018	0.0145	0.0090	0.8911
Articulation from Stage 1 features	0.1615	0.1878	0.0302	0.0889	0.0016	0.0210	0.0068	0.8903
Flow-matching articulation head	0.3615	0.4132	0.1263	0.3019	0.0021	0.1100	0.0190	0.8803
w/o within-part local attention	0.2485	0.3317	0.0416	0.1285	0.0045	0.0267	0.0093	0.8932
w/o global object token sequence	0.1655	0.1921	0.0285	0.0810	0.0016	0.0155	0.0097	0.8992
GT mask + random dilation / erosion	0.1491	0.1842	0.0264	0.0799	0.0016	0.0140	0.0043	0.9009
PAct — S = 20, Stage 2, regression	0.1395	0.1695	0.0202	0.0653	0.0016	0.0139	0.0011	0.9053

All on PartNet-Mobility with GT masks. (i) Aggregating more denoising steps helps, with diminishing returns — 20 steps is our default. (ii) Stage 2 features beat Stage 1 features: Stage 1 mainly encodes coarse part structure, while Stage 2 features are refined by object-level denoising and carry richer geometric and semantic information. (iii) Replacing the deterministic MLP with a generative flow-matching predictor does not help and adds complexity, so we keep the simple regression head. (iv) Removing within-part local attention is the most damaging architectural change; removing the global object token sequence in Stage 2 also hurts. (v) Randomly dilating or eroding the GT masks with a 5×5 kernel degrades results only mildly — the mask is a localization cue, not a geometric constraint.

Direct evaluation of joint parameters (PartNet-Mobility)

Method	Joint-Type Acc. ↑	Joint-Axis Err. ↓	Joint-Pivot Err. ↓
SINGAPO	0.9408	0.1357	0.0370
DIPO^†	0.9599	0.0206	0.1561
ArtFormer	0.3835	0.4187	0.2358
Articulate-Anything	0.8853	0.2094	0.3351
PhysX-3D	0.4360	0.9696	0.3229
PhysX-Anything	0.8020	0.3591	0.4628
PAct	0.9675	0.1065	0.0219

Our main evaluation uses motion-space metrics, since the final articulated state is determined jointly by joint type, axis, pivot, range and part geometry. For reference we also report per-parameter errors: PAct obtains the best joint-type accuracy and joint-pivot error, while DIPO's lower axis error benefits from observing two articulation states.

Controllable Decomposition

One image often admits several valid decompositions. The part mask makes that choice explicit instead of leaving it to the model.

Two cabinets, each generated twice from the same input image under two different part masks, producing either doors only or doors plus drawers while keeping appearance consistent.

Same image, different masks. In the first row one mask yields two doors, the other two doors plus two drawers — while appearance stays consistent and the joint configurations remain plausible. This is what reduces structural ambiguity on real-world photos.

How tolerant is the model to mask quality?

Each row shows an input image, a severely over-segmented mask and its resulting textured mesh, then a mask with random boundary dilation/erosion and its resulting mesh, in resting and articulated states.

Each row pairs a severely over-segmented mask with a mask carrying random boundary dilation/erosion. Because the mask is downsampled before being added to the image features, it acts as a coarse localization cue rather than a pixel-accurate constraint: moderate boundary noise is tolerated, but wrong part instances — over-segmentation or incorrectly grouped regions — change what the model is asked to build and the errors persist. That is also why a gap remains between the GT-mask oracle and the automatic VLM+SAM setting: what matters is recovering the right part instances, not pixel-perfect boundaries.

In-the-Wild Inputs

Beyond benchmark renderings: synthesized images and phone photos of everyday cabinets, desks, dressers, fridges and washing machines.

Despite the domain gap between curated dataset renderings and real photographs, PAct produces coherent part decompositions, plausible geometry and visually consistent textures. These examples are curated and have no ground-truth 3D assets, so they demonstrate practical feasibility rather than a systematic robustness study.

Straight into a Simulator

Two sequences in a physics simulator: a robot arm pulls open a drawer of a generated cabinet, and swings open the door of another generated cabinet.

Generated assets loaded into a physics simulator, where a robot arm pulls a drawer and swings a door open. Because articulation parameters come out of the same forward pass as the geometry, the asset is actuatable without manual rigging — the motivating use case for embodied-AI environments.

Limitations

Category and part-count scope. Training and quantitative evaluation concentrate on seven household furniture and appliance categories (≈ 3K objects after augmentation). Objects with many parts are rare in existing articulated-object datasets, and PAct struggles on them.
Input articulation state. Following SINGAPO, we assume resting-state or mostly-closed inputs. Large articulation and severe self-occlusion degrade results; handling them would require estimating the observed state jointly with the canonical joint parameters.
Unseen or occluded parts. We assume the relevant functional parts are visible. A drawer hidden behind a closed door is filled in from learned priors — plausible, but not guaranteed to match the specific instance.
Kinematic scope. The depth-1 formulation fits common shallow articulation trees. A preliminary graph-prediction extension to depth-N generalizes poorly at the current data scale; coupled joints and closed-chain mechanisms likely need explicit kinematic constraints.
Physical validity. Without explicit physical constraints, parts can collide or detach. Simulator-based validity feedback during generation or refinement is a promising direction.
Texture misalignment. Textures are back-projected from rendered Gaussian splats in the TRELLIS pipeline and can carry projection errors, particularly for hollow or occluded interiors. This is orthogonal to articulation modeling and should improve with stronger native texture backbones.

Representative failure cases

Two failure cases: a door colliding with a drawer, and a door detached from the base; base-only renderings reveal black interior regions from texture misalignment.

Physical-validity and appearance failures. Case 1 is a door–drawer collision; Case 2 shows a door detached from the base. Removing all movable parts and rendering the base alone exposes black interior regions caused by texture misalignment.

Predicted articulated states next to ground-truth references, illustrating joint-estimation discrepancies.

Predicted articulated states against ground-truth references. Single-view generation is inherently ambiguous, so the ground truth is one valid articulation rather than the only plausible one — small differences may be legitimate alternatives, while large discrepancies point to real geometry or joint-estimation errors.

BibTeX

@article{liu2026pact,
  title={{PAct}: Part-Decomposed Single-View Articulated Object Generation},
  author={Liu, Qingming and Yao, Xinyue and Zhang, Shuyuan and Deng, Yueci and Liu, Guiliang and Liu, Zhen and Jia, Kui},
  journal={arXiv preprint arXiv:2602.14965},
  year={2026}
}