New fine-tuning scripts and containers have been released for TRELLIS.2, a 3D generation model. These tools enable users to adapt the model to their specific 3D assets, addressing limitations of general-purpose models in specialized applications. The provided resources are designed for execution both locally and on Hugging Face Jobs.
3D generation is used in fields like robotics, simulation, media, and entertainment. A challenge in these areas is ensuring generated objects adhere to strict rules and standardizing model representations due to various file formats (e.g., GLB, PLY) and representations (e.g., NeRFs, 3DGS). TRELLIS introduced a unified latent representation to simplify this, with TRELLIS.2 further improving 3D structure representation. However, neither model previously offered an easy method for task-specific adaptation.
General-purpose models like TRELLIS.2 can produce errors in details crucial for specific applications. For instance, furniture models might require precise geometry that a pretrained model cannot provide without adaptation. An experiment using a subset of the Google Scanned Objects (GSO) dataset with 188 footwear objects showed that while pretrained TRELLIS.2 generated correct-looking shoes, the soles were often inaccurate. Fine-tuning aims to correct such specific deficiencies.
The fine-tuning process focuses on improving TRELLIS.2's performance on specific object categories, such as footwear from the GSO dataset. The approach involves selectively fine-tuning pipeline components related to detailed geometry while freezing others, as the pipeline separates structure, detailed geometry, and materials. Low-rank adaptation (LoRA) is utilized for fine-tuning, which trains small adapter matrices while keeping original weights frozen. This method reduces the number of trainable parameters and the memory required for gradients.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
Fine-tuning scripts and containers are now available for TRELLIS.2, a 3D generation model, allowing users to adapt it to specific 3D assets. These resources can run locally or on Hugging Face Jobs, addressing the need for task-specific accuracy in 3D model generation.