An overview of ViTA-Zero, a zero-shot visuotactile pose estimation framework. | Credit: Amazon Robotics
Object 6D pose estimation is a critical challenge in robotics, particularly for various manipulation tasks. While prior research combining visual and tactile (visuotactile) information has shown promise, Amazon Robotics said these approaches often struggle with generalization due to the limited availability of visuotactile data.
Amazon Robotics recently introduced ViTa-Zero, a zero-shot visuotactile pose estimation framework. Amazon Robotics said the key innovation lies in leveraging a visual model as its backbone and performing feasibility checking and test-time optimization based on physical constraints derived from tactile and proprioceptive observations. Specifically, the approach models the gripper-object interaction as a spring–mass system, where tactile sensors induce attractive forces, and proprioception generates repulsive forces.
Amazon Robotics said it validated the framework through experiments on a real-world robot setup, demonstrating its effectiveness across representative visual backbones and manipulation scenarios, including grasping, object picking, and bimanual handover. Compared to the visual models, Amazon said its approach overcomes some drastic failure modes while tracking the in-hand object pose.
In experiments, the researchers said the approach shows an average increase of 55% in area under the curve (AUC) of ADD-S and 60% in ADD, along with an 80% lower position error compared to FoundationPose. The team said it also conducted extensive ablation studies to evaluate the impact of various design choices, including different loss functions, refinement algorithms, and optimization initialization techniques. For future work, we plan to evaluate more object shapes and integrate our pose estimator into manipulation policies to assess its impact on the overall performance of manipulation tasks.
Download the paper below to unlock the full details behind ViTa-Zero’s breakthrough approach. You will learn about its spring–mass visuotactile interaction model to the experimental results that prove its zero-shot adaptability. If you’re interested in the future of robot perception and manipulation, see how ViTa-Zero pushes the boundaries of what’s possible in visuotactile artificial intelligence.
"*" indicates required fields
By clicking 'Submit' above and registering for this content, I acknowledge and agree to WTWH Media's Terms and Conditions and to WTWH Media's use of my contact information to communicate with me about offerings by WTWH Media, its brands, affiliates and/or third-party partners, consistent with WTWH Media's Privacy Policy. In addition, I understand that my personal information may be shared with any sponsor(s) of the content, and that they may contact me directly about their products or services. Please refer to the privacy policies of such sponsor(s) for more details on how your information will be used by them.
Tell Us What You Think!