Pinned
No teleoperation. No simulation. No RL.
Multi-fingered robot manipulation policies directly by watching videos of humans with Aria glasses on.
It was super fun working on this with @irmakkguzey's lead!!
1/n
00:00
Homanga Bharadhwaj
772 posts
Homanga Bharadhwaj
@mangahomanga
- I'll be joining the faculty @JohnsHopkins late next year as a tenure-track assistant professor in @JHUCompSci Looking for PhD students to join me tackling fun problems in robot manipulation, learning from human data, understanding+predicting physical interactions, and beyond!
- Woah SAMv2 is so good! Tried it on some robot manipulation evaluations from my prior works (Track2Act homangab.github.io/track2act/ RoboAgent robopen.github.io) and it is able to track the robots + intricate/tiny objects! Congrats to the @AIatMeta team!
00:00
00:00 - Happy to share that I've defended my PhD thesis a few weeks ago @CMU_Robotics ! Grateful to countless individuals for their selfless support over the years, in particular my advisors @gupta_abhinav_ @shubhtuls committee members @svlevine @Oliver_Kroemer all my collaborators...
- Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n
00:00 - Late update: I'm back @AIatMeta now as a Research Scientist with the Robotics team ( I'm based in scenic Seattle. If you're around, do say hi! )
- Gen2Act: Casting language-conditioned manipulation as *human video generation* followed by *closed-loop policy execution conditioned on the generated video* enables solving diverse real-world tasks unseen in the robot dataset! homangab.github.io/gen2act/ 1/n
00:00 - How can robots learn manipulation *just* by watching videos of humans in different unstructured settings? In our new paper, we develop a framework enabling zero-shot coarse robot manipulation from passive human videos (a 🧵) w/ Abhinav Gupta, @shubhtuls, @Vikashplus 1/N
00:00 - Excited to share that I will be joining @CMU_Robotics @SCSatCMU as a PhD student in Fall '21. Extremely grateful to my advisors, mentors, collaborators, in particular @animesh_garg @florian_shkurti @svlevine for their guidance and support in making this possible!
- HandsOnVLM: An in-context action prediction assistant for daily activities. It enables predicting future interaction trajectories of human hands in a scene given natural language queries. Evaluations across 100s of diverse scenarios in homes, offices, and outdoors! 1/n
00:00 - What is the relation b/w de-anonymization of authors through arXiv e-prints and their acceptance at a double-blind conference? Not a hot take: For ICLR, we find positive correlation between releasing preprints on arXiv and acceptance rates of papers by well-known authors 1/n
- Join us to work on creative problems in robotics, learning from human data, and beyond as a research intern in the Fundamental AI Research (FAIR) team @AIatMeta Apply through the portal below: metacareers.com/jobs/778113191…
- Happy to have received a Distinguished Dissertation Honorable Mention for my PhD @CMU_Robotics Grateful to my advisors @gupta_abhinav_ @shubhtuls committee @svlevine @Oliver_Kroemer and to all my supporters, collaborators, and mentors over the years!!!
- Excited to share our latest on generalizable zero-shot manipulation in the real world! We can train a single goal-conditioned policy that scales to over 100 diverse tasks in unseen scenarios, including real kitchens and offices. w/ @Vikashplus Abhinav Gupta @shubhtuls 1/n
00:00
