Suhas Kadalagere Sampath
A vision-guided deep learning framework for dexterous robotic grasping using Gaussian processes and transformers
Kadalagere Sampath, Suhas; Wang, Ning; Yang, Chenguang; Wu, Howard; Liu, Cunjia; Pearson, Martin
Authors
Dr. Ning Wang Ning2.Wang@uwe.ac.uk
Senior Lecturer in Robotics
Charlie Yang Charlie.Yang@uwe.ac.uk
Professor in Robotics
Howard Wu
Cunjia Liu
Martin Pearson Martin.Pearson@uwe.ac.uk
Senior Lecturer
Abstract
Robotic manipulation of objects with diverse shapes, sizes, and properties, especially deformable ones, remains a significant challenge in automation, necessitating human-like dexterity through the integration of perception, learning, and control. This study enhances a previous framework combining YOLOv8 for object detection and LSTM networks for adaptive grasping by introducing Gaussian Processes (GPs) for robust grasp predictions and Transformer models for efficient multi-modal sensory data integration. A Random Forest classifier also selects optimal grasp configurations based on object-specific features like geometry and stability. The proposed grasping framework achieved a 95.6% grasp success rate using Transformer-based force modulation, surpassing LSTM (91.3%) and GP (91.3%) models. Evaluation of a diverse dataset showed significant improvements in grasp force modulation, adaptability, and robustness for two- and three-finger grasps. However, limitations were observed in five-finger grasps for certain objects, and some classification failures occurred in the vision system. Overall, this combination of vision-based detection and advanced learning techniques offers a scalable solution for flexible robotic manipulation.
Journal Article Type | Article |
---|---|
Acceptance Date | Feb 24, 2025 |
Online Publication Date | Feb 28, 2025 |
Publication Date | Feb 28, 2025 |
Deposit Date | Feb 28, 2025 |
Publicly Available Date | Mar 4, 2025 |
Journal | Applied Sciences |
Electronic ISSN | 2076-3417 |
Publisher | MDPI |
Peer Reviewed | Peer Reviewed |
Volume | 15 |
Issue | 5 |
Article Number | 2615 |
DOI | https://doi.org/10.3390/app15052615 |
Keywords | dexterous robotic grasping; adaptive grasping; deep learning in robotics; transformer networks; Gaussian processes; vision-based force modulation |
Public URL | https://uwe-repository.worktribe.com/output/13826594 |
End hunger, achieve food security and improved nutrition and promote sustainable agriculture
Build resilient infrastructure, promote inclusive and sustainable industrialisation and foster innovation
Files
A Vision-Guided Deep Learning Framework for Dexterous Robotic Grasping Using Gaussian Processes and Transformers
(31.1 Mb)
PDF
Licence
http://creativecommons.org/licenses/by/4.0/
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
Head-raising of snake robots based on a predefined spiral curve method
(2018)
Journal Article
Enhanced teleoperation performance using hybrid control and virtual fixture
(2019)
Journal Article
Efficient 3D object recognition via geometric information preservation
(2019)
Journal Article
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search