Enhancing Tactile Sensing in Robotics: Dual-Modal Force and Shape Perception with EIT-Based Sensors and MM-CNN

Electrical Impedance Tomography (EIT)-based tactile sensors offer durability, scalability, and cost-effective manufacturing. However, simultaneously reconstructing force and shape from boundary measurements remains challenging due to EIT’s inherent location dependencies and image artifacts. This study presents a model-driven multimodal convolutional neural network (MM-CNN) for joint EIT-based force and shape sensing. The hybrid approach combines physics-inspired voltage preprocessing with an attention-based network to overcome EIT’s limitations. The preprocessing network applies a linearized one-step inverse solution with Tikhonov regularization to convert raw boundary voltage into a noise-reduced 2D image. The image reconstruction network uses an attention mechanism to focus on salient features, addressing location dependency issues. Quantitative metrics show that MM-CNN outperforms traditional EIT algorithms like NOSER and TV, reducing location dependency and improving shape discrimination. MMCNN enables unified force and shape modalities, validated through real-contact experiments, enhancing EIT tactile systems for human-robot interaction by incorporating physical knowledge with deep learning.