An electrostatic haptic display for the visually impaired

This completed study was sponsored by the National Science Foundation (NSF),
 grant number IIS-0081201. The project period was 9/1/2000–8/31/2004. The research was conducted at the Laboratory of Professor David Beebe, Department of Biomedical Engineering, University of Wisconsin–Madison, in collaboration with Mitchell Tyler and Kurt Kaczmarek, and the principal investigator was David Beebe. 

Project Summary

Access to non-textual material for persons with visual impairments has become increasingly important in recent years. The prolific use of Graphical User Interface (GUIs) based operating systems has made computer access more difficult for people with visual impairments and it is increasingly limiting their ability to participate in many vocational, educational and avocation activities. Speech synthesis is of limited use in presenting non-textual information and therefore low cost/effective tactile or haptic displays need to be developed to provide access to non-textual information. Tactile or haptic displays hold considerable advantages in term of non-textual information transfer, particularly for navigating in GUIs and in the transfer of graphic and spatial information (plots, bar graphs, etc.). We propose to evaluate a new type of haptic display based on electrostatic stimulation (as opposed to the more common electrocutaneous or vibrotactile modes of stimulation). Electrostatic displays have the potential to overcome many of the problems of electrocutaneous and vibrotactile displays. Electrostatic displays can be easily batch fabricated using micro fabrication techniques, the percept is one of texture (there is negligible direct current flow into the skin) and there are no moving parts to stick or wear. Significant preliminary results (average pattern recognition rates of 68% to 72%) have demonstrated the feasibility of electrostatic displays for the presentation of simple spatial patterns.

The proposed work will focus on the evaluation of the display for use in practical situations (such as the interpretation of business graphs) and will also lead to a better understanding of the perceptual mechanisms in electrostatic stimulation. Additionally, geometric and other modifications will be integrated into the display which should significantly increase the performance of the display. Specifically, the work will focus in two areas:

  1. evaluation of the display including learning effects, young vs. old, visually impaired vs. non-impaired, comparison to raised line drawings, pattern recognition, academic/business graphics and dynamic range,
  2. development of improved display technology guided by continuous evaluation experiments. These areas will be pursued in parallel with technological improvements being incorporated into the display as they are developed. This will allow human subject experiments to begin by the end of year one.

Specific evaluation experiments include threshold, resolution and pattern recognition including the effects of learning. The dynamic range of the display will be evaluated to determine the feasibility of displaying gray levels and color. Simple business, engineering and chemistry graphics will be presented in a study to evaluate the efficiency of information transfer as compared to raised line drawings. This focus on information transfer (academic graphics) and complex image issues (dynamic range and spatial dynamics) will allow evaluation of the displays’s usefulness for education as a technology that could improve the representation of students with visual impairments in science and engineering. Other experiments will investigate the parameters of shear force, pressure and friction and determine their role in the perception mechanism. Future work will include the incorporation of touch sensitivity, on-display signal processing capability and the incorporation of sound cues all within the basic framework of technology and evaluation developed under this grant.

Specific evaluation experiments include threshold, resolution and pattern recognition including the effects of learning. The dynamic range of the display will be evaluated to determine the feasibility of displaying gray levels and color. Simple business, engineering and chemistry graphics will be presented in a study to evaluate the efficiency of information transfer as compared to raised line drawings. This focus on information transfer (academic graphics) and complex image issues (dynamic range and spatial dynamics) will allow evaluation of the displays’s usefulness for education as a technology that could improve the representation of students with visual impairments in science and engineering. Other experiments will investigate the parameters of shear force, pressure and friction and determine their role in the perception mechanism. Future work will include the incorporation of touch sensitivity, on-display signal processing capability and the incorporation of sound cues all within the basic framework of technology and evaluation developed under this grant.

Project Results

The overall goal of this project is to develop better methods to communicate graphical/pictorial information to people using electrostatic stimulation of the sense of touch. We developed flat matrices of electrodes, using lithographic methods, that subjects scanned with their fingertips to receive information. The NSF-supported work concerned both instrumentation development and human subjects experiments to evaluate the perceptual properties of these electrodes.

  1. Instrumentation: We developed reliable means to fabricate electrostatic electrode arrays. These arrays comprise metallic traces deposited on insulated silicon wafers, with a means at the edge of the wafers to connect the electrodes to external electronics. The electrode arrays are covered with a thin layer of spun-on polymide to enhance the electrostatic sensation while preventing more than a tiny capacitive displacement current to flow in the skin. Understanding and careful manipulation of key variables in the fabrication process reliably yields high quality arrays with predictable properties. The major barriers we overcame were in producing a polyimide layer with uniform thickness and zero defects over the surface of the array and in reliably connecting a cable to the arrays.
  2. Surface texture: We developed methods to texture the surface in an effort to stabilize the sensation produced by electrostatic stimulation of touch. The sensation tends to vary intensity depending on the frictional properties of the sliding interface between the skin and the electrode array. While added texture was found to increase sensory thresholds, these thresholds were more repeatable (i.e., lower coefficient of variation). We also found an interaction between the surface texture and the texture of the fingertip (ridges).
  3. We investigated the relationship between sensory threshold and thickness of the polyimide insulator. It was found that this relationship was not predicted simply by the electrostatic force equation for a parallel-plate capacitor (i.e., with the polyimide as the dielectric). The sensory threshold appears to depend on both the polyimide thickness and the dielectric characteristics of the skin. A previously published model concerning the interaction of these two dielectrics was found to be invalid on theoretical grounds and a new model was proposed.
  4. We investigated the relationship between sensory threshold and the voltage waveform on the electrodes. Standard electrostatic theory states that the electrostatic force between two plates separated by an insulator is independent of polarity and this would predict that polarity would have no effect on sensation threshold for perception of electrostatic forces. Our results show a strong polarity effect, suggesting that one of the two dielectrics in question (polyimide, skin) has non-ideal dielectric properties. We showed that previously published research concerning the nonlinear conductive properties of skin explains this finding.
  5. We investigated the use of a new aromatic thermosetting polymer coating for the electrode array. Initial results suggest that the use of this material effectively reduces moisture uptake and charge buildup in the polyimide layer, both of which may result in unpredictable sensory thresholds.

Journal publications

  1. K. A. Kaczmarek, K. Nammi, A. K. Agarwal, M. E. Tyler, S. J. Haase and D. J. Beebe, Polarity effect in electrovibration for tactile display, IEEE Trans. Biomed. Eng., vol. 53, pp. 2047-2054, 2006.

Conference publications

  1. Agarwal, A.G. Nammi, KK, Kim, D, Kaczmarek, K.A., Tyler, M.E., & Beebe, D.J., A hybrid natural/artificial electrostatic actuator for tactile stimulation, (2002). Conference proceedings, Published Collection: Second International IEEE-EMBS Special Topic Conference on Microtechnology in Medicine and Biology Bibliography: May 2-4, 2002, Madison, WI, USA.
  2. Agarwal, A.K., Nammi, K., Beebe, D.K., An electrostatic haptic display for tactile stimulation, (2003). Conference proceedings, Published Collection: 2003 BME Seminar Bibliography: University of Wisconsin-Madison.
  3. A. K. Agarwal, J. Braun, K. Nammi, K. A. Kaczmarek, M. E. Tyler and D. J. Beebe, A microfabricated electrostatic haptic display: Effect of surface texture on tactile percept (abstract), in BMES Annu. Fall Meeting, Nashville, TN, 2003.

Theses

  1. Agarwal, A.K., A microfabricated electrostatic haptic display: A psychophysical study on the effect of dielectric thickness and surface texture on tactile percept, (2003). M.S. Thesis, University of Wisconsin-Madison.

Our Research

Founded in 1992, the Tactile Com­mu­nication & Neurorehabilitation Laboratory (TCNL) is located at the University of Wisconsin-Madison.

We are a research center that uses the experience of many different areas of science to study the theory and application of applied neuro­plasticity, the brain’s ability to re­or­ganize in response to new informa­tion, needs, and pathways.

Our research is aimed at developing solutions for sensory and motor disorder rehabilitation.