What is HFVE ?

HFVE (Heard & Felt Vision Effects) is an audiotactile vision information system for blind and deafblind people.  HFVE presents aspects of vision via a rich set of audio and tactile effects.

HFVE software  is just about complete, and enables totally blind and deafblind people to access many types of visual representation such as : still and moving images, graphs, diagrams, abstract shapes, and parts of a computer desktop.  Click here for demo videos.  Click here to try HFVE!

In the HFVE system, moving audio and tactile effects trace out shapes, with corners emphasised ("Tracers" and "Polytracers"), using buzzing sounds and other effects to clarify the shapes of items; and using groups of voices, speaking in unison, that rapidly convey the properties, and the approximate size and location, of items ("Imprints"). Audiotactile equivalents of item outlines and corners, color layouts, detected faces, motion etc. are presented via moving speech and buzzing sounds; via a moving force-feedback device (or via moving effects on a tactile array); and via braille.

A major feature of the system is presenting modified speech i.e. spoken word sounds that are changed, multiplied, and moved, in order to intuitively convey the location, size, shape, and other properties, of the items they are presenting.

"Multi-talker Focus effects" are a new feature : the system presents the items that are currently the primary focus of attention via crisp non-modified speech sounds. At the same time the system presents the speech sounds for items that are not at the focus of attention, but applies a distinct differentiating effect on them e.g. distinct types of voices (e.g. monotone, or with intonation), stereophonic location away from the primary item, and special effects such as echo and reverberation. The user is aware of the other items, but not directed towards them.

Key features

• Moving audiotactile effects trace out the shapes of key lineal features such as object outlines, medial lines, or symbolic tracers.
  
• Highlighted corners and "buzz tracks" clarify the shapes.
  
• Moving Polytracers and stationary Imprint effects help convey the content and location of objects.
  
• Multi-level multi-talker Focus effects allow several items to be presented and explored at the same time.
  
• Speech, braille, and tap code effects present visual features such as color, or identified objects.
  
• Computer vision techniques are used to present "blobs", motion, and recognized objects.
  
• Blind users can create audiotactile images via keyboard, pointer, touch-screen, or speech.
  
• Prepared guides can define how items are presented.
  
• Other apps that produce shapes and images can send visual representations for the HFVE software to exhibit, via a simple interface.

See the  HFVE software  page for more details.

I've done some  presentations  on the HFVE project.

I demonstrated the HFVE system at the  4th and  5th Interactive Sonification Workshop, showing the software running on a small tablet computer (and laptop); with tactile shapes etc. presented via an adapted Microsoft Sidewinder Force Feedback 2 joystick, and via a Logitech Wingman Force Feedback Mouse. Recently I presented the system at the  25th International Conference on Auditory Display (ICAD 2019) where my paper received Hyundai Motors Design Challenge Award Best Paper.

Contact HFVE  for more information.

 Demo videos

		Pentagon shape presented by HFVE, and drawn by user (51s)				Shape presented via joystick and force-feedback mouse (22s)
		Pentagon.mp4				Tactile.mp4
		Pentagon.wmv				Tactile.wmv
		HFVE OBJECTS - several tracer types; polytracers; and tracked items (41s)				HFVE REGIONS - regular rectangular areas, moved and zoomed (49s)
		Objects.mp4				Regions.mp4
		Objects.wmv				Regions.wmv
		HFVE IMPRINTS stepping round the colors in a scene (55s)				HFVE LOOKING for red items; and tracking some of them (52s)
		Imprints.mp4     Bubbles				Looking.mp4
		Imprints.wmv     Bubbles				Looking.wmv
		HFVE SHAPE and GRAPH tracers, controlled via an external app (63s)				EXPERIMENTAL - several pre-defined movie non-speech item effects (70s)
		Graphs.mp4				Movie.mp4
		Graphs.wmv				Movie.wmv
		CLIENT-SOURCED ENTITY, showing the Dewey-Decimal® levels of view, locking, etc. (81s)				HFVE MULTI-LEVEL EFFECTS, showing image levels of view, item locking, etc. (58s)
		Dewey.mp4				Levels.mp4
		Dewey.wmv				Levels.wmv
		HFVE MULTI-TALKER FOCUS EFFECTS, showing road map item locking, echo effects, etc. (115s)
		Map.mp4
		Map.wmv