roadmap based on CHI reviews

[derikon]

List of available features

Frontend

Technical aspects

• connect to multiple BLE devices used as tactile display (up to 10 possible)
• connect external input devices, i.e. gamepads

Vibrotactile pattern design

• real-time actuation (see mappings above)
• record pattern
• play back pattern (streamed to all connected clients)

Editing patterns (wip)

• adjust position of individual actuation blocks

• adjust duration of individual actuation blocks

• delete individual actuation blocks
  

Collaboration

• users in the same room have shared control of actuation

Backend

• stores rooms and vibrotactile patterns (JSON) in these rooms
• receives actuation signals from all collaborators in a room and broadcasts them in real-time

Hardware

Tactile display

We built a custom tactile display that connects via BLE to the host computer.

v1.0

• up to 8 ERMs (frontend configuration limits to 4 actuators)
• actuation is done via a BLE characteristic, i.e., writing an array of bytes each representing the actuation amplitude of the corresponding actuator

v2.0

• up to 4 ERMs, LRAs, or voice coils
• Frequency and amplitude are controlled via separate BLE characteristics

v3.0 (wip)

• add sensing capabilities for up to 4 analog sensors (e.g., flex, pressure)

Actuator mounting system

We started with kinesio tape to freely place the actuator on the body. This is straight forward and a fast process. However, replicating the same configuration is hard. Therefore, we propose a modular system that allows flexible and reproducible actuator configurations.

[derikon]

Features mentioned by reviewers

Interaction and design process

• how to control amplitude and frequency separately
• editing afterward (refine)
• copy, paste
• annotation / tagging
  • useful for synchronous collaboration?
  • colors should be used more to differentiate users at all stages

Collaboration

• keep focus on remote collaboration (distant multi-device collaboration is the most interesting)
• turn-taking (how does it work? what is needed?)
  • single user has control over actuators (jamming and recording)
  • single user has control over what is being played (playback)
  • enable recording different part of signal together (e.g., user 1 edits first half and user 2 edits second half)?
• solo mode
  • must enable sharing solo creation to multiple collaborators
  • build on top of collaborative designs (e.g., a group of designers created something, someone turns on solo mode to try stuff on this design, then decides to share that with everybody)
• expand on jamming (what does this actually need to be effective and efficient?)
  • non-recording mode (same as TactJam)
• recording modes
  • overdubbing
  • replacing
  • additive
  • subtractive
  • automation (i.e., like in DAWs with MIDI parameters)

Technical

• transferability to other actuator types (done with hardware v2.0)
• use MIDI as format for vibration patterns? (midi-file seems to work very nicely, and MIDI looks perfect for what we are doing)