Session 3: expression with robot embodiments

This is the result of session 3 in my course on Social Robot Design (2025/2026). The content of this session was mainly about robot expressivity. We got to play around with an SO-101 robot from huggingface where we utilized a program called TouchDesigner to manage the expressions. It is an intuitive software with drag and drop UI, a picture is shown below.

This was done with:

• Liz van Ginderen (s27349745)
• Anna Hornman (s3056600)
• Oyindrila Sen Gupta (s3697762)
• Sarah Mans (s2306379)

Touchdesigner

Figure 1: an illustrative view of Touchdesigner

Picture of setup

Figure 2: Our physical setup: the SO-101 arm on the table, laptop running TouchDesigner connected to it, and the group positioned around it as both operators and observers.

Assignment 1: Getting the robot to work

The first step was to make the robot work and connect the software. This was straightforward. Then we just played around with the buttons to make ourselves familiar with the software.

Assignment 2: Scaffolding

The goal here was to play a preconfigured motion, but with added expressiveness with “breathing” behavior. We tried multiple variations where they differed in aggressiveness.

Different aggressiveness:

Annotation: across the variations, the parameter we manipulated was mainly speed and amplitude of the movement. We tried faster, larger movements which we saw as more “aggressive,” slower and smaller ones read as more “calm” or “hesitant.”.

With breathing:

Annotation: the breathing behavior layered a slow, low-amplitude oscillation (up/down to simulate breathing) on top of the current motion the arm was carrying out. Even at rest, this small movement was enough to feel make the robot feel “alive”.

Assignment 3: Recording

Now we can make our own sequence! In this assignment we recorded our own sequence and played it back.

Recording:

Annotation: recording let us puppeteer the arm directly rather than relying on preconfigured motions, which gave us far more control over the exact trajectory and timing of each pose. It gave us a lot more freedom, but was also harder to repeat identically.

Assignment 4: Retargeting

In the last assignment we want to control the robot while it’s carrying out a sequence.

Recording with retargeting:

Annotation: retargeting meant we could nudge the arm’s movements live while a pre-recorded sequence was still playing. This is closer to how expressive behavior would need to work in a real interaction, where a robot has to adapt to what’s happening around it rather than carrying out a strict sequence.

Small project

In this small project we act out a small scenario: The robot lazily wakes up, notices an object in front, and approaches it to take a closer look. The robot is startled and it backs away in fear from the object.

Small project teleoperate/show:

Small project carried out:

Expressiveness Design Tool

Visualisation

Figures 1 and 2 above show the tool and our setup. The video annotations under each assignment are our motion sketches.

Structured observation

First we used TouchDesigner with parameters on top of an existing motion (aggressiveness, breathing) rather than defining the motion ourselves. The assignment on recording flipped that: we puppeteered the arm directly instead. Retargeting let us nudge a sequence while it was already playing.

The arm itself didn’t change much, it was the same underlying motion. It just got faster/slower and bigger/smaller, and that alone was enough to be perceived as aggressive or calm. Breathing showed that a tiny bit of continuous movement was enough to make the arm feel alive instead of static. Chaining the waking up → noticing → approaching in the small project turned short sequences into something closer to a little story.

Watching Sarah puppeteer live vs. watching the playback actually made a difference, even though the movement was the same. Live, we could stretch or shorten a section depending on how it was going, so it felt more like a performance. In the playback, the motion was basically the same but without Sarah visibly holding the arm it read as more autonomous.

Most of what made the movement “expressive” boiled down to a a few of low-level variables we could tune (speed, amplitude, periodicity), not the shape or pose itself. For example: breathing on its own made clear how much just a bit of motion adds to feeling alive.

Where I think it fell short, however, was that we could control speed and amplitude, but not the actual path the arm took (for the preconfigured motions). That was baked into the underlying motion where we could only speed it up or slow it down. There was also no support for shaping how one movement could smoothly transition into the next.

Reflection

Speed and amplitude alone are enough to communicate mood, with no face or voice. This was quite a surprising finding for me. Just like breathing can be a way to seem “alive”, a tiny bit of continuous motion is enough to stop the arm feeling switched off. I think this generalises well: any minimal arm could use the same trick. Maybe even with fewer degrees of freedom.

I’m less sure it holds outside short clips where everyone already knew what to look for, or on a different-shaped robot (maybe 2-DoF?), where fast movement might read as unstable rather than aggressive. The teleoperated vs playback comparison also made me wonder if some of the “aliveness” came from knowing a person was driving it, which won’t be true once the robot acts on its own or if it has another autonomous policy. If I changed the tool, I’d want direct control over the path itself, not just its speed, and some way to design transitions between movements instead of tuning them by hand.

Saerbeck and Bartneck (2010) found that a robot’s motion, specifically its acceleration and curvature, was enough on its own to communicate distinct affective content, independent of the robot’s shape or task ¹. This lines up with what we found in Assignment 2.

Saerbeck, M., & Bartneck, C. (2010). Perception of affect elicited by robot motion. In Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2010) (pp. 53–60). https://doi.org/10.1145/1734454.1734473