Together with friends, I made a Pong game that you can play with your body (videos below). This is how it happened.
At my university, we do a semester project for our computer science bachelor. Our project was to create and test an emergency brake mechanism for an RC Car. For detecting whether the car would run over a pedestrian, we had the opportunity to use a LiDAR: a super expensive laser scanner with very good precision. The LiDAR was mounted on the car, together with an ODROID minicomputer it was connected to. The ODROID ran Ubuntu and we used ROS (the Robot Operating System) for controlling the AC car.
We definitely wanted a visualization of the live LiDAR data to be able to understand it better. This data was just a cloud of points—with information about how well the LiDAR recognized each point. I wrote some software that livestreams this data to a website via WebRTC to visualize it. WebRTC (Web Real-Time Communication) is a lossy UDP-like network protocol. If you prefer data packets arriving fast over each single packet arriving and doing so in the right order, WebRTC is you protocol of choice. Usually, it is used to transmit webcam data, but it can transmit arbitrary data, like coordinates of points in our case.
Once all this was done, we could implement a simple cluster detection algorithm to analyze the point cloud and recognize something being in the RC Car's way. Then we connected all parts together to make the car actually brake, and tested everything rigorously.
Playing Pong with your body
This was a super fun project. Thanks to WebRTC, the game was actually quite responsive and playable. Relearning how to make a basic game engine was interesting. We also made the ball say ouch every time it hit something. Good times.