Modulating Matter with Intuitive Interaction
It is an experiment on the modulation of ferrofluid patterns controlled by both electromagnetic signals and a robotic interface. It modulates the patterns according to the audience interaction based on realtime face tracking data. Similarly to digital displays it is based on grid, but the material ferropixel presents a much richer gradient of emerging patterns.
Exhibited at Ars Electronica Center during Creative Robotics exhibition ( curation Johannes Braumann, Kristina Maurer) Linz, AT 09.05.2018- 07.2018
Maria Smigieska (CR UfG, mariasni.com ) design, robotics
Pierre Cutellic (CAAD ITA ETHz, compmonks.com ) design
Johannes Braumann (CR, UfG) robotics
Barlomiej Doros – chemistry advisor
music: Jo Noon “October”
video: Pierre Cutellic, Johannes Braumann
video edit: Maria Smigielska
Modulating Matter with Mixed Intelligences
Proteus is an installation that modulates ferrofluid patterns, with both human and machine intelligences, in a closed loop. Through an individual and prolonged visual experience, it immerses the visitor in an implicit interaction with the material through a brain-computer interface. A pre-trained dedicated machine learning model is informed by real-time neural signals, produced by the visitor’s gaze while being exposed to the rapid serial change of patterns without any explicit instructions to follow. Over the time of the gaze experience, visitors may witness a certain stabilisation of their own modulated picture of the material.
This model not only modulates the ferrofluid display over time but also slowly builds a digital motion picture of its behaviour. All the recorded sequences are then displayed separately to exhibit the variety of patterns collected throughout the period of the festival. The project infers on digitally encoded material properties with the use of a computational mixture of both human and machine vision models.
Maria Smigielska (mariasni.com)
Pierre Cutellic (compmonks.com)
Creative Robotics, UfG Linz
CAAD, ETH Zurich
Simulating Matter with Mixed Intelligences