Sensorimotor Proto-Objects: Bootstrapping Perception from Compensable Actions Loïc Goasguen, Sylvain Argentieri (2026). in IEEE Transactions on Cognitive and Developmental Systems. 📝PDF🌐DOI📚Bibtex
Sensorimotor contingencies theory provides a strong foundation for a mathematical formalization of the bootstrapping problem of robot perception. Currently, this framework lacks the ability to deal with the dynamics of the environment and, by extension, objects. This paper shows that a sensorimotor agent with prediction functions acquired by motor babbling can infer environment dynamics and detect primitive object notions. To do that, we formalize Poincaré's notion of compensation in terms of the agent's action group, establishing an isomorphism with the environmental displacements. This allows the agent to describe environmental spatial transformations in terms of actions. We then conduct two experiments applying those developments to a visual task to (1) infer environmental global displacements and (2) infer a single object's segmentation via its movement in the environment. We define a notion of sensorimotor proto-objects, as sensory elements whose information moves rigidly and coherently while maintaining shape and size. Here, the agent identifies the displacement by inferring the one that minimizes prediction error which is computed thanks to an internally built sensory metric. Results show that, in a simple 2D visual environment, the agent robustly infers the displacement observed for both the global and local case, but also accurately segments a single proto-object. By formalizing and inferring environmental transformation, this work introduces a new definition of sensorimotor proto-objects paving the way for a new understanding of how object perception can emerge from an agent's own actions.
Binaural Localization of Multiple Sound Sources by Non-Negative Tensor Factorization Elie Laurent Benaroya, Nicolas Obin, Marco Liuni, Axel Roebel, Wilson Raumel, Sylvain Argentieri (2018). in IEEE/ACM Transactions on Audio, Speech, and Language Processing. 📝PDF🌐DOI📚Bibtex This paper presents non-negative factorization of audio signals for the binaural localization of multiple sound sources within realistic and unknown sound environments. Non-negative tensor factorization (NTF) provides a sparse representation of multi-channel audio signals in time, frequency, and space that can be exploited in computational audio scene analysis and robot audition for the separation and localization of sound sources.Intheproposedformulation,eachsoundsourceis represented by mean of spectral dictionaries, temporal activation, and its distribution within each channel (here, left and right ears). This distribution, being dependent on the frequency, can be interpreted as an explicit estimation of the Head-Related Transfer Function (HRTF) of a binaural head which can then be converted into the estimated sound source position. Moreover, the semi-supervised formulation of the non-negative factorization allows to integrate prior knowledge about some sound sources of interest whose dictionaries can be learned in advance, whereas the remaining sources are considered as background sound which remains unknown and is estimated on-the-fly. The proposed NTF- based sound source localization is here applied to binaural sound source localization of multiple speakers within realistic sound environments.
Get in touch
You can find me at ISIR, office 56/66 309, or in the Master department, Esclangon building, 2nd floor, office 219 (Faculty of Science & Engineering, Sorbonne Université).
