Modelling the movement of clothing from every angle

4DHuman Outfit, or 4DHO for short, was the name given to this project, which researchers spent four years working on. Their goal was to develop a new tool that could be used by both scientists and manufacturers to model the movement of clothes. “There was previously very little reliable data, and very little data in general, on the movement of clothes”, explains Stefanie Wuhrer, a researcher with Morpheo, a project team at the Grenoble Alpes University Inria Centre. “We sought to address this issue by creating a set of real-life data on clothing dynamics for use in designing future models.”  

This was no mean feat: the first thing they had to do was to decide on the type of movements they wanted to record, before then capturing these movements and putting the data to use. 4DHO was formed in 2019 when Morpheo, a team specialising in perceiving and interpreting movements using multi-camera systems and the SED (Service expérimentation et développement - Experimentation and Development Department) from the Grenoble Alpes University Inria Centre teamed up with Naver Labs Europe, a laboratory for industrial research into artificial intelligence, which is also based in Grenoble. “Inria had worked closely with Naver Labs Europe for a number of years. Our work involves similar research subjects and our teams know each other well”, says Christophe Legras, project manager at Naver Labs. “Together we identified a pressing need within the scientific community for 4D data on people in motion with multiple degrees of variability.” 

©Kinovis

The method employed centred around 20 individuals with different body shapes carrying out 11 movements (9 of which were predetermined and 2 of which they were free to choose themselves) in 7 different outfits. The aim was to produce a data cube with each segment representing a combination of people, outfits and movements, all in 4D, i.e. in both space and time. 

This meant filming individuals while in motion, for which the Inria researchers relied heavily on the Kinovis platform, a 100 m² experimental space featuring 68 HD cameras that was ideally suited to this project. “There were data sets on the movement of clothing with minimalist outfits, fitting closely to the body”, explains Laurence Boissieux, an engineer with the SED. “But having such a space enabled us to gather far more extensive data with bigger movements.” 

Work began on image capture in January 2022. Combining different outfits and movements, 77 sequences were filmed for each subject, with half a day spent on each individual. This was then followed by a week of processing. “On the one hand we had to get white silhouettes on a black background by removing the green screen”, explains Laurence Boissieux. “Sometimes this meant correcting image fragments containing errors by hand, such as when someone was wearing too bright a colour, meaning it reflected the green of the background and rendered part of their outfit invisible. Images then had to be extracted from the video footage, at a rate of 50 images/second for each camera.”  

©Kinovis

Drawing on these two sources of data, algorithms were then used to recreate the shapes of the bodies and the clothes in movement at each moment in time (50 Hz). It took seven months to process all of the images, after which they were finally able to move on to putting the data to use. “Our aim was to present our data to the scientific community, accompanied by a number of reference models developed using that data, including models of the body without clothes and models of the clothes without bodies”, explains Stefanie Wuhrer. 

Morpheo and Naver Labs spent two years on this task, with support from PhD students. The project came to an end in June 2023 with the publication of a paper announcing that the data had been published online on a website designed for that purpose, in addition to their first models. However, only the data on 18 individuals, 6 outfits and 10 movements were made available to the public. “We will use the data that we have kept hidden to test those models developed using the data that we have published”, explains Stefanie Wuhrer. “We will ask them to simulate the movements of hidden clothes and see if the results match up with reality.” 

Given the wide range of possible applications, there is a lot of potential for 4DHumanOutfit.  “One example is virtual fitting, with one model capable of transferring an item of clothing for one body shape to another”, explains Laurence Boissieux. “The issue is that the data has to fit on a smartphone or a website, and you need to be able to acquire the desired body shape without having access to 68 cameras!”  

Another potential project involving the use of 4DHumanOutfit data involves transferring clothes modelled on different body shapes in order to generate the flats needed to actually produce these clothes. Other possible avenues include the development of more realistic avatars for use in cinema, advertising or video games. 

“Our data will enable models to recreate the movement of clothing in a more realistic way, moving away from simulation and towards a reproduction of reality”, explains Laurence Boissieux. Naver Labs intends to use the data that has been generated to perfect its body shape recognition models for use in complex situations (linked to variations in clothing and movements). Four years after 4DHumanOutfit was launched, Stefanie Wuhrer feels satisfied when she looks back on what they have achieved: “It was a lengthy undertaking, but it was very rewarding. We are happy with the data set we published, which is one of a kind. It would never have been possible without Kinovis and without the three of us working together.”  

4DHumanOutfit in video

• Presenting the 4DHumanOutfit project, Kinovis
• The 4DHuman Outfit catalogue, Kinovis
• Modelling complex objects will soon be much easier with Elan, Inria, 4/29/2020