Post-doctoral position (m/f/d): Volumetric Video Capture
Post-doctoral position (m / f / d) Volumetric Video Capture with RGB-D cameras
Summary of the research themes :
- Edge computing system,
- Graphic Quality Improvement,
- Volumetric Video Data encoding and compression
Looking forward to welcome you on our team for project ITHACA !
ITHACA : Immersion and Training with Holograms for Art, Craft and Audiovisual
Funded by the French State (France2030 program),
co-funded by the European Commission (Creative Europe).
Keywords : #volumetricvideo #virtualreality #3Dinteraction ; edge computing, optimization algorithms, gesture training...
1. Context
The production of volumetric videos opens up new markets in various fields such as immersive storytelling, video games, digital training in industry, crafts, and healthcare.
Despite being very costly, it remains limited to large-scale productions. The goal of ITHACA is to develop capture software based on RGB-D technology (depth cameras) and a post-production platform for affordable volumetric video production.
2. Research
A European consortium of four complementary entities :
- French Touch Factory, XR and Volumetric Video specialist,
- Research laboratory Center for Robotics Mines Paris - Virtual and Augmented Reality,
- Fraunhofer IPK (Germany),
- and Sergio Ochoa VFX Studio (Spain).
For the past year, the consortium has been developing a series of capture, post-production, fusion, and data export tools for volumetric video at 30 frames per second (fps).
The success of the project relies on the use of algorithms leveraging the GPU of various devices (PCs, PC VR, standalone MR headsets, Androids, etc.).
Summary : The processing steps for volumetric data are as follows :
1. Capture : Utilize capture devices to record volumetric video data.
Transitioning from a wired camera system connected to one or more master PCs to an edge computing approach based on Jetson Nanos.
By using for instance the new version of cameras ORBBEC Femto Mega or Bolt) 1
2. Registration : Align and synchronize multiple data sources or viewpoints to ensure spatial coherence.
Transitioning from traditional registration algorithms like Iterative Closest Point (ICP) to adaptive algorithms or those utilizing machine learning. 2
3. Post-production : Apply editing and enhancement techniques to the registered data.
Examples of Point Cloud Filters can be found in the point cloud library (PCL) 3 . AI can also be used to reach a better level of enhancement.
4. Fusion : Integrate the registered data sources to create a cohesive volumetric scene. Transitioning from simple point cloud fusion algorithms to more advanced methods. 4
5. Display : Present the volumetric content to users through suitable display technologies.
Taking advantage of new display methods such as Neural Radiance Fields (NeRF) or Gaussian splats for real-time hologram display. 5
6. Export : Prepare the processed data for use in various applications or platforms. Implementing compression formats for meshes and textures sequences. 6
7. Live Streaming : Enable real-time transmission of volumetric content for immediate viewing or interaction. Improving the encoding of volumetric data for live streaming and storage. 7
The R&D position involves at least one of these steps based on the doctor's interests and specific skills. Each point has already been studied in the state of the art, and various approaches tailored to specific use cases have been experimented with in the past, both by teams at French Touch Factory or its competitors worldwide, as well as by scientists.
The candidate with a PhD will conduct their research in close collaboration with the teams at French Touch Factory, applying it to real-world cases in sectors such as narrative XR, live performance, video gaming, digital training in crafts, industry, and healthcare many of which involve partners of ITHACA.
The goal is to test and enhance approaches already identified in the state of the art, ensuring they meet genuine business needs and specific use cases.
3. Requirements
- PhD in Computer Science, Virtual or Augmented Reality, Spatial Computing or similar relevant fields
- Strong knowledge of programming languages, C / C++, C# on Ubuntu environment and Windows, Python, Pytorch
- Motivation for working in a multidisciplinary research project at the crossroad between optimization,
computer science and 3D user interfaces
- A good level in English language (written and spoken)
- Additional knowledge in the following areas would be appreciated : game engine programming (Unity, Unreal), machine learning, shaders, conducting user studies, UX design
4. Hiring conditions
- 24-month full-time post-doctoral contract
- The successful candidate will be offered a salary commensurate to CV
- Start date : between May 2024 and January 1st 2025
- Location : Paris&Co startup incubator, 157 boulevard Macdonald, 75019, Paris, France
- Telecommuting up to 2 days a week
- 30 days of vacation from the first year of collaboration
- Refill of 75% of the subscription to the public transport
- Stimulating innovation ecosystem (startups, students, researchers)
5. How to apply
Candidates are required to send their applications including a CV with a publication list, a brief motivation letter explaining interests and qualifications regarding the position, prior research / work experience - all within a single pdf file - and contact information of two professional references to f.
Application deadline : 15 / 05 / 2024. Feel free to contact us for any questions regarding the position.
References
1 https : / / shop.orbbec3d.com / Femto-Mega
2 Colored Point Clouds Registration by Park 2017 : https : / / openaccess.thecvf.com / content ICCV 2017 / papers / Park Colored Point Cloud ICCV 2017 paper.
pdf RoCNet : 3D Robust Registration of Point-Clouds using Deep Learning : https : / / arxiv.org / pdf / 2303.07963
3 Point Cloud Library filtering : https : / / pointclouds.org / documentation / group filters.html
4 Function 4D approach : http : / / www.liuyebin.com / Function4D / assets / Function4D.pdf
5 Dynamic Gaussian splats : https : / / dynamic3dgaussians.github.io / ?fbclid IwAR0fTRPm5qYrgO77THyMZximTZwGqpZBpsktL1-jkk4lM6HUx9ueOYe Wm1w
Bullet time with NeRF : https : / / labs.laan.com / blogs / slow-motion-bullet-time-with-nerf-neural-radiance-fields /
6 WaveLet-based mesh sequences compression : https : / / webusers.i3s.unice.fr / fpayan / data / publications / Payan ICMI 2005.pdf
7 Mpeg Immersive Video - MIV : https : / / mpeg-miv.org /
Post-doctoral position (m/f/d): Volumetric Video Capture
Post-doctoral position (m/f/d) Volumetric Video Capture with RGB-D cameras. The goal of ITHACA is to develop capture software based on RGB-D technology (depth cameras) and a post-production platform for affordable volumetric video production. For the past year, the consortium has been developing a s...
Post-Doctoral Position H/F
The post-doctoral research will give a better understanding of the influence of dopants on local chemistry, electronic structure, phase composition, and their effects on material and ferroelectric parameters, including recrystallization temperature and remanent polarization. ...
Post-Doctoral Research Visit F/M Postdoctoral position in Quantum Information Theory
Post-Doctoral Research Visit F/M Postdoctoral position in Quantum Information Theory. The position will be funded through a QuantERA project involving a large european consortium: see the project website. Professional equipment available (videoconferencing, loan of computer equipment, etc. ...
Post-Doctoral open position
Genopole Thematic Incentive Actions) funding, ISTEM recruits a postdoctoral fellow as part of a research program entitled « From the biology of melanocytes to the development of innovative biotherapies for the treatment of skin pigmentation disorders by using human pluripotent stem cells« . ...
Post-doctoral position in Bayesian Causal Graph and uncertainty quantification – Application to the
This postdoctoral position is part of a collaboration between various CEA teams (LIAD, NRX – Nanostruc-tures and X-Rays Team in Grenoble), the university of Lorraine and the European Synchrotron (ESRF), eachbringing distinct skills such as AI, beamline physics, and instrumentation. ...
Post-Doctoral Research Visit F/M Post-doctoral position in mathematical and/or computational biology: Multiscale modeling of single cell-based dynamics of ovarian development
The postdoc position is open in the framework of the AI4scMED ("MultiScale AI for SingleCell-Based Precision Medicine") consortium (. The postdoc fellow is expected to use and develop the computational platform Simuscale (Bernard et al 2024) which enables one to simulate both intra- and extra-cellul...
Post-doctoral position in microbiota-host crosstalk in the early live
A post-doctoral position in microbiota-host crosstalk in the early live funded by collaborative project INRAE (National Research Institute for Agriculture, Food and Environment) –PiLeJe laboratory is available . The postdoc position will be focused in early life probiotic development. ...
Clinical Research Associate ( All Levels)
As a Clinical Research Associate at ICON, you’ll work within a large-scale, fast-paced environment alongside a close-knit team of highly qualified CRAs to identify, select, initiate and close-out investigational sites for clinical studies in phases II - IV, ensuring adherence to applicable regulatio...
Alternance - Data Scientist H/F - Paris
Informer régulièrement l’équipe Data Science et les autres équipes Data des progrès réalisés. Votre objectif est de créer des infrastructures de gestion de données robustes et adaptées, de les alimenter, de développer des algorithmes d’intelligence artificielle, de les mettre en production, mais aus...
Data Scientist - IA Défense F/H
Thales Digital Factory s’inscrit dans le programme de transformation de Thales qui a pour ambition de devenir un acteur incontournable et exemplaire du digital, et ce dans l’ensemble de ses marchés.Notre mission consiste à accélérer la transition numérique du Groupe Thales en s'aventurant dans de no...