The Spatial AI Lab is part of the Applied Sciences Group, a Microsoft research and development organization dedicated to creating next-generation human-computer interaction technologies leveraging the most recent AI developments and exploring new hardware capabilities and device form-factors.
Our team of scientists and engineers has strong expertise in computer vision and multi-modal AI, with a particular focus on spatial and embodied AI. We work on integrating AI capabilities in Microsoft products, ranging from new AI features in Windows applications, over core AI developments for the Windows platform, to exploring wearable form-factors and autonomous agents.
Founded in 2018 in Zurich, our lab is led by Marc Pollefeys, Professor of Computer Science at ETH Zurich, and serves as a hub for a strategic partnership between Microsoft and ETH.
Windows and AI
Our team helps develop multimodal foundation models, multimodal embeddings and generative AI models and incorporate this in Microsoft applications. Our expertise includes image and video analysis and generation, scalable training infrastructure, the training of models and deployment in quantized form to edge devices like Copilot+ PCs.
Spatial and Embodied AI
By integrating spatial and physical awareness into foundational models, we equip AI to understand and interact effectively with the real world, helping Copilot answer questions about the physical world but also enable embodied agents and robots to performs tasks.
Mixed Reality – HoloLens & VPS
Our team made fundamental contributions to Microsoft Mixed Reality and HoloLens (opens in new tab) ranging from object anchoring to Moving Platform (opens in new tab). We also co-developed Microsoft’s first cloud visual positioning system (VPS) for AR, known as Azure Spatial Anchors (opens in new tab). Designed to extend HoloLens’s mapping and localization capabilities it enabled shared experiences across multiple devices targeting industrial scenarios (opens in new tab).
The localization service also powered massively multiplayer outdoor AR phone games such as Minecraft Earth (opens in new tab). Today, the VPS system continues to support our Spatial AI and Robotics research.
Collaborations/Microsoft & ETH
Our lab maintains a strategic partnership with ETH Zurich. We work particularly closely with the Computer Vision and Geometry (CVG) group (opens in new tab). It provides ETH PhD, Master’s, and Bachelor’s students with the opportunity to work directly with Microsoft, gaining hands-on experience, contributing to diverse projects, and benefiting from mentorship.
