hero mantel

Reimagining the Future of Human Interaction

Innovative research to help build a civil, safe, and creative platform of shared experiences.

View Publications

Our Mission and Vision

Roblox Research informs how we accelerate innovation. Across distributed computing, natural language processing, artificial intelligence 3D content creation tools, data analytics, and interactive design we help to empower the next generation of shared experiences through scientific research.

VIEW CAREER OPPORTUNITIES

Recent Highlights

Roblox Tech Talk on Research Podcast

A discussion of our research vision between CEO David Baszucki and Chief Scientist Morgan McGuire.

Large-Scale Experimentation at Roblox

An overview of Data Science and Analytics team’s experimentation principles by Jay Martin, Boris Shang, Vincent Su, Kelly Cheng, and Xiaochen Zhang.

Publications

Iñaki Navarro, Dario Kneubuehler, Tijmen Verhulsdonck, Eloi du Bois, William Welch, Vivek Verma, Ian Sachs, Kiran Bhat, ACM SIGGRAPH 2021 Talk

Real time facial animation for virtual 3D characters has important applications such as AR/VR, interactive 3D entertainment, pre-visualization and video conferencing. Yet despite important research breakthroughs in facial tracking and performance capture, there are very few commercial examples of real-time facial animation applications in the consumer market. Mass adoption requires realtime performance on commodity hardware and visually pleasing animation that is robust to real world conditions, without requiring manual calibration. We present an end-to-end deep learning framework for regressing facial animation weights from video that addresses most of these challenges. Our formulation is fast (3.2 ms), utilizes images of real human faces along with millions of synthetic rendered frames to train the network on real-world scenarios, and produces jitter-free visually pleasing animations.

View Here

Quoc Le and Kip Kaehler, Data + AI Summit 2021

Machine learning is a key part of our ability to scale important services to our massive community. In this talk, we share our journey of scaling our deep learning text classifiers to process 50k+ requests per second at latencies under 20ms. We will share how we were able to not only make BERT fast enough for our users, but also economical enough to run in production at a manageable cost on CPU.

Zander Majercik (NVIDIA), Thomas Muller (NVIDIA), Alexander Keller (NVIDIA), Derek Nowrouzezahrai (McGill), Morgan McGuire (Roblox and McGill), ACM SIGGRAPH 2021 Talk

Interactive global illumination remains a challenge in radiometrically- and geometrically-complex scenes. Specialized sampling strategies are effective for specular and near-specular transport because the scattering has relatively low directional variance per scattering event. In contrast, the high variance from transport paths comprising multiple rough glossy or diffuse scattering events remains notoriously difficult to resolve with a small number of samples. We extend unidirectional path tracing to address this by combining screen-space reservoir resampling and sparse world-space probes, significantly improving sample efficiency for transport contributions that terminate on diffuse scattering events. Our experiments demonstrate a clear improvement — at equal time and equal quality — over purely path traced and purely probe-based baselines. Moreover, when combined with commodity denoisers, we are able to interactively render global illumination in complex scenes.

Download Article

Jack Buckley, Laura Colosimo, Rebecca Kantar, Marty McCall and Erica Snow, OECD Digital Education Outlook 2021

This chapter discusses how recent advancements in digital technology could lead to a new generation of game-based standardised assessments in education, providing education systems with assessments that can test more complex skills than traditional standardised tests can. After highlighting some of the advantages of game-based standardised assessment compared to traditional ones, this chapter discusses how these tests are built, how they work, but also some of their limitations. While games have strong potential to improve the quality of testing and expand assessment to complex skills in the future, they will likely supplement traditional tests, which also have their advantages. Three examples of game-based assessments integrating a range of advanced technologies illustrate this perspective.

View Here

Aditya Jonnalagadda (University of California, Santa Barbara), Iuri Frosio (NVIDIA), Seth Schneider (NVIDIA), Morgan McGuire (NVIDIA; now at Roblox), and Joohwan Kim (NVIDIA)

Game publishers and anti-cheat companies have been unsuccessful in blocking cheating in online gaming. We propose a novel, vision-based approach that captures the final state of the frame buffer and detects illicit overlays. To this aim, we train and evaluate a DNN detector on a new dataset, collected using two first-person shooter games and three cheating software. We study the advantages and disadvantages of different DNN architectures operating on a local or global scale. We use output confidence analysis to avoid unreliable detections and inform when network retraining is required. In an ablation study, we show how to use Interval Bound Propagation to build a detector that is also resistant to potential adversarial attacks and study its interaction with confidence analysis. Our results show that robust and effective anti-cheating through machine learning is practically feasible and can be used to guarantee fair play in online gaming.

View Here

Lily Brown, Andy Friesen, and Alan Jeffery, Human Aspects of Types and Reasoning Assistants 2021

Luau is the scripting language that powers user-generated experiences on the Roblox platform. It is a statically-typed language, based on the dynamically-typed Lua language, with type inference. These types are used for providing editor assistance in Roblox Studio, the IDE for authoring Roblox experiences. Due to Roblox’s uniquely heterogeneous developer community, Luau must operate in a somewhat different fashion than a traditional statically-typed language. In this paper, we describe some of the goals of the Luau type system, focusing on where the goals differ from those of other type systems.

Download Article

Michael Stengel (NVIDIA), Zander Majercik (NVIDIA), Benjamin Boudaoud (NVIDIA), Morgan McGuire (NVIDIA; now at Roblox), ACM Multimedia Systems Conference 2021

We present a networked, high-performance graphics system that combines dynamic, high-quality, ray traced global illumination computed on a server with direct illumination and primary visibility computed on a client. This approach provides many of the image quality benefits of real-time ray tracing on low-power and legacy hardware, while maintaining a low latency response and mobile form factor.

As opposed to streaming full frames from rendering servers to end clients, our system distributes the graphics pipeline over a network by computing diffuse global illumination on a remote machine. Diffuse global illumination is computed using a recent irradiance volume representation combined with a new lossless, HEVC-based, hardware-accelerated encoding, and a perceptually-motivated update scheme.

Our experimental implementation streams thousands of irradiance probes per second and requires less than 50 Mbps of throughput, reducing the consumed bandwidth by 99.4% when streaming at 60 Hz compared to traditional lossless texture compression.

The bandwidth reduction achieved with our approach allows higher quality and lower latency graphics than state-of-the-art remote rendering via video streaming. In addition, our split-rendering solution decouples remote computation from local rendering and so does not limit local display update rate or display resolution.

Download Article

Towaki Takikawa (University of Toronto, Vector Institute, and NVIDIA), Joey Litalien (NVIDIA and McGill), Kangxue Yin (NVIDIA), Karsten Kreis (NVIDIA), Charles Loop (NVIDIA), Derek Nowrouzezahrai (McGill), Alec Jacobson (University of Toronto), Morgan McGuire (McGill and NVIDIA; now at Roblox), Sanja Fidler (University of Toronto, Vector Institute, and NVIDIA), IEEE Computer Vision and Pattern Recognition 2021

“Neural signed distance functions (SDFs) are emerging as an effective representation for 3D shapes. SDFs encode 3D surfaces with a function of position that returns the closest distance to a surface. State-of-the-art methods typically encode the SDF with a large, fixed-size neural network to approximate complex shapes with implicit surfaces. Rendering these large networks is, however, computationally expensive since it requires many forward passes through the network for every pixel, making these representations impractical for real-time graphics applications.

We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs, while achieving state-of-the-art geometry reconstruction quality. We represent implicit surfaces using an octree-based feature volume which adaptively fits shapes with multiple discrete levels of detail (LODs), and enables continuous LOD with SDF interpolation. We further develop an efficient algorithm to directly render our novel neural SDF representation in real-time by querying only the necessary LODs with sparse octree traversal. We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works. Furthermore, it produces state-of-the-art reconstruction quality for complex shapes under both 3D geometric and 2D image-space metrics.”

View Here

Mak Batty and Simon Cooksey (UKC), Alan Jeffrey (Roblox), Ilya Kaysin and Anton Podkopaev (JetBrains), James Riely (DePaul U)

Program logics and semantics tell a pleasant story about sequential composition: when executing (S1; S2), we first execute S1 then S2. To improve performance, however, processors execute instructions out of order, and compilers reorder programs even more dramatically. By design, single-threaded systems cannot observe these reorderings; however, multiple-threaded systems can, making the story considerably less pleasant. A formal attempt to understand the resulting mess is known as a “relaxed memory model.’’ Prior models either fail to address sequential composition directly, or overly restrict processors and compilers, or permit nonsense thin-air behaviors which are unobservable in practice.

To support sequential composition while targeting modern hardware, we enrich the standard event-based approach with preconditions and families of predicate transformers. When calculating the meaning of (S1;S2), the predicate transformer applied to the precondition of an event e from S2 is chosen based on the set of events in S1 upon which e depends. We apply this approach to two existing memory models.

Download Article

Join Us in Shaping the Future

Join Roblox’s Research Team to solve unique challenges to advance the metaverse through open scientific inquiry and collaborative exploration.

FIND YOUR NEXT JOB