Rendering protein structures inside cells at the atomic level with Unreal Engine
Inventors & their inventions
Axial: https://linktr.ee/axialxyz
Axial partners with great founders and inventors. We invest in early-stage life sciences companies such as Appia Bio, Seranova Bio, Delix Therapeutics, Simcha Therapeutics, among others often when they are no more than an idea. We are fanatical about helping the rare inventor who is compelled to build their own enduring business. If you or someone you know has a great idea or company in life sciences, Axial would be excited to get to know you and possibly invest in your vision and company. We are excited to be in business with you — email us at info@axialvc.com
This paper presents a new approach to visualizing the intricate molecular world inside cells at an unprecedented level of detail and interactivity. By leveraging the power of the video game engine Unreal Engine 5 (UE5), the authors demonstrate the capability to render millions of biological macromolecules, such as proteins and lipids, at atomic resolution within their native cellular environment.
The motivation behind this work stems from the recent advancements in cryogenic electron tomography (CryoET), which has made it possible to identify and determine the structures of various macromolecules inside cells at near-atomic resolution. However, visualizing the complex cellular environment at the atomic level has remained a significant challenge due to the computational demands of rendering millions of molecules in real-time.
One of the main hurdles in cell visualization is the sheer number of triangles required to render protein structures at high resolution. For instance, while a high-quality mesh of a human body typically requires less than 100,000 triangles, rendering a single ribosome with a surface view at atomic resolution takes nearly 300,000 triangles. Even a small scene of a few proteins can contain tens of millions of triangles, exceeding the capacity of modern GPUs.
To overcome this challenge, the authors leverage Unreal Engine 5's new virtualized geometry system called Nanite. The Nanite system renders objects at different levels of detail automatically based on the distance of each object from the viewer, drastically reducing the number of triangles needed to display at any given moment. This allows for the rendering of billions of triangles simultaneously with consumer-level GPUs. The paper provides two compelling examples of biological systems rendered using this approach. The first example is a salmonella minicell infected by P22 bacteriophages, inspired by real CryoET tomograms. In this scene, the authors rendered 30 types of macromolecules with 580 different pieces of meshes, including the virus, bacterial flagella and pili, ribosomes, various chaperones, and proteins responsible for glycolysis and respiration processes. In total, the scene comprises more than 40,000 objects, composed of around 1.5 billion triangles, all rendered in real-time with a consumer GPU.
The second example explores the rendering of a piece of an eukaryotic cell, including parts of the Golgi apparatus and the chloroplast from a Chlamydomonas cell. In this scene, the authors focus on the processes of photosynthesis and carbon fixation, rendering the complex membrane systems and protein complexes involved. One of the main challenges in constructing this scene was the complexity of the membrane systems. The authors developed a programmatic approach to generate parametric descriptions of the membrane shapes and surface vectors, allowing them to piece together small square patches of lipids to build large organelles of various shapes. In the entire scene, which is roughly a 1μm cube in size, there are more than 20 million lipid molecules, each rendered at 3Å resolution, and more than 130,000 static mesh objects, comprising 11.6 billion triangles. To facilitate the rendering process, the authors developed tools for streamlined macromolecule placement within the game engine. They provide scripts that convert the location, orientation, and conformation of proteins determined from CryoET datasets into the game engine's coordinate system, allowing for automatic object placement. Additionally, they developed protocols for the placement of protein filaments with helical repeating patterns and membrane proteins within the lipid bilayers.
One of the key features of the paper is the integration of interactive tools for cell visualization. The authors developed in-game blueprints that enhance user control and interfaces, allowing users to navigate the complex cellular scene and annotate macromolecular features. These tools include movement control with mouse and keyboard, toggling pass-through-wall mode, and annotating objects by displaying their descriptions on the screen. Users can also "shoot" and destroy the first piece of a molecular model at the center of the view, providing a convenient way to "peel open" complex structures and reveal their inner workings.
The authors also discuss the display of protein movement within the game engine. While the current version of UE5 only supports static meshes, they present alternative approaches to rendering protein motion, such as modeling the rigid body motion of different molecular parts or creating blueprint actors that include the mesh objects and programming their movement. One of the limitations discussed in the paper is the lack of knowledge about the conformational dynamics of proteins at atomic resolution and the time scale at which these changes occur. The authors acknowledge that while the structures of ribosomes with and without tRNA binding are known, it is unclear how the tRNA is inserted into its site without colliding with other parts of the structure. To address this limitation, the authors are working on combining the UE5-based rendering system with heterogeneity analysis methods for CryoEM/CryoET, which could allow for the rendering of protein movement and conformational changes at atomic level in the future.
The paper also discusses future improvements and directions for this work. These include better integration with virtual reality (VR) devices, automatic membrane construction from tomogram segmentation, more realistic protein-lipid interactions, and the incorporation of coarse-grain simulation methods for large membrane protein complexes. Additionally, the authors highlight the potential of this interactive rendering approach as an educational tool for cell and molecular biology, inspiring the next generation of scientists in these fields. Overall, this paper presents a significant advancement in the visualization of the molecular world inside cells. By leveraging the power of a video game engine, the authors have demonstrated the capability to render complex biological scenes with millions of molecules at atomic resolution, overcoming the computational limitations of traditional visualization software. The interactive tools developed in this work provide an intuitive and comprehensive view of biological systems, enabling researchers to gain deeper insights into the interactions between macromolecules and the mechanisms of biological processes.
While there are still limitations and areas for improvement, this work paves the way for a new era of cell visualization, where researchers can immerse themselves in the microscopic world and explore the intricate details of cellular machinery with unprecedented clarity and interactivity. The potential applications of this approach extend beyond research, as it offers exciting opportunities for education and public outreach, allowing students and the general public to experience the marvel of the molecular world in a truly captivating and engaging way.
https://www.biorxiv.org/content/10.1101/2023.12.08.570879v1.full
