Jamie Moffa, host of In Plain English; Konrad Kording, Kaela Singleton and Arjun Raj
One of the pillars of science is the idea that experimental results can be replicated. If they cannot be reproduced, what if the findings of an experiment were due just to chance? Over the last two decades, a growing chorus of scientists has raised concerns about the “reproducibility crisis,” in which many published research findings can’t be independently validated, calling into question the rigor of contemporary science.
Two years ago, a group led by Konrad Kording, a Penn Integrates Knowledge Professor in Bioengineering and Neuroscience, founded the Community For Rigor (C4R) to build a grassroots movement to improve the rigor of scientific research.
Supported by a grant from the National Institutes of Health (NIH) and partners at Harvard, Duquesne, Smith College and Johns Hopkins, among other institutions, C4R creates educational materials that teach the principles of rigorous research, from data collection to pre-registration of results. “Everyone has done wrong things,” says Kording. “We’re all making these mistakes and we need to be able to talk about it.”
Last month, Kording appeared on In Plain English, a podcast devoted to making science more accessible, alongside Kaela Singleton, the co-founder and President of Black in Neuro; Arjun Raj, Professor in Bioengineering in Penn Engineering and in Genetics in Penn Medicine; and Jamie Moffa, a physician-scientist in training at Washington University in St. Louis, to discuss scientific rigor, including actionable strategies for students and faculty alike.
The conversation touched on everything from successfully managing the reams of data produced by experiments to the power of community to drive cultural change, as well as the difficulty of filtering useful feedback from the noise of social media. “I hope we can get to a point where people feel comfortable sharing what’s working and what’s not working,” says Raj.
Diagram of the optoPlateReader, a high-throughput, feedback-enabled optogenetics and spectroscopy device initially developed by Penn 2021 iGEM team.
For bioengineers today, light does more than illuminate microscopes. Stimulating cells with light waves, a field known as optogenetics, has opened new doors to understanding the molecular activity within cells, with potential applications in drug discovery and more.
Thanks to recent advances in optogenetic technology, much of which is cheap and open-source, more researchers than ever before can construct arrays capable of running multiple experiments at once, using different wavelengths of light. Computing languages like Python allow researchers to manipulate light sources and precisely control what happens in the many “wells” containing cells in a typical optogenetic experiment.
However, researchers have struggled to simultaneously gather data on all these experiments in real time. Collecting data manually comes with multiple disadvantages: transferring cells to a microscope may expose them to other, non-experimental sources of light. The time it takes to collect the data also makes it difficult to adjust metabolic conditions quickly and precisely in sample cells.
Now, a team of Penn Engineers has published a paper in Communications Biology, an open access journal in the Nature portfolio, outlining the first low-cost solution to this problem. The paper describes the development of optoPlateReader (or oPR), an open-source device that addresses the need for instrumentation to monitor optogenetic experiments in real time. The oPR could make possible features such as automated reading, writing and feedback in microwell plates for optogenetic experiments.
Left to right: Will Benman, Gloria Lee, Saachi Datta, Juliette Hooper, Grace Qian, David Gonzalez-Martinez, and Lukasz Bugaj (with Max).
The paper follows up on the award-winning work of six University of Pennsylvania alumni — Saachi Datta, M.D. Candidate at Stanford School of Medicine; Juliette Hooper, Programmer Analyst in Penn’s Perelman School of Medicine; Gabrielle Leavitt, M.D. Candidate at Temple University; Gloria Lee, graduate student at Oxford University; Grace Qian, Drug Excipient and Residual Analysis Research Co-op at GSK; and Lana Salloum, M.D. Candidate at Albert Einstein College of Medicine — who claimed multiple prizes at the 2021 International Genetically Engineered Machine Competition (iGEM) as Penn undergraduates.
The International Genetically Engineered Machine Competition (or iGEM) is the largest synthetic biology community and the premiere synthetic biology competition for both university and high school students from around the world. Hundreds of interdisciplinary teams of students compete annually, combining molecular biology techniques and engineering concepts to create novel biological systems and compete for prizes and awards through oral presentations and poster sessions.
The optoPlateReader was initially developed by Penn’s 2021 iGEM team, combining a light-stimulation device with a plate reader. At the iGEM competition, the invention took home Best Foundational Advance (best in track), Best Hardware (best from all undergraduate teams), and Best Presentation (best from all undergraduate teams), as well as a Gold Medal Distinction and inclusion in the Top 10 Overall and Top 10 Websites lists. (Read more about the 2021 iGEM team on the BE Blog.)
The original iGEM project focused on the design, construction, and testing of the hardware and software that make up the oPR, the focus of the new paper. After iGEM concluded, the team showed that the oPR could be used with real biological samples, such as cultures of bacteria. This work demonstrated that the oPR could be applied to real research questions, a necessary precursor to publication, and that the device could simultaneously monitor and manipulate living samples.
The main application for the oPR is in metabolic production (such as the creation of pharmaceuticals and bio-fuels). The oPR is able to issue commands to cells via light but can also take live readings about their current state. In the oPR, certain colors of light cause cells to carry out different tasks, and optical measurements give information on growth rates and protein production rates.
In this way, the new device is able to support production processes that can adapt in real time to what cells need, altering their behavior to maximize yield. For example, if an experiment produces a product that is toxic to cells, the oPR could instruct those cells to “turn on” only when the population of cells is dense and “turn off” when the concentration of that product becomes toxic and the cellular population needs to recover. This ability to pivot in real time could assist industries that rely on bioproduction.
The main challenges in developing this device were in incorporating the many light emitting diodes (LEDs) and sensors into a tiny space, as well as insulating the sensors from the nearby LEDs to ensure that the measured light came from the sample and not from the instrument itself. The team also had to create software that could coordinate the function of nearly 100 different sets of LEDs and sensors. Going forward, the team hopes to spread the word about the open-source oPR to other researchers studying metabolic production to enable more efficient research.
Lukasz Bugaj, Assistant Professor in Bioengineering and senior author of the paper, served as the team’s mentor along with Brian Chow, formerly an Associate Professor in Bioengineering and a founding member of the iGEM program at MIT, and Jose Avalos, Associate Professor of Chemical and Biological Engineering at Princeton University.
Key to the project’s development was the guidance of Bioengineering graduate students Will Benman, David Gonzalez Martinez, and Gabrielle Ho, as well as that of Saurabh Malani, a graduate student at Princeton University.
Like space shuttles using booster rockets to breach the atmosphere, lipid nanoparticles (LNPs) equipped with the new molecule more successfully deliver medicinal payloads. (Love Employee via Getty Images)
Inspired by the design of space shuttles, Penn Engineering researchers have invented a new way to synthesize a key component of lipid nanoparticles (LNPs), the revolutionary delivery vehicle for mRNA treatments including the Pfizer-BioNTech and Moderna COVID-19 vaccines, simplifying the manufacture of LNPs while boosting their efficacy at delivering mRNA to cells for medicinal purposes.
In a paper in Nature Communications, Michael J. Mitchell, Associate Professor in the Department of Bioengineering, describes a new way to synthesize ionizable lipidoids, key chemical components of LNPs that help protect and deliver medicinal payloads. For this paper, Mitchell and his co-authors tested delivery of an mRNA drug for treating obesity and gene-editing tools for treating genetic disease.
Previous experiments have shown that lipidoids with branched tails perform better at delivering mRNA to cells, but the methods for creating these molecules are time- and cost-intensive. “We offer a novel construction strategy for rapid and cost-efficient synthesis of these lipidoids,” says Xuexiang Han, a postdoctoral student in the Mitchell Lab and the paper’s co-first author.
The bright white spots represent tiny clusters of proteins detected by CluMPS. (Photo by: Thomas Mumford)
Penn Engineers have pioneered a new way to visualize the smallest protein clusters, skirting the physical limitations of light-powered microscopes and opening new avenues for detecting the proteins implicated in diseases like Alzheimer’s and testing new treatments.
In a paper in Cell Systems, Lukasz Bugaj, Assistant Professor in Bioengineering, describes the creation of CluMPS, or Clusters Magnified by Phase Separation, a molecular tool that activates by forming conspicuous blobs in the presence of target protein clusters as small as just a few nanometers. In essence, CluMPS functions like an on/off switch that responds to the presence of clusters of the protein it is programmed to detect.
Normally, says Bugaj, detecting such clusters requires laborious techniques. “With CluMPS, you don’t need anything beyond the standard lab microscope.” The tool fuses with the target protein to form condensates orders of magnitude larger than the protein clusters themselves that resemble the colorful blobs in a lava lamp. “We think the simplicity of the approach is one of its main benefits,” says Bugaj. “You don’t need specialized skills or equipment to quickly see whether there are small clusters in your cells.”
How does the placenta keep harmful substances away from developing babies while still providing proper nutrition?
(Photo: Getty Images)
The exact mechanisms remain unknown, which is why the University of Pennsylvania, Rutgers University, Tulane University, the University of North Carolina at Chapel Hill and the University of Rochester have joined together to launch a research center dedicated to solving this mystery and ensuring healthy pregnancies.
A $5 million grant from the National Institutes of Health (NIH) will help fund the Integrated Transporter Elucidation Center (InTEC), which will operate from the Rutgers Biomedical Health Sciences campus in Piscataway under the leadership of Lauren Aleksunes, a professor of pharmacology and toxicology at Rutgers’ Ernest Mario School of Pharmacy and resident scientist in the Environmental and Occupational Health Sciences Institute (EOHSI).
“Since my time working as a community pharmacist, I have found the lack of high-quality information about the safety of everyday products on the health of a pregnancy frustrating,” says Aleksunes. “People need to know whether the chemicals in their diet, personal care products and medications can impact their babies. Our goal at InTEC is to better understand how these chemicals travel in and out of the placenta and if they can reach the baby and influence their development.”
Aleksunes will study how transporter proteins carrying nutrients, dietary supplements, medications and toxic chemicals work during pregnancies. Some of the work will test how individual placenta cells respond to various stimuli in the laboratory. Others on the team will examine how environmental factors influence placental transporters during healthy and unhealthy or complicated pregnancies.
Key to this work will be Dan Huh, Associate Professor in Bioengineering in Penn Engineering, who will lead a team with an innovative approach to modeling the transfer of molecules across the human placenta.
As a pioneer of organ-on-a-chip technology, the Huh group will use a novel microengineered system in which maternal tissue engineered from a layer of primary human trophoblasts is grown adjacent to a three-dimensional network of perfusable fetal blood vessels to mimic the human placental barrier. This microphysiological system will be employed as an in vitro platform to simulate and quantitatively analyze the exchange of various substances between maternal and fetal circulation without the need for laboratory animals or placenta explants.
Jina Ko (left) and Kevin Johnson (right), both from the School of Engineering and the Perelman School of Medicine with appointments in Bioengineering, have received the National Institute of Health Director’s Award to support their “highly innovative and broadly impactful” research projects through the High-Risk, High-Reward program.
The National Institutes of Health (NIH) has awarded grants to three researchers from the University of Pennsylvania through the NIH Common Fund’s High-Risk, High-Reward Research program. The research of Kevin B. Johnson, Jina Ko, and Sheila Shanmugan will be supported through the program, which funds “highly innovative and broadly impactful” biomedical or behavioral research by exceptionally creative scientists.
The High-Risk, High-Reward Research program catalyzes scientific discovery by supporting highly innovative research proposals that, due to their inherent risk, may struggle in the traditional peer-review process despite their transformative potential. Program applicants are encouraged to think “outside the box” and pursue trail-blazing ideas in any area of research relevant to the NIH’s mission to advance knowledge and enhance health.
Two Penn Bioengineering faculty, Johnson and Ko, are among 85 recipients for 2023.
Johnson, the David L. Cohen University Professor of Pediatrics, is a Penn Integrates Knowledge University Professor who holds appointments in the Department of Computer and Information Science in the School of Engineering and Applied Science and the Department of Biostatistics, Epidemiology, and Informatics in the Perelman School of Medicine. He also holds secondary appointments in Bioengineering, Pediatrics, and in the Annenberg School for Communication. He is widely known for his work with e-prescribing and computer-based documentation and, more recently, work communicating science to lay audiences, which includes a documentary about health-information exchange. Johnson has authored more than 150 publications and was elected to the American College of Medical Informatics, Academic Pediatric Society, National Academy of Medicine, International Association of Health Science Informatics, and American Institute for Medical and Biological Engineering.
Ko is an assistant professor in the Department of Pathology and Laboratory Medicine in the Perelman School of Medicine and Department of Bioengineering in the School of Engineering and Applied Science. She focuses on developing single molecule detection from single extracellular vesicles and multiplexed molecular profiling to better diagnose diseases and monitor treatment efficacy. Ko earned her Ph.D. in bioengineering at Penn in 2018, during which time she developed machine learning-based microchip diagnostics that can detect blood-based biomarkers to diagnose pancreatic cancer and traumatic brain injury. For her postdoctoral training, she worked at the Massachusetts General Hospital and the Wyss Institute at Harvard University as a Schmidt Science Fellow and a NIH K99/R00 award recipient. Ko developed new methods to profile single cells and single extracellular vesicles with high throughput and multiplexing.
The model of tubule packing developed by the Hughes Lab shows the tubules repelling each other and shifting around.
A recent study by Penn Bioengineering researchers sheds new light on the role of physics in kidney development. The kidney uses structures called nephrons and tubules to filter blood and pass urine to the bladder. Nephron number is set at birth and can vary over an order of magnitude (anywhere from 100,000 to over a million nephrons in an individual kidney). While the reasons for this variability remain unclear, low numbers of nephrons predispose patients to hypertension and chronic kidney disease.
Now, research published in Developmental Cell led by Alex J. Hughes, Assistant Professor in the Department of Bioengineering, demonstrates a new physics-driven approach to better visualize and understand how a healthy kidney develops to avoid organizational defects that would impair its function. While previous efforts have typically approached this problem using molecular genetics and mouse models, the Hughes Lab’s physics-based approach could link particular types of defects to this genetic information and possibly highlight new treatments to prevent or fix congenital defects.
Alex J. Hughes, Assistant Professor in Bioengineering
Louis Prahl, NIH F32 Postodctoral Fellow
During embryonic development, kidney tubules grow and the tips divide to make a branched tree with clusters of nephron stem cells surrounding each branch tip. In order to build more nephrons, the tree needs to grow more branches. To keep the branches from overlapping, the kidney’s surface grows more crowded as the number of branches increase. “At this point, it’s like adding more people to a crowded elevator,” says Louis Prahl, first author of the paper and Postdoctoral Fellow in the Hughes Lab. “The branches need to keep rearranging to accommodate more until organ growth stops.”
To understand this process, Hughes, Prahl and their team investigated branch organization in mouse kidneys as well as using computer models and a 3D printed model of tubules. Their results show that tubules have to actively restructure – essentially divide at narrower angles – to accommodate more tubules. Computer simulations also identified ‘defective’ packing, in which the simulation parameters caused tubules to either overlap or be forced beneath the kidney surface. The team’s experimentation and analysis of published studies of genetic mouse models of kidney disease confirmed that these defects do occur.
This study represents a unique synthesis of different fields to understand congenital kidney disease. Mathematicians have studied geometric packing problems for decades in other contexts, but the structural features of the kidney present new applications for these models. Previous models of kidney branching have approached these problems from the perspective of individual branches or using purely geometric models that don’t account for tissue mechanics. By contrast, The Hughes Lab’s computer model demonstrates the physics of how tubule families interact with each other, allowing them to identify ‘phases’ of kidney organization that either relate to normal kidney development or organizational defects. Their 3D printed model of tubules shows that these effects can occur even when one sets the biology aside.
Hughes has been widely recognized for his research in the understanding of kidney development. This new publication is the first fruit of his 2021 CAREER Award from the National Science Foundation (NSF) and he was recently named a 2023 Rising Star by the Cellular and Molecular Bioengineering (CMBE) Special Interest Group. In 2020 he became the first Penn Engineering faculty member to receive the Maximizing Investigators’ Research Award (MIRA) from the National Institutes of Health (NIH) for his forward-thinking work in the creation of new tools for tissue engineering.
Pediatric nephrologists have long worked to understand the cause of these childhood kidney defects. These efforts are often confounded by a lack of evidence for a single causative mutation. The Hughes Lab’s approach presents a new and different application of the packing problem and could help answer some of these unsolved questions and open doors to prevention of these diseases. Following this study, Hughes and his lab members will continue to explore the physics of kidney tubule packing, looking for interesting connections between packing organization, mechanical stresses between neighboring tubule tips, and nephron formation while attempting to copy these principles to build stem cell derived tissues to replace damaged or diseased kidney tissue. Mechanical forces play an important role in developmental biology and there is much scope for Hughes, Prahl and their colleagues to learn about these properties in relation to the kidney.
Other authors include Bioengineering Ph.D. students and Hughes Lab members John Viola and Jiageng Liu.
This work was supported by NSF CAREER 2047271, NIH MIRA R35GM133380, Predoctoral Training Program in Developmental Biology T32HD083185, and NIH F32 fellowship DK126385.
Eight researchers from the Perelman School of Medicine have received research grants designed to invest in high-risk, high-reward projects.
Bushra Raj, Assistant Professor of Cell and Developmental Biology in the Perelman School of Medicine and member of the Penn Bioengineering Graduate Group, was one of three Penn winners of the NIH Director’s New Innovator Award for independent projects developed by early-career investigators. More additional Penn scientists who received NIH Director’s Transformative Research Award for a project focusing on cancer research.
Raj’s project focuses on “testing a novel technology that uses CRISPR/Cas gene-editing tools to genomically record inputs from two signaling pathways in the developing zebrafish brain.”
Established in 2009, the Transformative Research Award promotes cross-cutting, interdisciplinary science and is open to individuals and teams of investigators who propose research that could potentially create or challenge existing paradigms.
Neuroscientists frequently say that neural activity ‘represents’ certain phenomena, PIK Professor Konrad Kording and postdoc Ben Baker led a study that took a philosophical approach to tease out what the term means.
Neuroscientists use the word “represent” to encompass multifaceted relationships between brain activity, behavior, and the environment.
One of neuroscience’s greatest challenges is to bridge the gaps between the external environment, the brain’s internal electrical activity, and the abstract workings of behavior and cognition. Many neuroscientists rely on the word “representation” to connect these phenomena: A burst of neural activity in the visual cortex may represent the face of a friend or neurons in the brain’s memory centers may represent a childhood memory.
But with the many complex relationships between mind, brain, and environment, it’s not always clear what neuroscientists mean when they say neural activity “represents” something. Lack of clarity around this concept can lead to miscommunication, flawed conclusions, and unnecessary disagreements.
To tackle this issue, an interdisciplinary paper takes a philosophical approach to delineating the many aspects of the word “representation” in neuroscience. The work, published in Trends in Cognitive Sciences, comes from the lab of Konrad Kording, a Penn Integrates Knowledge University Professor and senior author on the study whose research lies at the intersection of neuroscience and machine learning.
“The term ‘representation’ is probably one of the most common words in all of neuroscience,” says Kording, who has appointments in the Perelman School of Medicine and School of Engineering and Applied Science. “But it might mean something very different from one professor to another.”
Also coauthor on the paper is Benjamin Lansdell, a data scientist in the Department of Developmental Neurobiology at St. Jude Children’s Hospital and former postdoctoral researcher in the Kording lab.
Funding for this study came from the National Institutes of Health (awards 1-R01-EB028162-01 and R01EY021579) and the University of Pennsylvania Office of the Vice Provost for Research.
In 2005, John Ioannidis published a bombshell paper titled “Why Most Published Research Findings Are False.” In it, Ioannidis argued that a lack of scientific rigor in biomedical research — such as poor study design, small sample sizes and improper assessment of the significance of data— meant that a large percentage of experiments would not return the same results if they were conducted again.
Since then, researchers’ awareness of this “replication crisis” has grown, especially in fields that directly impact the health and wellbeing of people, where lapses in rigor can have life-or-death consequences. Despite this attention and motivation, however, little progress has been made in addressing the roots of the problem. Formal training in rigorous research practices remains rare; while mentors advise their students on how to properly construct and conduct experiments to produce the most reliable evidence, few educational resources exist to support them.
Konrad Kording, a Penn Integrates Knowledge Professor with appointments in the Departments of Bioengineering and Computer and Information Science in Penn Engineering and the Department of Neuroscience in Penn’s Perelman School of Medicine, has been awarded one of the initiative’s first five grants.
“The replication crisis is real,” says Kording. “I’ve tried to replicate the research of others and failed. I’ve reanalyzed my own data and found major mistakes that needed to be corrected. I was never properly taught how to do rigorous science, and I want to improve that for the next generation.”
