Measuring Chaos: Using Machine Learning to Satisfy Our Need to Know

by

How do we measure chaos and why would we want to? Together, Penn engineers Dani S. Bassett, J. Peter Skirkanich Professor in Bioengineering and in Electrical and Systems Engineering, and postdoctoral researcher Kieran Murphy leverage the power of machine learning to better understand chaotic systems, opening doors for new information analyses in both theoretical modeling and real-world scenarios.

Humans have been trying to understand and predict chaotic systems such as weather patterns, the movement of planets and population ecology for thousands of years. While our models have continued to improve over time, there will always remain a barrier to perfect prediction. That’s because these systems are inherently chaotic. Not in the sense that blue skies and sunshine can turn into thunderstorms and torrential downpours in a second, although that does happen, but in the sense that mathematically, weather patterns and other chaotic systems are governed by physics with nonlinear characteristics. 

“This nonlinearity is fundamental to chaotic systems,” says Murphy. “Unlike linear systems, where the information you start with to predict what will happen at timepoints in the future stays consistent over time, information in nonlinear systems can be both lost and generated through time.”

Like a game of telephone where information from the original source gets lost as it travels from person to person while new words and phrases are added to fill in the blanks, outcomes in chaotic systems become harder to predict as time passes. This information decay thwarts our best efforts to accurately forecast the weather more than a few days out.

“You could put millions of probes in the atmosphere to measure wind speed, temperature and precipitation, but you cannot measure every single atom in the system,” says Murphy. “You must have some amount of uncertainty, which will then grow, and grow quickly. So while a prediction for the weather in a few hours might be fairly accurate, that growth in uncertainty over time makes it impossible to predict the weather a month from now.”

In their recent paper published in Physical Review Letters, Murphy and Bassett applied machine learning to classic models of chaos, physicists’ reproductions of chaotic systems that do not contain any external noise or modeling imperfections, to design a near-perfect measurement of chaotic systems to one day improve our understanding of systems including weather patterns. 

“These controlled systems are testbeds for our experiments,” says Murphy. “They allow us to compare with theoretical predictions and carefully evaluate our method before moving to real-world systems where things are messy and much less is known. Eventually, our goal is to make ‘information maps’ of real-world systems, indicating where information is created and identifying what pieces of information in a sea of seemingly random data are important.” 

Read the full story in Penn Engineering Today.

Looking to AI to Solve Antibiotic Resistance

by Nathi Magubane

Cesar de la Fuente (left), Fangping Wan (center), and Marcelo der Torossian Torres (right). Fangping holds a 3D model of a unique ATP synthase fragment, identified by their lab’s deep learning model, APEX, as having potent antibiotic properties.

“Make sure you finish your antibiotics course, even if you start feeling better’ is a medical mantra many hear but ignore,” says Cesar de la Fuente of the University of Pennsylvania.

He explains that this phrase is, however, crucial as noncompliance could hamper the efficacy of a key 20th century discovery, antibiotics. “And in recent decades, this has led to the rise of drug-resistant bacteria, a growing global health crisis causing approximately 4.95 million deaths per year and threatens to make even common infections deadly,” he says.

De la Fuente, a Presidential Assistant Professor, and a team of interdisciplinary researchers have been working on biomedical innovations tackling this looming threat. In a new study, published in Nature Biomedical Engineering, they developed an artificial intelligence tool to mine the vast and largely unexplored biological data—more than 10 million molecules of both modern and extinct organisms— to discover new candidates for antibiotics.

“With traditional methods, it takes around six years to develop new preclinical drug candidates to treat infections and the process is incredibly painstaking and expensive,” de la Fuente says. “Our deep learning approach can dramatically reduce that time, driving down costs as we identified thousands of candidates in just a few hours, and many of them have preclinical potential, as tested in our animal models, signaling a new era in antibiotic discovery.” César de la Fuente holds a 3D model of a unique ATP synthase fragment, identified by his lab’s deep learning model, APEX, as having potent antibiotic properties. This molecular structure, resurrected from ancient genetic data, represents a promising lead in the fight against antibiotic-resistant bacteria.

These latest findings build on methods de la Fuente has been working on since his arrival at Penn in 2019. The team asked a fundamental question: Can machines be used to accelerate antibiotic discovery by mining the world’s biological information? He explains that this idea is based on the notion that biology, at its most basic level, is an information source, which could theoretically be explored with AI to find new useful molecules.

Read the full story in Penn Today.

Largest-Ever Antibiotic Discovery Effort Uses AI to Uncover Potential Cures in Microbial Dark Matter

by Eric Horvath

Credit: Georgina Joyce

Almost a century ago, the discovery of antibiotics like penicillin revolutionized medicine by harnessing the natural bacteria-killing abilities of microbes. Today, a new study co-led by researchers at the Perelman School of Medicine at the University of Pennsylvania suggests that natural-product antibiotic discovery is about to accelerate into a new era, powered by artificial intelligence (AI).

The study, published in Cell, the researchers used a form of AI called machine learning to search for antibiotics in a vast dataset containing the recorded genomes of tens of thousands of bacteria and other primitive organisms. This unprecedented effort yielded nearly one million potential antibiotic compounds, with dozens showing promising activity in initial tests against disease-causing bacteria.

“AI in antibiotic discovery is now a reality and has significantly accelerated our ability to discover new candidate drugs. What once took years can now be achieved in hours using computers” said study co-senior author César de la Fuente, PhD, a Presidential Assistant Professor in Psychiatry, Microbiology, Chemistry, Chemical and Biomolecular Engineering, and Bioengineering.

Nature has always been a good place to look for new medicines, especially antibiotics. Bacteria, ubiquitous on our planet, have evolved numerous antibacterial defenses, often in the form of short proteins (“peptides”) that can disrupt bacterial cell membranes and other critical structures. While the discovery of penicillin and other natural-product-derived antibiotics revolutionized medicine, the growing threat of antibiotic resistance has underscored the urgent need for new antimicrobial compounds.

In recent years, de la Fuente and colleagues have pioneered AI-powered searches for antimicrobials. They have identified preclinical candidates in the genomes of contemporary humans, extinct Neanderthals and Denisovans, woolly mammoths, and hundreds of other organisms. One of the lab’s primary goals is to mine the world’s biological information for useful molecules, including antibiotics.

Read the full story in Penn Medicine News.

Artificial Intelligence to Accelerate Antibiotic Discovery

Using AI for discovery of new antibiotics.

The growing threat of antimicrobial resistance demands innovative solutions in drug discovery. Scientists are turning to artificial intelligence (AI) and machine learning (ML) to accelerate the discovery and development of antimicrobial peptides (AMPs). These short strings of amino acids are promising for combating bacterial infections, yet transitioning them into clinical use has been challenging. Leveraging novel AI-driven models, researchers aim to overcome these obstacles, heralding a new era in antimicrobial therapy.

A new article in Nature Reviews Bioengineering illuminates the promises and challenges of using AI for antibiotic discovery. Cesar de la Fuente, Presidential Assistant Professor in Microbiology and Psychiatry in the Perelman School of Medicine, in Bioengineering and Chemical and Biomolecular Engineering in the School of Engineering and Applied Science, and Adjunct Assistant Professor in Chemistry in the School of Arts and Sciences, collaborated with James J. Collins, Termeer Professor of Medical Engineering and Science at MIT, to provide an introduction to this emerging field, outlining both its current limitations and its massive potential.

In the past five years, groundbreaking work in the de la Fuente Lab has dramatically accelerated the discovery of new antibiotics, reducing the timeline from years to mere hours. AI-driven approaches employed in his laboratory have already yielded numerous preclinical candidates, showcasing the transformative potential of AI in antimicrobial research and offering new potential solutions against currently untreatable infections.

Recent advancements in AI and ML are revolutionizing drug discovery by enabling the precise prediction of biomolecular properties and structures. By training ML models on high-quality datasets, researchers can accurately forecast the efficacy, toxicity and other crucial attributes of novel peptides. This predictive power expedites the screening process, identifying promising candidates for further evaluation in a fraction of the time required by conventional methods.

Traditional approaches to AMP development have encountered hurdles such as toxicity and poor stability. AI models help overcome these challenges by designing peptides with enhanced properties, improving stability, efficacy and safety profiles, and fast-tracking the peptides’ clinical application.

While AI-driven drug discovery has made significant strides, challenges remain. The availability of high-quality data is a critical bottleneck, necessitating collaborative efforts to curate comprehensive datasets to train ML models. Furthermore, ensuring the interpretability and transparency of AI-generated results is essential for fostering trust and wider adoption in clinical settings. However, the future is promising, with AI set to revolutionize antimicrobial therapy development and address drug resistance.

Integrating AI and ML into antimicrobial peptide development marks a paradigm shift in drug discovery. By harnessing these cutting-edge technologies, researchers can address longstanding challenges and accelerate the discovery of novel antimicrobial therapies. Continuous innovation in AI-driven approaches is likely to spearhead a new era of precision medicine, augmenting our arsenal against infectious diseases.

Read “Machine learning for antimicrobial peptide identification and design” in Nature Reviews Bioengineering.

The de la Fuente Lab uses use the power of machines to accelerate discoveries in biology and medicine. The lab’s current projects include using AI for antibiotic discovery, molecular de-extinction, reprogramming venom-derived peptides to discover new antibiotics, and developing low-cost diagnostics for bacterial and viral infections. Read more posts featuring de la Fuente’s work in the BE Blog.

Herman P. Schwan Distinguished Lecture: “Seeing the Unseen: How AI Redefines Bioengineering” (Dorin Comaniciu, Siemens Healthineers)

Dorin Comaniciu, Ph.D.

We hope you will join us for the 2023 Herman P. Schwan Distinguished Lecture by Dr. Dorin Comaniciu, hosted by the Department of Bioengineering.

Wednesday, December 13, 2023
1:00 PM ET
Location: Wu & Chen Auditorium (Levine 101)
The lecture and Q&A will be followed by a light reception in Levine Lobby.

Speaker: Dorin Comaniciu, Ph.D.
Senior Vice President
Artificial Intelligence and Digital Innovations
Siemens Healthineers

About Dorin Comaniciu:

Dr. Comaniciu serves as Senior Vice President for Artificial Intelligence and Digital Innovation at Siemens Healthineers. His scientific contributions to machine intelligence and computational imaging have translated to multiple clinical products focused on improving the quality of care, specifically in the fields of diagnostic imaging, image-guided therapy, and precision medicine.

Comaniciu is a member of the National Academy of Medicine, the Romanian Academy, and a Top Innovator of Siemens. He is a Fellow of the IEEE, ACM, MICCAI Society, and AIMBE, and a recipient of the IEEE Longuet-Higgins Prize for fundamental contributions to computer vision. Recent recognition of his work includes an honorary doctorate from Friedrich-Alexander University of Erlangen-Nuremberg.

He has co-authored 550 granted patents and 350 peer-reviewed publications that have received 61,000 citations, with an h-index of 102, in the areas of machine intelligence, medical imaging, and precision medicine.

A graduate of University of Pennsylvania’s Wharton School, Comaniciu received a doctorate in electrical and computer engineering from Rutgers University and a doctorate in electronics and telecommunications from Polytechnic University of Bucharest.

He is an advocate for technological innovations that save and enhance lives, addressing critical issues in global health.

About the Schwan Lecture:

The Herman P. Schwan Distinguished Lecture is in honor of one of the founding members of the Department of Bioengineering, who emigrated from Germany after World War II and helped create the field of bioengineering in the US. It recognizes people with a similar transformative impact on the field of bioengineering.

Harnessing Artificial Intelligence for Real Biological Advances—Meet César de la Fuente

by Eric Horvath

In an era peppered by breathless discussions about artificial intelligence—pro and con—it makes sense to feel uncertain, or at least want to slow down and get a better grasp of where this is all headed. Trusting machines to do things typically reserved for humans is a little fantastical, historically reserved for science fiction rather than science. 

Not so much for César de la Fuente, PhD, the Presidential Assistant Professor in Psychiatry, Microbiology, Chemical and Biomolecular Engineering, and Bioengineering in Penn’s Perelman School of Medicine and School of Engineering and Applied Science. Driven by his transdisciplinary background, de la Fuente leads the Machine Biology Group at Penn: aimed at harnessing machines to drive biological and medical advances. 

A newly minted National Academy of Medicine Emerging Leaders in Health and Medicine (ELHM) Scholar, among earning a host of other awards and honors (over 60), de la Fuente can sound almost diplomatic when describing the intersection of humanity, machines and medicine where he has made his way—ensuring multiple functions work together in harmony. 

“Biology is complexity, right? You need chemistry, you need mathematics, physics and computer science, and principles and concepts from all these different areas, to try to begin to understand the complexity of biology,” he said. “That’s how I became a scientist.”

Read the full story in Penn Medicine News.

Cesar de la Fuente On the “Next Frontier” of Antibiotics

César de la Fuente
César de la Fuente

In a recent CNN feature, César de la Fuente, Presidential Assistant Professor in Bioengineering, Psychiatry, Microbiology, and in Chemical and Biomolecular Engineering commented on a study about a new type of antibiotic that was discovered with artificial intelligence:

“I think AI, as we’ve seen, can be applied successfully in many domains, and I think drug discovery is sort of the next frontier.”

The de la Fuente lab uses machine learning and biology to help prevent, detect, and treat infectious diseases, and is pioneering the research and discovery of new antibiotics.

Read “A new antibiotic, discovered with artificial intelligence, may defeat a dangerous superbug” in CNN Health.

Why is Machine Learning Trending in Medical Research but not in Our Doctor’s Offices?

by Melissa Pappas

Illustration of a robot in a white room with medical equipment.Machine learning (ML) programs computers to learn the way we do – through the continual assessment of data and identification of patterns based on past outcomes. ML can quickly pick out trends in big datasets, operate with little to no human interaction and improve its predictions over time. Due to these abilities, it is rapidly finding its way into medical research.

People with breast cancer may soon be diagnosed through ML faster than through a biopsy. Those suffering from depression might be able to predict mood changes through smart phone recordings of daily activities such as the time they wake up and amount of time they spend exercising. ML may also help paralyzed people regain autonomy using prosthetics controlled by patterns identified in brain scan data. ML research promises these and many other possibilities to help people lead healthier lives.

But while the number of ML studies grow, the actual use of it in doctors’ offices has not expanded much past simple functions such as converting voice to text for notetaking.

The limitations lie in medical research’s small sample sizes and unique datasets. This small data makes it hard for machines to identify meaningful patterns. The more data, the more accuracy in ML diagnoses and predictions. For many diagnostic uses, massive numbers of subjects in the thousands would be needed, but most studies use smaller numbers in the dozens of subjects.

But there are ways to find significant results from small datasets if you know how to manipulate the numbers. Running statistical tests over and over again with different subsets of your data can indicate significance in a dataset that in reality may be just random outliers.

This tactic, known as P-hacking or feature hacking in ML, leads to the creation of predictive models that are too limited to be useful in the real world. What looks good on paper doesn’t translate to a doctor’s ability to diagnose or treat us.

These statistical mistakes, oftentimes done unknowingly, can lead to dangerous conclusions.

To help scientists avoid these mistakes and push ML applications forward, Konrad Kording, Nathan Francis Mossell University Professor with appointments in the Departments of Bioengineering and Computer and Information Science in Penn Engineering and the Department of Neuroscience at Penn’s Perelman School of Medicine, is leading an aspect of a large, NIH-funded program known as CENTER – Creating an Educational Nexus for Training in Experimental Rigor. Kording will lead Penn’s cohort by creating the Community for Rigor which will provide open-access resources on conducting sound science. Members of this inclusive scientific community will be able to engage with ML simulations and discussion-based courses.

“The reason for the lack of ML in real-world scenarios is due to statistical misuse rather than the limitations of the tool itself,” says Kording. “If a study publishes a claim that seems too good to be true, it usually is, and many times we can track that back to their use of statistics.”

Such studies that make their way into peer-reviewed journals contribute to misinformation and mistrust in science and are more common than one might expect.

Read the full story in Penn Engineering Today.

Penn Bioengineering Senior Discusses Remote Research Experience

Yi-An Hsieh (BE 2023)

Yi-An Hsieh, a fourth year Bioengineering student from Anaheim, California, worked remotely this summer on a team that spanned three labs, including the Kamoun Lab at the Hospital of the University of Pennsylvania. Hsieh credits her research on kidney graft failure with enriching her scientific skill set, exposing her to machine learning and real-time interaction with genetic datasets. In a guest post for the Career Services Blog, Hseih writes about her remote summer internship experience. “It showed me that this type of research energy that could not be dampened despite the distance,” she writes.

Read “Exploring How Amino Acid Polymorphisms Affect Graft Survival” in the Career Services Blog.

Training the Next Generation of Scientists on Soft Materials, Machine Learning and Science Policy

by Melissa Pappas

Developing new soft materials requires new data-driven research techniques, such as autonomous experimentation. Data regarding nanometer-scale material structure, taken by X-ray measurements at a synchrotron, can be fed into an algorithm that identifies the most relevant features, represented here as red dots. The algorithm then determines the optimum conditions for the next set of measurements and directs their execution without human intervention. Brookhaven National Laboratory’s Kevin Yager, who helped develop this technique, will co-teach a course on it as part of a new Penn project on Data Driven Soft Materials Research.

The National Science Foundation’s Research Traineeship Program aims to support graduate students, educate the STEM leaders of tomorrow and strengthen the national research infrastructure. The program’s latest series of grants are going toward university programs focused on artificial intelligence and quantum information science and engineering – two areas of high priority in academia, industry and government.

Chinedum Osuji, Eduardo D. Glandt Presidential Professor and Chair of the Department of Chemical and Biomolecular Engineering (CBE), has received one of these grants to apply data science and machine learning to the field of soft materials. The grant will provide five years of support and a total of $3 million for a new Penn project on Data Driven Soft Materials Research.

Osuji will work with co-PIs Russell Composto, Professor and Howell Family Faculty Fellow in Materials Science and Engineering, Bioengineering, and in CBE, Zahra Fakhraai, Associate Professor of Chemistry in Penn’s School of Arts & Sciences (SAS) with a secondary appointment in CBE, Paris Perdikaris, Assistant Professor in Mechanical Engineering and Applied Mechanics, and Andrea Liu, Hepburn Professor of Physics and Astronomy in SAS, all of whom will help run the program and provide the connections between the multiple fields of study where its students will train.

These and other affiliated faculty members will work closely with co-PI Kristin Field, who will serve as Program Coordinator and Director of Education.

Read the full story in Penn Engineering Today.