Kyle’s List of Ongoing Research Questions (In Progress)

Since finishing my Physics PhD in June 2023, I became very interested in machine learning. I've noticed that all the research questions that have floated through my brain in the past ~18 months share some component of using machine learning to make progress in some area of science, whether that be in areas like Earth systems science, astrophysics, neuroscience, or psychology. Obviously, I’m not an expert in all those domains, especially machine learning itself, but I must admit that the growth of these AI models over the past year and a half has given me hope that we will see scientific, technological, and engineering breakthroughs in areas related to these questions, and more! A future where everyone will be able to use AI to help them solve whatever the “Holy Grail” problem is to them on a personal or professional level, and where there will be greater access for more people to be involved in research and development. Because of this, I feel totally at ease sharing some of these questions, because maybe there are undoubtedly others out there who have had the time and space to think more deeply about some of these issues than me, considering that #1 and #2 are literally what I work on for my day job. I’m ecstatic at the prospect of being a truly interdisciplinary scientist augmented through AI, and I hope other scientists will feel similarly!

 The List

1.    Can we develop a robust machine learning algorithm that can accurately identify points in a lidar point cloud as belonging to certain categories (buildings, low, medium, high vegetation, bridges, electrical powerlines etc.) and apply this algorithm at scale to United States 3D Elevation Program lidar data?

2.    How can we develop a robust deep learning algorithm to perform semantic segmentation on cloudy pixels, non-cloudy pixels, and cloud shadow pixels in both NASA geostationary (GEO) and low-Earth orbiting (LEO) data that effectively discriminates among the classes at high confidence regardless of time of day, geographic location on Earth, and Sun orientation angle?

3.    How can artificial intelligence (AI) improve the lives of those diagnosed with autism spectrum disorder (ASD) and their loved ones? Can a multi-modal AI system learn behavioral patterns, evaluate physiological state, and improve the flow of communication between those with ASD, especially those who are non-verbal?

4.    Is it possible to create synthetic astronomical data to train a robust machine learning algorithm to perform black hole mass measurements on central supermassive black holes found in the centers of galaxies and compare this with traditional gas-dynamical or stellar-dynamical approaches?

5.    Why is deep learning so effective? For example, how is it able to make better predictions about the behavior of chaotic systems such as the weather without “understanding” the underlying equations?

The Unsettling Pace of AI Progress: A Physics PhD’s Perspective

As a physicist watching the artificial intelligence (AI) revolution as a curious bystander, I find myself in an increasingly familiar state of awe and unease. While I am not an AI expert, the recent developments have left me questioning my prior assumptions on the timeline of AI development.

The jump from OpenAI's o1-mini, o1-preview, and o1-full models to o3 has been nothing short of staggering. We're witnessing performance improvements that I hadn't expected to see for another 2-5 years – particularly the leap from 25-30% to 76% on the ARC-AGI benchmark (reaching 88% at an exorbitant cost of compute). This isn't just incremental progress; it's like watching a toddler’s cognitive abilities develop into a teenager’s in a matter of months.

My perspective on AI capabilities changed with OpenAI’s o1 series released in mid-September. As someone who likes to solve mathematics and physics problems for fun, I found myself humbled by these models' abilities. Problems that would take me hours to devise a solution for were aced by o1 in minutes. The skeptic in me wanted to attribute this to sophisticated pattern matching, but that argument is starting to collapse.

Perhaps most shocking to me is o3's performance on the FrontierMath benchmark, recently released in November 2024. While previous state-of-the-art AI models struggled to achieve 2%, o3 broke the grading curve and achieved 25%. A 10x improvement over previous state-of-the-art models isn't just progress – it's a paradigm shift. These are mathematical problems that challenge the world's best mathematicians, problems that demand days of deep contemplation. Even Terence Tao, 2006 Fields Medalist and in my opinion, the greatest living mathematician, said that these math problems “are extremely challenging…I think they will resist AIs for several years at least.” (so, what does one think when it’s making this kind of progress in less than 2 months?) These aren’t longer simple language models performing next-token prediction anymore; this is new AI-model territory that even those of us with non-AI technical backgrounds struggle to fully comprehend.

What worries me most isn't just the capabilities these models demonstrate, but their peculiar limitations. How can an AI model that impresses Nobel laureates and Fields Medalists simultaneously fail tasks a child could solve? This inconsistency, along with the fundamental lack of interpretability in these models, raises serious concerns about their deployment on a large scale.

When ChatGPT 3.5 emerged in November 2022, I talked with friends about how this is just the start of what’s to come. Yet I doubt any of us truly grasped the speed at which these AI models would improve. As a recently minted (June 2023) Physics PhD, I've spent years learning and thinking about complex systems, but this rapid evolution of AI is something I’ve never conceived of outside of Terminator movies and Avengers: Age of Ultron.

The deployment of these AI systems feels inevitable. While I'll continue my AI testing and analysis on YouTube, I do so with growing uncertainty about my ability to fully grasp the implications. Perhaps my concerns are unfounded – and part of me hopes they do. But in an era of such unprecedented technological advancement, I believe voicing my concerns is not just justified but necessary.

The question is not whether these systems will reshape our world because I think they already are, but how prepared are we for the transformation they'll bring? How can we adapt to such a rapid pace of development when we humans evolved on timescales of tens of thousands to millions of years? And while I may not have all the answers, I know enough to understand that this is a conversation we need to be having now, not later.

ChadGPT: Caregiver helper for autism dynamics – a Generative Personalized Teammate

Autism spectrum disorder (ASD) affects 1 in every 36 children, including my brother Chad. Individuals with ASD as well as their loved ones and caregivers often struggle with communication, especially if the individual with ASD is non-verbal. Struggling to understand a loved one’s needs can take an enormous toll on a family as resources and support for children with ASD can be scarce. However, with the rapid improvement of artificially intelligent (AI) systems and the increasing amounts of available data through medical records, behavioral reports, pictures, videos, and the Internet of things, AI could provide a non-trivial amount of relief. For example, large language models (LLMs) supported with retrieval-augmented generation (RAG) could serve as a knowledgebase of personalized information for an individual with ASD. This knowledgebase can document years of behavioral patterns, preferences, and signs of distress that could take weeks, months, and even years for new caregivers to learn through experience. Thus, while there is little research on how LLMs can help individuals with ASD, the limited amount of it suggests that it can improve barriers with communication (Jang, Moharana, Carrington, & Begel, 2024) (Iannone & Giansanti, 2024).

ChadGPT stands for Caregiver helper for autism dynamics – a Generative Personalized Teammate. As the name suggests, it is designed to help caregivers understand their loved one or client with ASD better. As a prototype, it is a RAG application that is built using Llama3.1-8b, Pinecone, and Docker (inspired by the Cerebras example on the inference docs), and leverages Cerebras’s superior inference time compute, which could be vital for making real-time decisions in emergency situations. The system uses PyPDFLoader for processing documents, Cerebras's llama3.1-8b model to provide caregivers advice and instructions, and searches Pinecone to find the most relevant information chunks. A Pinecone and Cerebras API key are needed to run the system and are entered in a sidebar panel.

In its prototype phase, caregivers just need to import a document or set of documents that contain information regarding their client’s medical needs, behavioral patterns, hobbies, preferences, and even anecdotal experiences. While it is still in the prototype stage and relies on text retrieval and generation, the ultimate form of ChadGPT could potentially be multi-modal. By combining an up-to-date foundational knowledgebase with live video and audio, real-time behavioral, emotional, and physiological analysis is possible. While the realization of this kind of technology is still distant, the foundational RAG-augmented LLM will be a core component in the final design.

ChadGPT GitHub Repo

Bibliography

Iannone, A., & Giansanti, D. (2024). Breaking Barriers—The Intersection of AI and Assistive Technology in Autism Care: A Narrative Review. PubMed. https://pmc.ncbi.nlm.nih.gov/articles/PMC10817661/

Jang, J., Moharana, S., Carrington, P., & Begel, A. (2024). "It's the only thing I can trust": Envisioning Large Language Model Use by Autistic Workers for Communication Assistance. arXiv. https://arxiv.org/html/2403.03297v1

 Pictures of Me and Chad ~2005-2006 and in 2024