Kyle’s List of Ongoing Research Questions (In Progress)

Since finishing my Physics PhD in June 2023, I became very interested in machine learning. I've noticed that all the research questions that have floated through my brain in the past ~18 months share some component of using machine learning to make progress in some area of science, whether that be in areas like Earth systems science, astrophysics, neuroscience, or psychology. Obviously, I’m not an expert in all those domains, especially machine learning itself, but I must admit that the growth of these AI models over the past year and a half has given me hope that we will see scientific, technological, and engineering breakthroughs in areas related to these questions, and more! A future where everyone will be able to use AI to help them solve whatever the “Holy Grail” problem is to them on a personal or professional level, and where there will be greater access for more people to be involved in research and development. Because of this, I feel totally at ease sharing some of these questions, because maybe there are undoubtedly others out there who have had the time and space to think more deeply about some of these issues than me, considering that #1 and #2 are literally what I work on for my day job. I’m ecstatic at the prospect of being a truly interdisciplinary scientist augmented through AI, and I hope other scientists will feel similarly!

 The List

1.    Can we develop a robust machine learning algorithm that can accurately identify points in a lidar point cloud as belonging to certain categories (buildings, low, medium, high vegetation, bridges, electrical powerlines etc.) and apply this algorithm at scale to United States 3D Elevation Program lidar data?

2.    How can we develop a robust deep learning algorithm to perform semantic segmentation on cloudy pixels, non-cloudy pixels, and cloud shadow pixels in both NASA geostationary (GEO) and low-Earth orbiting (LEO) data that effectively discriminates among the classes at high confidence regardless of time of day, geographic location on Earth, and Sun orientation angle?

3.    How can artificial intelligence (AI) improve the lives of those diagnosed with autism spectrum disorder (ASD) and their loved ones? Can a multi-modal AI system learn behavioral patterns, evaluate physiological state, and improve the flow of communication between those with ASD, especially those who are non-verbal?

4.    Is it possible to create synthetic astronomical data to train a robust machine learning algorithm to perform black hole mass measurements on central supermassive black holes found in the centers of galaxies and compare this with traditional gas-dynamical or stellar-dynamical approaches?

5.    Why is deep learning so effective? For example, how is it able to make better predictions about the behavior of chaotic systems such as the weather without “understanding” the underlying equations?

Look at the Old Code!

As I type this, I’m processing data on NASA’s Pleiades supercomputer for work. I just spent the past 3 days (today being Saturday) engaged in a test of my patience and drive to succeed. I found a bug in my programs which led me on a wild goose chase of sorts to find the right answer. Stackoverflow, code documentation, and Google were scoured by me in an attempt to solve my problem. Yet, it was not until today that I realized that 90% of my solution could be found in previous iterations of this code that I had written. A few copy, pastes, and subsequent error debugging over the span of an hour paid off with the correct answer. Like a phoenix rising from the ashes, my old code proved itself to be useable after all! The moral here is, don’t forget to consult yourself when solving a problem!

The Unsettling Pace of AI Progress: A Physics PhD’s Perspective

As a physicist watching the artificial intelligence (AI) revolution as a curious bystander, I find myself in an increasingly familiar state of awe and unease. While I am not an AI expert, the recent developments have left me questioning my prior assumptions on the timeline of AI development.

The jump from OpenAI's o1-mini, o1-preview, and o1-full models to o3 has been nothing short of staggering. We're witnessing performance improvements that I hadn't expected to see for another 2-5 years – particularly the leap from 25-30% to 76% on the ARC-AGI benchmark (reaching 88% at an exorbitant cost of compute). This isn't just incremental progress; it's like watching a toddler’s cognitive abilities develop into a teenager’s in a matter of months.

My perspective on AI capabilities changed with OpenAI’s o1 series released in mid-September. As someone who likes to solve mathematics and physics problems for fun, I found myself humbled by these models' abilities. Problems that would take me hours to devise a solution for were aced by o1 in minutes. The skeptic in me wanted to attribute this to sophisticated pattern matching, but that argument is starting to collapse.

Perhaps most shocking to me is o3's performance on the FrontierMath benchmark, recently released in November 2024. While previous state-of-the-art AI models struggled to achieve 2%, o3 broke the grading curve and achieved 25%. A 10x improvement over previous state-of-the-art models isn't just progress – it's a paradigm shift. These are mathematical problems that challenge the world's best mathematicians, problems that demand days of deep contemplation. Even Terence Tao, 2006 Fields Medalist and in my opinion, the greatest living mathematician, said that these math problems “are extremely challenging…I think they will resist AIs for several years at least.” (so, what does one think when it’s making this kind of progress in less than 2 months?) These aren’t longer simple language models performing next-token prediction anymore; this is new AI-model territory that even those of us with non-AI technical backgrounds struggle to fully comprehend.

What worries me most isn't just the capabilities these models demonstrate, but their peculiar limitations. How can an AI model that impresses Nobel laureates and Fields Medalists simultaneously fail tasks a child could solve? This inconsistency, along with the fundamental lack of interpretability in these models, raises serious concerns about their deployment on a large scale.

When ChatGPT 3.5 emerged in November 2022, I talked with friends about how this is just the start of what’s to come. Yet I doubt any of us truly grasped the speed at which these AI models would improve. As a recently minted (June 2023) Physics PhD, I've spent years learning and thinking about complex systems, but this rapid evolution of AI is something I’ve never conceived of outside of Terminator movies and Avengers: Age of Ultron.

The deployment of these AI systems feels inevitable. While I'll continue my AI testing and analysis on YouTube, I do so with growing uncertainty about my ability to fully grasp the implications. Perhaps my concerns are unfounded – and part of me hopes they do. But in an era of such unprecedented technological advancement, I believe voicing my concerns is not just justified but necessary.

The question is not whether these systems will reshape our world because I think they already are, but how prepared are we for the transformation they'll bring? How can we adapt to such a rapid pace of development when we humans evolved on timescales of tens of thousands to millions of years? And while I may not have all the answers, I know enough to understand that this is a conversation we need to be having now, not later.

The Distracted Mind

I’m scatter-brained right now. In the past week, I’ve reviewed concepts from quantum mechanics, artificial intelligence, fluid dynamics, biomechanics, quantitative finance, computer programming, numerical analysis, novel writing, and chess; my poor brain can’t keep track of it all! It is why I’m writing now. Writing helps slow down my mind when it feels as if it’s moving at the speed of light. I try to give each word I write the attention it deserves, and in doing so, the swarm of thoughts must slow to a trickle. Deep breaths after each sentence also help.

It feels as if there is too much to learn, and not enough time. While this has always been true, it feels even more true today in 2024 than ever before in human history. We have so many distractions to choose from. From our phones, tablets, and computers, there are millions of apps, videos, articles ready to eat up some of the 86,400 seconds that make up our day. Not only are they there, but I am aware that they are there. That awareness itself consumes some of the finite amount of energy available for me to spend. Ignorance is bliss is now making more sense with each passing day.

What is the solution? Do I merge with AI? Get an implant in my brain to increase my functional capacity to memorize and learn? I sure hope not. At least not yet. Would merging with AI take away the joy of learning, or increase it exponentially? Is the joy of learning derived by cutting through the struggle of ignorance in your previous state? It certainly feels that way to me, but maybe because it is the only way I’ve known how to learn. The amount of available information to learn is growing every day, and my ability to learn it all remains stagnant, and will decline as I age. Maybe that’s the issue. The thought that it is even possible to learn it all. It’s not. At least until we merge with AI maybe. Until then, I think the best I or anyone like me can do is to accept the limitations of the human brain. Choose your battles wisely. Don’t overload yourself with too many topics. Pick a few at a time and focus intently. It sounds nice in theory but will be much more difficult to implement in practice. I know myself too well. Even still, I will try to reduce my cognitive load. Distraction is omnipresent nowadays, and I need to be able to focus if I’m going to learn even a small fraction of all the knowledge I want to learn by the time I’m no longer here.