Kyle’s List of Ongoing Research Questions (In Progress)

Since finishing my Physics PhD in June 2023, I became very interested in machine learning. I've noticed that all the research questions that have floated through my brain in the past ~18 months share some component of using machine learning to make progress in some area of science, whether that be in areas like Earth systems science, astrophysics, neuroscience, or psychology. Obviously, I’m not an expert in all those domains, especially machine learning itself, but I must admit that the growth of these AI models over the past year and a half has given me hope that we will see scientific, technological, and engineering breakthroughs in areas related to these questions, and more! A future where everyone will be able to use AI to help them solve whatever the “Holy Grail” problem is to them on a personal or professional level, and where there will be greater access for more people to be involved in research and development. Because of this, I feel totally at ease sharing some of these questions, because maybe there are undoubtedly others out there who have had the time and space to think more deeply about some of these issues than me, considering that #1 and #2 are literally what I work on for my day job. I’m ecstatic at the prospect of being a truly interdisciplinary scientist augmented through AI, and I hope other scientists will feel similarly!

 The List

1.    Can we develop a robust machine learning algorithm that can accurately identify points in a lidar point cloud as belonging to certain categories (buildings, low, medium, high vegetation, bridges, electrical powerlines etc.) and apply this algorithm at scale to United States 3D Elevation Program lidar data?

2.    How can we develop a robust deep learning algorithm to perform semantic segmentation on cloudy pixels, non-cloudy pixels, and cloud shadow pixels in both NASA geostationary (GEO) and low-Earth orbiting (LEO) data that effectively discriminates among the classes at high confidence regardless of time of day, geographic location on Earth, and Sun orientation angle?

3.    How can artificial intelligence (AI) improve the lives of those diagnosed with autism spectrum disorder (ASD) and their loved ones? Can a multi-modal AI system learn behavioral patterns, evaluate physiological state, and improve the flow of communication between those with ASD, especially those who are non-verbal?

4.    Is it possible to create synthetic astronomical data to train a robust machine learning algorithm to perform black hole mass measurements on central supermassive black holes found in the centers of galaxies and compare this with traditional gas-dynamical or stellar-dynamical approaches?

5.    Why is deep learning so effective? For example, how is it able to make better predictions about the behavior of chaotic systems such as the weather without “understanding” the underlying equations?

ChadGPT: Caregiver helper for autism dynamics – a Generative Personalized Teammate

Autism spectrum disorder (ASD) affects 1 in every 36 children, including my brother Chad. Individuals with ASD as well as their loved ones and caregivers often struggle with communication, especially if the individual with ASD is non-verbal. Struggling to understand a loved one’s needs can take an enormous toll on a family as resources and support for children with ASD can be scarce. However, with the rapid improvement of artificially intelligent (AI) systems and the increasing amounts of available data through medical records, behavioral reports, pictures, videos, and the Internet of things, AI could provide a non-trivial amount of relief. For example, large language models (LLMs) supported with retrieval-augmented generation (RAG) could serve as a knowledgebase of personalized information for an individual with ASD. This knowledgebase can document years of behavioral patterns, preferences, and signs of distress that could take weeks, months, and even years for new caregivers to learn through experience. Thus, while there is little research on how LLMs can help individuals with ASD, the limited amount of it suggests that it can improve barriers with communication (Jang, Moharana, Carrington, & Begel, 2024) (Iannone & Giansanti, 2024).

ChadGPT stands for Caregiver helper for autism dynamics – a Generative Personalized Teammate. As the name suggests, it is designed to help caregivers understand their loved one or client with ASD better. As a prototype, it is a RAG application that is built using Llama3.1-8b, Pinecone, and Docker (inspired by the Cerebras example on the inference docs), and leverages Cerebras’s superior inference time compute, which could be vital for making real-time decisions in emergency situations. The system uses PyPDFLoader for processing documents, Cerebras's llama3.1-8b model to provide caregivers advice and instructions, and searches Pinecone to find the most relevant information chunks. A Pinecone and Cerebras API key are needed to run the system and are entered in a sidebar panel.

In its prototype phase, caregivers just need to import a document or set of documents that contain information regarding their client’s medical needs, behavioral patterns, hobbies, preferences, and even anecdotal experiences. While it is still in the prototype stage and relies on text retrieval and generation, the ultimate form of ChadGPT could potentially be multi-modal. By combining an up-to-date foundational knowledgebase with live video and audio, real-time behavioral, emotional, and physiological analysis is possible. While the realization of this kind of technology is still distant, the foundational RAG-augmented LLM will be a core component in the final design.

ChadGPT GitHub Repo

Bibliography

Iannone, A., & Giansanti, D. (2024). Breaking Barriers—The Intersection of AI and Assistive Technology in Autism Care: A Narrative Review. PubMed. https://pmc.ncbi.nlm.nih.gov/articles/PMC10817661/

Jang, J., Moharana, S., Carrington, P., & Begel, A. (2024). "It's the only thing I can trust": Envisioning Large Language Model Use by Autistic Workers for Communication Assistance. arXiv. https://arxiv.org/html/2403.03297v1

 Pictures of Me and Chad ~2005-2006 and in 2024