Imagine you're a rover on Mars, trying to find your way around. There's no towering skyscrapers or bodies of water to tell you where you are. What do you look for? The original natural landmarks: changes in land elevation that denote mountains and craters. On this week’s episode, Dr. Danielle DeLatte, a Space Systems Engineer at Draper, came in to talk about her PhD work using machine learning and computer vision to calculate the ages of craters and their possible role for rover navigation in the future.
Read on for the transcript!
Monte 0:07
Hi, I’m Monte Zweben, CEO of Splice Machine. You’re listening to ML Minutes, where cutting-edge thought leaders discuss Machine Learning in one minute or less.
This episode, our guest is Dr. Danielle DeLatte, a Space Systems Engineer at Draper. Danielle earned her PhD in aerospace engineering from the University of Tokyo, where she researched space applications of machine learning. Before earning her PhD, Danielle worked as an aerospace engineer at NASA Goddard. Welcome, Danielle.
Danielle 0:40
Thank you so much, Monte, I'm so happy to be here.
Monte 0:43
Well, this is great. I love speaking to fellow space people, former NASA people. I'd love to hear your journey: tell me about how you got to where you are now.
Danielle 0:53
Sure, I was going for the most degrees on most continents that I could possibly do. So I was extremely fortunate to study aerospace engineering at MIT, they have an absolutely phenomenal program. And when I graduated, I started looking at what what else was out there. And I found this really interesting program in in France called the International Space University, where we got a really interdisciplinary education. And that ended up being a theme for my career: interdisciplinary research, and projects. From there, I went to Goddard, as as you mentioned, then out to the University of Tokyo.
Monte 1:31
Excellent. Excellent. So tell us a little bit more about your PhD work: What are some of the applications of machine learning for space?
Danielle 1:38
One problem that I was really interested in was a planetary geology problem. Planetary geologists are really interested in how old a region is. So for example, on Mars, how old is a crater? Or how old is a region? So maybe there's a dry riverbed that's of interest. And so one of the really interesting parts of that was the interdisciplinary nature of the research, I got to work with Jack's Professor Elizabeth Tasker, planetary geologist, Dr. Sarah Crites, and machine learning researcher, Dr. Nicholas Guttenberg, to explore this problem and see how could machine learning help?
Monte 2:14
Excellent. And from a planetary geology perspective? How did you use machine learning to assess how old something is? On a planet like Mars or some other planetary object?
Danielle 2:27
That's a great question. So the way planetary geologists can tell how old a region is, is they they look at the number and the size of different craters. So what we were able to do with machine learning is look at Mars imagery and first of all, detect the craters and then count the ones in different sizes. And then from there, planetary geologists have come up with algorithms to determine at least the relative age for something like Mars. If you were looking at lunar images, though, you can actually use the samples that were brought back from the Apollo missions to do real age dating. But on Mars, we at least can do the relative ages.
Monte 3:03
Excellent. And that imagery, what frequencies or wavelengths [of light] were you using for that imagery; was that in the visible wavelength, or were there other other kinds of imagery being used?
Danielle 3:17
So we actually use infrared imagery. And there are a lot of different types of data sets that you could use, you could use digital elevation models, for example. So looking at how high each each pixel is, or, or you can, of course, use visible imagery. But we what we liked about the infrared is that there are less shadows and shadows turned out to be a really big challenge when we're looking at the craters. Because as you go toward the poles, the the craters look more and more extreme in visible imagery, but infrared had less of that issue.
Monte 3:50
Excellent. How did you formulate the machine learning problem?
Danielle 3:55
Well, we started with trying to detect where the craters were. So we were trying to detect the rims and basically find find a circular or elliptical outline of each crater. From there, we used computer vision methods to look at what the actual size was for each crater, and then use that to bin them and then determine the age.
Monte 4:17
Excellent. Did you use supervised learning to label edges that you knew were edges, and use that to train models?
Danielle 4:27
Yes, we fortunately, planetary geologists have been studying this problem for a long time. And so there are some fantastic labeled datasets. And so we we started with a really incredible data set that actually had 300,000 labeled craters on Mars. And that was, that was a fantastic start for for our machine learning algorithm.
Monte 4:47
That is fantastic. 300,000 labeled images that were probably labeled manually,
Danielle 4:54
Yes, labeled manually. If you can imagine for each of those craters in order to label manually, the researcher clicks on three points around the crater, or clicks in the center and then drags it out. So there there are some tools that help you do that. But it is a very manual process that we certainly benefited from in in the machine learning research.
Monte 5:16
That's very interesting. And now, with respect to the techniques that you used to model this supervised learning problem, did you use a deep learning neural network approach? Or did you use other algorithms? What was your basic approach to the classification problem?
Danielle 5:34
We used a segmentation algorithm called UNet. And what's unique about it among the compositional neural networks, is that from some of the training that happens in the early layers, comes back in the end layers. And so you there's an adding or merging process that happens. And so some of that earliest knowledge comes back at the end and helps helps bring the segmentation together.
Monte 5:59
Excellent. Thank you. With respect to the convolutional neural network, did you use tools to build that? Or did you build it from scratch?
Danielle 6:10
So we used Keras. And to be quite honest, I think if we were doing it again, today, we would probably use PyTorch. But at the time, Keras was the right tool. And we were also very cognizant of our audience. So we were hoping that planetary geologists might be able to pick this up and Keras had a great interface in Python. So we were we were hoping to make it as friendly and accessible to, to planetary geologists. But I think if you were to start from scratch today, I would probably recommend PyTorch.
Monte 6:43
Excellent. Yeah, I think many data sciences pick their framework, pi torture character based on the capabilities and others are picking it based on accessibility and popularity. There are all kinds of reasons for these frameworks. It's great that there are so many frameworks now that we can use out there. Going back to the planetary science for a moment, why is it important to be able to recognize craters?
Danielle 7:11
Well, for planetary geologists that wants to know how old an image is, it's it's a great application. There are actually a lot of other really interesting applications. One that I'm really excited about and have seen papers on this this last year is on looking, looking into using craters as navigation. So Lena Downes from MIT wrote, it wrote a paper in 2020, and called Deep Learning Crater Detection for Lunar Terrain, Relative Navigation. And I think that's just an absolutely phenomenal idea. Because while we may not be confident enough in that today, the more the more data we get, as we return to Mars and the moon, the more will we might actually be confident enough to use craters as the actual navigation for our spacecraft. So I think that's a really cool application.
Monte 8:06
I agree, that sounds like a really cool application, maybe we can double click a little on it just to help our audience understand that. So are you saying that by creating a better understanding of the location of all the craters, you can use them as sort of a guide way or a pathway for path planning?
Danielle 8:32
Maybe I mean, that's, that's what's so cool about about the research, there's, there's a lot of a lot that it could it could cover. And we can kind of imagine, if you if you look at a city skyline, our brains recognize that, you know, maybe something Seattle or Boston or Chicago, usually when you're pretty familiar with the city line, but you can, you can imagine something similar happening with craters, because there are so many and so many different sizes, that these could actually become fingerprints for different areas that that a spacecraft might want to go.
Monte 9:05
I always told my kids, having grown up in New York City, that when they come out of the subway, they should look for the Empire State Building, and look for other buildings down south. When I was a kid, it was the World Trade Center, now it's the Freedom Tower. So it sounds like perhaps when we're exploring other planets, we can have drones or other kinds of flying objects communicating with rovers and maybe giving some guidance as to where the rover is by looking at the craters. Is that a good way of thinking about this?
Danielle 9:39
I think it is. I mean, I don't think we're quite there yet. But research like like Lena's is is definitely a huge step in that in that right direction.
Monte 9:49
Excellent. Well, when you were doing this research on the planetary geology and using UNet and the convolutional neural networks underlying UNet, what was the most significant challenge you faced on that project?
Danielle 10:10
I think one of the biggest challenges was finding the right data set, both the right imagery data set and the right labeled data. There are, there are so many rich space databases in the industry. But sometimes accessing the data requires special software or detailed knowledge of the instruments that were used to take each image. So for that, finding the right people to collaborate with is key. And in my research, it was vital to have a planetary geologist, she already knew what how to how to access the data, she already had the tools. And that turned out to be absolutely invaluable. A second challenge once you found the right data set is understanding its specific strengths and limitations. So a specific example from our research is the global mosaic of Mars actually had missing pixels. So that required some thought about how to actually incorporate that in in our model.
Monte 11:07
Well, how did you overcome that, when you did find that you add missing pixels? What was your compensation method?
Danielle 11:14
Well, we ended up coming up with a very simple solution, which was to add the average gray value to the missing data, because the missing data was just showing up as zeros. And fortunately, our technique was was robust to the missing data once once we had applied that that gray value. So it turned out to be a much simpler problem than we thought. And the network turned out to be more robust than we thought. But the training was really key to make sure that it could be robust to those kinds of those kinds of changes.
Monte 11:45
So you had to create a pipeline that took the raw data, smoothed out the images, where there were missing pixels with the averaged out gray values, and then submitted that to the segmentation layer. Is that right?
Danielle 12:00
Yes, that's it.
Monte 12:02
Excellent. Well, looking forward, what do you see coming next in space applications of machine learning? What's a big area you see, that's coming next.
Danielle 12:15
Another really interesting example is look using machine learning, with satellite imagery to improve things like food security on Earth. So Professor Hannah Kerner, from the University of Maryland, for example, does does that type of remote sensing and machine learning hybrid
Monte 12:34
That's excellent. We also had another podcast guest on ML Minutes that talked about using Earth-facing satellites in the infrared, and also other wavelengths to find sustainable forests, supplies in Palm oil production. And also, another podcast guest actually used Earth satellites to talk about the use of the imagery to predict the amount of moisture in the forest and predicting forest fires. So I think that's a fantastic area for future research.
Danielle 13:17
Absolutely. I think one of one of the places that machine learning really excels is when you have huge amounts of data and a specific problem. And I think some of these some of these things with Earth observation are just absolutely perfect for that.
Monte 13:31
Oh, that's pretty cool. That sounds great. Well, Danielle, it's been a pleasure. Thank you so much for joining us on ML Minutes. I hope you had a good time.
Danielle 13:39
Thank you so much for having me Monte, it's great to be here.
Monte 13:49
If you want to hear Danielle talk about her work on NASA's Artemis project putting people on the moon, check out our bonus minutes. They're linked in the show notes below and on our website, mlminutes.com. Next episode, we'll be exploring how machine learning can improve patient outcomes and experiences in health care. To stay up to date on our upcoming guests and giveaways, you can follow our Twitter and Instagram @MLMinutes. ML Minutes is produced and edited by Morgan Sweeney. I’m your host, Monte Zweben, and this was an ML Minute.