The Center for Data Innovation spoke to Miriam Huijser, chief research officer and co-founder of Aiir Innovations, a Dutch company that uses uses image recognition algorithms to spot damage to the insides of aircraft engines. Huijser discussed how AI can help maintenance crews detect faults in maintenance checks, and how future developments in AI and quantum computing might enable broader use of AI.
This interview has been edited for clarity.
Nick Wallace: Flying commercially is relatively safe, and has been for a long time, which suggests maintenance crews have been doing a good job. How does AI improve aircraft maintenance compared to how mechanics have traditionally inspected and maintained aircraft?
Miriam Huijser: To answer that question I think I should first explain what maintenance currently entails for aircraft engines. Basically, what they do now is they take the engine off the wing, and they put a camera—called a borescope—into the engine, so it’s like an endoscope for machines. They slowly rotate the engine, they get the video up on a screen, and they look through it for hours and hours. It can take 24 hours for an engine to be inspected. There are about 800 blades in one engine, and they need to be inspected from six different viewpoints, because you cannot see the whole blade in one go. You can imagine that if you were doing that, you’d get very tired and start making mistakes.
The main problem right now is first that it takes a lot of time, and second that different mechanics produce different results, so it’s very inconsistent. That means if there is a mechanic who misses something, or an incoming inspection that misses something, then the things that are caught are fixed, but then at the outgoing inspection they will see something else, which means the engine will have to go all the way to the front of the line again, which causes a lot of delays. Plus, all the things that are missed by inspectors are still in the engine when it’s back on the wing, which means there could be some catastrophic events if that fails mid-flight. These blades are quite strong, but they rotate at hypersonic speeds, so if there are small cracks then at some point they can break off and shoot through the entire engine. If that happens mid-flight, it’s a big problem.
So there are a couple of facets to this problem. First, you say flying is very safe…yeah, but if something does happen, it has catastrophic results. Second, engine inspection takes a lot of time, and if they miss something it has to happen again, so it’s very labor intensive. Basically, what we see is we go into an airplane and we lift off, but we don’t see everything that goes on behind the scenes.
Wallace: So Aiir adds image recognition technology to engine inspection—can your technology be used for anything else?
Huijser: Yes. Maintenance is very broad. But we are a company that consists entirely of AI experts, so we can easily take on other domains than maintenance. We develop very general approaches in anomaly detection, which can be used in almost anything. It’s just right now we have a data source that pertains to engines, so that’s why we’re focusing on it now.
Our vision as a company is to take every mundane inspection task with whatever kind of sensor and to automate that, or to at least assist in a way that becomes a lot less labor intensive. Another thing that we do right now is use hyperspectral cameras to automatically classify and segment different kinds of plastic so they can be more efficiently recycled. The traditional way to do this is to burn a piece of plastic and smell it. With hyperspectral imaging, you can automate that. In terms of aircraft engines, we want to move towards using x-ray, and we also do stuff with thermal imaging. Basically, anything that outputs an image and where humans are still looking at a screen to analyze the results and deciding what needs to happen, that’s where we come in and say “we can help you” or “we can automate this task.”
Wallace: Your academic background is firmly rooted in artificial intelligence. What aspects of AI did you focus on when you were studying, and how did you end up working on aircraft maintenance?
Huijser: I specialized in machine learning and computer vision. Then we had a project where KLM, the Dutch airline, came to the university and said “we have this borescope inspection task, can you automate it?” We worked on a proof-of-concept for four or five weeks, and then we went back to them and said we could see it working—we showed them some early detectors—but we did not have enough data for full-scale deep learning. They said, “great, but who’s going to build this for us?” That’s when we decided to build a company and start doing it, even before we’d graduated.
Wallace: What are your priorities as chief research officer? What areas are of greatest interest for Aiir’s future development?
Huijser: We’re collecting a lot of data, and that is needed for full-scale, supervised deep learning. The issue is that usually it’s very difficult to obtain a lot of data in a short time, especially where you need experts to label the data. We built a species of software now that automatically generates reports when the client uses it, but also generates data for us. But the thing is, it needs to be used by many customers before we get to a level where supervised deep learning is the best approach, and for the moment we only have one.
What I’m researching now is state-of-the-art anomaly detection on images, and domain adaptation. Anomaly detection is where basically you learn what is normal and what a normal engine should look like, and you start to detect deviations from the normal standard. That way, you can start detecting defects. Domain adaptation is a field where you go from one domain to the next. For example, you have a lot of images that are synthetically generated: so we’ll have a 3D model of the engine and will add some damage to it, and then we also have the real engine footage—doesn’t necessarily need to match exactly—and then the algorithm can make that synthetic data look more realistic. You basically create your own data then. With that data, which is also annotated because you know where you put the damage, you can then do supervised deep learning, and bundle it with your real data to obtain a better model.
Wallace: Have you come across any other challenges in maintenance or aviation that you think AI could help tackle in the near future?
Huijser: I’m becoming more and more fascinated with artificial general intelligence. That’s basically having a system that can completely automatically learn by itself and can really do many different tasks. What we have now is more like shallow artificial intelligence, that is very specific to certain tasks. Every time you train a model, it’s for a specific task, maybe two, and that’s it. I think it would be really great to have something that can look over the shoulder of a mechanic and really learn about what to do, and then just do it.
It’s difficult to say how far away that is—the moment that it happens, it happens very fast. I think quantum computing would really help, maybe in the next five years, but I don’t know if that’ll actually happen. We need a speed-up like that to take the next step. Deep learning models right now are being run on GPUs (graphics processing units), because GPUs allow you to parallelize a lot of computations, whereas CPUs (central processing units) are mostly sequential. With quantum computing, you can parallelize even more. That would greatly speed up machine learning. I don’t have a background in quantum computing—I’m going to start learning soon—but I think that would be great to have for AI.