Fault handling of self-driving vehicles with remote human support

For a special edition of Popular Science we invite Erik Branzén and Jesper Englund, currently finishing their master theses at ITRL and Scania on the role of the control tower in the diagnostic process and diagnostics-based decision-making in autonomous vehicles. Their theses are titled:

“Implementation of fault scenarios in autonomous vehicle” (Erik Branzén)

“The control tower role in fault scenarios for driverless vehicles” (Jesper Englund)

Due to the projects close connection to one another, we have decided to group them together for this interview. The projects are also a part of a larger project at Scania which is about studying what should replace the driver when they disappears, the diagnostic process, and how to maximize uptime.

What is the project about?

Erik: My project is about implementation of algorithms in autonomous vehicles to ensure a reliable diagnostics of complex fault scenarios that can occur in a truck or vehicle. We have these symptoms that a person can detect fairly easily such as vibrations or noise, but these kinds of symptoms are not really integrated into the diagnostic systems in conventional vehicles. When we take away the person in the vehicle how could we then guarantee that this vehicle will operate safely and reliably? I have found a set of processes that could ensure this.

Jesper: My project is about defining, to some extent, the role of a proposed control tower in driverless vehicle diagnostics and give suggestions on what the control tower interface could look like.

What will be the end results?

Erik: Autonomous vehicles need to have some sort of decision making process react since faults will always occur. In a theoretical world you could avoid all faults but that is not going to happen, so how should the vehicle react? A good knowledge base is vital in informing the decision-making of the vehicle. Finding the active fault through a diagnostic algorithm is the first step in my proposed system. The next step is prognostics. For example, if a minor fault occurs, and the diagnostics found that there is a small air leakage in the suspension, then the prognostics will tell me how I should react to this small fault. The active fault might propagate into a more severe fault or a set of severe faults. The knowledge of this potential future can inform the decision-making process.

Jesper: For me, it’s partially a prototype of the interface but I’ve also been able to draw conclusions around technical requirements on the vehicle and the competency requirements for the operator in the control tower. In addition, conclusions were reached about how this new system would fit into a larger organisation around the diagnostics process. How the different stakeholders can be involved, that is.

What could some of those operator competencies be?

Jesper: One of the key takeaways would be that talking about a single control tower operator, which is what we did in the beginning of the project – is kind of misleading because for this kind of diagnostics process you would need at least two different roles in the control tower. One would be more of a transport leader, having more of an overview and is focused on the planning. The other would have a more analytical role, digging deep into the actual faults and have a technical background.

Have there been any surprising results?

Erik: Just how hard it is to create this type of model from intuition. We were given a lot of data from interviews with experts from Scania, people who drive vehicles, service technicians, and people who knows the trucks in and out, and see how these people would associate these symptoms with different faults. Based on that I created a mathematical model and from that I draw the conclusion that in order for this general methodology to infer faults from symptoms you really need to have a lot of data. It needs to be a combination of intuition from experts and data driven models to make this type of general methodology to work.

Jesper: One of the surprises was that you need two different operators. Also, the concept of the control tower, might not be as useful as you might think. What I’ve found is that the control tower ought to be seen as a bigger system or an information hub with various access points for different stakeholders. It should be available from Scania’s side of things, like central support instances at Scania, but also from workshops, service centres, or transport companies. Additionally, I don’t know where one would place this control tower, should it be at Scania or at the transport companies? It has to be flexible in that sense.

How has the collaboration with ITRL and Scania looked like?

Erik: On our end it has been mainly through our supervisor Lina Rylander (PhD student at ITRL and Scania) and some of her colleagues. We’ve set up a lot of interviews and discussions with different people at Scania and I’ve talked a lot with people here at ITRL.

Jesper: Through Lina we have been able to get access to different people at Scania so it’s been a luxury in that sense, when you talk to other master students they might not have a close connection to the company that they’re doing the thesis for.

Erik: We are also using the SVEA platform, which is a small platform for autonomy to test the algorithms and the control tower integration, so we are testing the software on ITRL hardware.

What is the take home message?

Erik: One really important takeaway is the idea that creating these autonomous platforms is a process, a very long process of integrating new technologies. We realized that the driver in conventional vehicles is the key actor in making the vehicle work, you can’t just put path following algorithm in the vehicle and then it is going to work. When Jesper’s control tower developed we realized that there are so many different rules that a driver has to follow, this is really hard to replace, and we still have human actors working with autonomous vehicles. The end goal is that we can replace more and more processes with autonomous processes.

Jesper: The end goal of level five autonomy is a pretty big threshold to reach, this is only a way to lower that bar, and from this being the first step towards that you can then incorporate more and more autonomy.

While their work isn't finished yet, you can read more in Erik's midway presentation (pptx 2.1 MB) . Feel free to contact Erik  and Jesper  for more information about the project.