Anna Olejniczak-Serowiec Nov 13, 2023 - Interior sensing

Who’s in charge? The issue of taking control of an automated vehicle

Author: Anna Olejniczak-Serowiec, Senior Project Owner @ Robotec.ai

Living the change

What was once a product of writers’ and filmmakers’ imagination is now close to becoming our daily life. Self-driving cars used to be a thing of science fiction. Today, however, we are faced with the fast-proceeding development of automatized vehicles, and the first fully autonomous cars were already cleared for testing on public roads last year (Hope, 2022, Mitchell, 2022). The automotive industry puts much effort into the development of advanced driver assistance systems and autonomous functions. We are witnessing this change; we are making this change!

Source: Image by Julius H. from Pixabay

The honorable status of change participants implies not only the privilege to feel, taste, and try the new driverless quality of transportation but also the fact of facing all the intermediate steps the industry takes on its way toward this future.

Like any change in this world, the transition from manually operated to autonomous cars is a successive process that unfolds over time. Society of Automotive Engineers defined six levels of vehicle automation (SAE: J3016_201806), which are now widely referred to in the industry.

Level 0 describes traditional cars operated manually. Level 1 describes cars with driver assistance functions, like cruise control. Level 2 refers to partially automated vehicles, in which the driver is responsible for driving safety and thus monitors all the functions and needs to take control of the vehicle at any time. Level 3 describes cars, in which many functions are automated, but the driver can easily take over the control whenever needed. At level 4, the car operates fully automatically. However, if the situation becomes too complicated for the system to handle, the driver is required to take over. Level 5 refers to fully automated driving.

Is autonomous transport even possible?

First things first, is this really possible? Is autonomous transport our future? Experts discuss the possibility of achieving a fully circumstances-proof autonomous driving system. Their development encounters challenges of various types, which are being meticulously solved over time, each of which represents a step toward the once-set goal—autonomous transportation. Meanwhile, most researchers, industry experts, and society agree that we will be able to take advantage of high automation in the not-so-distant future. In the meantime, we have access to more and more automatized vehicles. Automatized functions are oriented towards increasing traffic safety and decreasing human driver cognitive load, enabling safe travels regardless of the driver’s state and events going on in their lives.

Paradoxically, new challenges arise from the need for smooth interaction between the human driver and the automatized systems. Automatized functions relieve the driver from many tasks and the need for constant processing of the surrounding environment. At level 4,  where supervision of functions is no longer required, it is possible for the driver to be engaged in non-driving tasks during the ride. Thus, the driver can participate in work or indulge in the pleasures of reading or listening to music, to give a few examples.

Source: Image by diana.grytsku on Freepik

Still, the driver is required to take over control of the vehicle when such a need occurs. To deal with decreased driver attention paid to the vehicle, level 3 and 4 cars release requests to intervene (RTI) when the system fails to deal with the situation. The RTI calls the driver to take control and transfers the responsibility for decision-making and operating the vehicle back to the driver. In fact, this means that the driver needs to pay attention to the road and the vehicle, quickly and effectively restore their situational awareness, and make decisions related to safe driving.

The tricky part of the automation – human driver deal

Here comes the trick: as it turns out, a lower need for constant control over a vehicle can lead to more mind wandering (Gouraud et al., 2018) and a decrease in environment monitoring effectiveness (Biondi et al., 2018). This means that the driver who is physically present behind the wheel might be cognitively engaged in some other activities and thus not ready to take over immediately upon request. An even more evident challenge comes from possible engagement in non-driving tasks during the ride. Being cognitively and possibly also physically engaged in some other tasks, be it reading, watching a movie, e-mail writing, or knitting, to give some examples—might prove it difficult to jump into the driving task quickly.

Situational awareness is a key factor in safe driving. It refers to perceiving the situation on the road, understanding it, and anticipating its consequences, thus enabling effective decision-making and adequate response to the situation. In its initial shape, situational awareness described and justified the need for continuous monitoring of the dynamic road situation and making constant corrective decisions to maintain safe driving (Endsley, 1988; Gugerty, 1997, 2011). The delegation of some driving tasks to the automatized systems and the possibility of engaging in non-driving related tasks might significantly decrease the situational awareness of the driver (Capallera et al., 2022, de Winter et al., 2014, McKerral et al., 2023) who upon an RTI, needs to quickly restore it, make effective decisions, and successfully execute them.

Control take-over-related challenges for the contemporary automotive industry

The change in the approach towards situational awareness—from constant maintenance to quickly restoring when required—raised a bunch of new questions in the automotive industry. How do we construct the RTI to make sure that the driver understands it? How do we parametrize the RTI, which will be salient enough to catch the driver’s attention but simultaneously not disturbing enough to cause distress and uncontrolled, hasty actions? Under which circumstances will the RTI be useful, and when will it provide more issues than benefits for traffic safety? How much time is needed for the driver to restore situational awareness? Which conditions need to be satisfied to ensure effective driver decisions upon request? The above provides a set of examples from a wildly broad and surely not yet complete catalogue of questions that HMI specialists are now trying to address (cf.: An et al., 2020, Huysamen et al., 2021, Rodak et al., 2022, Yang et al., 2018).

Source: Image by Rawpixel.com on Freepik

Assuming that the RTI parameters constitute the heads of the RTI coin, we cannot ignore its tails either. The other end of the challenge is surely devoted to whether the driver can react to the RTI appropriately. Given that they are allowed to engage in other activities, the safety systems need to constantly monitor the driver’s state and assess their availability and fitness to react. Is the driver cognitively present behind the wheel? Is the driver not too drowsy to react safely? Is the driver…? The questions multiply; nonetheless, they refer to the question of driver availability monitoring. Our common sense and experiences in driving, as well as years of traffic safety research, jointly form a long list of variables influencing driver availability, which the industry is now trying to address in a step-by-step manner, driver distraction and drowsiness monitoring proving to be the first highly significant steps on this bumpy road towards safe, automatized traffic future.

At Robotec.AI, we are highly committed to developing human-centered traffic safety systems and thus engage in testing and validating driver/occupant monitoring systems in terms of various driver states: distraction, drowsiness, and control takeover effectiveness included.

If you have interesting insights on this topic or would like to discuss potential projects with us, contact us via humanfactors@robotec.ai!

References:

An, S., Maeda, M., Mao, X., & Itoh, M. (2020). Effects of request-to-intervene message contents on driver performance and safety under conditionally automated driving system. IFAC-PapersOnLine, 53(5), 203-206. https://doi.org/10.1016/j.ifacol.2021.04.099

Biondi, F. N., Lohani, M., Hopman, R., Mills, S., Cooper, J. M., & Strayer, D. L. (2018, September). 80 MPH and out-of-the-loop: Effects of real-world semi-automated driving on driver workload and arousal. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 62, No. 1, pp. 1878-1882). Sage CA: Los Angeles, CA: SAGE Publications. https://doi.org/10.1177/1541931218621427

Capallera, M., Angelini, L., Meteier, Q., Abou Khaled, O., & Mugellini, E. (2022). Human- Vehicle Interaction to Support Driver’s Situation Awareness in Automated Vehicles: A Systematic Review. IEEE Transactions on Intelligent Vehicles. https://doi.org/10.1109/TIV.2022.3200826

De Winter, J. C., Happee, R., Martens, M. H., & Stanton, N. A. (2014). Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transportation research part F: traffic psychology and behaviour, 27, 196-217. https://doi.org/10.1016/j.trf.2014.06.016

Endsley, M. R. (1988, October). Design and evaluation for situation awareness enhancement. W: Proceedings of the Human Factors Society annual meeting (Vol. 32, No. 2, pp. 97- 101). Sage CA: Los Angeles, CA: Sage Publications. https://doi.org/10.1177/154193128803200221

Gouraud, J., Delorme, A., & Berberian, B. (2018). Out of the loop, in your bubble: mind wandering is independent from automation reliability, but influences task engagement. Frontiers in Human Neuroscience, 12, 383. https://doi.org/10.3389/fnhum.2018.00383

Gugerty, L. J. (1997). Situation awareness during driving: Explicit and implicit knowledge in dynamic spatial memory. Journal of Experimental Psychology: Applied, 3(1), 42. https://doi.org/10.1037/1076-898X.3.1.42

Gugerty, L. (2011). Situation awareness in driving. Handbook for driving simulation in engineering, medicine and psychology, 1, 265-272

Hope, G. (2022). Europe’s First Driverless Car Test Completed. IOT World Today. https://www.iotworldtoday.com/transportation-logistics/europe-s-first-driverless-car-test-completed [Access: 28.08.2023]

Huysamen, K., Collins, M., & Wardle, A. (2021) General Safety Regulation: Technical Study to Assess and Develop Performance Requirements and Test Protocols for Various Measures Implementing the New General Safety Regulation, for Accident Avoidance and Vehicle Occupant, Pedestrian and Cyclist Protection in Case of Collisions. Driver Availability Monitoring Systems (DAMS). Final Report. TRL Ltd. Under the contract with European Commision. Luxembourg: Publications Office of the European Union

McKerral, A., Pammer, K., & Gauld, C. (2023). Supervising the self-driving car: Situation awareness and fatigue during highly automated driving. Accident Analysis & Prevention, 187, 107068. https://doi.org/10.1016/j.aap.2023.107068

Mitchell, R. (2022). Waymo says it’s bringing robotaxis to L.A. Los Angeles Times. https://www.latimes.com/business/story/2022-10-19/here-come-the-robotaxis-waymo-set-to-deploy-in-la [Access: 28.08.2023]

Rodak, A., Kruszewski, M., & Sztandera, B. (2022). Does the Driver Understand the Warning? Comprehension of the Request to Intervene. Applied Sciences, 12(19), 9451. https://doi.org/10.3390/app12199451

Yang, Y., Karakaya, B., Dominioni, G. C., Kawabe, K., & Bengler, K. (2018, November). An HMI concept to improve driver’s visual behavior and situation awareness in automated vehicle. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC) (pp. 650-655). IEEE https://doi.org/10.1109/ITSC.2018.8569986