01
Solving for
Semi-AutonomyCockpit HMI UX
Overview of learnings, achievements, and mobility design solutions at Toyota & Lexus. Focused on designing for semi-autonomy.
Client
Woven by Toyota
Responsibility
Design Lead, Strategy, Design Direction, Interactive Design, IA, 3D design and animation.

Project Introduction

Solving for semi-autonomy requires trust, clarity, understanding, clear expectations.

At Woven by Toyota, I led teams in creating organizational change through innovative UX design, focusing on semi-autonomy solutions. This page offers insights into our approach to creating, discovering, and validating solutions for various mobility projects I spearheaded or contributed to.
As proprietary work cannot be shown in full, what you see here is work in progress or reworked versions of the final output. I am happy to discuss any aspects that interest you in more detail. 
Solving for semi-autonomy

‘ Am I in control, or

is the car in control...? ’

— What you don’t want to be questioning while on the move
Designing for
Mode Confusion
Our goal is to create intuitive semi-automation mental models that reduce cognitive load, enhance safety and feature adoption. We have developed clear driver expectations to set and deliver industry-leading standards for automation interactions.
Education is vital in semi-autonomy to prevent overtrusting the technology and ensure safety. We strived to solve complex autonomy issues and design mobility solutions in a way that provides clear user value and competitive advantages for Toyota and Lexus.

It is also important to note that designing motility experiences that are rewarding enhances user adoption, retention, and engagement.
Challenges
Organizational Challenge Working effectively with various stakeholders across such a large organization and building a shared ideal of the importance of prioritizing user experience (UX) in future mobility solutions, particularly within an engineering and manufacturing context.

Cultural Integration Establishing a collaborative culture and earning trust as newcomers in the organization.

Understanding Complexities Grasping the intricacies of user needs, capabilities, and regulatory requirements and frameworks.
Approach
Establishing a strong Design Foundation We began by working with other teams to establish a good design practices to build towards effective collaboration and effective output. This included  defining a shared vision, values, first principles, design direction, a collaborative and fun culture.

Holistic & Data-Driven Methodology We established a holistic, data-driven approach at Toyota to address the complex physical and digital driving experience, ensuring our solutions are technically sound and aligned with real-world scenarios.
Collaborative Culture & Focus on Excellence By setting clear goals and principles, we fostered a collaborative, excellence-driven culture, crucial for navigating internal politics and facilitating innovation.
Top center section is the most noticeable area for users to see and take action (tested and validated).

Left Section User defined content area - for non-driving critical info such as music, efficiency, larger details, etc.

Center Dynamic Situational Awareness - shows users current coverage of what the vehicle can see and do.

Right Section Speedometer and driving-critical related information.

Surrounding real-estate Instrument Cluster peripheral - telltales, additional info such as time, gauges for fuel, ODO, temp. etc.

Bottom Center Supporting information prompt space for temporary non-actionable items e.g. auto-parking prompts, traffic condition updates, vehicle status updates etc.
Holistic ADAS & Navigation integrated Situational Awareness View
Further explorations and progress shots below.
User is primarily in control
The vehicle is primarily in control

Mobility Design Framework

Three main areas of focus we defined for designing mobility experiences
01
— Safety
Design
Objective safety is prioritized over comfort and communication. Perceived safety in less critical circumstances needs to be designed for to build trust and demonstrate capability. If the vehicle intends take action then the user needs to know what is happening, why it is happening and what is expected of them.
02
— Service
Design
Service design aspects of the vehicle should be intuitive & frictionless, allowing users to navigate functions smoothly and easily. This approach ensures features are accessible designed with strong mental models to enhance overall satisfaction and convenience. Ensures a balanced overall experience rather than every touchpoint needing to 'wow' the user.
03
— Experience
Design
These are the remarkable moments; good experiential design forms a lasting impression. Efficiency is no longer the primary goal. The aim of experiential design is to create experiences that delight, surprise, engage, empathize and take the user on 
a journey. How you deliver, at times, is as important as what you're delivering. Positive friction can be a good thing.
Design Initiatives to test Hypothesis & Framework
Many design iterations were needed to optimize for functionality, user attentiveness and clarity.
Design Initiatives to test Hypothesis & Framework
Different methods of communicating what the car is seeing and what action it will take next.
Design Initiatives to test Hypothesis & Framework
Curating modalities for common incident circumstances; for varying levels of criticality - low, medium, high and critical.
User Cognition Insights
Effects of Curated

Multi-Modal Feedback
Our goal was to demonstrate our capability to solve complex autonomy issues internally, while strategically designing mobility solutions that offer clear user value and competitive advantages for Toyota and Lexus.

Through a thorough set of design and research cycles we were able to understand how the optimal curation of modalities could improve a user's response times, effectiveness of understanding and actions taken.
Voice
Haptics
Tones
Visual
Resulting in faster user response times with higher levels of success
400ms
300ms
220ms
+ Clarity & Understanding
Human Cognition Process
By understanding how people perceive and process actions on the road we can more effectively provide solutions that improve their overall experience. We can then learn from their behavior to help encourage new and more effective driving habits.
01
Awareness
Recognizing surroundings and hazards
02
Recognition
Identifying the significance of what's observed.
03
Decision
Choosing the appropriate action based on recognition.
04
Action
Physically carrying out the chosen course of action.
05
Outcome
Observing consequences of the action taken.
06
Learn
Reflecting on outcomes to improve future responses.
Rewarding User Value Interaction Loop
To create mobility experiences that increase user adoption, engagement, and retention we need to design from a deep understanding of our users, how they think and how they behave to create a framework that can consistently produce effective results.
01
Intention
What do they intend to achieve? What do users really want? What pain is your product relieving?

(Understanding our users, their lifestyle, their behaviors & their needs)
02
Trigger
What brings users to your service?

(internal or external trigger)
03
Pre-action
How can we communicate clear value proposition so users think of Toyota or Lexus first when wanting to address a mobility related pain point?

(Motivation and ability)
04
Action
Design products with intuitive HMI mental models for effortless and rewarding user actions.

(Effectively designed HMI mental models)
05
Reward
Design effective feedback loop that creates a wanting & satisfaction in the user. Are users fulfilled by the reward, yet left wanting more?

(Resolve feedback loop 
with variable rewards)
06
Investment
What “bit of work” do users invest in your product; of time, data, effort, social capital or money to make their experience better?

(Data collection, profile personalization, social connection)
Success & Growth Strategy
To produce effective products for our users, alignment among Product, Marketing, Business, Design, Research, Data, Engineering, and Manufacturing is crucial. The goal is to create a partnership where users build lasting trust and take pride in owning a car that serves as a dynamic and supportive driving companion. Intuitive connected mobility, evolving through empathy and understanding, creates memorable experiences that users will cherish and share.
01
Acquisition
Attracting potential customers or users through various marketing channels.
02
Activation
Ensuring new users have a positive first experience and understand the added value of the new mobility product.
03
Retention
Keeping users engaged and coming back over time by continuously delivering value through OTA updates.
04
Monetization
Convert active users into paying customers by using the vehicle as a purchase portal for connected services, offering value through the vehicle to diversify income beyond the point of sale.
05
Referral
Encouraging satisfied customers to refer new users.

Leverage platform specific network effects: through shared profiles, connected journeys and more.
06
Feedback
Collecting and analyzing feedback from users to make informed improvements to the vehicle and services provided.
Cockpit Hierarchy
of Information
Initiative Categories
Hover over the titles on the right to discover areas of focus on how to design holistically to produce an effective mobility solution that will resonate with people.
Design Philosophy
Autonomy Principles
Active Driving Partner
Learning System
Intuitive ADAS Execution
Enhanced ADAS Capability
Network Effects
Connected Experiences
Curated Experiences
Business Partnerships
& New Revenue Streams
Intuitive Innovation: UX Design Philosophy
Our Intuitive Innovation design philosophy at Toyota and Lexus redefines the user experience (UX) in automotive design. By blending practicality with futuristic aspirations, we aim to deliver emotionally resonant experiences. Our goal is to create intuitive connected mobility that evolves through empathy and understanding, fostering a partnership where drivers take pride in their dynamic 
and supportive driving companion.
A clear design philosophy guides teams, helps to facilitate positive and effective collaboration around a shared vision 
and purpose for a cohesive product development process. Product, marketing, design, engineering, research and manufacturing need to be aligned in order to create products that users will love to use and live with.
First Principles to connect humans with the machine
Through many design and research iterations, it was apparent that we needed to define what success meant 
for us and our users. These first principles were developed, shared and used to provide clarity when designing a product for Toyota or Lexus. One major breakthrough when collaborating with UX teams across Toyota was defining what it was we
were trying to create; That the aim of developing self driving / semi-autonomous vehicles is to reduce the mental and emotional labor of driving. This helped to focus efforts on features and functionality that were in service of realizing that principle. Principles were derived from intuition, experience, insights from user research and data.
Enhance Safety with driver-in-the-loop HMI
Offer users proactive driving support to better aid them in understanding & trusting their vehicles. Take users beyond a base level understanding of their vehicles safety systems and teach them defensive driving skills, better ways to drive to improve passenger experience and safety or help increase track times.
The car can observe all user behavior; gaze tracking, response times, response triggers, concerns, various driving habits to guide users where it is most helpful and adapt to their behavior to support rather than to intrude or interrupt.
Title
Create an adaptive learning system for Toyota and Lexus vehicles that builds a unique user profile based on input 
and ongoing road data. This profile adjusts to the user's behaviors, preferences, and driving ability, providing tailored support, information, and training on features like ADAS. Users experience personalized interaction, accessing their profile in any Toyota or Lexus vehicle for a seamless and customized driving experience.
Users can benefit from the behavior of the Toyota and Lexus fleet - from analyzing the optimal drive lines across any stretch of public road, the vehicle can grow more confident in it’s autonomy and literally expand it’s scope 
of capability from learning from real world circumstances. This will change how we design and test for ‘outlier’ scenarios and make the road safer and more predictable for our systems.
Title
Implementing a focused ADAS strategy enhances user experience by making the system intuitive & rewarding to use. Unified cockpit design clarifies driver support capabilities. Configurable safety settings increase user control, encouraging adoption. Clear semi-automation standards and scalable mental models improve situational awareness and reduce mental effort. Multimodal alerts, incorporating lights, tones, haptics, and voice, enhance 
user awareness and response times for a more natural communication approach.
This allows users to build trust in Toyota as opposed to being dependent the capabilities of a single car - use collected and real-time data gathered from fleet to increase vehicle vision and user awareness, be able to share real time road conditions to better adapt to changing conditions, improved automated driving behavior.
Dynamic camera movements to optimize driver focus
Natural transitions between Hands ON & Hands OFF modes
Title
With big data, the ADAS and interaction capabilities of mobility platforms could continually evolve and adapt to our user’s needs and preferences. Driving confidence of ADAS systems will increase dramatically leading to increased trust, normalization of ADAS usage and higher adoption rates.
Reducing the cost of HD maps by learning from all drivers who have ever shared the road before - reduces the singular reliance on sensors on each users car by distributing data of all ADAS capable Toyota and Lexus vehicles in the area to enhance confidence of real-time conditions as well as let the user experience a more comfortable and stress free drive.
Gaze activated automated Lane Change Assist
AR HUD visualization to show user what the car sees
Drive Better Together
User is able to enjoy improved driving experiences with a vision enhanced by surrounding vehicles, not just reliant on the limitations of a single vehicle. Would be able to see occluded objects, more confident view, be able to plan further in advance.
Users would benefit from other Toyotas in the area. Find available parks they have seen. Toyota and Lexus can build further trust in the brand rather than have that trust be limited and exclusive to the users own vehicle.
Gaze activated automated Lane Change Assist
AR HUD visualization to show user what the car sees
Title
With Vehicle-to-Vehicle (V2V) connectivity, driving becomes a social experience. V2V enables safer convoy modes for higher-speed travel, while sharing eco-stats, unique routes, and even multiplayer games. Real-time hazard alerts enhance safety, and entertainment options make cars hubs for socializing. V2V liberates drivers from isolation, transforming the road into a vibrant space for connection and collaboration.
Not only can you make mobility richer with people to people moments, being able to be connected to local shops and services could add tremendous convenience to users. Join local meet ups, use your phone as a connected input device, be part of local online communities and share tips and experiences. Seize opportunities to connect people 
and create moments that matter.
Lexus Experiences
Lexus members enjoy exclusive curated experiences that enhance their mobility journey, offering enhanced convenience, comfort, and excitement. From bespoke road trips to all-inclusive holiday planning and thrilling "fun-to-drive" moments, members benefit from a seamless and luxurious mobility experience. With smart trip planning tools, navigating journeys becomes effortless, ensuring every moment behind the wheel is optimized for enjoyment.
These experiences extend beyond the vehicle, allowing members to create and share memorable moments with loved ones, enriching their overall Lexus ownership experience. Through these curated offerings, Lexus redefines luxury vehicle ownership, fostering a sense of community and connection among its discerning clientele, and making every journey truly memorable.
Title
Open up Toyota and Lexus vehicles to enable business partners to easily integrate their services. Uber, Lyft, Japan Taxi, delivery services, passenger experiences, real-estate agents, and the list can continue to grow! Offer an integrated experience rather than having users tack their mobile device on the dashboard. Toyota can profit off of each transaction, license usage and provide continual development support.
Augment the boring parts - offer services for mechanics to pick up, service and drop off vehicles for routine maintenance, easily schedule time to fix and understand what work is done and why. Trusted cleaning services can also be requested by 
the user. A more well looked after and well maintained vehicle is more trustworthy, safer and will be cared for.

Design Explorations

Notable Accomplishments
at Woven / Toyota
01
Understanding Toyota
Understanding the mobility space
02
Lexus Teammate improvement
Instrument Cluster bucketing
IC architecture restructure
UX Autonomy Model v1
Rated safest ADAS system (IIHS)
03
Implementing a data driven design process at Toyota
04
Change through communication
Building relationships
Building team culture
05
Learning
Diving deep to understand aspects of Trust, safety, delight, Levels of autonomy, technical feasibility, conceptual thinking, innovation. 
Ideation - full mobility experience.
06
Failing - Lots of failing
Happy to share any of our failures in communication, strategy, design hypothesis, organizational change, learning about the mobility space. Feel free to ask.
07
Japan Motor Show
Lexus LF-30 Concept Interface 2019
Lexus LF-ZL Concept Interface 2023
08
Arene Interface v.1
09
Company Branding
TRI-AD, Arene, Woven Planet, Woven Core, Woven Capital, Woven Alpha & including almost all collateral.
10
ADAS & Mobility UX
New ADAS Mental-Model, Steering wheel redesign for 1-button ADAS activation, Onboarding Design, Developing Scalable solution, 2025 Cycle Production Design Direction, Integrated SA with Navigation, Defined Design Philosophy, Modality configurations for varying levels of criticality, Take over request design, Auto-park UX, Delivering Innovative ideas and POCs to Lexus, implementing dynamic Situational Awareness view to production at Toyota
and Lexus.
See Next Project

Please view on a desktop. It'll be more fun I promise.

Or mobile portrait works okay too...