Amplifying the Automotive Human-Machine Interface Design with Generative AI – Case Rightware

The use of AI in the automotive industry is on the rise. In collaboration with Rightware, an innovative automotive HMI software and services company, we're using generative AI to enhance the development of Human-Machine Interfaces (HMI).

Rightware is the pioneering provider of automotive graphics software tools and services, and their
‍Kanzi One toolchain is trusted by over fifty automotive brands worldwide. Kanzi One is the first all-in-one toolchain for the creation of signature UIs in the modern, intelligent cockpit. Kanzi One provides the most capable automotive-centric graphics engine, the deepest integration with Android, and a new UI framework for optimal workflow and performance.

Setting the scope – optimizing HMI design workflows for greater efficiency

When partnering with Silo AI, Rightware was looking to optimize their HMI design workflows for greater efficiency. They began by defining their scope through a series of questions that helped them determine exactly what they wanted to achieve.

Why 

Rightware had to determine their motive for developing generative AI by answering the question of why. While the technology had shown promising results and evolved significantly in the visual context previous years, they did not want to develop AI just for the sake of developing AI. As a technology-focused company, Rightware had a deep interest in exploring AI, and their primary objective was to deliver substantial value through innovation.

How 

The next step was to determine how to proceed. Rightware had some internal capabilities, but, not the time to focus fully on AI. The options were to either prioritize internal development, hire new expertise and gradually expand the team, fully outsource, or find a collaboration model with a partner. After careful consideration, a longterm, strategic partnership with Silo AI was agreed. Silo AI’s world-class AI expertise made it an obvious choice to pair with Kanzi One’s capability to quickly implement innovative features. It made for a “best of both worlds” solution.

What

Finally, before kicking off the development, Rightware needed to clarify their objectives and determine what they wanted to achieve. They began by defining how to add value to end-customers by incorporating AI functionality into HMI design workflows. The collaboration’s ultimate goal was to recognize automotive UI elements and use generative AI as a seamless part of the creative and development process.

Project walkthrough – AI development for HMI design

After Rightware had clarified the whys, hows, and whats of their desired outcomes for the generative AI development for HMI design, we kicked off the collaboration with the following steps.

1. Defining the Concept

Defining a clear concept was the first crucial first step. We faced the challenge of identifying the elements in the HMI design. Our primary questions focused on breaking down the design into a more semantic model. We aimed to transform this semantic model into a practical format that would be compatible with Rightware's Kanzi One toolchain. Our ultimate goal was to reconstruct the digital cockpit display image into something entirely different and visually distinct.

2. Annotating and Training

The development process began by annotating hundreds of diverse cluster images. Initially, we trained the model to identify the different components present in the cluster images, such as speed, RPM, navigation, and others. We also trained the model to differentiate between various visual styles, including modern, minimalistic, and rich designs. To improve its functionality, we integrated text recognition into the model.

Our focus was on developing user-friendly tools that could extract readings from different elements identified by the model. These tools were crafted to integrate seamlessly with Rightware's model, ensuring practical usability in real-world scenarios. At the same time, we also integrated support for deconstructed images back into their flagship product, Kanzi Studio.

3. Innovative Design Generation

In the next phase, we leveraged the extensive training data accumulated from thousands of images in the initial phase. This helped us create an AI model that can generate visually appealing cluster designs. Designers were given increased control over the generative AI to enable them to define not only the visual appearance but also the layout and boundaries of the generated design. Once again, a user-friendly interface was at the core of the development, combining graphical user interface control, UI control, and parameters with freeform prompts. 

Bringing designs to life: the tool in action 

With the Kanzi One AI feature, designers can easily specify the number of gauges and choose their preferred bezel or frame. The generated designs adhere to the boundaries of the original guiding image or pseudo-image. The system can produce various contemporary cluster designs in just a few seconds. This efficiency is particularly noteworthy compared to traditional methods that typically require weeks of manual effort from designers to create such visual content. AI-driven design generation significantly accelerates the creative workflow, offering a paradigm shift in how designers conceptualize and produce visual content. This workflow can be seen in the following images, showing the steps to the multiple design options in the final picture, with the whole process taking seconds to complete:

The video below shows the tool in action, demonstrating how it works and behaves. As the video demonstrates, it is possible to experiment with various visual styles and get inspiration for upcoming projects by inputting prompts. Parameters to explore different styles suitable for different regions and states across the Americas, Europe, and Asia are easy to adjust. The AI feature in Kanzi One also recognizes elements such as cars, speed, etc., making it simple to identify and remove them and even replace them with real-time 3D models. Additionally, designs can be made more interactive and engaging by replacing static UI elements with dynamic counterparts. The tool balances between automated and manual control to refine and organize visual elements seamlessly. 

What sets this AI feature apart is its seamless integration with Rightware's production runtime. This means that moving from prototyping to production is fast. Design concepts can be taken directly into a production vehicle to test and refine them in a real-world setting. However, it's important to note that while this demonstration offers a glimpse into the rapid transition from idea to functional prototype, there are additional considerations involved in getting to a production-ready program. This prototype is evidence of the efficiency and speed the tool brings to the creative process.