AI Vision for Field Service Technicians

Empowering technicians with AI and AR to see, diagnose, and fix problems faster — an Apple + IBM collaboration that cut diagnostic time in half and boosted first-time repair accuracy by 30 %, showcased on Apple.com and at the IBM Think Conference.



Solution

Apple + IBM Partnership

Customer(s)

Apple + IBM Partnership

Deliverables

Mobile App

Introduction

As part of the Apple × IBM MobileFirst partnership, I led the design of an AI-powered mobile solution that helps field service technicians diagnose and repair complex machines using computer vision, machine learning, and AR guidance.

Before this work, repair crews relied on manuals, tribal knowledge, and phone support — costing hours per job. The opportunity was clear: use Apple hardware and IBM Watson intelligence to create an assistant that learns from every fix, turning expert knowledge into an intuitive, guided workflow.

Impact

↓ 50 % diagnostic time through automated issue detection
↑ 30 % first-time repair accuracy across pilot groups
Showcased at IBM Think Conference and featured on Apple.com
Generated new enterprise pilots with multiple Fortune 500 clients

By merging intelligence with design craft, the app demonstrated how mobile AI can directly elevate frontline productivity and customer satisfaction.

Project gallery

Product Story — The Golden Thread of Assisted Work

The experience was built around a “day-in-the-life” narrative that keeps the technician’s flow unbroken:

Understand — The app prioritizes daily jobs and surfaces equipment history before arrival.
See — Computer vision identifies parts and likely faults in real time, showing confidence levels for transparency.
Act — AR overlays walk the technician through repair steps hands-free.
Learn — Each interaction improves the model and future recommendations.

Key design decisions included:

Context-first interface for noisy, low-light environments
Confidence scoring UI to build trust between AI and user
AR repair guidance that visualizes physical tasks rather than text lists
Feedback loop that turns technician corrections into new training data

Reflection

This project embodied what I call designing intelligence around people, not in place of them.
It redefined enterprise service from a manual workflow into a collaborative dialogue between human skill and machine insight — setting the foundation for future AI-first onboarding and assistance systems.

No items found.

Interested in working together? Get in touch today.

Ready to bring clarity, strategy, and momentum to your next big initiative? Let’s talk about how we can make it happen.

