Modernizing mission planning software for the US Navy and Air Force — and what 13 active duty pilots taught us about the cost of software friction.
I arrived at Naval Air Station Oceana in Virginia Beach, badged in with my CAC card, and met with the 2Circle team — a veteran-owned defense technology firm — to plan the day. Our goal was specific: we weren't testing usability in the traditional sense. We were testing discoverability.
Thirteen participants filed in. F-15, F-22, and E-2 pilots alongside intelligence officers. They were handed a high-stakes strike planning scenario and tasked with building a mission plan from scratch using the Unified Planner.
I sat directly behind the pilots as they worked. My job was to watch, not to help. I tracked every moment of friction — where they paused, where they backtracked, where they leaned forward looking for something that wasn't where they expected it.
One thing I didn't anticipate: how quickly the pilots apologized. When the software confused them, their first instinct was to assume they had done something wrong. We reassured them constantly. That dynamic alone told us something important.
Users struggled to remove shapes, threats, and entities across multiple contexts. One participant renamed an item "delete me" as a workaround — then still couldn't find the deletion function and moved on.
Deletion interaction patterns were flagged as a priority redesign. Right-click context menus and clearer destructive action affordances were scoped for Q1 implementation.
Users were forced to learn the system's internal structure before they could begin planning. Prescribed step sequences didn't match how pilots actually think about building a mission.
Workflow flexibility became a core design principle — users needed to start from what they knew and fill in details as the picture developed, not follow a software-imposed sequence.
When automated route generation from ATO data worked, the reaction was immediate and genuine. The capability existed — but buried under friction that prevented most users from discovering it.
The ATO import flow was identified as a high-value feature requiring a discoverability redesign — not a rebuild, but a clearer path to a capability users genuinely valued when they found it.
The system grouped friendly and threat COA data in ways that violated how operators mentally separate these categories. This wasn't a preference — it was a fundamental IA mismatch.
Blue force vs. red force separation was added to the IA backlog as a structural issue affecting multiple areas of the system — not just the COA builder.
Multiple users raised low-bandwidth deployment scenarios — ships, forward operating locations. Tools relying on heavy data transfer become effectively worthless at sea. This dimension wasn't accounted for in the design environment.
Bandwidth-sensitive design considerations were escalated as a pre-deployment risk — particularly for features relying on real-time data sync and map rendering.
The timeline was the single most consistently criticized element across all participants. Multiple users couldn't complete timeline-related tasks without help. 2–3 participants had nearly identical failure experiences.
The timeline tool was scoped for a full redesign in Q1 — one of two direct, named design actions that resulted from the NAS Oceana session findings.
The post-session guided discussion sharpened what observation had surfaced. The most pointed moment came from the 2Circle lead running the event.
The 2Circle lead — with deep operational knowledge — argued that the product had enough fundamental issues that pilots actually needed an airwing-specific planner. The Unified Planner should be used only when joint coordination was specifically required. That's not a UX note. That's a product strategy question.
The debrief surfaced a pointed question that lingered: at what point does repeated user feedback become undeniable enough to change product direction? Thirteen participants surfaced the same themes independently. The signal was consistent.
The ability to reference and build on another planner's work was called out positively. Real-time visibility into what other users were changing was meaningful. The ATO route automation, once discovered, was the session's clearest moment of delight.
F-22 pilots apologized when the software confused them. This pattern — expert users assuming personal error rather than system failure — is a strong signal that feedback and affordances are not meeting the moment.
In a pre-deployment government contracting environment, impact looks different than a consumer product dashboard. The pilots who struggled at NAS Oceana influenced design decisions made in reviews weeks later.
Timeline tool — full redesign scoped for Q1. The most consistently criticized element across all 13 participants. Named directly in the post-session report.
Map inconsistencies — dedicated task force formed to address cross-cutting rendering and interaction issues surfaced during the session.
New capabilities and user stories written based on synthesized findings, presented to the full LUXCE team and scoped for Q1 implementation.
Observational notes compiled across all 13 participants covering navigation struggles, task failures, moments of delight, and verbatim quotes.
Guided debrief facilitated with active duty pilots — structured discussion to surface themes not visible during observation alone.
Findings synthesized and presented to the broader LUXCE cross-organizational team, with prioritized design implications and open questions flagged.
This project shaped how I think about the relationship between user expertise and software design in ways a consumer product project never could have. Elite users are often the worst served by complex software — because their expertise gets blamed for their confusion. When an F-22 pilot can't find a target selection tool, nobody assumes the software failed them.
It also taught me something about research craft. Restraint is a skill. Sitting behind a pilot who is visibly frustrated and not intervening — because their struggle is the data — requires discipline you only develop by doing it. The temptation to help is the enemy of insight.
Working within a cross-organizational team across MTI, MITRE, and Northrop Grumman — each with different processes, priorities, and communication styles — was its own education in navigating complexity. Good design in that environment isn't just about the work. It's about alignment, communication, and knowing when to push and when to listen.