back
Virtual Device System Architecture
2019 - 2020
Tulip Interfaces / Enterprise Governance & Edge Computing
A logic model for programming connected tools at scale.
Product Design Lead
Problem
The user experience was suffering from technical debt stemming from how physical devices were modeled in the software. Conversely, how physical devices were modeled in the software was limiting the user's ability to manage fleets of devices.
User Stories
As an app builder, I want to be able to define app events that relate to what's happening at my station. I want to be able to say "do this when this breakbeam gets crossed" as opposed to using the language of GPIO pins.
As an app builder, I want to be able to define app actions using the reality of output devices at my station. I want to be able to "show error indicator on the andon" instead of thinking about the hardware involved in this process.
As an app builder, I want to my apps to be separate from the physical configuration at my station so that I can use the same app at a different station, even if the same station is wired differently.
As an app builder, I want to use two devices of the same driver connected to the same gateway/player.
View Persona Map
View Platform Diagram
What is a "device"?
Problem Architecture Map
Cross-functional alignment requires explicit documentation to drive conversations forward effectively (and efficiently). Using the existing and proposed solutions identified during the workshop, I decided to construct a map of the problem space. The goal was for everyone to see where their solutions were located in relation to a problem architecture so that we could identify the most promising course of action. I started with the various internal definitions of a device and mapped down to their impact on internal teams, unfolding the technical and user challenges in-between. By making the endpoint the impact to internal teams, groundwork was also laid early on to be further decomposed into objectives and key results (OKRs) as the project concretized.
1
This project was both difficult technically and tricky organizationally. To scale the platform quickly, each piece of a "device" was historically built and owned by a different team. We ended up with a classic case of Conway's law impinging on the end experience.

To kick-off the project, I ran a cross-functional workshop to understand everyone's perspective (problem definition and solution ideas) when it came to feature enablement. This would help me align stakeholder concerns to push the project through when it came to making engineering and design trade-offs downstream.
Learning 1
Each team had a different definition of a device.
Learning 2
Broadly construed, devices were defined in different ways within the platform, leading to a fragmented user experience.
Learning 3
The IT archetype needs to be strategically managed. There are some tensions here as devices are managed by OT but present a large threat surface which is IT's responsibility to manage. The working intersection between OT and IT is still evolving within manufacturing.
2
To learn more about how each internal team defined devices, I ran eight rounds of open and closed card sorts to generate more specific mental models. This forced the participants to concretize and work through their models instead of giving me unanchored facts (and opinions).
Head of Product's Card Sort (Left)
IoT Engineer Card Sort (Above)
The results successfully reflected how each person interacted with the concept of devices in ways that were reflective of their job within the organization. For example, our Head of Product categorized devices according to how form related to function because he was thinking of how the user would assume an interaction with the platform. Meanwhile, our IoT engineer thought of devices in terms of how they "behaved," or how their data needed to be configured and managed in relation to the Tulip platform.
Learning 4
Network protocols are tied to devices and are the first touchpoint to set-up, but only 1 stakeholder mentioned them explicitly as part of a "device." They're also the part of devices that IT cares about most.
3
Having gathered the technical requirements and challenges strewn across various teams, it was time to synthesize. The first pass at a device architecture took all our devices, internally and externally packaged and manufactured, and positioned them at various levels of abstractions from the network level down to the computational level.
Learning 5
As a part of this exercise, I identified the need for a middle layer doing conversions between different units. For example, a user would want the "weight" for a part, but "weight" could be in pounds or kilograms, and may be user-defined or manufacturer-defined by default. Handling this for an operator at an admin level would reduce errors, simplify set-up, and streamline reporting, especially as factories were often located around the world and data needed to be rolled up. We often think of internationalization as the translation of language, but units as well as timezones are primitives that can also be considered to fall under the same umbrella.
Device Architecture v.1
The first pass at a device architecture took all our devices, internally and externally packaged and manufactured, and positioned them at various levels of abstractions from the network level down to the computational level.
What is a "camera"?
Device Architecture v.2
With all the devices structured and positioned, I started to simplify the model by creating an enumerated type system of different device types. These enums were defined according to how the user wanted to use our platform, which goes beyond the use of the device itself. This ended up mirroring the relationship the device had with data which is rather intuitive in hindsight given that our clients used Tulip to digitize their operations.
Learning 6
I was unconvinced that "machine vision cameras" were differently different from "cameras." While they were developed for a specific use case in Tulip (operator guidance), my intuition was that we needed to de-couple them from use cases. While "cameras" and "machine vision camera" are similar forms, image and video data are fundamentally different from tabular data from a processing standpoint. The space of what a Tulip user could potentially do with images and videos is also fundamentally different. For lack of time, I put a pin in it. The question of image and video data still haunts me.
Device Conceptual Model
The device architecture was collapsed into a simplified conceptual model. Edge cases identified during the card sorts were included in its development as breakpoints for modular design. While I had taken a first stab at the enums, pulling the edge cases back in demonstrated that some of them had to be composable into higher order objects that mapped back to our custom kits packaged for common manufacturing use cases. While the conceptual model was initially developed with explainability to the user in mind, it also ended up helping our internal teams map strategy back to technical product requirements.

This conceptual model also took a first pass at the product terminologies breaking down the amorphous mass of devices into smaller concepts that mapped to the platform architecture. All this would help users orient themselves to the connected world that is the Tulip platform.
4
The device architecture and enums represent how our platform needed to interact with devices to enable our core value prop of "digitization" (data processing), but they were not representative of how our users saw them. Although our users wanted to manipulate data, the majority understood its computation as arithmetic; they were not data literate in a procedural programming sense. Devices were still conceptually rooted in function and purpose: what they did and why in relation to manufacturing lines. How a device processed data was only a means to an end for our users.

This presented some challenges as the function and purpose of devices are wildly open-ended. Defining all of them in advance would be equivalent to defining all the possible products anyone would ever make in advance: a foolish task. I needed to develop a conceptual model that balanced technical requirements with functional/formal intuition to present a system image that would guide our users' understanding of the platform. Currently, users all had different mental models of the platform, impending their ability to effectively accomplish tasks and troubleshoot errors.
Device Concept Definition
Along with distilling the structure of the device architecture further, I approached the concept definition as an integration problem and included all the attributes that would provide an endpoint linking a Tulip device into other parts of the platform. To start managing the complexity and provide some form of progressive disclosure downstream, I started considering how some of these "attributes" could be pulled out into their own modules. I tested the concept across multiple devices to assess their fingerprints and find any misfit scenarios.

Concepts are my absolute favorite thing to design. Properly translating the things-in-the-world into a coherent JSON blob allows you to skip "lean" development and go straight into sane roadmapping. You're able to get ahead of the little tiny feature requests and swiss cheese data structures that make the user confused and the product bloated. That devices were not assessed as critical concepts within the platform early on led to many of the problems we encountered today. It doesn't take any lean development paradigm to figure it out: just look at the domain and start there.
5
Finally (!), after gathering and structuring all the technical requirements into a conceptual model, I could telescope in and start design!... of a Tulip device concept that would describe the device to a user and an application.

As a course of this research, it was evident how little control we had over a device. Most of the challenges were from the heterogenous ways that manufacturers had decided to set up their device configurations and defaults. This pointed to how definitions would need to be surfaced and aligned from the manufacturer, how we would have to introduce some of our own for usability and findability, and how the user should be able to edit the definitions of most parameters for the use cases they were trying to enable after some smart defaulting on our part. It also showed how we would have to continue to modularize devices into configurations that could be shared across the set.
Learning 7
Network protocols are killing me. After creating the device concept, There was still something crunchy about them, a sign that they probably needed to be pulled into their own object, especially since it was being managed by IT. My only concern was whether this would slow down OT deployment initially.
Conceptual Data Model v.*
After working the network protocols a bit more and relating them to the other existing and proposed objects in our system to shake out the crunchiness, I developed an object model that covered the surface area of the device attributes identified. I find that working at two levels of abstractions simultaneously is necessary for creating a system, and working quickly allows you to jump up / down without breaking mental continuity. Sketching is the quickest and best medium for this.
Conceptual Data Model, Solidified
After the objects and relationships are hammered out, it needs to be documented for team assessment. Certain design cases and product opinions surfaced during research that would be relevant to the engineering team are called out. For example, we can see in the relationship between displays and stations how the business bakes its strategic opinions directly into the data model. Being able to have a diagrammatic view of what is linearly represented in the code / text base opens up new collaborative avenues for discussion about structural product decisions. Proposed changes to any of these parts are understood in relation to the whole, allowing for more fluid and precise mental simulations of alternative systems for risk mitigation.
Object Definitions, Solidified
In keeping with working at two levels of abstraction, all proposed objects are defined as well. Each object is given a purpose that is tied to a user's job-to-be-done. Designing objects explicitly around users at this stage ensure that flows and interactions are streamlined and handled in the backend for smoother frontend development and sane design evolution. With this structure in place, design thinking and agile development processes can proceed without incurring outsized engineering and design debt. While the data model is primarily a result of internal constraints and product vision, the user's conceptual model of the system is reflected in the objects' purpose, its data attributes, and relationships. These directly influence the information that can be displayed and how easily they can be updated.