2016 - 2018
About
Workflows is a net-new data orchestration solution to be introduced into our platform. It stitches and scales existing capabilities across our technical suite -- including integrations that would repeatedly push / pull data to query for marketing segmentation, analytics, and syndication -- and enables our clients to orchestrate real-time data pipelines to modularize and digitize their domain operations for repeatability, scalability, and collaboration.
Advanced data orchestration functionality needed to be migrated from a command line prototype and unified with our GUI-based product ecosystem. However, the original architecture was designed for a highly technical user while our new market vertical demanded a tool that would fit less technical users. Data objects had to be audited and re-calibrated to bridge operations between roles involved in interacting with different parts of the same data infrastructure.
Data Management
Data Management
Data Science
Data Science
Start-Up
Start-Up
World Assemblies
World Assemblies
Democratizing Data Operations
The entire data operations team would rely on Workflows and therefore objects need to be designed with a suite of users and attendant skills in mind. These users had varying levels of data savviness as defined through their native tools (i.e. an IDE vs. Tableau). Competitive research done on common tools like Marketo revealed that analysts often hit a ceiling on what they could do and wanted more power. Given that our buyers were both SMBs desiring turnkey solutions and enterprise customers who needed high levels of configurability, our product goals were to design an experience that would meet the needs of both by strategically architecting an experience that could be generalized into extensible templates in the future while maintaining our core strength of enterprise-grade configurability.
Platform Components
Workflows is an engine that allows our clients to orchestrate real-time data pipelines to modularize and digitize their domain operations for repeatability, scalability, and collaboration. It stitches together capabilities across our suite of products, including integrations for pushing / pulling data from multiple sources and queries for analytics.
Users Matrix
I led the development of a user skills matrix to understand the make-up of teams and organization prior to kicking off this project (bottom). Depending on the size of the organization, Workflows may be maintained by a data engineer, but marketing / data analysts as well as data scientists may need to build their own pipelines for optimizing campaigns as well as for building machine learning models.
Upskilling Analysts
The entire data operations team would rely on Workflows and therefore objects need to be designed with a suite of users in mind. These users had varying levels of data savviness as defined through their native tools (i.e. an IDE vs. Tableau). Competitive research that analysts often hit a ceiling on what they could do and wanted more power with industry-standard tools like Marketo.
Given that our buyers were both SMBs desiring turnkey solutions and enterprise customers who needed high levels of configurability, our product goals were to design an experience that would meet the needs of both by strategically architecting an experience that could be generalized into extensible templates in the future while maintaining our core strength of high configurability in the present.
Given that our buyers were both SMBs desiring turnkey solutions and enterprise customers who needed high levels of configurability, our product goals were to design an experience that would meet the needs of both by strategically architecting an experience that could be generalized into extensible templates in the future while maintaining our core strength of high configurability in the present.
Data Management Platform
Data Management Platform
System Audits
To get a grip on what I was working with, I audited the technology, the interface, and the experience.
Enterprise Technology: Power & Simplicity Trade-offs, As Usual
Working in B2B SaaS, our enterprise clients had complex configuration needs as defined by IT. Our product is able to flex with these needs with custom configurations, but the consequences of such power is increasing complexity in sane information management, a proliferation of edge cases, and the challenge of needing to work at a higher level of abstraction from the underlying raw data sources which incurs cognitive costs and can be unpredictable.
Requirements 1
Given the complexity allowed by workflows, maintenance can become challenging, especially as employees turnover and take their knowledge with them. The system needed to maintain its flexibility for us to meet our enterprise business goals, but we also needed to provide an experience to help users reduce the number of edge cases by helping users understand the orchestration system they were designing so that they could get ahead of and manage existing and introduced complexity.
Requirements 2
Information needs to be architected for navigation that would passively onboard users into the system and help them understand its components.
Requirements 3
Information needs to be chunked to give users relevant context at the point in which they are executing on a task.
Engineering-Led Interfaces: Links to Everything, Everywhere, All at Once
A quick information architecture audit of the engineering prototype demonstrates (not surprisingly) that great attention was paid to the system architecture to ensure the robustness and performance of our client's data operations, but little attention was given to how a user would navigate it.
User Experience: Not Enough Words in the World
A series of user interviews were performed with beta testers to capture feedback on the prototype.
User feedback indicated that data analysts frequently misunderstood the concept of a ‘$session’ -- a far more dangerous
problem than simply not understanding what a concept is.
User feedback indicated that data analysts frequently misunderstood the concept of a ‘$session’ -- a far more dangerous
problem than simply not understanding what a concept is.
1
$session already has a meaning outside the product.
Users had pre-defined ideas of a workflow $session as being like a $userSession and/or $browserSession which, while similar, is still significantly different in purpose.
2
$session is incompletely understood within the product.
When a new $session is created, users relates $session to processing logic ($workflow), but not to data being processed ($sessionTime). Concept of $session includes both $sessionTime and $session because $sessionTime identifies the $session.
3
The purpose of a $session is especially unclear with less technical users who do not understand how to operationalize data.
$session’s usefulness is indeterminable until contextualized in error handling because it was designed to prevent data duplications and data drops. However, users need to define the $session in order to run the workflow.
Requirements 4
Data operations is pure software play and software objects are invisible. Unfortunately, this means that software companies like to come up with fancy new words to brand their not-so-new offerings. This creates a cluttered ecosystem. Treasure Data has an open source DNA so we preferred to stick to the way the ecosystem describes things. If
$session is overloaded as word to describe the object, we need to pick a new word that organically occurs with use, one that, ideally, reduces the multi-syllabic word salad problem that often occurs when trying to describe the things in the system.Requirements 5
$session and $sessionTime need to re-named and/or re-architected for clarity. Words are important in that our sales and sales engineering team will repeatedly use them to describe what is happening, and a poorly chosen name impedes learnability. Further, it will show up in documentation and a poorly chosen name becomes obvious in technical writing. System Audits
System Audits
Workflow Object Lifecycle Analysis
Workflow Object Lifecycle Analysis
Once the system components were re-factored for learnability, a task analysis was performed on the workflow object to identify user and product gaps.
Gap Analysis for Tasks
Questions were identified for each user task associated with a
$workflow.
Workflow Object Lifecycle Analysis
Requirements 5
The design of information architecture needs to be structured around monitoring because it was the stage in which failures can occur once a
$workflow is in production. Enabling users to resolve issues quickly is synonymous with good navigation.Generated Questions
• What do users what to confirm?
• What do they want to be notified of?
• Are there spectrums of severity?
• What are the action steps to resolve a problem?
• Who does it and why?
• What do they want to be notified of?
• Are there spectrums of severity?
• What are the action steps to resolve a problem?
• Who does it and why?
Error State Object Inheritance
Error State Object Inheritance
Error State Object Inheritance
The state of objects in the platform communicate errors in the system to users.
Object-Relationship & Error State Inheritance Analysis
The system components tied to a
Through this exercise it became clear that a
$workflow are hierarchical with multiple levels of states across different objects. These were structured to understand the direction of state inheritance.Through this exercise it became clear that a
$workflow is an unideal object to communicate child states-- there are too many of them for each client based on a quick query of the average number of workflows created per account. At the same time, $project were created to organize $workflows, but they have no communicative state properties that could be surfaced. 
Requirements 6
Since users need to understand at what layer an error occurred to resolve an issue, we needed to design for effective system communications. Since
$workflows was too proliferate to be meaningful, I designed a new property to the higher level $project that would communicate the state of a set of $workflows to the user, so that they could have a bird’s eye view of how they were operating. Adding a summative state property to the $project object would also make the purpose of the $project object more immediately clear to users by providing more context to the $workflows inside, creating an entry point for rapid drill-down.$project was later re-named to $group to be more agnostic about the intent for the collection of items.Generative Design Workshops
Interface ideas were generated in workshops across the organization to assess for potential user / product needs from internal stakeholders.
Product Design Explorations
With the architectural problems identified and resolved, the backend could meaningfully support a front-end experience that meets user needs.
Monitoring Iterations
The top-level page for workflows was designed for monitoring, and iterations focused on structuring complex information for drill-down given the re-architecture. Early versions focused on an object-oriented (projects / workflows / tasks) vs. process-oriented (workflows / sessions / runs) display.
Workflow Visualization Explorations
I explored how workflows might be visualized in a drag-and-drop interface. While most of our users were currently power users, we anticipated that the construction of workflows in the future might need a visual editor. Our workflow run definition overlay should display the visualization parallel to its YAML configuration to help with spot-checking structure and passively training analysts for upskilling.
Product Line Vision
Product Design Explorations
Product Line Vision
I produced a medium-fidelity vision for the product line that included the full set of features that would then be scoped for execution based on appetite and ambiguity.
0.0 Workflows Monitor
0.1 Workflows Directory
Information Architecture
The process-oriented timeline and object-oriented directory views were combined into a toggle. States remained persistent across both views to aide in error handling.
$group was chosen as the top-level object on the root screen given the number of existing $workflows. A filter was introduced to navigate across $workflowStates. Tags were introduced for bottom-up navigation to provide a mechanism to organize similar $workflows across $groups.0.1.1 Workflow Session History
0.1.2 Workflow Definition
Workflow Session History
Cross-Navigation
Cross-Navigation
In directory mode, clicking on a
The choice of an overlay visually reinforced a
$workflow surfaces all its $sessions in an overlay. Terminology was carefully calibrated on the overlay to shape the concepts associated with a $workflow. Through columnar adjacencies, the user is able to understand a $workflow has a $session history with $runs. Two date-time parameters are shown to reinforce that there is a difference between the data scoped to $sessionTime and the data processing kick-off of $runTime. The $workflow definition is shown at the same level of the $workflow $session history to indicate that the definition is shared across $sessions.The choice of an overlay visually reinforced a
$session as a data management middle layer. 0.1.1.1 Workflow Session Run
0.1.2 Workflow Definition
Product Line Vision
Workflow Session Run
Drill-Down
Drill-Down
Clicking into a
The
Users can click on the
Timeline UI details changes subtly to indicate a shift in scale.
$workflow on the timeline would maintain monitoring mode on-click to show the progression its latest $run at a $task and $subTask level. With the $run as the lowest-level object and site of failure investigations, the user is able to bypass selecting intermediary $sessions to target specific $runs.The
$run page reflects a $session in the header component to simplify the mental model further. The $runId and $attempt are treated as details instead of first-class objects.Users can click on the
$session's $workflow to surface the same $session history overlay that displays a list of all $runs.Timeline UI details changes subtly to indicate a shift in scale.
Product Line Vision
State Design
States for $groups, $sessions, $runs, $tasks, and $subtasks were designed for coherence and structural consistency.
State Design
Existing States & Visual Language
New States & Visual Language
New State Inheritance
State Design
