Visualization and UX Challenges in Utilizing Complex Multimodal Data

While tackling storage, harmonization, and computation of complex multimodal data poses significant challenges, as discussed in previous entries, ultimately, the data and results are meaningless unless users are able to translate them into insights. Producing usable visualizations and user-friendly UX for complex multimodal data presents hurdles as large as data management itself, especially when diverse users with varied use cases access the same data for vastly different tasks.

Disparate Data, Disparate Visualization Needs

The inherent nature of multimodal data, characterized by its diverse data types, presents a significant visualization challenge. It’s clear that a visualization solution suitable for visualizing someone’s genomics won’t work for their medical imaging. While the logical response is to employ the solution most appropriate for each data type, seamlessly integrating these solutions into a streamlined UX is far less obvious. The ideal UX for multimodal data types should minimize task-switching for users and present diverse visualizations in a way that intuitively highlights the interrelationships of the underlying data within a sample, patient, or population.

Diverse Users, Same Data

Even within the same data type, the needs of one user are often very different from those of another. Imagine two individuals accessing sets of genomics data from the same sequencing assay. A researcher investigating population genomics would require a relatively ”raw” view across many samples, in contrast with a clinician looking for a prioritized list of relevant and actionable variants with annotations about their effects and associated treatments for a specific patient.

As with differing visualization needs across modalities, meeting the needs of every user with a single visualization solution can be highly challenging. However, many visualizations can often be customized for specific user profiles, revealing or concealing details as needed. For example, a tabular view of genomics data might expose population frequencies to a researcher rather than the clinical trial lists displayed for a clinician.

Diverse Tasks, Same Data

Adding to the complexity, users are likely to perform vastly different tasks with the same data. For a clinician, data visualization might be the primary interaction with the UX, whereas a researcher might need to conduct cohort searches and execute workflows. While integrating every user’s tasks seamlessly within a single UX design may not be feasible, a flexible UX framework can facilitate embedding user-specific UX elements within a common design and backend implementation.

Fusion of Multimodal Data

At some point, visualizing individual data modalities may no longer be sufficient to fully grasp their interrelationships. Developing visualization strategies that reveal these connections is a challenge in itself and will be explored in a dedicated entry in this series. However, these views often need to combine graphical representations of summaries and/or key data points from each modality (e.g., genomic profiles, imaging-based disease characterizations, and relevant events from EHR records).

Customization

Finally, some users may simply require visualizations outside the scope of any out-of-the-box solution. While anticipating such needs is inherently difficult, a flexible UX framework can simplify providing support for these custom tools.

It’s clear that no single solution exists to address the visualization and task execution needs of every data type and user involved with complex multimodal data. However, this very complexity underscores the need for flexible and modular UX and backend designs to adapt to these diverse requirements.