November 5, 2020

Towards a Paradigm for Visual Modeling

Augmented intelligence for compositional problem-solving with large-scale systems

Fahd Husain 

Significant research at Uncharted in recent years has focused on the union of visualization and augmented intelligence techniques. We strive to create interactive systems in which human and machine intelligences can complement each other in real time to solve problems that are compositional in nature. Following a previous engagement for a workshop for Artificial Intelligence Applications to Agriculture hosted by the Texas A&M Institute of Data Science (TAMIDS), I had the privilege of delivering a talk at the TAMIDS Fall 2020 Seminar Series, representing the work across two Uncharted teams. “Towards a Paradigm for Visual Modeling” describes how an overarching philosophy of augmented intelligence guides our efforts. Specifically, it details our work developing systems for exploring, assembling and engaging large, complex graphical models that span multiple domains or involve diverse representations. The impetus for this ‘visual modeling’ work comes from two pioneering DARPA fundamental research programs: World Modelers and Automating Scientific Knowledge Extraction (ASKE).

“Towards a Paradigm for Visual Modeling” at Texas A&M, September 25, 2020.

The World Modelers program aims to model socio-natural complex systems, such as those centered around drought and famine events. Its goal is to predict the causal consequences of such systems and support policy efforts looking to provide aid to affected populations. For this program, Uncharted developed Causemos, which helps modelers quickly and iteratively build computational graphical models of real-world complex systems. This is done visually and interactively: modelers can explore and curate knowledge graphs and analyze data from open source indicators and high-resolution spatial outputs. They can then stitch together this knowledge and data to assemble computational and probabilistic graphical models for prediction and intervention analysis. With the support of DARPA, Luma Consulting and the Bill & Melinda Gates Foundation, Causemos has been successfully trialed in a number of user engagements with stakeholders in Ethiopia, applied to issues of food security.


A knowledge graph (center) in Causemos constitutes a collection of causal assertions extracted from a corpus of thousands of documents. Modelers can filter the graph using the common attributes (left) and drill down to pieces of evidence underlying causal relationships.

The DARPA ASKE program aims to automate the extraction and discovery of scientific knowledge at scale. For science in the time of pandemic, this is a task of immediate urgency. The global explosion of COVID-19 research has produced a deluge of literature and models; both governments and the scientific community are struggling to synthesize the knowledge produced over the last ten months. Against this backdrop, Uncharted is prototyping an integrated visual modeling platform for exploring, augmenting, executing and analyzing a multitude of models extracted from the corpus of scientific literature. Our approach is built on a set of primitive graphical model representations—from simple node-link graphs to enriched hierarchical structures—that can articulate a range of common biological and epidemiological models. With this platform, we look to help modelers interactively interrogate diverse models for source knowledge, parameterize them for execution, and eventually, compare and contrast them in a multi-model space.


A subgraph of a COVID-19 biological model (left) of molecular-scale interactions, with visible links representing protein-protein relationships that have been validated. Disconnected nodes represent agents to be investigated by modelers and subject to causal checking. A core subroutine (right) from a functional graph composed of the CHIME (COVID-19 Hospital Impact Model for Epidemics) model, a compartmental version of the SIR (Susceptible, Infected, Recovered) model. Modelers can analyze components, trace computational flow and surface metadata and backing knowledge interactively.

Both World Modelers and ASKE are large, ongoing programs that continue to engage fundamental research questions with immediate social impact. We continue to refine our visual modeling approach via these research efforts. Our ultimate goal is to provide platforms that enable modelers to accelerate the discovery, contextualization, assembly, and utilization of diverse models and have impact across domains such as food security and pandemic analysis. Keep watching this space for updates on the evolution of our work.

Many thanks to DARPA, Luma Consulting and the Gates Foundation for continued support, as well as to Texas A&M University for the seminar invitation.

  Learn more: