AI-Assisted Authoring of Text and Charts for Data-Driven Communication

Charts and text go hand in hand during data-driven communication. A well-crafted visualization can help understand complex data, but it often needs accompanying text — titles, captions, annotations — to provide context and emphasize key takeaways. Despite this, most tools treat text and charts as separate elements, leaving users to manually align them for effective storytelling. Bridging this gap, we present Pluto, a new AI-assisted system designed to seamlessly integrate text and charts, making data communication more intuitive and insightful.

The Problem: Disconnected Text and Visuals

Text plays an important role in helping readers interpret data visually. A well-written caption or annotation can draw attention to trends, explain anomalies, or provide necessary background information. However, existing tools provide little support for jointly authoring charts and their accompanying text. This lack of integrated authoring support often results in inconsistencies, where the text doesn’t fully align with the visualization, making it harder for audiences to grasp key insights.

Introducing Pluto: A Mixed-Initiative Tool for Authoring Semantically Aligned Text and Charts

Examples of text and chart suggestions in Pluto. (A) Auto-generated title and description based on the chart. (B) Sentence completion suggested based on multimodal input from the preceding text and a selection on the chart. (C) Sorting and highlighting a chart to improve coherence with the description.

Noting the tight coupling between text and charts, Pluto uses features of a chart’s construction (e.g., visual encodings) as well as any textual descriptions a user may have drafted to make suggestions about the content and presentation of the two modalities. Specifically, Pluto offers different types of authoring assistance including:

• Generating text based on chart data and selections — given a chart, Pluto suggests titles, descriptions, and annotations that reflect the underlying data trends and promotes a good reading flow. Offering more granular suggestions, Pluto also offers sentence-level suggestions based on user interactions like clicking or brushing on the chart.

Example of Pluto suggesting sentence completions based on the underlying data and chart selections. As the author types in the first statement and writes “However,” Pluto detects a data trend that is an exception to the first statement and suggests including a fact about values for Texas. Subsequently, when the user selects New York, the sentence completion picks up on the tone of the preceding statements and generates a new statement about New York to help continue the description.

• Interactive text verification — Pluto promotes an iterative workflow by actively inspecting the consistency between any drafted text and the underlying data to flag potential discrepancies. Users can interactively verify the discrepancies to update their content for clarity.

Pluto flags statements that require manual verification. Authors can inspect the corresponding reference in the chart and appropriately update their text. Here, Pluto identifies a potential error in the description, flagging to the author that the reference to the “peak” value for one of the series in the line chart may be incorrect. By interactively verifying the statement, the author is able to update the text to contextualize the takeaway in the description to a specific time frame.

• Recommending chart modifications based on textual descriptions — If a description emphasizes certain data points that aren’t visually prominent, Pluto suggests design changes such as annotating or sorting the chart to improve clarity.

Example of Pluto suggesting chart annotations to emphasize points conveyed in the text. Here, noticing that the text focuses on two series in the chart (Action and Animation) and the time period 2013–2017, Pluto suggests annotating the relevant portion of the chart.

How does Pluto generate its suggestions?

To enable this interactive authoring experience, Pluto models a conceptual schema that captures key components of the chart (i.e., data, specification, selections, visual embellishments) and text (i.e., title, description, textual annotations along with the semantic information they convey). This schema is then used as part of a recommendation engine that blends pre-configured heuristic-approaches with LLMs to generate authoring suggestions. More details about the recommendation generation process are available in the accompanying research paper.

Overview of Pluto’s system architecture. Given the active interface state, Pluto uses a combination of a heuristic parser and large language models (LLMs) to process information and generate recommendations.

Looking ahead

Initial user studies suggest that Pluto helps bootstrap the authoring process and supports different authoring strategies. For instance, commending the tool’s ability to provide an initial draft, a participant noted, “I would love to use these as a first draft. Just helps get my juices flowing.” Participants also appreciated the ability to leverage multimodal interactions with the chart to generate text - “It was nice to be able to point at things and have the system give the text. It saved me a lot of time.” and “the selection ensured the generated text was about something I want to talk about.” Finally, the study participants found the chart design suggestions useful for reflecting on their content before publishing, stating, “It’s great that it [Pluto] highlighted some elements in the chart based on my text. It made me see things from a reader’s perspective and go back and make additional changes.”

Encouraged by the initial feedback, we envision continued exploration of this research space, extending AI-assisted authoring tools to accommodate a broader array of visualization types and narrative styles, wherein text and visuals are not mere complements but coequals in data-driven communication.

________________________________________________________
 

This post summarizes the research paper “Pluto: Authoring Semantically Aligned Text and Charts for Data-Driven Communication,” co-authored by Tableau research scientists,  Arjun Srinivasan and Vidya Setlur, and Prof. Arvind Satyanarayan from MIT. You can also refer to an accompanying video overview more elaborately illustrating the examples discussed in this blog. The paper recently appeared in the proceedings of the 2025 ACM Conference on Intelligent User Interfaces held in Cagliari, Italy from March 24–27, 2025.

To learn more about the Tableau Research team and other research projects, visit research.tableau.com.