Synthesizing Serendipity

Originally Posted on November 13, 2024

Learn from everyone; follow no one; look for patterns; and work like hell. – Scott McCloud

A couple of years ago, I compiled a list of 25 Books Every Technologist Should Read. It was an effort to synthesize some of the most important influences on my thinking. It received a good reception (including from some of the authors). A couple of months later, I started working on a similar list of shorter pieces, but never published it because I got pulled into other projects.

Like all curated bibliographies, both lists were exercises in synthesis. I charted how these books and articles related to each other and to the project of augmenting our intelligence through the careful construction of technologies and human systems. I created this Venn diagram of the book list.

The Original Book List Venn Diagram from 2023

I started playing with Google’s NotebookLM (co-developed by Steven Johnson, who is on the book list with Where Good Ideas Come From). NotebookLM is the closest thing to my Knowledge Navigator concept currently in the LLM space.

As I have written in the past, the real breakthrough with Large Language Models is their ability to synthesize disparate chunks of ideas without human ontologies. I was interested in exploring how well NotebookLM synthesized abstract ideas and what kinds of serendipity it might produce. I wanted to see how well it broke categories.

We learn to categorize things from a very young age. Remember the “One of these things is not like the other” segment on Sesame Street? It taught us that everything belonged in a category. As an adult, I’ve learned to reject that lesson and recognize the interconnectedness of things.

For me, when I am told that something “doesn’t fit,” I take it as a challenge to figure out how it does. I want a tool that makes it even easier for me to do this.

When I created the list, I could see the connection between Vannevar Bush’s seminal article “As We May Think” from 1945 and Kate Raworth’s “Doughnut Economics” concept from 70 years later. They are both connected by systems theory, but it’s a long and complicated road to get there.

I’ve been writing for quite some time about the idea of a Knowledge Navigator, which would take any data and synthesize it into a visual map of information, allowing the user to find hidden patterns and surfacing buried information.

The Knowledge Navigator Concept

One of the key aspects of the Knowledge Navigator idea is the ability to explore serendipity to form new connections and ideas. I was curious how close NotebookLM could come to replicating this idea.

We rely on other humans to give us opportunities for serendipitous synthesis. I do this professionally in a variety of ways as a trainer, teacher, writer, and consultant. However, the effort requires ongoing and active conversations.

I’ve found some of the most rewarding times in my life come when brainstorming ideas with people whose ideas challenge and expand my paradigms. Processing these inputs and turning them into useful new ideas requires synthesis.

However, getting people and their attention spans into that kind of alignment is hard work. Conferences are rare and expensive (and don’t always create space for this to happen). I have developed techniques to achieve these kinds of interactions, but in-person has a decided edge when it comes to the all-important side conversations that drive thinking.

As someone who spends most of his hours working on ideas alone (as many of us do), I’ve often found my creative synthesis stunted by a lack of conversation. This creates an echo chamber, where I find myself struggling with the same ideas.

I was interested in exploring NotebookLM’s potential as a conversation “partner.” Being a true partner requires that it does far more than respond to queries. It must be capable of having a conversation with my mind.

I found this serendipity in an unexpected place. One of NotebookLM’s features is an ability to generate a podcast from the information in your collection.

In order to explore synthesis, I asked it to generate a podcast based on my 25 articles. I could tell it was struggling to synthesize the diverse strands of my warped mind, but it eventually produced a 25 minute conversation. As usual, I’ve explored the edges of what the intended capabilities of the tool are.

The interesting thing about this conversation was how much it gave me an alternate synthesis to the one I already had constructed in my mind. This was synthesis and serendipity. Then I had it build two more “podcasts” (Version 2 and Version 3) for even more serendipitous fusions.

There are many problems with these podcasts. They are bland and rather white (and more than a little sexist in how they use the male and female voice). However, if you get past that and analyze how it is synthesizing the information with a critical eye, every version holds some aspect of fascination (and differs from my synthesis).

To think critically and look for opportunities, you must look at an idea from a wide variety of angles. Perspective is the difference between good and boring photography. It’s also the difference between good thinking and lazy thinking.

The podcasting function on NotebookLM synthesizes information from a different perspective than mine. The model has unusual rules about what makes “sense” so it challenges my sense making.

Sometimes this verges on parody, but even that has its uses. A few weeks ago, I mentioned the power of “parody” to shift “paradigms” (a juxtaposition that occurred to me as the result of a voice transcription AI getting my dictation serendipitously wrong).

Paradigm shifting is key to critical synthesizing, but it is hard to do systematically. You must read widely, understand what you’re reading in context, and be prepared to shuffle that context at will.

Without this skill, our civilization will not survive. Complex problems ranging from climate change to democracy in a world of information overload require a paradigm-shifting approach. We need better synthesizing tools to meet these kinds of challenges.

NotebookLM points the way, but it’s only the first step. The analyses in the NotebookLM podcasts suffer from a somewhat myopic view of the information.

NotebookLM podcast’s perspectives are incomplete. They are also, by necessity, linear, because it is trying, even struggling, to make a linear conversation out of a scatter plot of readings.

Iteration mitigates these problems, but you still must devote the time to listening to 20-25 minute “conversations.” As thought-provoking as they are, they’re still a time sink. (An integrated transcription function would be nice.) We need to synthesize much more quickly and comprehensively than the NotebookLM platform does.

The Knowledge Navigator would address this challenge by providing a two-dimensional map of the information instead of a linear narrative structure. Creating a map allows us to create a constellation of ideas, not just to ram them into a linear storyline. (See below for a comparison.)

We need better thinking tools to engage the problems of the world today. Better thinking tools are better synthesizing and serendipity tools. It’s time to get busy building them.

A Practical Example:

This exercise demonstrates the limits of textual thinking in organizing the ideas expressed in the 25 Articles list:

First of all, here is a graphic grouping of them manually created using Miro:

Article List Venn diagram from 2024

When I tried to get NotebookLM to organize these ideas (even with my categories), it didn’t go so well (the podcasts it generated did a better job of synthesizing).

My query: Group these articles into one of these seven categories: 1) designing technology, 2) designing systems, 3) understanding systems, 4) information network design, 5) narrative design, 6) human design, and 7) designing new realities

NoteBookLM response:

Here is a grouping of the articles based on your provided categories:

Here is a list of titles of the categories you provided, populated with titles of the sources you provided in our previous turn:

• Designing Technology:
  • “Man-Computer Symbiosis”
  • Excerpts from “Design Principles Behind Smalltalk”
  • “The Augmented Knowledge Workshop”
• Designing Systems:
  •  “Augmenting Human Intellect: A Conceptual Framework”
  •  “Leverage Points: Places to Intervene in a System”
  •  “Computer Lib/Dream Machines”
• Understanding Systems:
  • “Pace Layering: How Complex Systems Learn and Keep Learning”
  • “The Evolution of Large Technological Systems”
• Information Network Design:
  •  “As We May Think”
• Narrative Design:
  • Excerpts from “Science Fiction Prototyping: Designing the Future with Science Fiction”
• Human Design:
  • There are no sources in this category.
• Designing New Realities:
  • Excerpts from “1982violacondominiumsindataspace.pdf”
  • Excerpts from “Design Unbound: Designing for Emergence in a White Water World”
  • Excerpts from “Dynamic Difficulty Adjustment in Games”
  • Excerpts from “thinkingbackward.pdf”

And its underlying logic:

• Designing technology:
  • This source consists of a list of references from “Man-Computer Symbiosis,” focusing on technological advancements in areas like speech synthesis, chess-playing programs, and pattern recognition.
  •  This excerpt focuses on the user interface (UI) design of Smalltalk, an object-oriented programming language. It discusses design principles and the process of creating a user-friendly environment for exploration and learning.
  • These excerpts are from a paper titled “The Augmented Knowledge Workshop.” It describes the features of the NLS system, which was designed to augment human intellect. The features include structured text, hyperlinking, multiple windows for viewing, and support for collaboration and teleconferencing.
• Designing Systems:
  • These excerpts come from Doug Engelbart’s “Augmenting Human Intellect: A Conceptual Framework.” They propose an augmentation system to enhance human problem-solving. It emphasizes structuring information with techniques like list structuring and linking, enabling efficient browsing, manipulation, and comprehension of complex concepts.
  • These excerpts from Donella Meadows’ “Leverage Points” offer a framework for intervening in complex systems. It presents a hierarchy of leverage points, ranging from parameters and numbers to the system’s goals and the paradigm it operates within.
  •  These excerpts come from “Computer Lib/Dream Machines,” a book exploring the potential of computers in various domains. It covers hypermedia, hypertexts, computer graphics, and interactive systems, emphasizing the need for user-friendly designs that represent the true content and structure of human thought.
• Understanding Systems:
  • This excerpt from “Pace Layering: How Complex Systems Learn and Keep Learning” introduces the concept of pace layering as a mechanism for adaptability and resilience in complex systems. It proposes six levels of pace and size for a healthy civilization.
  •  These excerpts are from “The Evolution of Large Technological Systems,” discussing the characteristics and evolution of these systems. They highlight momentum, hierarchical structures, and the interplay of various components, including physical artifacts, organizations, and scientific knowledge.
• Information Network Design:
  • This excerpt from Vannevar Bush’s “As We May Think” proposes the “memex,” a hypothetical device that allows users to create associative trails between pieces of information. It envisions new forms of encyclopedias and information retrieval systems based on the concept of interconnected knowledge.
• Narrative Design:
  • These excerpts explore the use of science fiction narratives in urban futures and design. It highlights the role of storytelling in shaping the imagination, facilitating discourse, and engaging communities in the design process, particularly in urban planning.
• Human Design: This category doesn’t have any corresponding sources.
• Designing New Realities:
  •  This excerpt suggests new ways to visualize and interact with information. It proposes moving away from linear, hierarchical structures to a more spatial, explorable “data space” that mirrors the conceptual structures of the brain.
  • These excerpts from the book “Design Unbound” present a framework for “designing for emergence,” emphasizing the design of contexts rather than things and the need for a transdisciplinary approach to address complex problems.
  • These excerpts highlight the concept of “Flow” in video game design and discuss its relationship to dynamic difficulty adjustment (DDA). It explores the application of Flow principles in enhancing player engagement and creating an optimal gaming experience.
  • These excerpts emphasize the limitations of traditional text and paper metaphors for knowledge organization and propose moving towards a more interconnected and dynamic knowledge network. They advocate for systems that enable flexible manipulation and recombination of ideas, drawing inspiration from thinkers like Ted Nelson.