Launching Struct Knowledge Base

Chat platforms today are inherently broken. Slack and Discord, the most commonly used chat platforms, while useful for real-time communication, create knowledge black holes, hindering the effective storage and retrieval of information.

This article is divided into 3 main sections. In section 1, I dissect the challenges plaguing existing chat platforms. In section 2, I introduce the CRISPY framework that outlines six underlying principles all chat platforms should strive to uphold.

And finally in section 3, I introduce Struct -- a new, innovative chat platform embodying this framework. Struct challenges the status quo, turning real-time communication into accessible, lasting knowledge and reinventing chat as we know it.

This blog post made it to the front page of Hacker News, resulting in a very fruitful discussion.

1. Communication Does NOT Equal Knowledge

Since the dawn of civilization, humans have been communicating. Exchanging stories over the warm glow of campfires, and sharing thoughts over a communal meal, these exchanges form the bedrock of human connection.

These conversations, while essential for social bonding and creating shared experiences, do not always translate into lasting knowledge. While knowledge is shared in these exchanges, without any effective capture process, they are rendered as mere memories: forgotten or irrecoverably altered over time like a game of telephone.

Capturing knowledge and maintaining it for the future required us to invent writing and printing. It involves the process of capturing, documenting, organizing, and storing information effectively. Once captured, a written word can be handed down over generations and centuries, informing all future communication.

The current chat platforms like Slack, Discord, and others allow us to communicate. But, these platforms fail at converting these conversations into knowledge. And therein lies the biggest problem that millions of users of these platforms suffer on a daily basis.

Chat Platforms are Knowledge Black Holes

As the founder of Dgraph Labs, the fastest-growing open-source Spanner-like graph database -- I was running a growing and very active Slack community of Dgraph users. We had over 3000 members in Slack. It was the main source of support for our user base. And while Slack provided a real-time mode of communication -- its problems were manyfold.

Horrible Search: After dedicating over six years to Google Web Search infrastructure, I've come to appreciate the intricate balance between content, context, and 'meatiness' that makes any document fully searchable. Imagine each conversation as a document. It's not just an isolated message or two, but a group of related messages, all encapsulating the same train of thought.

In the case of Slack, however, this ideal of a "document" is dramatically scaled down to a single chat message. This limitation inherently strips away the rich context and substantial content that make a conversation ripe for efficient searching. Consequently, this is why searching within Slack often feels like a wild goose chase.

New Message Whac-a-Mole: There is an endless stream of bold-font channels to chase. In Slack, a channel with bold fonts means it has a new message. Every new message in the channel would cause it to become bold. It was a tiring job as an open-source maintainer, playing Whac-a-Mole with new messages popping over an array of channels.

The N+1 Channel Conundrum: Given that creating a channel is the sole means of establishing the context within Slack, there always seems to be a need for 'just one more channel' to cater to slightly different contexts. This approach results in a channel explosion, making navigation and organization an uphill battle.

The Echo of Repetition: Slack struggles with capturing and making knowledge readily available, leading to recurring questions that endlessly loop back into the conversation.

Notification Avalanche: The barrage of notifications becomes relentless, descending on laptops, iPads, phones, and smartwatches without mercy. This constant bombardment is the only way to keep up with the real-time flow of messages -- because Slack just isn't designed for a 'later.'

Impossible Pricing Model: When running thousands of users on Slack, you have a choice between paying exorbitant amount of money to Slack or losing your chat history.

These are among the key reasons that led us to shut down our Slack instance, pivoting instead to Discourse. However, while Discourse is a useful platform, it's an asynchronous forum, not a real-time chat platform, making it an unsatisfactory substitute.

In fact, these issues aren't exclusive to Slack. The same complaint could be lodged against virtually any chat platform that exists today, whether it be Discord, IRC, or WhatsApp.

We can do Better.

I have thought about this problem for the past 5 years -- and arrived at a set of principles the ideal chat platform should aspire towards. And I've formulated all that into, what I call, the CRISPY framework.

Before I introduce the CRISPY framework, let's define what a conversation is. A conversation is a series of related chat messages exchanged between two or more participants. These messages, covering a single topic, together form a complete conversation.

2. CRISPY: Framework for an Ideal Chat Platform

CRISPY is a framework that defines what one should expect from a platform that they use for meaningful communication. A platform that can not only support effective real-time communication but also capture knowledge and make it accessible.

The CRISPY framework has 6 principles:

• Contextual Clarity: Each conversation should have a defined title and summary for immediate understanding.

• Real-time Interactions: Streamline short-form chats and implement real-time indicators to make the conversation lively and engaging.

• Isolation: Enable clear separation of concurrent conversations to maintain focus and avoid confusion.

• Spam Control: Implement robust measures to filter and prevent unwanted or disruptive messages.

• Progressive Learning: Utilize AI to learn from the knowledge shared in past conversations, predict and suggest responses, and improve the conversational experience.

• Yieldful Search: Incorporate an advanced search feature for effortless retrieval of past conversations.

Each of these principles serves as the cornerstone for a platform that not only facilitates effective and real-time communication but also enriches them, making them accessible and leveraging their knowledge for future interactions.

How CRISPY are Existing Platforms?

Existing chat platforms like Slack and Discord fare poorly when looking from the prism of the CRISPY framework.

In fact, after searching over the years, I've come to realize that no existing chat platform abides by the CRISPY framework. And that's not just because I came up with the framework. While I've formulated them into an acronym, these are reasonably expected principles from any knowledge-communicating platform.

For example, many email platforms and forums can give you 4 out of 6. Contextual Clarity, Isolation, Searchability and Spam Control. They can't provide real-time interactions. And there's probably work going on for progressive learning.

Given any decades-old email platform can score a 4.5, chat platforms are objectively worse. You get real-time interactions, and you pretty much give up on everything else. With a score of 1.5, chat platforms are a horrible solution.

Lately, Slack and Discord, have both added threads to allow some level of isolation. But, to achieve that, they had to break away from what made them special -- the liveness of conversations. They have taken a live, real-time chat platform, with typing indicators, and hidden it all behind a number. It's a hacky design, which doesn't align with their emphasis on real-time communication.

At this point, some readers would say -- you communicate in the chat platform. And then you capture that knowledge in other places. That's a very common practice. And I won't dwell much into it, but just say one thing -- it's an obvious "faster horses over cars" argument.

We can do better. Much better.

3. Introducing Struct Knowledge Base

For the past 6 months, my team and I have been tirelessly building a new kind of chat platform – Struct. Struct fully embodies the principles of the CRISPY framework, reshaping the very concept of what a chat platform can be.

For Struct Knowledge Base, we focused on bringing three core principles of CRISPY – contextual clarity, searchability, and progressive learning – to existing chat platforms like Slack and Discord. To accomplish this, we've developed innovative bots that extract threaded conversations and funnel them into Struct.

Struct then utilizes AI to

• generate the title, summary, and resolution status of the conversation for context,

• generate vector embeddings for each page to power search and similar threads,

• create an SEO-optimized 'knowledge' page for each thread making the page accessible forever,

• conversational responses to users' queries utilizing captured knowledge.

This generates a knowledge base out of existing conversations in Slack and Discord, with every thread getting a server-side rendered page dedicated to that conversation. As an example, see the same Slack thread (as above) when served by Struct.

We're launching Struct in two stages:

• Stage Knowledge Base launching today: Ability to funnel conversations from Slack and Discord into Struct, to build a knowledge base. Useful for groups who can't move away from Slack or Discord.

• Stage Chat Platform launching later: Ability to use the Struct chat platform, replacing a Slack or Discord instance. Useful for groups that can move away from Slack / Discord and adopt Struct fully.

You too can transform your Slack or Discord communication into a comprehensive knowledge base.

In essence, Struct is transforming what is currently a knowledge black hole into an intelligent, interactive, searchable knowledge base.

Struct has Amazing Users

Already, Struct is enhancing communication for several online communities: Chatwoot, Grafbase, Infisical, OpenObserve, Supertokens, Tooljet, and Typesense. This is what Jason Bosco, co-founder of Typesense, had to say about Struct: