Mar 15, 2022
Bringing Conversational AI to Slack, powered by Struct and OpenAI
In my last blog post, I introduced Struct — the Chat Platform we're building to bridge the gap between communication and knowledge. The post was very well received, making it to the front page of Hacker News. And we got quite some interest — 60+ orgs installed the Struct bots, resulting in conversations spanning 13,500 threads and 22,000 users.
We also got some confused questions — Is Struct a platform, or is it a bot? Let me quickly clarify that. Struct Chat is the knowledge-rich chat platform that we're building to replace Slack and Discord, both of which leave a lot to be desired (see CRISPY framework in the last blog post). We're hoping to start trialing the platform by the end of this quarter. You can sign up for the waitlist here.
In the meantime, we've built Struct bots -- for Slack and for Discord. They capture threads in the respective platform to generate a knowledge base. You can see current examples of knowledge bases running here:
Struct Bot captures Knowledge from Slack
In this post, I'd talk specifically about the bot that we built for Slack.
Struct bot picks up Slack threads and processes them through AI to generate a title, summary, resolution status for the conversation, and vector embeddings. All these allow us to generate knowledge document per thread -- which we render as an easily accessible, searchable, SEO-optimized web page.
Moreover, Struct generates vector embeddings per query, so it can search not just by the keyword, but also by the similarity of the search term — doing a hybrid search to find the most relevant threads for your search query.
Note that Struct doesn't pick up individual messages in channels so far, only threads. The former are too diluted to be used for knowledge capture.
Generative AI, Vector Embeddings, and Hybrid Search
Struct uses OpenAI APIs to generate the title, summary, and resolution status. To generate document and search query embeddings, we use Microsoft's E5 model running on Typesense search engine.
Why not just use OpenAI's embeddings? We initially used to. But, the latency turned out to be too unpredictable for generating embeddings for search queries. Thus, we rely upon OpenAI only for indexing time requests (like generating titles, summaries, etc.), while the runtime requests (like search queries) are processed through the open-source E5 model, where we control latency tightly.
In fact, we rely upon Typesense to auto-generate the embeddings by specifying the specific model we want in their schema. This model is also applied to search queries to generate embeddings for them in real time. Typesense then calculates a hybrid score, a mix of keyword search and cosine similarity search, to identify the most relevant documents corresponding to the query. We use the same mechanism to power our "Similar Threads" search as well.
Conversational AI within Slack
All this technology has allowed us to take a giant leap. Starting today, our Slack bot can directly answer users' questions, learning from the knowledge it has captured from your Slack conversations.
After installation, the bot would create a channel called #ask-struct-ai. If you have restrictions on who can create a channel then the bot might not be able to create this channel. In that case, post install, the mods can create this channel, and add @struct.ai bot to it. Any questions asked in this channel would get a response from the AI bot. See it in action here:
To achieve this, Struct runs a hybrid search for the user's query across all the captured knowledge pages so far. It picks the top k pages and generates a query prompt. It sends this query prompt to OpenAI, while specifically asking it to provide references for the generated response. The response from OpenAI is then messaged back over to the user as a "threaded" response, within Slack. We create threads to avoid unnecessary clutter.
The responses are surprisingly accurate if the topic had been previously discussed to any extent, even if it was a few chats within some thread. If there isn't enough information, as in, the topic was never previously discussed, the response tends to admit its limitations so as not to mislead the user.
Seeing this in action, I must admit, is very exciting. This is also a MAJOR step in achieving the P in CRISPY -- the framework I introduced in my last blog post.
As engineers, we spend a lot of time sharing knowledge within Slack and other chat platforms -- most of which act like knowledge blackholes -- never surfacing all this knowledge in any meaningful way. For the first time, thanks to the Struct bot, the knowledge shared in Slack can not only be captured but also be used to respond to users' queries. This is exciting and very promising.
In the future, we could crawl and index the documentation, support tickets, or other channels of communication, hence, improving the positive response rate of the AI. The possibilities are endless.
But, What About Privacy?
I'm sure the next natural question is. "But, what about privacy?" This might be OK for an open-source project, but we don't want any confidential company information to be used to train a publicly available AI model.
At Struct, we take privacy very seriously. Since our launch, we have introduced access control to the Knowledge Base. When you install Struct bots, the threads picked up by Struct would be accessible only to Slack members by default. This is done by enforcing channel-level permissions, accessible through Workspace Settings by the moderators. Struct then automatically limits which threads the conversational AI would use to generate the responses, based upon access level.
But to generate the title, summary, and responses to user's queries, we're sending that data over to OpenAI -- what about that?
For that, refer to the API data usage policy of OpenAI:
"Starting on March 1, 2023, we are making two changes to our data usage and retention policies: OpenAI will not use data submitted by customers via our API to train or improve our models, unless you explicitly decide to share your data with us for this purpose. You can opt-in to share data. Any data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law)"
To summarize, OpenAI does not use the data submitted via their paid APIs to train or improve their models.
So you get all this amazing functionality while also maintaining privacy. Next time, when you have to ask a question, before bugging a colleague, just ask the Struct.AI bot. It might already know the answer.
Conclusion
Our goal with Struct is to capture the knowledge that's shared in everyday communication and make it accessible. Today, we take a giant leap in that direction with the introduction of our conversational AI bot.
Our new bot uses the knowledge pages generated from Slack threads to respond to user queries. This means you don't have to keep asking the same questions on Slack over and over again. You can just ask our bot!
You can install Struct bots by following these links:
This is a game changer for how we work together and share knowledge. In the future, we want to make our bot even smarter by letting it learn from more places, like documentation and support tickets.
Thank you for being part of this exciting journey to knowledge accessibility in chat platforms.
Image credits: Source
Gutenberg's printing press, an invention that revolutionized history by vastly increasing book accessibility, resulted in European book production soaring from a few million to about a billion copies in less than four centuries.
At Struct, we are similarly invigorated about the infinite potential that comes from capturing everyday conversations, making them accessible for future use. We believe in the power of unlocking this untapped knowledge, much like the transformative impact of Gutenberg's press on information sharing.
Manish R Jain
Founder, CEO
About the author
Manish Rai Jain is a distinguished entrepreneur and the visionary founder of Dgraph Labs, Inc., leading the company to become the most popular open-source graph database on GitHub with a $1M ARR and a global team of 50+ members. Prior to Dgraph, Manish contributed instrumental work to Google's Knowledge Graph, co-leading the development of Google's graph indexing and serving system. These innovations continue to support the freshest and largest collection of web content that fuels Google to this day.