I did a lot of reading this weekend for my “book library” MVP. I dug deep into retrieval augmented generation (RAG) and learned some helpful things:

• There are different variations of RAG: GraphRAG, StructRAG, LightRAG, etc. New versions have been introduced every few months this year. Which is the right one depends on your use case.
• Normal RAG isn’t great for a large data set like a book. It struggles to make connections when presented with a lot of data.  
• RAG’s results are better when you feed it relationships in a data set via a schema. GraphRAG, StructRAG, and LightRAG try to make up for this by using a knowledge graph to index the information better, which leads to understanding the data better and providing better results.

I’m realizing that how the information gets indexed in large data sets is critical, especially if I want to query across lots of dense data sets like books. Thinking about why entrepreneurs with photographic memories have an edge, I decided that their minds have done a superior job of indexing everything they’ve consumed and making nonobvious connections across the data. Those connections lead to unique insights that lead to creative solutions to problems or actions to get closer to their goal.

This weekend highlighted that I need to focus on and understand how information gets indexed as I evaluate RAG and other alternatives.

Today I read an article about a hack that helps you learn better. It suggests writing a one-page summary after you’ve read ten pages of a book because this increases your retention by 50%. The idea is that to learn, you must stretch yourself. You’re forced to do that if you have to process and organize what you consumed and then express your understanding in writing.

A few months back, I was writing a series of five or so blog posts for every biography or autobiography I read. I created a digest of each book and then shared the important parts via the blog series. Then, I began using the blog series to create a podcast series.

I was doing multiple levels of distillation, which helped me uncover insights from each book and retain more about each founder’s journey that I’d read about. Doing all of this weekly wasn’t sustainable, and I want to find a more sustainable process for doing the same thing. That’s why I’ve created an MVP to help me create book digests.

I’m a fan of writing. It’s a powerful practice that helps me organize my thoughts. When I write about what I’ve read, it feels like a superpower. I identify core concepts and insights I wouldn’t have found by reading passively, I retain them, and I can share them with others.

Writing about what you’ve read is something everyone can do, but most won’t. It’s more work, but I think the reward is worth the effort.

Last week I talked to an entrepreneur who’s shuttering his start-up after several years. The company is at breakeven, but it’s not growing. He shared with me what he’d learned from his journey.

The market is the main thing he wishes he’d paid attention to in the beginning. The market he entered looked good at first glance because it’s large. However, it’s slowly shrinking, and shrinking market forces have made it difficult for him to acquire customers because he has to steal market share from other companies. He now recognizes the huge role the market played in his company’s trajectory, but he didn’t understand it until he was in deep. If he knew then what he knows now, he wouldn’t have started a company in that market.

Markets matter a lot for entrepreneurs, especially if they aim for outsize success. Growing markets are good, and new, rapidly growing markets are the best. This founder is talented, but that wasn’t enough to overcome a shrinking market. He recognizes this now and says his next company must be in a growing market.

Shuttering a company isn’t ideal, but it happens often in entrepreneurship. This founder gained valuable wisdom from his first company. I’m excited to see what the next leg of his journey looks like.

This week, I committed to reading my highlights from David Allen’s Getting Things Done and Tiago Forte’s Building a Second Brain. It’s been a few months since I read them. Today, I read highlights from Getting Things Done. A few thoughts:

• This is a framework book (mostly). It teaches you a method you can use when executing your work to get more done with less stress.
• In school and at my first job, I was never taught how to work; I was just expected to get a lot of work done. Looking back, I was pretty inefficient, and I wish I’d known about this framework. I suspect most people are never taught how to work efficiently. They work hard but might not be efficient or strategic in their efforts.
• Context switching is a challenge for many. Often, the ramp-up period when starting a new task is a pain point. Some of this book’s methods can resolve this.
• Teaching this framework to employees at smaller companies could increase the velocity (which matters more than speed) of the company’s execution.
• A lot of entrepreneurs approach managing execution as a top-down activity. Some end up micromanaging because of this approach. This framework is more bottom-up—it empowers the employee and removes the need to micromanage.
• I like the idea of equipping employees with training on this framework, combined with a strong vision/mission and a goal-setting framework (such as OKRs or EOS), to create a company with high execution velocity.

I’m looking forward to fine-tuning how I execute the ideas in these books.

Current Project: Reading books about entrepreneurs and sharing what I learned from them

Mission: Create a library of wisdom from notable entrepreneurs that current entrepreneurs can leverage to increase their chances of success

Cumulative metrics (since 4/1/24):

• Total books read: 36
• Total book digests created: 14
• Total blog posts published: 217
• Total audio recordings published: 103

This week’s metrics:

• Books read: 1
• Book digests created: 1 (using technology)
• Blog posts published: 7
• Audio recordings published: 0

What I completed this week (link to last week’s commitments):

• Read David Ogilvy’s autobiography
• Read highlights from David Allen’s Getting Things Done
• Read highlights from Tiago Forte’s Building a Second Brain
• Read two resources on prompt engineering
• For my “book library” MVP, created a separate agent and RAG to index each book instead of one agent and RAG to index multiple books
• Tested prompts and system instructions to improve the quality of responses from the “book library” MVP
• Created one book digest using the “book digest” MVP
• Tested prompts and system instructions to improve the quality of digests created using the “book digest” MVP

What I’ll do next week:

• Read a biography or autobiography
• Test alternative agent setups with RAG for the “book library” MVP
• Ask AI developers about RAG alternatives
• Create a book digest for David Allen’s Getting Things Done using the “book digest” MVP

Asks:

• None

Week two hundred forty-one was another week of learning. Looking forward to next week!

Current Project: Reading books about entrepreneurs and sharing what I learned from them

Mission: Create a library of wisdom from notable entrepreneurs that current entrepreneurs can leverage to increase their chances of success

What I struggled with:

• No material struggles this week.  

What I learned:

• Getting the “book library” MVP to provide quality results that add value is the priority. After that’s accomplished, I can start thinking about how to put it in the hands of other users. Trying to figure out the path to allowing others to use it publicly was premature. I need to get this thing working and adding value first; then I can figure out how to share it.
• I’ve been reading up on retrieval augmented generation (RAG) because the MVP isn’t working as intended. RAG has more limitations when you feed it a ton of information (e.g., multiple books) than I initially thought. It struggles to make connections between related information, but that’s essential. If the MVP can’t do that, providing value-added responses will be hard.
• There’s a good chance that the Google Cloud Platform throttling of my account is impacting the depth of results I get. This is frustrating because you don’t get a warning or confirmation of throttling.
• AI is good at many things, but it isn’t yet good at making sense of large, unstructured text data sets like books. Creating a structure or taxonomy for this kind of data could unlock what AI can do with it.
• Google makes it easier for nontechnical people to test with and tune Gemini large-language models (LLMs). The throttling has me thinking about adding LLMs from other companies into the testing.
• The more I learn from this project, the more I respect the human brain and its ability to store and process information from books.

Those are my struggles and learnings from the week!

This week, I worked on the “book library” MVP. My goal is for the MVP to mimic an entrepreneur who’s an avid reader with a photographic memory. I want to query across multiple books, and I want the MVP to make connections that uncover new insights. I also want it to recall any book’s details quickly.

I’m using retrieval augmented generation (RAG) and Gemini LLMs in the MVP. Last week’s testing yielded responses that weren’t detailed enough and didn’t uncover new insights. I was happy to have something working (it felt like a big milestone), but it still needed work. This week I tried using a different RAG setup—a separate agent and RAG to index each book instead of one agent and RAG to index multiple books. I also tested different prompting and system instructions. The changes didn’t improve the responses, which was frustrating. Still too high-level and unable to make insightful connections.

I’m not sure why this is happening. My developer friend and I have a few theories. It could be a limitation of RAG not being great at indexing entire books. It could be limitations with Gemini LLMs, technical limitations imposed by Google Cloud Platform (GCP), or something else. Given that the output I’ve been able to generate from Google AI Studio for an individual book has been pretty detailed, we think there’s a high probability it could be a GCP limitation.

This wasn’t the outcome I was hoping for at the beginning of the week, but that’s part of the process when you’re building something that hasn’t been done before. Definitely frustrating, but such is life. We’ll do more testing to try to figure this out.

I’m using Google AI Studio to run one of the MVPs for my book project. Google’s AI is called Gemini, and there are eight different Gemini large-language models (LLMs). Determining which would yield the best result was a concern. Google had thought it through, though: AI Studio has a compare feature: you can ask a question and select two LLMs, and Gemini will provide responses from both of them in a side-by-side view.

I’ve been testing prompting and system instructions this week, and the compare feature has been helpful. Seeing how the different LLMs respond to the same question is helping me narrow my choices faster.

Google AI Studio has limitations, but it’s a good tool for someone who is nontechnical and wants to fine-tune their AI experience.

I’ve spent the last few weeks diving into Google’s AI Studio, NotebookLM, Vertex AI Agent Builder, and various other AI-related tools from Google and other companies. A developer friend has helped me a lot. I was aware of some of these technologies from reading about AI and LLMs in general, but now that I’m trying to use them to create solutions for my personal project, my understanding of them has gone much deeper.

I have a clear idea of what I want the technology to do. I’m trying to figure out if it can specifically do what I want. If so, what are all the ways? What are the implications of each option? What I learn sticks in my memory. This is different from my normal approach of poking around to understand a tool’s general capabilities, which doesn’t result in good retention.

I’ve also noticed that when I seek help from technically oriented people to learn new technologies, describing the problem and how I want to solve it helps tremendously. It gives them a better idea of where to start, and the conversation is more focused on solutions to my problem than on a broad overview of the technology.

I’m not sure which, if any, of these technologies will be part of the solutions I build. But I’ve learned something: If I have a problem I’m excited to solve, I should try using new technologies to create a solution. Worst case, I’ll gain a better understanding of the technologies. Best case, I understand the technologies better and create a solution to my problem.  

The IPO market for technology companies has been slow (see here). I’ve been curious why that’s the case (see here). Klaviyo is a known technology company that IPO’d in September 2023. I came across an interview with the CEO and co-founder, Andrew Bialecki. The interview caught my attention because he discusses initially bootstrapping and growing to over $1 billion in revenue and a market capitalization (i.e., valuation) of over $10 billion as of this writing.

One section of the interview addressed what he thinks the criteria are for technology companies to go public or, said differently, what a company needs to demonstrate to get public market investors to buy its stock and have a successful IPO. Here are the criteria:

• Positive free cash flow – the company needs to generate, not consume, cash.
• Sustainable business – The company provides a product or service that customers will value in future years.
• Durable growth – The company must be growing at a healthy rate. The smaller the revenue base, the higher the growth rate investors want to see. The growth rate must also be durable for the next four or five years.

Growing at a rapid rate that’s durable while not burning money isn’t easy to do. Many technology companies can achieve high growth rates, but they burn a ton of cash to accomplish this.

Bialecki’s perspective on the current IPO market for tech companies is valuable, given he’s one of the few who has successfully completed a technology IPO in the last two or so years.

He shares other great nuggets during the interview. If you want to hear just the section on his thoughts on IPOs, see here, but I found the entire interview worthwhile.