What Do You Put In Your Database?

First post here, so be gentle, lol.

I'm having a hard time wrapping my perception about what kind of data you can, or should put into yours for your AI memory. I've been messing around with computers and by extension data manipulation, since the 70s. I have a good understanding of how a relationship database runs.

But we're not building that kind of ecosystem(?) are we? We can go bigger.

Before I dug deeper, I always picture an LLM like Chatgpt, as having this huge massive brain, which held all of the Internet, and when I asked would wave it's virtual hands to say "Here it is". I know that's incorrect.

I'm using AI as a research assistant and junior co-writer. I'm doing a non-fiction book what kind of skills you'll need to make money in the next 20 years. I'm doing a ton of digging for trends and possibilities and...all of you have a good idea of what that means, I'm sure.

I went into this thinking it would be more like a Wiki compiled form all my research and conclusions but is that what I want? Seems like there so much more than just a glorified book list. I do want to have a folder style system, with the full transcripts, complete articles, or other important documentation that I need. That's doable too.

But recently, I wanted to start at least collecting the base data. If I don't start doing that, I'm just digging my hole deeper. So I laid down a basic schema, and then asked GPT to pull me 5-6 highlights that it thought should go into the database. Once I get the workflow built, I pretty sure I'll try to automate it, so as I research, the AI formats and stores any highlighted data I come across.

It pulled those six, then another three from our discussions on the first six, and we were in a side chat at the time, the main chat was on markdown files. We got 10 entries from that. So I had nineteen.

I haven't done any more, but when I look at what I have, I get this weird vibe that the majority of them aren't on the book subject, but are more of how the LLM views the way I work? That's a poor description of it, I hope it works.

Anyway, someone please explain where I'm seeing things wrong. Something isn't clicking. <frowns>

12 comments