Fable's gone...now what?

This whole "Fable banned" situation raises so many important points that can really set you apart from others going forward. I think as a community it's a good idea to come together and share our thoughts, opinions, and feelings so that we can all grow from this and seize whatever opportunities it creates.

Here are mine.

Importance of owning your own inference

This a big one. I've long felt that getting comfortable with using, and even hosting, open models has strategic importance for multiple reasons.

Manage compute cost
Data Privacy
Realize/Understand how powerful they are

What happened yesterday helped me appreciate a new reason.

If you don't own your inference, at any point an outside actor can destroy your whole business with a letter. These frontier models are amazing but due to the political climate we're in, no matter how amazing they are, if they can be shut down overnight and your business is not model agnostic, your whole foundation is build on sand that can quickly be eroded out from under you.
@Jake Van Clief
has been telling us for a long time the importance of being model agnostic. If this isn't a wake up call, I don't know what is! Many of us were I'm sure looking forward to playing with Fable all weekend. Now that we can't, here's a suggestion. If you haven't worked your way through the courses yet to understand how the whole folder structure system works, here's your chance. Take advantage of it.

Opportunities owning your own inference creates

How many opportunities seemed off limits due to HIPAA or other governmental or privacy regulations. If you are able to offer a solution that exists on-prem and neve sees the internet with self hosted open models that's a pretty massive market to cater to.

The clock as been ticking for how long these frontier models were going to remain affordable. Even if Fable wasn't banned, who of us were going to be able to afford it after June 22? $10/M Tok in and $50/M Tok out? Yikes dude! We've been getting seduced by these frontier models that have largely been made affordable by billions in funding coming in from private equity and venture capital. A friend of mine calculated the actual cost of his token spend with his $200 max plan. He estimated it to be around $2400. That means these Pro and Max subscription plans are not sustainable and would either get way more expensive, or way more limited. Homer Simpson said it best (see the attached video).

If anything, this is a wake up call for all of us. Time to shake loose of the spell these frontier models have on so many people and realize there are other options. They may not be as powerful as these frontier models but that's ok. I'd say that easily, 70%-80% (if not more) of what most offices need can be handled by the current open models that are out there, and they're only gonna get better from here. According to Jensen Huang, they are only 6 months behind, and that gap is shrinking. (To be clear, those "6 month behind" models aren't things most of us could host on our own equipment but it's only a matter of time)

I've thoroughly enjoyed driving the Bugatti of LLM's for the past year for Kia prices. It's been really fun and I will continue to enjoy it as long as I can. But I'm also looking for a solid Toyota or Honda I can pivot to as a daily driver knowing I can rely on it and it won't let me down, or surprise me with a $30,000 bill for an oil change, or a $42,000 bill for a new set of tires.

I'm anxious to hear everyone else's thoughts...let's use this as a chance to grow!

2 comments

Fable's gone...now what?