|
I invite you to upgrade to a paid subscription. Paid subscribers have told me they have appreciated my thoughts & ideas in the past & would like to see more of them in the future. In addition, paid subscribers form their own community of folks investing in improving software design—theirs, their colleagues, & their profession.
I remember walking to the bus from high school, staring at a Motorola 6800 instruction set manual. I didn’t really understand what I was looking at—boolean expressions, instruction encodings, timing tables—but I was obsessively fascinated by the mechanism of it all. Here was this complicated machine where if I understood it I would have power & control.
I feel the same way about AI models right now. I don’t claim to understand the details, not yet, but I’m fascinated by the mechanism of it all. I’m interested in both:
How models work but also,
The machinery that makes a model.
It’s this latter topic, how a model gets constructed, that I will begin to explore in this post (& possible followups).
Baking
 |
I love baking. You take ingredients in one form & transform them to a totally different form. The ingredients aren’t palatable in themselves but what you create from them is delicious. Oh and also baking is sensitive to initial conditions—you can make a small change early in the process & it will have a large consequence later.
I’ve been experimenting with cold proofing, where you let the yeast do its work overnight in a refrigerator. As I was working to understand models it struck me that there’s an analogy there to creating models, at least as I understand the process so far (please correct me in the comments if I’ve gotten something wrong).
First, though, a progressively revealed story about what we mean by a model. I’ll over-simplify but then reveal more complexity a little at a time.
Zooming Out
By “model”, we mean a computer system that can converse with a human (see, told you I’d over-simplify—hope you stay with me, this won’t take long).
 |
Actually it’s a little more complicated than that. The model is split into 2 parts:
A user interface that takes care of formatting inputs & outputs & sequencing & authentication & all that stuff.
The model proper where the magic happens.
 |
The user interface is built using conventional programming techniques (even though “convention” has changed radically in the last 2 years). The model proper, though (our topic of exploration) is built using radically different techniques.
 |
Model
A model is a bag of numbers. For today’s purposes that’s enough (maybe some day we’ll get to how those numbers are used when the model is producing useful words, but I’d have to understand it a little first).