Knowing and Doing

My memory is getting fuzzy, but this is how I remember it. Back in 2004 or so, I bought a new iBook. At the time, Apple was running a promotion that laptop purchasers could buy an iPod Shuffle for the special price of $50. They would even engrave it for free.

In college, I had worn out a couple of cheap Walkman knock-offs listening to radio and cassette tapes while walking to and from classes, and later a cheap Walkman CD knock-off. Why not try out this tiny little curiosity, filled with tracks ripped from CDs I owned?

I started walking or biking the two-and-a-half miles to work back in 2012, and every walk to campus was a 30-minute opportunity to listen to songs from the Shuffle's gigabyte of music, usually on shuffle but sometimes in playlist order. There weren't many options, given the spare interface. I loved it.

Jump ahead twenty-plus years, and I was still listening to that Shuffle. When I purchased my current laptop, a 2023 MacBook Air, I found that Apple Music no longer supported the device, so I had to refresh the music from my previous MacBook Pro. I did that every so often because 1GB isn't a lot of music, and a little variety is nice.

I've known the end was coming for a while. A few years ago, the battery stopped holding much of a charge, and cold winter walks sapped it even more than usual. But... I loved, loved, loved it: the size, the shape, the limited interface. The way it clipped to my shirt pocket, my collar, my shoulder bag. So I charged the battery more often and kept listening.

Thursday night, I charged the battery after using it for an hour that day. As I headed out across campus on Friday afternoon, I turned the Shuffle on to calm my mind, and it went dead almost immediately. No amount of fiddling could coax it to play. The end had finally arrived.

Photos of my Shuffle show more dings and scratches around the edges than I ever noticed while using it. In my mind, it's still the brand-new little device I remember from twenty years ago. I suppose that's like a lot of things as I grow older, including the face I see in the mirror every morning.

I have an iPod Touch from 2019, which I purchased for a trip to Europe for a Dagstuhl workshop. It's mostly been sitting unused the last few years. Yesterday, I loaded it up from my Daily Listening playlist and will give it a go walking across campus this week. I'm sure it will suit me fine, even if I have to fiddle with more interface than I'm used to — or care to, really. I just like the music.

Farewell, old friend. I will miss you.

Courtesy Joan Westenberg, another way in which outputs are not enough:

Bach copied Vivaldi for years before the Brandenburgs. Picasso painted in classical mode for two decades before cubism. Joni Mitchell played other people's standards in coffee houses for years before Blue. Hunter Thompson typed out The Great Gatsby and then spent fifteen years writing journalism that no one could ever mistake for Fitzgerald.

These are long, long stretches of work that looked, from outside, like nothing was happening. Inside, the inputs were being broken down into their components, sorted, and rebuilt as something the practitioner could call their own. The temptation, especially now, is to skip this phase by trusting a model to deliver the surface without the years. That temptation should be refused for the same reason a virtuoso refuses to lip-sync: the work that bypasses the apprenticeship produces no apprentice, only an output. And an output is not enough.

When we train a model, the inputs are broken down into their components, sorted, and rebuilt as the model. That process doesn't change us, though. The model's output is just another input for us to consider.

Several colleagues and former students have told me that LLMs have helped them learn a new area more quickly than they might otherwise have been able. One element common to all their stories is that they have deep background knowledge they can use as they process the new information. Another is the time and energy they spend processing the new information, engaging with it — and with more reliable sources — in ways that successful learners always do.

Our new tools may lead to more personalized education, but they will not eliminate the need to do the work.

But Shannon entropy and Kolmogorov complexity are measuring fundamentally different things. Shannon entropy measures the statistics of outputs. Kolmogorov complexity measures the structure of the generating process — the program, the mechanism, the cause.

Deep learning learns the outputs. It does not learn the program.

The quoted passage is from Shannon Got AI This Far. Kolmogorov Shows Where It Stops. It explains — better and more completely than I've ever articulated to myself — why neural networks and statistical approaches have never appealed to me.

Even back in the late 1980s and early 1990s, these systems could be very good at producing answers. Today's LLMs operate at a new level and really do produce amazing outputs.

But even at their most impressive, they interest me only as a possible substrate for something more. The results are cool, but they don't answer the question that motivates me: What's the program?

One thing I like about the Shannon/Kolmogorov article is that it does a reasonable job of describing how the current statistical models, despite their limitations, might help scientists and creators: These systems may well be useful tools for humans operating at higher rungs on Judea Pearl's ladder of causation. In doing so, they will magnify the abilities of those humans best able to operate at the higher rungs — and especially those able to move with some facility among the rungs.

I'd still rather work with other humans myself.

Here are two passages from an old conversation with musician Henry Rollins about writing and creating that stood out to me as I wrap up another year of teaching:

I'm a shipbuilder. I don't want to sail in them. I want you to sail in them. I'm just happy that they leave the harbor so I can have an empty workplace.

And:

That hesitation, that's what holds a lot of people back. That's why I never say, "I'm a writer," because I don't want to shoulder that. I just want to do some writing. "What would a writer do in this situation?" I don't know, man. Ask one. And don't tell me what he said, I'm busy.

Hat tip to Jen Myers, who quoted yet another blunt passage in her recent newsletter, along with this piece of advice, "Sometimes, you just need Henry Rollins to set you straight, you know?"

I need to write titles that distinguish This and That posts, so here you have the January 28 edition.

It's Books

A delightful paragraph from Linda Liukas:

At Hatchard's I was waiting for B. who had vanished into the first and modern editions section. An older husband was already exasperated: I've been calling you several times, he sighed down the stairwell. His wife emerged, unbothered, brushing past him: Oh, it's books, darling, as if that settled not only the argument but the entire question of how to live.

"... as if that settled not only the argument but the entire question of how to live." Beautiful.

Hat tip Robin Sloan.

Money on the Table

Joan Westenberg in Why My Newsletter Costs $2.50:

The instinct to leave money on the table in exchange for a better relationship with your audience is neither naive nor unsophisticated.

Most CS faculty leave money on the table to work with students in a way that only colleges and universities offer.

Way to Sell the Downside

From Dan Wang's 2025 letter:

While critics of AI cite the spread of slop and rising power bills, AI's architects are more focused on its potential to produce surging job losses. Anthropic chief Dario Amodei takes pains to point out that AI could push the unemployment rate to 20 percent by eviscerating white-collar work. I wonder whether this message is helping to endear his product to the public.

I was glad to see Dan write a 2025 letter after taking a break last year to work on his book. This letter is worth a read, as always. As much as I learn from and think about his writing on China and American policy, I really enjoy the parts about books and culture more broadly.

A few items from recent weeks...

Writing Code Is Fun

David Celis, in a post of the same name:

When someone's primary job is to figure out and write requirements or manage the entities who are actually producing the code, we don't usually call that person a software engineer. We call them a product or project manager.

Not, as Celis goes on to say, that there's anything wrong with that. But I like to write code. For that purpose, looking up an answer in language documentation or on StackOverflow serves me fine.

I'm not ready to turn writing code over to an assistant programmer. When I do want to work with an assistant, though, I work with a student. I'd rather help a student learn to design and write code than help somebody's LLM gather more data.

Being a Beginner

Asha Dornfest, in You, too, can be an urban sketcher:

There's a spark of aliveness that comes with being a beginner. A combo of shock and giddiness when you do something you thought you couldn't do. The intense focus that comes when there's no prior experience to fall back on. The possibility of new and exciting things within your grasp, like finding hidden treasure inside your house.

Hat tip to Daniel Steinberg.

More Confident a Year Ago

From comments by an exec at Salesforce:

All of us were more confident about large language models a year ago...

This is a Senior Vice President of Product Marketing, talking about why Salesforce is rethinking its "heavy reliance on large language models after encountering reliability issues". It turns out that "predictable 'deterministic' automation" is more reliable.

Huh. It turns out regular old programs do many tasks really well.

I'm not sure where I first heard about /ai pages. Damola Morenikeji may have been the first to explain the idea: generative AI is getting so good that readers won't be able to tell if the text they are reading was written by a person or generated by an LLM. One way for a writer to engender trust is to be transparent, by linking to a page that tells readers how AI is used on the site.

I ran across Derek Sivers's /ai page again recently and decided I would have one, too.

So, I have created https://www.cs.uni.edu/~wallingf/ai.html.

The one-line answer to "How does Eugene use AI on this site?" is: not at all, and certainly not on this blog.

If you see any text here that isn't a quotation of another person's work, then I wrote the text myself. That page elaborates a bit on my thinking, but it all boils down to the fact that I like to write and don't want to outsource my writing to a program. If I want to have a writing assistant, I'll work with a student.

My /ai page does describe one time I used an LLM as part of a research project:

I have used large language models (LLMs) in a research project with a student. The student and I worked with a prospective author to train an LLM on the public writing of a well-known educator from history. We then queried the model to see what the educator might say about some modern issues in education and public policy. For me, this is just the sort of project for which LLMs offer a potential benefit that would be hard to attain in another way.

By the way, this is a really cool project... I've been meaning to write more about it here, but it's not ready yet for public exposure. Besides, I'm sensitive to the externalities imposed on creators and on the environment by VC-backed generative AI systems, so I'm reluctant to promote their use any more than the current AI bubble is promoting them.

Anyway, I have an /ai page now. I doubt my stance on generative AI is likely to change significantly in the future. I like to write, I like to program, and I like for the presence of my name on a piece of work to reflect my personal investment in that work. If that changes, I will update the page.

Recently, a link to the 2013 Computer History Museum article Adobe Photoshop Source Code was going around my social media feed.

That first version of Photoshop was written primarily in Pascal for the Apple Macintosh, with some machine language for the underlying Motorola 68000 microprocessor where execution efficiency was important. It wasn't the effort of a huge team. Thomas said, "For version 1, I was the only engineer, and for version 2, we had two engineers." While Thomas worked on the base application program, John wrote many of the image-processing plug-ins.

I'm not sure why a 12-year-old article about a 35-year-old software application popped back into everyone's attention, but it brought back good memories for me. My history with Photoshop does not go all the way back to the beginning of Photoshop, but it does go back to my beginning as a faculty member.

In 1992, I started as a brand-new assistant professor. A colleague worked with me to set up my office and lab computing equipment: a Macintosh Quadra 950, a massive Apple display (*), and a bunch of complementary hardware and software, including a flatbed scanner from LaCie, OmniPage OCR software — and Photoshop. I felt like I was living in the future.

(*) This display got warm... really warm. One day that year, I was down the hall teaching an AI lab section when we all smelled burning wire. My display had spontaneously combusted. Fortunately, we turned it off soon enough to avoid setting off the alarm and inviting a visit from the fire department.

With permission from Adobe, the Computer History Museum provides access to the source code of Photoshop 1.0.1 from 1990:

All the code is here with the exception of the MacApp applications library that was licensed from Apple. There are 179 files in the zipped folder, comprising about 128,000 lines of mostly uncommented but well-structured code. By line count, about 75% of the code is in Pascal, about 15% is in 68000 assembler language, and the rest is data of various sorts.

Pascal — another connection to 1992 Eugene.

That first semester as a prof, I taught Pascal in our intro course, something I did again for the next couple of years. I had fun. Teaching programmers to beginners is a challenge that rewards you every time the light goes on in a student's eyes. Besides, Pascal had become one of my go-to languages halfway through my undergrad years, and I loved writing Pascal programs every day. It remains a favorite to this day, though I haven't written any Pascal in many, many years.

In the coming year, I intend to dive into the Photoshop source and see what I can learn from it. I like reading source code. This line from the CHM article says it all:

Software source code is the literature of computer scientists, and it deserves to be studied and appreciated.

... or, "How a Pair of Smart Glasses Jogged My Memory"

Earlier today, a student in my department tried to take a final exam while wearing Ray-Ban Meta smart glasses. Fortunately, the prof noticed.

Now we get to monitor our students' eyewear. Good times.

We now range from students who can't afford to buy textbooks in their effort to learn, to kids who can afford to buy smart glasses in their effort not to.

As someone commented on Mastodon,

the thing that really sucks is that smart glasses ... can help with accessibility and have very valid use cases, but they are already so misused that any wearing of them is suspect now.

Likewise, LLMs may also have a valid use in support of student learning (*), but right now their dominant use among students seems to be as the sort of crutch that inhibits learning — which accounts for the desperation come exam time.

(*) modulo their negative externalities, of course

Unfortunately, a student who does not confront their lack of learning until exam time, even if only by being unable to perform, generally ends up paying a steep price.

After all these years teaching, I have a lot of sympathy for students who feel desperate enough to cheat. Yes, they put themselves in a bad position, often after their instructor has made a significant effort to inform them about appropriate ways to use technology in support of learning and to explain the risks of inappropriate uses. And yes, they are accountable for their own behavior.

Still, a part of me thinks of the adage, "There but for grace of God go I..."

The year is fourth grade; I am taking a science test. I want to say that the test was on the six simple machines, but it might have been on a biology topic. At this great distance, memory is unreliable...

In any case, I forgot one item from a list. Either I knew that I was underprepared for the exam, or I had simply rushed in the moments before it started, because my science notebook was open on top of the pile of papers inside my flip-top desk. I opened my desk, saw the list, and filled in the final answer.

The teacher noticed.

Mrs. Bell came to my desk, looked at my paper, and asked what had had happened. I told the truth. She shook her head and told me she was disappointed in me.

That was like a dagger. I was the sort of student who wanted to please all authority figures. But this wasn't me disappointing just any teacher. I loved Mrs. Bell. She remains to this day my all-time favorite teacher. She was a huge influence on me.

(Here's another memory: I remember once saying to Mrs. Bell something like, "I don't know if I'll go to college." She smiled and said, "Eugene, college was invented for people like you." From that moment forward, I never doubted my educational future.)

Anyway: I don't recall what else she did or said in that moment. Did she dock me points? Make me take a new test? I don't even know if she told my parents, but I don't think so. If she did, they never said anything about. They, like Mrs. Bell, knew me well enough to know how I embarrassed I was by doing something so wrong and, yes, being caught.

Whatever else she did or said, I remember her calm demeanor and her personal response to me.

The cost of failing a test in the fourth grade is significantly less than the cost of failing a final exam in college, or of failing a course. So, I may have been saved from paying a much steeper price later by succumbing to a temptation much earlier and paying the price then. I was also saved by a generous and loving teacher who knew just how to respond to the particular student in front of her at that moment. What she did and how she treated me obviously made an impression.

I can only hope that my colleagues and I have the wherewithal to respond in such situations in a way that both holds our students accountable (yes, that's important) and helps them learn from what could be a traumatic experience. Unfortunately, I think we are going to have plenty of opportunities.

Ask, and ye shall receive.

In yesterday's post, I reported on an attempt to solve the classic Fizzbuzz problem in a functional style. Near the end of the post, I wrote:

This solution is fun but feels a little unsatisfactory. I'm not as fluent with functional programming as I am with OOP. Perhaps there is a more idiomatic FP way to do this? Let me know.

Thanks to everyone who responded with ideas and conversation!

Chris Johnson offered a couple of different approaches and ended up with a slick solution that is fully functional: create an array containing the different values to print, then use cycled values for Fizz and Buzz to construct an index into the array.

Here's an elegant solution in Haskell, courtesy of Chris:

pick :: (Int, Int, Int) -> String
pick (f, b, n) = [show n, "Fizz", "Buzz", "FizzBuzz"] !! (f + b)

main = do
  let fizzes = cycle [0, 0, 1]
  let buzzes = cycle [0, 0, 0, 0, 2]
  mapM_ putStrLn $ take 30 $ map pick $ zip3 fizzes buzzes [1..]

Beautiful. 1 (= 1 + 0), 2 (= 0 + 2), and 3 (= 1 + 2) select the appropriate word to print in cases divisible by 3 and/or 5, and 0 (= 0 + 0) selects the number as a default. Beyond that, the code is a couple of maps on top of a zip.

This technique reminds me of a similar trick Joe Bergin used when we ran the Polymorphism Challenge workshop back in 2005, solving one problem by looking up in an array an object to send a message to. I'd completely forgotten that solution until I saw Chris's!

Racket is not as concise as Haskell for a problem like this, but a Racket solution is nice, too.

First, here's choose:

(define (choose f b n)
  (let ((answers (vector (number->string n) "Fizz" "Buzz" "FizzBuzz")))
    (vector-ref answers (+ f b))))

Then we define the cycles:

(let ((fizzes (cycle '(0 0 1) size))
      (buzzes (cycle '(0 0 0 0 2) size))
      (numbers (range 1 (add1 size))))
  ...)

Finally, the code to implement the functional idea is:

(map displayln
     (map (lambda (L) (apply choose L))
          (zip fizzes buzzes numbers)))

I have to apply choose to the zipped lists because Racket doesn't have a destructuring define form. Of course, Racket is a language for making languages, so we can create one if we'd like! I've written a three-place form hardcoded to handle (define-match (choose (f b n)) ...), but I'll have to do some work to generalize it to take lists of any size.

Thanks again to everyone who wrote in response to my post, and especially to Chris for the fine solution. Readers of this blog continue to teach me new things. I hope they find some value in my posts, all too rare these days. In any case, I have much to be thankful for on this Thanksgiving Day.