CS GPT

Fri, 20 Jan 2023 08:57:09 -0400

This should be my last post about chatGPT.

The GPT in chatGPT stands for Generative Pretrained Transformer. Not something that you're going to cover in an early college CS class. I wanted to get my head around it and the technologies are typically covered in an advanced machine learning undergraduate elective so something you take after you take that 400 level machine learning class. Looks like I'm going to have to do some review first since I haven't looked at any of the prerequisite material in around a dozen years.

So, your K12 students won't be building their own GPT but that doesn't mean that there's nothing worth talking about.

There are technical things that you can teach that will help lead the way to understanding today's AI systems and also a lot of surrounding issues.

On the surrounding stuff one can certainly talk about how chatGPT works in a general sense. How it breaks down the input, the basic ideas behind a neural net, how it's trained on a data set etc.. Of course, that lets us talk about the data set. Where did it come from? This came up with a similar technology - GitHub's Copilot. You pose a programming question in a comment and it fills in the code. Where did it get the code? From all the GitHub users. The training set consists of basically all the code on GitHub. What about for other systems? Well, Google has all their users email + docs + photos + everything else and we all signed off on giving them the okay to use it way back when. Same thing for all those other "free" services. This raises some big questions that some of us were asking years ago. As they say "If you are not paying for the product, then you are the product." Lots of good fodder for discussion here and perhaps we'll raise the next generation to be more aware of what they're giving away.

Another issue is a systems reliability. We've seen many examples where chatGPT produces incorrect answers. What about Copilot? What if code generated by Copilot seems to work but leads to catastrophic failure later on? Who's to blame?

I don't remember the details but was told by an older computer scientist years ago about an expert system - that is, a much more primitive AI question answering system that performed medical diagnoses. Apparently it did better than residents and as well as doctors but it was never used. Why not? I was told due to liability issues. A computer program can't hold malpractice insurance.

This leads to a whole set of ethical questions

Another is ownership. If chatGPT generates a story or a song, who owns the rights? What about AI generated art?

One more on the ethical front, how about the Microsoft AI that was let lose in the wild and was quickly trained to spew racist garbage?

The ethical and societal issue abound.

There's so much rich discussion to be had with students of all ages here but there's also a technical side that can be introduced. As I said, you're not writing your own GPT system in a CS class prior to an advanced undergraduate elective but there are things you can introduce to pave the way and give students an inking of what's possible and what's to come if they continue to study CS.

One of my favorites is word chain or Markov chain text generation (see this older post). It's not the same but it is text generation and if you play around with it, use flowery text like poetry of the Book of Psalms you can get some fun results. It also lets you talk about concepts similar to overfitting which they'll learn all about when they do machine learning.

You can also do a bunch of introductory bag of words based natural language processing topics. Again, these are in no way in the neighborhood of a GPT but it's really good CS and it sets the stage.

Clustering is another topic that can be approached early that has some similar base concepts.

There are many more.

Just because you're not going to do a Transformer Neural Net in K12 doesn't mean you can't do cool things that get the kids both ready and thinking about important issues. One of my favorite topic to cover in CS0 or CS1 is a Caesar Cipher Decoder. It's not smart or even a necessary program but it's a cool unit that starts to scratch the surface of how more advanced systems work. It sets the stage.

So I think this is it for chatGPT posts for a while. Hope you all found these useful, interesting, or in some way positive.

GPT on C'est la Z

CS GPT