AI on C'est la Z

Teachers learning, using, and teaching AI

Wed, 31 Jul 2024 16:33:08 -0400

How do we prepare teachers to teach AI to their students, teach their students to use AI effectively, and how to effectively use AI themselves.

I was having this discussion with a friend this morning and I'm sure many educators have been having this and similar conversations over the past year or so.

While the discussion around AI is different from the discussion of CS, after all an AI writing assistant can be used with zero understanding of AI and in any subject area but AI is a subset of CS and there are a number of parallels when thinking about preparing teachers in both cases. Unfortunately, we're still having the same discussions about CS so while we can look at what's been done so far, everything's still too new to know what works or not.

Should students learn about AI in all of their classes or in specific AI or CS focused classes? What about in a specific "Using AI" class? Can "all teachers" learn sufficient AI to teach their students or do we need specialists? Are the answers different for the early grades than from high school? Will teachers learn on their own? Should we provide professional development? What would that look like? Or, do we need to integrate all of this into our teacher preparation programs.

These are all issues we've been dealing with in CS education and we don't have any definitive answer.

I don't have any final answers but I though I'd share some of my current thoughts both on delivery and also on training content.

In terms of delivery, the two options are Professional Development (PD) or preservice coursework.

While PD can be very valuable, quality PD is extremely rare, at least if you listen to teachers. On the other hand, if you're in a state with minimal teach requirements or the powers that be can't convince the colleges and universities that prepare teachers to bring in AI (or CS for that matter), then you're left with PD.

A local department of education, even one as large as NYC can also roll out PD much more quickly and at a wide scale like what NYC has done in their CSforAll program without requiring any changes at the state level, in legislation or by including colleges or universities.

A good example of some good work being done here is what Nora Burkhauser and Nick Yates are doing in Maryland. I wrote a bit about it here. They run a 2 part PD series. It doesn't prepare teachers to use AI effectively as it's meant to educate them on what AI is but it looks to do a great job at that. Part 1 is a great introduction for non CS teachers of all grades and part 2 goes even further. Ideally there would be even more for CS teachers, as I wrote about in my earlier post, but that's a limitation of the PD model and they're doing all they can do under their current system. They're also doing more than so many others.

On the other hand, I've had an opportunity to really see the pros and cons of the PD offered in NYC for their CSforAll program - here PD is generally offered by a PD provide - that is, someone like code.org or the College Board.

I got to do a deep dive when I evaluated applications for the CS Certification programs at Hunter College. I reviewed hundreds of applications over a three year period. If I remember correctly over 100 each year and close to 200 multiple times. There was a small amount of overlap each year in applicants but not much.

Applicants needed rudimentary programming knowledge in any text based language. Write a program that used a loop, had variables, a conditional, a function, and used an array or list. That's it.

The vast majority of the applicants had NYC CSforAll training and many, if not most had multiple years of this PD.

In years one and two I could barely accept 20 per year. In the third year I was able to accept close to 50 but that was by additionally offering multiple bootcamp sessions.

Clearly, the PD model wasn't working. We had a large number of teachers having gone through multiple years of training by the big name players in the field yet far too many couldn't tell the difference between a for loop and a forklift.

Sometimes PD is the best you can do and often it's all you can do right now but it's not the long term answer.

If this doesn't convince you I'll add that we all know that the attrition rate for teachers is among the highest of all professions and every time a teacher leaves the profession after two or three years, all that effort is lost. A PD model at best is a never ending treadmill.

On the other hand, if you work AI into your preservice courses, you have more time to cover it, the students (teachers) have some accountability (which rarely exists in PD sessions as one basically becomes certified by just showing up), and teachers enter the profession with AI as part of their skill set. Of course, ongoing PD after that then becomes a bonus.

Of course the challenge then becomes convincing states to require it and educational institutions to offer it.

In my Hunter programs, there's certainly room for both AI as a CS subject and also how to leverage using AI but my program is really made for high school CS teachers. I've had elementary school teachers but it goes so far beyond what they need to know in terms of content and there's less focus on their needs. It's also made for CS specialists so we'll never get the history teacher for example.

That said, the topics and ethics CS classes both give platforms to learn about AI. Even in 2020 we had a unit in the topics class on neural nets and many in the ethics class on bias. Meanwhile the methods classes and curriculum development class give a platform for exploring and using the tools, although, again the focus in on a CS class.

I can see a number of places we could work AI, from a generalists point of view into a typical preservice program and, between you and me, there's enough cruft in most programs that nothing would be lost. Even if you do have to cut something, AI isn't going anywhere so methods classes, for example, really will have to deal with AI tools if they really want to prepare teachers.

In any case, the next question is what to teach the teachers. I can think of a number of activities, both on the computer and unplugged which would give teachers a basic understanding of some key AI concepts. While not AI, a Markov Chain text generator show how you can use statistics and probability to generate text. Now and LLM is much more complex and uses a neural net on the inside but an unplugged activity around this simple generator can go a long way in demystifying an LLM and once demystified, they can be used more safely and more effectively.

I've done a number of other proto-AI lessons like "Who Played Spiderman?" or my Caesar Cipher unit which can be converted into unplugged activities to demystify other aspects of AI.

There's also a good amount of low hanging fruit for other uses of AI like generating discussion topics or having groups of students using AI to generate or research content and then have them discuss or debate.

Another advantage of doing this in preservice is that you can spread the AI over many courses and you'll have the time to also discuss issues like bias and dangers like using an AI tutor at the expense of developing teamwork and interpersonal skills with your classmates. In A PD time is much more limited and while there's good work like the Maryland project I spoke of above, far too much is given by vendors who's bottom line is selling a product. On top of that, things like bias is really beyond AI - yes, AI exhibits all sort of bias but the root is societal bias and where an AI PD is unlikely to be able to sufficiently handle that, an entire masters program can at least make a dent.

So, those are my thoughts for now. They'll probably change and evolve and in any case, given my current role as "retired guy" I'm not sure when I'll be able to act on any of this, though I do have some thoughts.

AI without guardrails

Fri, 26 Jul 2024 14:37:10 -0400

CSTA was only a week ago but I've already been trotting out things learned there in conversations. Specifically, during the panel keynote, one panelist, while discussing the affects of AI cited a study.

Two groups of programmers were tasked with writing some safe and secure code. One group used AI tools like copilot and chatGPT. The other didn't

When all was said and done, the group that didn't use the AI tools produced more secure code. The group that did use the AI tools not only produced worse code but they were actually more confident in their code.

To be honest, this really didn't come as a surprise.

Today I was thinking about this again. My friend Bethany Crystal wrote up a post on leveraging AI. Lots of good stuff in the post, one point was when Bethany talks of when she was with StackOverflow and how it became such an important part of developers workflows and how AI is now becoming the same but beyond coding.

The post is a great post to get people thinking about using AI in their workflows but we also have to be reminded that AI without verification can be dangerous.

Bethany gives an example of using chatGPT in a similar vein to StackOverflow - as a timesaver to generate content from which she would then build her actual work.

To do this, she asks chatGPT:

Hi, I'm a principal at a public high school in the South Bronx. Can you help me make a case about the importance of project-based learning to teachers who are skeptical about why we need to teach kids how to code?

If one is an educator and already knows their stuff on Project Based Learning (PBL), the results are a nice start if you're building a presentation, going to address your faculty, or a number of other tasks.

On the other hand, if you're someone who just got out of Teach for America and are now making the next greatest education startup but don't really know the real deal in education, it's not nearly as helpful. I ran this query twice, the second time, the answer didn't talk at all about some of the problems with PBL - class size and time issues, project storage over days for physical projects, prep time needed to set up each experience and many more. The first time I ran it, it mentioned some of the issues with PBL but just in the most generic sense "make sure teachers have the support they need."

Both answers also conflated coding and PBL so if the person using chatGPT doesn't have as strong education and in this case programming background the chatGPT solution can do more harm than good.

I then, for kicks, tried this prompt:

Hi, I'm a principal at a public high school in the South Bronx. Can you help me make a case about the importance of the use of lectures in teaching who are skeptical about why we need to teach kids how to code?

I had to change the wording a bit but it's basically the same ask but with the use of lectures instead of projects.

While parts of the response are very specific to lectures, the entire answer actually reads very similarly to the PBL answers and its justifications are just as strong (or weak depending on your point of view).

All of my queries resulted in some incorrect assertions.

Now, I'm not saying at all that one shouldn't use these AI tools - we should - if we already know about the domain we're using them in. They can be great at generating code for well understood algorithms - we just have to make sure to verify what's been generated. They can help phrase a difficult paragraph or reframe an argument and assist us in many other ways. The important thing, and the thing that that first group of programmers from the experiment up top forgot, is that we have to verify what these tools give us. We end up using them like a curated library and it isn't.

Tools like chatGPT look at a corpus of documents and probabilistically generate its content. This means it's basing its results on the documents out there.

In the case of code, we know that there's a lot more bad code out there than good. This means that while chatGPT can probably easily generate code for a really well known algorithm like a sort, we've got to question and test anything it puts out.

Likewise for education issues, there's a lot of bad material on education out there. A lot of garbage has been published and treated like "the true way" to teach and to run schools. Things that "reformers" love because they cut costs but teachers know are crap. Things that are likewise pet theories of education that have become popular but are no better and are frequently worse than what teachers know are tried and true methods.

What's more a chatGPT response can lack nuance. I can picture someone who's starting a school - a non educator let's say, heard from a person that PBL is good. Now, I know some people who swear by PBL and say to use it all the time - those are people in super selective private schools with tremendous resources and ultra small classes. The reality is that PBL is good and should be used as appropriate but there are other good techniques that will be better under some circumstances and worse under others. This person starting the school then dives into the AI with queries like I made on PBL and get a very one sided view. Now we have a problem.

I loved Bethany's post and it got me thinking more of how I can leverage AI, of course the problem for me is that I'm currently between gigs so don't have any work to apply it to. It also reminded me of the dangers of using it blindly and without guardrails.

Teachers want to teach AI but how will we teach the teachers?

Tue, 05 Mar 2024 09:58:42 -0400

Saw this earlier as posted on Linkedin by Pat Yongpradit of code.org. According to the article, nine out of ten educators think that students should be taught how AI works prior to graduating from high school. How it works, not just how to use it. This is a good thing. Sure, opinions vary as to when to teach it - elementary, high school, or across the board but still, a good thing.

It did make me think though about a few things.

First, in a way this is really just saying "kids should learn computer science." This too is a good thing but the bad thing is that a lot of people talking about learning about AI don't get this.

AI isn't some new magic mystically appearing. It's algorithms applied on data on computers with a user interface. Sounds like CS to me, more specifically a subset of CS. Sure the data sets are HUGE and the algorithms, more complex than the bubble sort but at a technical level, it's a next step.

AI applications can do some really impressive things but then so could earlier computer solutions.

AI can be filled with bias but computer systems have always been infused with our societal biases (and we have to do better).

AI can either take away jobs or enhance productivity but the same can be said with earlier computer innovations.

So, the call to teach how AI works is a good one but there's a level of concern that people don't see that in a lot of ways this isn't a revolution but a next step and as my readers know, I fell we've gotten some serious things wrong in the roll out of teaching CS, without acknowledging that this comes from CS begs the question, what will we do differently, or more specifically better.

This brings me to my next thought, and this is the bigger one - if teachers are going to teach all our students how AI works, how are we going to teach all those teachers?

It's clear, at least to me, that the professional development, short form training model favored by so many for CSforAll efforts is far from ideal. It sets up content providers and politicians to be able to say "we've trained all our teachers in just n years" with n being some small number but the truth is, those teachers don't really know what they're teaching and are doing a disservice to their students in the long run ¹. We also see Computational Thinking efforts where elementary school teachers relabel things they already do with CT labels - debugging for error fixing, for example. This can be a stepping stone to true CT and CS concepts but I've seen implementations where just the labeling is considered good enough. Worse, at times the teachers delivering these CT enhanced lessons aren't using the CT definitions correctly.

CSforAll in New York City has been going on for nine years. Some states are further along in bringing CS to their schools and some started later. Still, after a decade, we have an extremely long way to go. The number of high school teachers that have a deep knowledge of CS and how to teach it still pales in comparison to other subject areas and while elementary school teachers don't require the same depth of knowledge, they, in my opinion, as a group are also far behind where they should be when CS is a subject equal to all the others.

I'm not blaming anyone for this (other than what I already said about the acceptance and in fact enthusiasm of the short term PD model) but rather noting that creating a new subject area from whole cloth where those you intend to teach it never studied the subject themselves is a multi decade process.

What's worth thinking about now though, is what are we going to do with AI? Are we going to double down on short form PD and pay lip service to this emerging technology? Are we going to make some serious decisions and decide that some meaty material has to be included in per service programs?

I'm a skeptic. In a country where public school teaching is regularly derided and even in New York there's talk about removing the Masters degree requirements I'm not hopeful. Will we, as a society, decide to do what's right for all of our students or will we pay lip service while those who are well resources continue to get further ahead.

Time will tell.

PD as a first step is often necessary and can be a good start but without meaningful long form preparation as a follow up it's pure style over substance]

Hosting an AI panel at Stuy

Fri, 09 Feb 2024 11:15:47 -0400

Went back to school yesterday. Specifically, I was back at Stuy running an AI panel discussion for the Stuy HS Alumni Association. I figured it's time for me to get back in the CS / CS Ed game.

Plans started a few months ago when a member of my Stuy CS family who's also in the SHSAA board asked about doing something. I was game but a little apprehensive, mostly because there have been, let's say, issues with the way the SHSAA had operated in the past. Still, this was a new board with a new staff so I figured, worth exploring.

Turns out the new Alumni Association ED is a former student of mine so that rocketed the trust level of the association way up and we were off to the races.

Setting up any panel or tech event can be tricky. If you're doing something for your own classes it's much easier - you've got a good idea as to the classes experience as well as interest in whatever event you're planning. A panel like this, not so much. I had no idea on the audience size but since it would be mostly Stuy grads along with a few Stuy students and a low cost or free event it meant that the room would be filled with nimble minds. The challenge was that those nimble minds would be all over the place with regards to tech experience. I was guessing we'd have adult tech pros and hard core CS people, students who were into CS just finding their way, adults wanting to learn more but who had no background in either CS or AI, and probably a few alumna who just wanted to show up for an alumni event. Turns out I think I got the audience range right.

How to make a panel for that? You want to be able to do something of a deep technical dive but you don't want to lose the non-techies. You also want to spend time on all sorts of societal issues - talking AI is a lot more than just tensor flow, neural nets, transformers, and LLMs.

I reached out the my StuyCS family, my alumni network for four specific individuals. Oh, if you're an AI person and part of the family and wondering why I didn't ask you - remember - this was a live NY event so all you Pittsburgh, Seattle, and California AI folks, sorry - geography made that decision.

Alphabetically, first up was Ben - he was one of my technical heavyweights. I was confident he could handle any technical curveballs from the audience. He also has worked at both large companies and has built startups so he could also give some management and entrepreneurial context. Next, Doug - strong tech background but a lot of experience as a technical manager. Been at a number of major and minor companies from small to large. I figured he would anchor the business/manager side. Eugene was third - another technical anchor to balance Ben but Eugene has more formal academic training and also spent his entire career in a single major tech company. Finally, Jamil who also brought startup and small/mid company experience but has always been big on civic tech and equity related issues.

Of course, all four overlapped. I joked that Jamil was the conscience of the panel but truthfully those societal issues were important to all of us. Likewise, all have had some managerial experience etc.

From the limited feedback I got so far, I think the mix worked. We started "defining" AI, modern AI, LLMs (in lay terms) etc. and then shifted to issues like the use of other peoples data in training, bias in AI, and of course, recommendations for the youngsters just getting started. The audience Q and A got more deeply into these issues along with future jobs issues and if it's even worth learning to code anymore.

I won't recap all the specifics but a highlight to me was how the panel painted the picture to show that while AI was new and shiny and is opening new doors with both opportunities and challenges, at the core, so many of the important issues are societal based, not technical. Ben mentioning that in a startup the big cost drivers are still things like personnel and that we're still very much in the same startup/funding/scaling model just with new tech and Jamil and others emphasizing that it's not that there are these new potential dangers with bias, for instance with AI but rather that the bias is already baked into our system and AI is the next frontier - a wake up call that we can't just sprinkle some equity sugar on AI but rather have to deal with the hard societal issues.

All in all, I think it was a terrific event.

At the end I had a chance to informally spend time with both panelists and audience and it was great to see a number of old friends. Between the panelists and audience, I had students from the class of '95 up to some of my most recent graduates.

It was a fun first event to come out of my retirement hibernation. Not sure whats next but this one was both educational and fun.

AI For All

Fri, 31 Jan 2020 07:00:01 -0400

Yesterday we had another Professional Development Workshop for High School CS Teachers and as usual, I wnat to express my thanks to Digital Ocean for continuing to provide space, food, and great overall support.

This time though, instead of JonAlf and I having to run the show we had a guest speaker. We were joined by Sarah Judd of AI4ALL.

Sarah gave an overview of what AI4ALL was up to and why but the core of her presentation was taking us through some of the exercises they have been developing at AI4All. Specifically, they're developing what they call AI4All Open Learning which consists of curricular materials aimed at high school students.

As we only had a short time, Sarah took us on a whirlwind tour of what they're building from the baseline concepts of AI that she feels (and I agree) that every students should know up to specific AI topics like neural nets. Obviously you can't cover even a single course in one evening professional development session but Sarah did a great job giving both a taste and an overview.

The activities were all engaging and and covered both ethical/societal issues as well as the technical/CS. I appreciated that the materials weren't presented as canned experiences but rather tools that a teacher would use to deliver their own experiences. On the other hand I'm skeptical of the website's claims of "No AI or computer science experience necessary." I'm a firm believer that a CS teacher must know both CS and how to teach it - maybe not at first due to necessity, but ultimately. Regardless, Sarah has both real teaching experience and CS knowledge and I put more faith in what she's saying than a PR blurb on a web site.

Now you'll notice I haven't said anything specific about Sarah's presentation or AI4All and that's because Sarah will be giving a variation of it at SIGCSE in March. If you teach high school and are going to be at SIGCSE, I highly recommend you check out her session.

If not, perhaps I'll blog more specifics after the event.