Amiee Winchester:
Yeah, talking about what does it mean to be a learning organization and what does it mean to build improvement habits across the organization and what would it look like if we operated in this way as a way to solve some of our most persistent problems?

Alec Patton:
This is High Tech High Unboxed. I’m Alec Patton, and that was the voice of Amiee Winchester. Amiee is the Director of Continuous Improvement at Baltimore City Public Schools. In this episode, I talked to her and Zack Jaffe, who’s a Manager of Continuous Improvement also at Baltimore City Public Schools. I wanted to talk to Amiee and Zack about their improvement work on middle school and high school literacy, and particularly about the challenges and rewards of doing continuous improvement within a large urban school district.
Now I’ve got a pretty good theoretical handle on continuous improvement, but I learned so much from this conversation about what it looks like in real life. Amiee and Zack are exceptionally thoughtful, honest, and pragmatic on how they think about helping kids learn and how to do that better at a system level. It was a joy to talk to them and I’m so glad we can share this interview with you. Let’s get into it.
Amiee Winchester, Zack Jaffe, welcome to High Tech High Unboxed. Coming here from Baltimore City Public Schools, which we will now be calling city schools, because that’s what the cool kids say, and we’re here to talk about improvement. And first of all, can you just tell me what your roles were? Actually, can you both tell me what your roles were when you started this and if it’s changed what you’re doing now?

Amiee Winchester:
Yeah, so I started as the Program Director for Continuous Improvement and didn’t quite know what that meant entirely, but essentially I was brought on because we had a grant to kind of do this thing called continuous improvement in city schools and try to figure out what it could look like in various aspects of the organization.

Alec Patton:
All right. Zack, over to you. How’d you end up in city schools?

Zack Jaffe:
I was a teacher for 15 years, a high school English teacher. My wife was becoming a teacher, we worked together for a little while at a library at Johns Hopkins, and it looked really interesting when she got into the game so I started taking classes too. Ended up doing my student teaching at a Baltimore High School, I was really fascinated by it and got a job right after that. Stuck with it for all of those 15 years. And then I became a literacy coach as part of the blueprint that Amiee was mentioning before. They were hiring literacy coaches at a select few schools and I got to go in and work with the literacy improvement from that lens.

Alec Patton:
Okay, so at that point, if I said to you, “Hey Zack, continuous improvement. What is it? I keep hearing about it. I hear the Gates’ just gave a bunch of money to city schools for continuous improvement. What’s going on?” What would you have told me?

Zack Jaffe:
Yeah, I would’ve told you I knew nothing about it at first. It wasn’t until maybe six months in the job that they said, “Hey you, all the literacy coaches, you’re going to get this continuous improvement training.” And we ended up sitting there in a nice conference space and hearing all about, from the Carnegie Foundation, what continuous improvement was. And I remember sitting there with one of the first exercises that we did in looking at variation and just saying, “Well, sometimes the average is not the whole story but there’s a lot of people on one end and a lot of people on the other end and you can learn from that.” And I was like, “Wow, that seems pretty cool. That seems like a good way of approaching things.” And then they had us trying to investigate, back at our schools, like what are some literacy issues that are going on there and try and generate some solutions. And so, that was my first exposure to it.

Alec Patton:
At that first event, if you were to do your own chart of intrigued and excited versus suspicious because it’s like another thing coming from the district, I’m not making any assumptions about your particular district, but I do know how education is generally, it’s probably not the first thing that had come down the line that you’d heard about. Where were you?

Zack Jaffe:
Yeah, I would say that I didn’t know what I was getting into at first. And just having no clue, naturally there is a little bit of suspicion like, “What’s this all about?” But from that very first session it felt to me like it was something different, because it was just a way of approaching our work so that we could learn from it rather than the usual professional development which is like, “Well, here’s the best practice that we already know and now you must go forth and implement it.” And so, that was exciting to me from the beginning.

Alec Patton:
Was that your responsibility? Did you do that, Amiee?

Amiee Winchester:
Actually, when I was going through the interview process for my job, I used Zack’s data that he had created from his own kind of system analysis at his school. He was kind of a legend by the time I came on, because he was one of the people that ran with it.

Alec Patton:
Now I’m confused. So the Continuous Improvement Program was happening when you got your job, Amiee?

Amiee Winchester:
So it started about six months before, because we got the grant and then like Zack mentioned, before we kind of did any sort of PDSA testing or anything like that, they did the launching the network. So they did empathy interviews and they process mapped and they kind of did all of these tools. And then at that point they realized, because we were getting into a new school year, that this was overwhelming the literacy coaches because they were really the ones that they wanted to launch this work with.
And so then they brought me in alongside some other team members to really kind of run this network, because they realized that like, “Hey, this is going to take a lot of work and we might have to have additional people that could do it.” So I came in about six months later when they were able to, like as a district things move slowly sometimes and you have to learn your way into something that’s new. And so I came in after the literacy coaches had gotten some of these initial trainings.

Alec Patton:
So you were kind of, in a way you were the result of an early PDSA cycle.

Amiee Winchester:
Yeah.

Alec Patton:
Where people were like, “Oh, you know an issue that we’re spotting?”

Amiee Winchester:
Completely.

Alec Patton:
“Is that we don’t have coordination right.”

Amiee Winchester:
Yeah, I think my boss, again, one thing that I really appreciated about her was that she had a lot of future thinking of where this could go. And I think her, alongside some of our other leaders were really intentional about saying, “Hey, we can’t necessarily do what this is going to entail, but what if we were to bring in?” So myself and then the network lead who was a part of the literacy office, we came and kind of became this team alongside some analysts that we partnered with. So that didn’t happen until about six months after Zack started.

Alec Patton:
Got it.

Amiee Winchester:
So by that time, we all knew Zack Jaffe, because he really, like he found some really amazing stuff in his school by using these tools. And I think looking back now, he internalized it pretty quickly and was able to leverage it in a way that I think now that we look back are like, “This is how we want people to operate with improvement.”

Alec Patton:
You knew that you were going to be looking at literacy? That was-

Amiee Winchester:
That the grant was focused on improving outcomes for eighth and ninth grade on track. And our theory was, what would it look like if we had the lever of literacy as a focus? So that was always their focus from the beginning.

Alec Patton:
And why literacy? What was the initial theory there?

Amiee Winchester:
The initial theory I think was that because we had at the time, this new strategy coming from the district, one of the pieces in that strategy was literacy. That was one of the pillars. And we knew that literacy is a key lever for student outcomes, and we knew that that was an area that our students could continue to see improved outcomes in. And also, we knew that we wanted to launch this literacy coach kind of initiative, and so understanding like, “Hey, how could improvement really help us learn here?” I think was the ultimate thing. Also, I think as a district, when you see opportunities to align grant funding with what you’re trying to do, that doesn’t hurt either.

Alec Patton:
Of course.

Zack Jaffe:
We also had a new literacy curriculum K through eight at the time, and so it was naturally kind of building upon that. There was also a lot of work going on just to really investigate the science of reading the foundational literacy skills, and this is a good chance to say, “Well, let’s not forget about our older students as well and let’s try and figure out something for them.”

Amiee Winchester:
While we did start in literacy, there was always a goal, which is something that I held from the beginning of, what does it look like to be improvers in an urban school district?

Alec Patton:
Are you talking like every single school?

Amiee Winchester:
Yeah. Talking about, what does it mean to be a learning organization and what does it mean to build improvement habits across the organization, and what would it look like if we operated in this way as a way to solve some of our most persistent problems?

Alec Patton:
All right, I want to hear about Zack’s legendary findings. So, what did you do?

Zack Jaffe:
It was nothing super mysterious, I empathy interviewed my students. We were all investigating certain literacy problems, I had already determined that one of the big focus areas for the high school I was working at was writing just from looking at our test scores. That jumped right out. And once I learned about empathy interviews, I was like, “Okay, I got to go talk to the students.” And so, I pulled the eighth grade writing scores of some of our students and compared it to their 10th grade writing scores and said, “I want to learn from these particular students right here, who seemed to be on such a good track in eighth grade and then when they got to high school, their scores in writing dipped,” to find out what was going on. What’s their experience with writing instruction? How do they approach a test in writing versus the everyday-to-day writing that they do, whether it’s in school or out?
And so, just going to talk to them really made me realize that they had a really positive attitude around writing but also felt that academic writing was something that their teacher kind of did with them or for them sometimes, and they didn’t really feel empowered to do it on their own. So I mean, I think that was the big “Aha!” moment that I got out of it and maybe brought to other people. Which is that, boy, no matter what we teach around writing, we have to work on confidence, we have to work on independence, all of those things that make it from just a skill you do when your teacher tells you to do it to something that you can actually use in life.

Amiee Winchester:
I’ll add to that. Just kind of thinking about what we often hear now five years on, empathy interviews seem to be the tool that’s most profound for people and I feel like it’s probably the most part of what we do. I hear now often, and I’m sure you do, it’s like, “Oh, let me go empathy interview so-and-so.” But also, you looked at data, you looked at the variation. I remember, because I had to analyze his stuff, that you were looking at data differently. It wasn’t just aggregated data of this many students, but you were kind of looking at it from different lenses, which I thought was really interesting because it wasn’t something that I had seen very often before.

Alec Patton:
Well, sure. One thing that I’m hearing is that when you selected students for the empathy interviews based on a pretty thoughtful approach to looking at the data, that these were kids who you’d said that they kind of had this particular attitude and confidence in eighth grade and you saw this shift. So you were pretty strategic about who you interviewed, because I think, I mean, empathy interviews is always valuable. Any time a teacher does that, like takes the time to actually talk to kids in a non-threatening, non-judgmental way, they’re going to learn a lot.
But there is something particular about saying, “Oh, I want to know… I’m seeing a story here. This data’s telling me a story, I can’t actually interpret it, I’m going to talk to those kids to find out.” And that’s something that doesn’t always go together, that I think sometimes you have a kind of a, “Oh, we seen the data so we know the story.” Or, “I’m not really interested in numeric data, I just want to talk to the kids and find out.” But it’s like one or the other, and actually it’s that, the power really lies in that combination.

Zack Jaffe:
Yeah, it was really about trying to find the story behind the data. And once I was looking at the names of the students, it’s like I knew a lot of those students. It was a small school, I knew who their teachers were and how they approached teaching, and I knew that I could go talk to them and pull them out of class. And they were a little worried at first, they’re like, “What’s this all about?” But just approached them in a way that it’s like, “Hey, I just want to hear what you have to say.” And I thought that that was the group who had the most interesting stories to tell. We’re always wondering what happens with students when they receive a particular type of instruction, especially as they grow throughout their years? And they just had really interesting stories to tell in that respect.

Alec Patton:
Yeah. So confidence in writing and maybe a way that, and possibly a way that teacher supports were actually reducing confidence, was something that you were seeing. So at that point, were you crafting an aim statement? What happened then after those empathy interviews?

Zack Jaffe:
In the meantime, the network as a whole, all 14 schools, we were working really deeply to understand what an aim statement could look like. What are some different avenues that we could create in a driver diagram? And so for me, that was one part of the driver diagram that we created, which was really focusing on writing improvement. There were other ones too, but I felt like that was the one I really wanted to tackle for a big project going forward the next year and saying, “Hey teachers, this is what our students are telling us. What do you think about the instruction that you’re doing? How could it be related to what the students are telling us? What improvements do you think we could make?” It just made me really excited as we kind of finished off creating that aim statement and driver diagram in June to come back the next year and really dig into it with teachers.

Alec Patton:
Yeah. And so you’re in a position where you’re an instructional coach, you have a little more time available to you to be doing this, to be working on this, which is an obvious, I think, kind of strategic point of, that there’s going to be stuff you’re going to be able to do that the teachers aren’t going to be able to do. How did you get teachers involved?

Zack Jaffe:
One thing I did was show them what their students were saying. They took a good long look at it and circled key phrases and they kind of came to the same realization that I did without me having to say anything. And from there, it was just a matter of like, “Well, let’s brainstorm.” Let’s bring all the teachers starting with the English department into the work and say, “What do you think we can do to get to the students to the point where they feel more like they can do it independently, like they don’t need you to hold their hand through the writing process?” And so, they were excited to try that out at the beginning of the school year, to brainstorm like, in little groups of two or three, what could be a change idea? What could be a little PDSA cycle that we run?

Alec Patton:
And now this is where I think the structures of improvement get interesting, because I think it’s safe to say that every school in the country starts the year saying, “Hey, we’ve noted some interesting things. We’ve got some interesting ideas.” And teachers get excited about it and it’s like, “Yeah, that would be cool. Let’s try doing that.” And then two weeks in, nothing more happens. So tell me about what happened next to make that into something that was meaningful?

Zack Jaffe:
Yeah, I think that this is where me having that little bit of freedom in the school that you talked about, and in essence, I was like a middle level leader at the school. Where I had that freedom to say, “Well, I am running the department meetings, we’re meeting once a week as a collaborative team.”

Alec Patton:
This is the English department?

Zack Jaffe:
In the English department. “And this is how we’re going to explore it. Rather than us sitting here and just looking at policies that I could easily send an email about, we are going to spend our time doing PDSA cycles.” And because we are one of the 14 schools as part of this larger network, the school leadership was bought into that as well. And we had some pretty good learning for the first semester, just kind of sharing what we learned, trying different things, making suggestions. I could go into classrooms and watch the teachers and give my instructional advice as well and say, “Maybe if we try it this way.” And there was a lot of really, really good learning that was happening because we just made that the focus of our department meetings.

Alec Patton:
Can you talk me through what one of those meetings, how those meetings went?

Zack Jaffe:
Yeah. In the beginning of the year when we’re starting off with brainstorming change ideas, then typically I would come in with a little bit of research. “Here are some best practices in writing instruction. What do you think we could tackle?” And putting people in twos or threes to get together and talk about what that looks like. And then if we came back in another week, it would be, “All right, let’s really solidify what our change idea is going to be. Let’s think about what’s going on in the curriculum right now and think about how we can fit this idea in.”
And the change ideas varied. Anything from giving students more weekly independent writing time during Fridays to actually having a paragraph planning strategy. And because there was at least two teachers doing each change idea, then they got some time during those department meetings to talk to each other and give each other some advice as well. And then we would come back and actually look at a little bit of data, bring in some student work. What are you seeing here? Is it making a difference? Are your students able to act more independently or do they still need as much help from you as they did before? So it felt really generative to have everybody working on these problems together.

Alec Patton:
Amiee, were you coming to any of these meetings?

Amiee Winchester:
No, because at that time, so something happened where from January until June they were doing the network launch. So it was like, what’s our aim? What’s our driver diagram? They were kind of using all the improvement tools. And then a lot of the literacy coaches who were a part of those initial 14, they were sharing that this was too much. They’re like, “With the curriculum, with this being a new role that I just started in my school, I can’t also own this improvement work.”
Zack was kind of the anomaly, because he was one of the coaches that kind of figured out how to make it work within his context. And so by the Fall of the following year, we actually had shifted completely away from literacy coaches and they created a new kind of group called Improvement Champions. And these were just random people from the schools that the Principal designated. We had a math coach, we had an AP, and then some people were related to literacy in some way.

Alec Patton:
Why were the literacy coaches having a tough time with this do you think?

Amiee Winchester:
I don’t know entirely, Zack might have some insights. I will say that when we first started improvement in the district through this network, it was the quintessential technical, “First you do this and then you do that and then you do this.” And it was kind of what I feel like the experience is when you’re learning improvement for the first time, and that’s really overwhelming because you don’t yet know how to adapt it for your context, you’re kind of just doing this extra thing.
And sometimes, I know from talking to some Principals at the time, it didn’t feel aligned to what they were already doing in their schools. And so I would maybe suspect that some coaches were struggling with that, that it was on top of what they were already doing. I know that some coaches were new in their school, so they were just building relationships with the teachers. So I’m sure there was a lot of root causes for why that happened, I just know that when I came in it was like, “No more coaches. Now champions.” And that was who I started with when I first came up.

Alec Patton:
Except for the legend that was Zack Jaffe.

Amiee Winchester:
Well, Zack was always a part of it because he was, I would say, this is Amiee’s personal, right? I think he was able to kind of see it for what it was and then adapt it in a really humanistic way that I don’t know that every coach had the privilege or opportunity to do.

Zack Jaffe:
I would say, yeah, thinking about it as not just a bunch of tools that we’re using but as a way to get teachers really excited and involved in their own learning was my approach to that. I would also say like, being at a high school, again, there was a new curriculum K through eight that was a big burden on a lot of those middle school literacy coaches that I didn’t have to deal with. And so, I think it did feel to a lot of them like, “Okay this is interesting stuff, but it’s not the core of my work.” Whereas as I would-

Amiee Winchester:
Well, I don’t have time to because I have to do these other things.

Zack Jaffe:
Yeah. Right. Yeah. And I was trying to approach it like, “Wow, this could really be the core of what we do here.”

Alec Patton:
Yeah. I was wondering, I was going to ask you how something would be not aligned? Because I was kind of like, “Well, the whole sort of like,” my initial thought was like, “Well, you want literacy to improve at your school, so that’s aligned with every school,” but I can see that if you’re literally adopting a brand new-

Amiee Winchester:
That’s right.

Alec Patton:
Unfamiliar curriculum, then like-

Amiee Winchester:
Your position’s new.

Alec Patton:
Yeah.

Amiee Winchester:
We just rolled out this whole new strategy. Essentially, if you think about these 14 coaches, they’re on the front lines of this new strategy and they’re kind of shepherding this change into their schools. And to be fair to them, improvement back then was overwhelming. I was overwhelmed. I think I cried the first time I had to interact with that driver diagram, because it was so theoretical and it was just so big. I think there was a learning curve for me personally of understanding that as a leader of this work, I am allowed to have a vision, and that might mean that we adapt the way we do it. It doesn’t have to be exactly the way it was kind of packaged to us.

Alec Patton:
Yeah, so I’m hearing two things here. One is just that I think for middle school instructional coaches and literacy, they’re like, “Yeah, I know my aim statement. My aim statement is implement the new curriculum that none of us understand. Done. I don’t need to empathy interview a bunch of students, I just know that that’s my job.” And so, that’s pretty obviously kind of a, and also to be like, “Okay, if the new curriculum doesn’t actually meet these other goals, then that’s going to be an issue. But for the time being, I know that people spent a lot of money on this curriculum and I know that that’s what I’m supposed to do.” That’s a hard position to be in, I think.
The other side I think is just that there is this paradox with improvement I feel like, that it’s about deeply listening to the people who are often least empowered. It’s about empowering the people closest to the problem and the belief that they’re closest to the solution, as Deming said. And so it’s this intensely bottom up, humanistic, as you said, way of working. At the same time, it has all these pretty impenetrable, it’s like somebody who’s really into Dungeons and Dragons being like, “This is amazing because you get to tell your story and it’s all empowered.” And you’re like, “Oh, that sounds so cool.” And they’re like, “Well, the thing you need to understand is the hit points and the 12 sided dice.”

Amiee Winchester:
That’s right. That’s exactly right.

Alec Patton:
“Oh but no, it’s actually the eight sided dice.”

Amiee Winchester:
Yes.

Alec Patton:
And you’re like, “Forget it. Forget it. I’m just going to read a book, because I can’t deal with this.” And so I think there is that paradox really deeply built in there. And so you identifying that as an issue-

Amiee Winchester:
It was obvious. Because here’s the thing, our literacy coaches, they were some of the smartest people that I ever got to work with. And when you say to them like, “Okay, so now we’re starting this network, this improvement network, and you just spent the last six months doing all these things, now you’re responsible for taking this and putting it into your school.” There’s a difference between being able to be an improver versus facilitating improvement. And I think very quickly we all learned that whoever was sitting in this role, at least for right now, had to be facilitators. And in order to be a facilitator, you have to deeply know what you’re doing, because you have to be able to make those pivots.
And so I think we knew that early on. I think then we spent the next six to eight months trying to figure out what that could look like. So in the summer was when the district transitioned from literacy coaches were going to lead this improvement work in their school to now we have these champions and literacy coaches are going to be on the improvement team, but they’re not going to be responsible for leading it. And so the champions was where I started. They kicked off in the Fall of 2019.

Alec Patton:
So how did you find your champions?

Amiee Winchester:
So I was still not here yet, I came a couple months after they came on board, but they literally went to Principals and said, “You need a champion. Someone’s got to do this work in your school.” And so the principal designated somebody, and that’s where we got champions that we ended up working with.

Alec Patton:
Didn’t every Principal first say, “Oh, the literacy coach. They’ve been doing it.”

Amiee Winchester:
No actually, usually as is the case, they pick somebody that they trusted and that they also knew would be able to do the work. So it was oftentimes other instructional coaches, an AP, or very rarely was it a teacher because teachers didn’t have the time to come out of the classroom.

Alec Patton:
Yeah, of course. And I mean, there must’ve been some variation there in-

Amiee Winchester:
Totally.

Alec Patton:
Enthusiasm and like-

Amiee Winchester:
Actually, for the most part, and what’s really interesting and amazing now is the majority of, or at least half of the first 14, are still people that we work with in some capacity today.

Alec Patton:
That’s amazing.

Amiee Winchester:
One of them is now leading one of our other networks in central office. He was the math coach that was a part of that. So, I think that-

Alec Patton:
The math coach who was leading the literacy program?

Amiee Winchester:
The math coach who was leading the literacy improvement work for his school. That’s right. Yes.

Alec Patton:
Right on.

Amiee Winchester:
He did a great job too.

Alec Patton:
If that guy’s doing a great job, then you know that that’s somebody to keep an eye on.

Amiee Winchester:
He did. So I think there was a little bit more of a buy-in because while the principal did designate, the person still had to agree to doing it. And so they showed up and we also really, we did a lot of professional development with them so that they had those opportunities. But again, we didn’t really launch anything for the next couple of months. They did some PDSA cycles, but it was very minimal. So we didn’t actually get into any work until after the pandemic had happened.

Alec Patton:
Interesting. One thing that’s always a puzzle to me is that it’s very easy to say, “Oh, we did lots of professional development,” but that is by no means a guarantee that things were in a good position. I could imagine I’m an AP or a coach, I’m not necessarily, literacy is not necessarily something that I’ve spent that much time on. I feel a little bit like, “Ah, I don’t know, I’ve always been a little nervous about that, I’m glad it’s not my job to do it.” Suddenly it is my job and I’m kind of going, “Okay, this improvement thing, this sounds kind of cool. I’ve heard about Zack, he talked to a bunch of kids, I like talking to kids, that sounds like a nice experience. Great, cool.”
Come out of a professional development like, “Oh my good, this is completely overwhelming. I thought this wasn’t complicated, but now I’ve got no idea. I’ve got 30 pages of notes, I can’t make sense of any of them. All the diagrams and the arrows go in all kinds of different directions. What is going on?” So I could really see that making things worse rather than better. How did you make professional development work to help people feel confident and have the abilities to match that confidence?

Amiee Winchester:
So they came on in, let’s say September of 2019, and I came on in the beginning of November 2019, so a couple months later. Up until that point, they were trying some PDSA cycles, but they weren’t, like nothing was really happening with any sort of reliability. And then in December we were given the opportunity to do a training from the Carnegie Foundation around, “What does it mean to be facilitators of improvement?”
Now, something to keep in mind, and this is something that Zack and I have been kind of on a journey with the improvement work in the district overall, is that if you’re doing improvement and it takes you a year to figure out what you’re focused on, it’s not improvement. Or at least it’s not helpful in a district. And so we were hoping to launch again in January of 2020, because we were adding another 14 schools. Because we had to go up to 26 in total. So we-

Alec Patton:
But nothing had happened yet, except for Zack.

Amiee Winchester:
That’s right. Well, and Zack was like, only reason I knew about Zack is because I used his data for my performance task when I was getting hired. And other folks who worked with him knew that too. But yeah, nothing had really happened, we were still trying to make sense of what the driver diagram was, what are our measures? And so-

Alec Patton:
You were like, a meta improvement project was happening where you were like, “Well, that didn’t work. So now-”

Amiee Winchester:
We were aspirationally a network. We really wanted to improve literacy, but there was so much variation and I think we were trying to encompass all of that in what we were trying to accomplish. And it wasn’t until actually a couple months after I started, we met with myself and Zack’s predecessor, the person who had his role before. We met with Ryan here from High Tech High, and we were like, “Okay Ryan, what do we have to know about leading a network?” And the one thing he said to us that resonated and still does to this day is he said, “You have to determine what your non-negotiables are.”
And for us at High Tech High, the non-negotiable is the aim, what we’re trying to accomplish. And then from there, people can choose if that is a outcome that they are interested in being a part of this community. And I think when we kind of talked to him in late winter, that really helped kind of start us on this path of like, “Okay, what is the aim of the district?” And not so much trying to figure it out for every single school, because naturally it was different. And that was where we were finding it hard to find a common place in which we were all engaging in improvement. So yeah, not a lot of stuff started until a few months after the pandemic when we were able to kind of get really clear and narrow about what it is we were trying to achieve.

Alec Patton:
Yeah. And when you say person in Zack’s role, you mean Zack’s current role.

Amiee Winchester:
That’s right. Yes.

Alec Patton:
Which is, we didn’t say that before I realized.

Zack Jaffe:
Yeah. Manager of Continuous Improvement.

Alec Patton:
Got it.

Amiee Winchester:
So he took over as the network lead. Because the way that we structured is that you had a content lead who was the network lead, you had an improvement lead which was me, and then we had a partnership with the local university and they were like our analytic capacity because we didn’t have it internally.

Alec Patton:
Got it. Sure. So all right, you have that meeting with Ryan. What was your non-negotiable?

Amiee Winchester:
Our non-negotiable was the aim. We had to figure out what it is we were trying to accomplish. And so January we launched, again, because now we’re adding more schools. And I think I cried that day because we literally had like 15 different small groups and we were trying to group people together in these common things and it just wasn’t working.

Alec Patton:
So did you actually add the extra 14 schools? Like you went ahead-

Amiee Winchester:
We did.

Alec Patton:
So now you’ve got 28 schools.

Amiee Winchester:
Yeah.

Alec Patton:
But the schools, new schools were like, “So what’s going on with this thing?” And the old schools are like, “I don’t know.”

Amiee Winchester:
That’s right. It was a lot of learning about, what does it mean to launch an improvement network? And I think one of the tensions that you experience when you work in a district is the tension between needing to move forward and having time to think about where it is you want to go. And sometimes you take two steps forward and you learn something really valuable and then you take two steps back because you didn’t get as much time to really reflect on what you’re doing. And I think that was kind of the journey that we were on at that point.

Alec Patton:
Now let me ask, and either one of you can answer this or both of you. Do you wish in that, in early 2020, that you could have had a do-over and just be like, “Let’s just do another year with these 14 schools and press pause,” or do you think it was better going up to 28?

Zack Jaffe:
Well, I think the real learning is like, I think what Amiee was noticing is that there’s a lot of people involved in this work and they don’t all necessarily have the time for it, they don’t all necessarily approach it in the same way, and there was a lot of learning that happened in that that really led to a big improvement that was forced on us by Covid, which was the teacher fellowship and working directly with the teachers. So regardless of what happened and expanding to more schools and things like that, I think it was the rethinking of the whole nature of, who do we work with? Who has the time and the commitment and the interest and the expertise on the ground? That was the really big change.

Amiee Winchester:
Yeah, and I will say that when you work in a district you become acutely aware of all the external voices that say, “Look, I’ll build a strategy for, I know the answer, have the answer.” And we had to go through that to get to a place where we had the right aha moments. But what that did help us to do is to condense the launch of new networks in the future, right? So we had to go through this process in order to understand, “Okay, when you’re launching a network, you need an aim and the aim needs to be grounded in what you know about your system and so forth.” And when the pandemic hit we were trying to figure out, where do we go now? Is this dead? Is it still alive? What can we do with it?
And we realized very quickly that that also gave us an opportunity to dig into our own data. And that was where what Zack had found six months before around writing at his school, we saw that across the district. We saw that about a third of our students were not writing anything at all on the state assessment, and that gave us the opportunity to go, “Whoa, this is important and we really want to focus in on here.” And we also found something around about 50% of our students were entering middle grades without having their foundational literacy skills. So that gave us the ability to pause for a moment and validate some of the hunches that we had about where we could focus, but then it also gave us the opportunity to invite teachers in because now teachers were on the front lines of what was happening and it made a lot of sense to incorporate them in a way that we hadn’t been able to before.

Alec Patton:
Yeah. So you mentioned the teacher fellowship. What was that?

Zack Jaffe:
Yeah, so when the pandemic hit, coaches, assistant principals, or anybody really couldn’t be involved in the improvement work anymore. It was just all hands on deck for survival. How do we teach Zoom classes? How do we do Google Classrooms? All of that kind of stuff. And it wasn’t until over the summer when we were taking a breath again and Amiee contacted me and said, “Hey, we’re starting up these teacher fellowships. We want to explore different things and we’d love it if you were able to be a part of this small network that’s going to investigate some best practices in virtual learning. Because we anticipate we’re coming back the next year, it’s still going to be virtual, and we need to get people who are on the ground and doing the work to be really thoughtful and creative and try and figure out what’s going to be best for their students.” I was like, “Okay, let me get involved in that as an advisor to be working with these teachers.” And meanwhile, there were a couple other affinity groups that were starting up with other areas of literacy.

Amiee Winchester:
Well, and what Zack’s leaving out is that Zack had created his own kind of mini network around virtual engagement. And so we had heard about it, because some of the teachers that we were working with were like, “Oh, I’ve been working with Zack and they have their own data collection.” And so again, here he is kind of living and breathing these improvement principles in his work. And so, it was natural to kind of pull that into the network. But at the same time, we wanted to understand more about writing and we wanted to understand more about these foundational skills, specifically phonics, vocabulary, and fluency. And at the same time all of this is happening, we’re starting to understand how improvement can actually be a value add to the work that we’re doing now that we’re in a space where there’s a lot that is out of our control.

Alec Patton:
And what was that? What was the value add?

Amiee Winchester:
Well the value add was the ability to learn quickly, and it was the ability to apply some discipline into how you were going to go about testing out potential solutions to see if it led to outcomes that you wanted. And I think that while it feels very intuitive, it’s not always, right? Because it does require some of that technical know-how in order to say, “Okay, so if I’m looking at,” I think I remember your spreadsheet you created with Gina, one of the teachers, where it was like, “Okay, we want to see how many assignments kids submitted and we want to see how many assignments kids submitted after they were asked to submit if they hadn’t.” And so they had all of these kind of process measures that helped them to understand what the impact might be of what the teachers were doing in those virtual settings.

Alec Patton:
So were teachers involved with that?

Amiee Winchester:
Yeah. So essentially what we realized after we identified what the outcome is or the levers we wanted to address, we actually went out to the literacy coaches and we said, “Hey, we’re starting this teacher fellowship around improvement, we want to know which teachers would you put up for this?” And so they gave us names of teachers and we emailed them and were like, “Hey, starting a network, want to join?” And I think it also helped that as former teachers, we sat down and we thought to ourselves, “If I were a teacher being asked to do this, what would I want?”
And so we’re like, “Well, a stipend would be nice.” And so we really thought about what could be the things that a teacher could get if they were a professional in this way? And so yeah we, I think started out with 21 teachers in these four affinity groups that we had and had no idea what we were doing but our goal was to connect them to measurements, to connect them to literacy experts, and to start trying some PDSAs. And that’s really where the network’s activities took off.

Alec Patton:
Yeah. I think with improvement with teachers, you have the classic difficult sell of, “This is going to be more work for a little bit, but it is going to make your life easier.” How did you get people to go along with that?

Amiee Winchester:
We actually framed it, because this is how we were coming at it, which is that, “Hey, we have these curiosities about fluency, about phonics, about vocabulary, virtual engagement writing. We don’t know the answers, but we know you do, so would you partner with us and help us figure out the answer?” And I think that was a really important way to approach it, because it was authentically how we felt.
As former teachers, we were in a place where we’re like, “They know the answer.” If something isn’t working, they could probably tell us why it’s not working. And so what if we were to invite them into this improvement journey as opposed to them kind of being bystanders who sometimes passively gives feedback? And I think approaching it that way, and we’ve heard this from our teachers we’ve worked with, that that adds a level of wanting to be a part of it because you’re looking at me as the professional that I am with the expertise that I bring. And so, we didn’t have a hard time finding teachers. It also didn’t hurt that we were giving a stipend, which should pay people for what they are worth.

Alec Patton:
That’s huge. Yeah. One thing about teachers is that teachers have been conditioned to get very nervous when the word data is used. And here you are coming in talking a lot about data and sometimes it’s like, “Hey, talking to the kids is data.” But it’s also looking at the kids’ test scores as data, and that’s literally one of the things that you’re looking at here in those blanks on that assessment. How did you approach that?

Amiee Winchester:
One of the most important things that we started saying from the beginning was that, “Data’s a flashlight and not a hammer.” One of the things that I learned early on from one of our longtime partners, Jared Bolte, was that there’s a difference between data for improvement and data for accountability. And I think internalizing that was really important because when you look at data for improvement, you ask different questions and there isn’t judgment. It’s literally there for you to learn and have curiosity and to propel you on your journey for learning.
And accountability is what we’re all conditioned to do in education, because that is how we operate, that is the way the system is currently designed. And so I think one of the things that I’ve seen is teachers kind of going on this journey of starting with skepticism, rightfully so, to realizing that this data is honestly for us to be able to learn together. And so I think that was the biggest thing that I had seen and experienced with teachers.

Zack Jaffe:
And really thinking about what types of data they were collecting. At the time, being part of this virtual learning group, we’re asking the teachers, “So what do you think would be a good measure of how your students are engaging in the work?” “Oh, well maybe it’s how many times they participate in the chat in the Zoom.” “Okay, great. That’s the data you’re going to collect.” And I think when people see that it’s so close to the work, that we’re not looking at these accountability measures that only happen once a year or something like that, but they really see it’s part of their day-to-day processes and they had a hand in selecting what that data would be, that helped to have some buy-in too. Because then it was like, “When we come back in two weeks, you’ll have something to share and it’ll be a little bit of learning that we can all benefit from.”

Amiee Winchester:
Yeah. And also, I think one of the most critical things in improvement, especially when you’re engaging voices that maybe historically don’t have a voice or might be the ones who are blamed when something doesn’t go right, is that we came in with measures we cared about. So it’s like, “Hey, we want to see if students oral reading fluency increases. We want to see what happens on a writing assignment. We want to see what happens with turn-in rates for virtual assignments.”
And by doing that, it is not about what that teacher, what their outcomes are in their classroom, it’s about them contributing to what we know about this particular measure that we’re interested in. And so, I think those practical measures and being very clear about what those are helps to build some of that trust. That it’s not about us looking into what you’re doing, but it’s about you adding to the conversation of what we’re trying to understand.

Alec Patton:
Yeah. One of the difficult things in a school system is that you have all these people who are used to using data for accountability. You’ve promised a bunch of people that you’re not using that data for accountability, but then that data is kind of sitting around and, or somebody goes, “Wait, actually looking at this, I actually have some serious concerns about this teacher based on this.” How do you protect the data that’s not for accountability from turning into data, because there’s nothing about that data that makes it innately non-accountability data, it’s how you’re using it.

Amiee Winchester:
I do have to say without it sounding kind of, I don’t know, like cliche maybe, but our leaders, especially our superintendent, my boss, all of them in our pipeline, they were really in support of us trying this out. And so I think that we had the privilege and the opportunity to be a little bit shielded from some things, because we had leaders who created space for us to build this bubble and try out what could be.
I say that because I think oftentimes as a middle leader, at least for me when I came into this system, I had an assumption of how leadership operated. And that assumption was that you wait for your higher level leader, whoever that is, to tell you where you’re going and then you work within the confines of that. And what I appreciate about my leaders looking back on it, and that’s something I had to learn, is that they created the space and said, “Go forth and build something and have a vision for what it can be.” And also, we didn’t have expectations other than a grant, right? So we had the ability to kind of do things that might be counter to how they would normally be done.

Zack Jaffe:
There’s a flip side to that too, which is that if your data looks good but you’ve been operating in a bubble, how do we know that we can generalize from this?

Amiee Winchester:
Yeah.

Zack Jaffe:
You’ve got this little boutique group of probably really committed, enthusiastic teachers, now we don’t necessarily trust that the data you’re finding-

Amiee Winchester:
Maybe it’s just the teachers. Yeah.

Zack Jaffe:
Right. Can spread to other areas. So there’s freedom-

Amiee Winchester:
And we experienced that too.

Zack Jaffe:
Yeah. There’s freedom within that bubble to explore, but then there’s often skepticism about, “Okay, now you want to break out of the bubble? I’m not sure that’s going to work.”

Alec Patton:
Was there a point that you remember where you thought, “Oh wow, this is working.”

Amiee Winchester:
Yeah. I mean, I feel like I had a couple of those. Obviously when we started the fellowship and a couple months in and we do these huddles, learning huddles that happen every few weeks. And as the teachers were engaging and we were learning, and I think there’s a moment where you’re like, “Wow, this must be what it feels like when you’re doing improvement in kind of a human centered way.” Not just throwing out these tools, but you’re doing something that seems more transcendent than that. I think the moment where I was like, “Holy cow, I actually think we moved something in the system.” That was about a year and a half in and we were, for fluency, looking at oral reading fluency, which is the number of words correct per minute a student can read and how accurate they are reading in hopes of helping them with their comprehension.
And we were looking at this chart, and I’ll never forget it because it was looking at sixth, seventh, and eighth grade. And there are national norms, so we were able to look against those, and we were looking at two lines. And the first line was a line of what I would expect students to do if they started out in this place, which is where our students started. And then the other line was where our kids actually were. And it was just twice as big as we thought it would be. And I remember texting my old boss, Sarah, and I was like, “We did it. We did it. This is the manifestation of your vision and all of those that came before us that wanted to see things differently.” And so I think that was the moment where I was like, “This is something special,” and I felt really honored to be a part of it.

Alec Patton:
That’s awesome.

Zack Jaffe:
I think the moment for me has to do more with the teacher excitement and the internalization. It was when we were sort of in our last year of doing this fluency focus fellowship, when one of the teachers we’d been working with for a while said, “Okay, I want to start one of these up at my school.” And to have her come to us and say, “I am ready to transition from being someone who just does this and experiments to leading the work at my school and getting my Principal involved.” That was the moment for me where it was like, we are seeing success in making teachers feel like professionals and doing real learning that they feel is valuable enough to spread to other people at their school.

Alec Patton:
All right. To be continued. Zack, Amiee, thank you so much.

Zack Jaffe:
Thank you.

Amiee Winchester:
Thank you.

Alec Patton:
High Tech High Unboxed is hosted and edited by me, Alec Patton. Our theme music is by Brother Herschel. Huge thanks to Amiee and Zack for this conversation. As promised, there will be a sequel to this episode. Look out for it. Thanks for listening.