Ask Matt: Episode 4
In this episode “Assessment and Planning”, we talk to Matt about assessment—formative, summative, performance and other types. Matt suggests that it doesn’t matter what you call the stuff, it's how it's used by teachers that is important. We also have a very good discussion about the science and engineering practice—planning and carrying out investigations, and how the planning part is so important.
Listen to Episode 4 of ‘Ask Matt’ on Apple Podcasts, Google Podcasts, Spotify, or below:
The transcript of the episode is also provided below.
[Intro music]
Eugene: Hello and welcome to ‘Ask Matt’. I’m Eugene Cordero, professor of meteorology and climate science at San Jose State, and Founder and Director of Green Ninja. Green Ninja is an educational initiative where students use science and engineering to solve real world problems.
I’m here with Matt d’Alessio, geology professor from Cal State Northridge and chief author of the 2016 California Science Framework. Matt has dedicated over 20 years of his life to science education and is a national expert on how to make science learning more effective and engaging. Matt was a high school science teacher, runs a sustainability education program at an elementary school, and spent a year as a stay-at-home dad. Matt cares deeply about the environment and he’s also recently received a Distinguished Teaching Award from Cal State Northridge.
I met Matt about three years ago and he agreed to help our team at Green Ninja with advice as we created our own NGSS curriculum. Today we are an approved science publisher in California, in large part because of Matt’s guidance and advice. We all learned a lot from our work with Matt, and I thought it would be great to share some of his wisdom and insights with others. So here we are.
The format is I ask Matt about a range of subjects from NGSS and science teaching to professional training and science content. And hopefully, we’ll learn some more about how to make this transition to NGSS easier and more rewarding for both teachers and students.
If you have any of your own questions, just send them to info@greenninja.org and I’ll share some of them with Matt in a future episode. So let’s get started. Thanks for joining us today, Matt.
Matt: Hey, good to be here again!
Eugene: So before we get started, do you have any special news to share with our listeners today about your wife’s recent participation in the special election for a seat on the city council?
Matt: We had an election here and it was a very exciting night a couple weeks ago on Tuesday. Lorraine was in second place for a while, and then after all the ballots came in, we eventually find out that she finished as the top finisher in the primary election, and so she’s moving on to the runoff!
Eugene: Wow, that’s amazing. Congratulations. So what’s the next step right now?
Matt: We are preparing to get her message out about a cleaner, more sustainable Los Angeles to the entire community for the election on August 13, which is going to be kind of in the middle of the summer. There’s nothing happening, except for it and it’s gonna be hard to get people out, but we hope that we can energize people because it’s a very important race.
Eugene: Wow, anyhow, that’s really amazing. Congratulations to her and to you and your whole family and all the folks supporting that work. Does she have any special position on education?
Matt: Well, the city council in Los Angeles doesn’t actually have much purview over education. She’s a very strong supporter of public education and has been working with our teachers, trying to support our teachers as they’ve struck recently in Los Angeles, and trying to get fair contracts and things like that. But, unfortunately, as a city council member, all she can do is create a city climate that is supportive of education, but has no actual authority over our public school system. So, no specific positions, but you can ask her about it.
Eugene: [laughs] Okay, maybe I will next time. So let’s turn our attention to education and NGSS, and I’d like to start off with the focus on assessment. So, what do you think about assessment and what do the California Framework say about how to assess science learning?
Matt: So, the purpose of assessment is to get information about how good a job we did at teaching things. The framework tells us, really, that we need to be using the results of assessment or there’s no point in doing assessment. And so, how do we use them? We have to have both the tools to interpret the assessment results, and we also need to take the time and effort to do that. And finally, when once we actually have looked at the assessment, we actually need to use those to inform and change our teaching.
And so, it’s really about looking at how well you are doing. You’re assessing yourself as much as you’re assessing the students and how you’re meeting each student’s needs. The only thing that gets that individual results from individual gets you is information about how well you’re reaching each of those different students and how you’re going to need to tailor your instructions to help those individual students. But from our perspective and writing the framework, we were really very teacher centered about this and saying, “These results are for you to use. For assessment of all types, you need to be actually taking advantage of that opportunity and using that to change your instruction in the future.”
Eugene: So, what are the different types of tools? I mean, we hear about formative and summative assessments. You know, are those kind of the primary tools that teachers are going to be using?
Matt: Well, if you remember from education literature and education classes, there’s this word called ‘formative assessment,’ where you’re sort of in the middle of your teaching, and ‘summative assessment’ is supposed to be at the end where you, at the end, sort of check to see how well you did in the end. And in an original draft of the framework, I actually had a heading labeled, “Formative versus Summative Assessment: A False Distinction,” and I talked--
Eugene: Oh, alright.
Matt: I talked in that about how, remember our purpose is to use the results of assessment, and the only difference between what is traditionally called formative assessment and summative assessment is the time scale over which you’re making the changes. So, if you are doing something that’s traditionally called a formative assessment in your classroom-- you ask something and your students reveal some sort of misconception. Then in the next five minutes, you might be modifying your teaching. If you give an end of unit test at the end of the unit, that might inform something about how well you did, so that next time you teach this particular unit, you might need to make some modifications on that time scale. And then at the end of the year, you get some picture from some statewide assessment and that gives you a big broad picture of how well your entire middle school program is doing at helping teach the three dimensions.
And so, you’re going to be making changes at a different scale. So it’s really our crosscutting concept of scale here. These different assessments are used on different time scales, but always the thread that ties them between is if you’re not using it to change what you do in your classroom for next time or for the next minute or for the next week, then there’s no point in actually performing that assessment task. It’s just a waste of time.
Eugene: So, it’s not the idea that you would assess your students, send the data to the students and maybe the parents, and then never kind of reflect on that.
Matt: That’s right. We need to make sure that we’re closing the loop is what it’s called in the assessment literature and that involves making changes to our instruction to improve. And many teachers do this intuitively and naturally on a minute by minute basis and we are trying to support that and give them some more formal structures to actually capture that formative assessment data so they can go even further with it.
Eugene: And in the framework, we see the notion of performance assessments, and is that different than traditional assessments?
Matt: Yeah. So, what you actually-- how you actually collect that information about how your students are doing and what they’ve gotten and not gotten and to what degree they’ve gotten things, that it should be thought of perhaps a little bit differently. Remember, our purpose in this NGSS is to help our students to think like scientists, to act like scientists, to know things like scientists do. And so it’s more authentic than we did in the past, which was sort of classroom learning, which is assessed by a classroom instrument like a test.
We want something more authentic and oftentimes that involves some sort of a performance task the students are doing where they’re actually creating something that is more authentic. They’re actually doing an investigation of something they’ve never seen before. And in the process of doing that investigation and analyzing their results, they’re actually applying the knowledge that they gained in the teaching that you did in the last couple of weeks, and we can actually see how well they do it, how well they’re actually able to put it into practice. And that’s the type of information we want. We don’t care whether they can spit it back on a test. We care, can they put it into practice? And that’s what performance task gets us.
Eugene: And how do teachers best assess performance activity?
Matt: Well, a lot of the time, we end up with these rubrics and the NRC actually put out a document called something like, “Assessing Three Dimensional Learning,” and it’s a very nice document. We use that to help write the assessment chapter of our framework because it’s really insightful. And one of the things that it emphasizes is that-- and I said it just a few moments ago-- that it’s not getting or not getting things. There’s actually a progression of getting things and you can see that within the NGSS from K to 12.
And so, we try and give a rubric to say how far along this progression are they? Do they understand the idea that matter is conserved? Yes or no? And then a more detailed thing is, can they actually track that matter and show you exactly where it’s gone? That’s a more sophisticated level than just knowing that matter should be conserved, but to actually follow us the next level up.
And so, we try and have these rubrics that help place people along this continuum of how much do they get it for every single disciplinary core idea, for every single science and engineering practice. Are they asking questions at the elementary school level or the middle school level or the high school level or the college level? Those are different level questions and we should be sort of scoring our students based upon where they are on that progression.
Eugene: You know, we’ve been doing a bit of professional training with teachers who are going to be piloting Green Ninja in the upcoming academic year, and I do get this question a lot from teachers who give an end of unit assessment, basically like an end of chapter quiz or exam. And how do you help when-- if a teacher says, “Oh, if there isn’t one of those, I feel really uncomfortable or the parents are going to, you know, might give us a hard time.” Do you have any suggestions for how to help teachers kind of think about, you know, how a performance assessment or we’re trying to assess where the students are thinking like scientists and how that might be a little bit different than the traditional multiple-choice and the chapter kind of instrument that had been used previously?
Matt: Yeah. Well, definitely one of the things that I see as a pitfall in our performance task that some of the teachers put together when they’re first starting off NGSS is they’re going to give this a try. If I convince them, you’re gonna have this culminating activity and they’re going to make a poster about things, and the scoring needs to be three dimensional as well. So, a lot of times they’ll sort of score based upon how the poster looks or something like that. But you actually want to be checking to see are there specific content, disciplinary core ideas that are properly conveyed in that poster and making sure that we identify that. And if it’s missing, that means that they didn’t-- they either don’t have it or they just didn’t know to put it on the posters.
This is one of the reasons why we-- during this performance task, we’re circulating around the room and looking at things and checking to see that the fact that it’s not on the poster is not-- is just an omission-- I guess, trying to distinguish between ‘do you not know it?’ or ‘do you just not put it on the poster because you didn’t get asked about it?’ And so we can give them that feedback as we’re walking around, so we can make sure that’s on there. And having those rubrics ahead of time saying, “I’m looking for this, this, this, and this,”--
Eugene: Mhm.
Matt: --can be really helpful. And when I say this, this, this, and this, what are the this is? Those are often revealed in our evidence statements that Achieve has put out for each NGSS performance expectation.
So we actually have a list of all the things that are core ideas that the student needs to have conveyed on that poster and those come to us through Achieve’s website. So that’s a nice tool to use as you’re trying to create those rubrics to really make sure that you’re not just having kids make a cute poster and having fun. Yeah, they’re gonna love doing it because it’s artistic--
Eugene: Mhm.
Matt: --it’s fun, but it needs to be rigorous, and so we need to actually be scoring all three of the dimensions.
Eugene: Now, I have obviously looked at those evidence statements, Matt, and I find them a little scary to be honest, that there’s a lot of stuff in there like-- you know, you’ll have the standard, the performance expectation, and then you have the different practices and crosscutting concepts and core ideas. And when you look at the evidence-- when you click on the evidence statements, there could be a lot more content in there.
Matt: There can be quite a bit. I mean, some of these performance expectations are, you know, covered in several weeks of instruction and so it’s like one sentence in the PE, but it’s really representing weeks worth. And I give the classic example from Earth and space science. I think it’s ESS2-1 in both high school and middle school, and it’s basically, “Understand everything about how the world works inside and out, and how--”
Eugene: [laughs]
Matt: “--it was constructed and [inaudible].” And I call it the Earth science performance expectation. It really is intended to encompass most [laughs] of what I consider Earth science.
Eugene: [laughs]
Matt: And that’s all in one PE. So, I hear you and I understand that, but remember, we’re sometimes only assessing a single PE after weeks worth of instruction--
Eugene: Yeah, yeah.
Matt: --to put in that context.
Eugene: And I know we talked about the California science test before, but it does come up in with talking to teachers a lot about, you know, is this approach aligned with the statewide testing? And you’ve-- I know you haven’t looked at it and haven’t been involved in the development of any of that testing, but in your experience, what would you say to a teacher?
Matt: Yeah. Well, the thing that the test does well is that it does include a little bit of context from these new problems that people have never seen before and that it probes that one little part of the-- if you were to imagine a performance task that you were doing in real life that might take a couple days, it has one little piece of that that the students are working on, and I think it’s-- considering the constraint of trying to assess in a matter of two hours, the entire middle school curriculum, I think it does a pretty neat job of doing that in many cases.
It’s a challenge and so we can actually take CAST-like items, and we can have our students practice some of those either as a-- you can have those as a test, but more usefully, we can put those in the middle of our units and use those sorts of things to-- as a formative assessment, sort of jump off and say, “Here’s this challenging question. Let’s see how we would address it.” And then, use that to go from there and actually follow it with more instruction, and I think they-- I find this in my own classroom. A lot of my best sort of items for assessment are much better as formative assessments because they’re-- the discussion that they generate. If they’re really good questions, they should generate like a discussion. You should be able to debate! Is it this one or this one? And there’s a little bit of truth in each of those multiple-choice responses. And we want to be able to have those discussions and you can’t do that on the actual test, unfortunately.
Eugene: Yeah, that’s right. And I think that-- I mean, I think that one of the hopes in NGSS is that, you know, students are going to be able to figure things out, and we’re giving them the tools, some experience, and practice in doing that. And it seemed like in looking at some of the CAST questions that students are given scenarios or situations or, like you said, a context and then they have to work with that, and they might not have studied that particular topic before that scenario. But hopefully, they’ve been gaining practice in figuring out, like, how to look at these two different graphs or whatever it is, and that by giving, you know, students practice and opportunities doing that, that will reflect hopefully well in the exam.
Matt: Yeah.
Eugene: We’d like to talk about shifts in instruction, and during the last couple of episodes, we talked about flipping the classroom upside down, which I thought was a nice way to think about how we think about instruction. I’d like to continue that, but now focusing on the idea of failure. And what I mean by that is, what happens when you’re a teacher and you run a demo and it doesn’t work? Or what if you have a student who’s creating something that designs an engineer design thing that it doesn’t do what it’s supposed to do? Is that a good thing?
Matt: Well, I’m actually running a summer institute right now in the other room and our teachers are asking that exact same question. It’s a very-- they’re really worried about their students giving up because they’re failing and they sort of turn off at some point when the failure-- when they get too much failure, if you will. And so we really have to be working very vigorously on changing our classroom culture and climate, and I think the inroad to doing this that’s both understandable for teachers and for students is with engineering design.
We talked about this a lot and our teachers seem to be very comfortable teaching it in an engineering design context because we often talk about the engineering design processes as sort of planning things, constructing them, and then the third pillar is revising. And, you know, the revision is a huge part of that engineering design process and people are very comfortable with that. Of course, it’s not perfect on the first try, but I want to bring everyone’s attention to the fact that, that revision process is actually in almost all of our practices that we have.
So, when we’re talking about developing and using models, we often have our students come up with an initial model, and then come back to it and revise it later, and that’s a huge part of it. And so, just creating an idea of instead of failure, it’s talking about we had a certain level of understanding and now we’re revising it now that we know more. And we really want to try and bring that into our explanations. We’re going to construct explanations, but we have first drafts and then we go from there. We want to go further with them and get them better. And so, talking about getting better, improving, revising, even our questions.
When I asked students to ask questions, we start with initial questions that are sort of good starting point questions, but then we refine our questions and make more detailed ones. And so, that’s the kind of thinking that I want teachers to get across to their students is, is that it’s all part of this progression from kindergarten through high school and you’re going to be making a lot of revisions along the way, and that’s how science works. And we’re going to have to get used to when things don’t work right or when we can’t, you know, when we didn’t find the answer the first time, we’re going to have to revise our thinking.
Eugene: And kind of following up on that-- and that’s, like, I think that’s a great point and I’m interested that your teachers are asking you very similar questions. As NGSS-- and if you go into the integrated model, you might be teaching subjects you haven’t taught before, for example, Earth science-- what happens when a student asks you a question that you don’t know the answer to?
Matt: This should happen all the time, if you’re doing NGSS right, in the sense that we are-- should be probing some really interesting and complicated problems that we don’t necessarily know the answer to. I think we talked about this in a previous episode where we were looking at a graph showing sea ice and it had some crazy ups and downs, and I asked you, “What’s causing those?” and you said, “I’m not entirely sure. It could be this or this, but I don’t know!” And that’s exactly the type of thing that we want our teachers to get into that experience of having them asking questions.
And so, I talked about the research on curiosity, and when a teacher expresses curiosity or asks questions about things and acts as the chief explorer in their classroom, that rubs off. The research shows that our students ask better questions when we are examples of explorers. And so, that’s the only thing I got to do is you got to be brave, just like the, you know, the great explorers of the earlier times. You are an explorer in the NGSS world and you just gotta go out there and say, “I’m going to give this a try and I want to learn,” because most of us became science teachers because we love learning new things!
Eugene: Mhm. Yep.
Matt: Sometimes, that means learning it right there with your students.
Eugene: That’s great. We do like to talk about the SEPs and CCCs -- science and engineering practices and crosscutting concepts -- and one of the SEPs is planning and carrying out investigations. So what’s that about?
Matt: Well, many of our teachers have been having their students do stuff hands-on and conducting investigations, but the big shift here I feel like from NGSS is the planning, the addition of the planning part where-- when I was teaching high school, I used to have these really, really detailed step by step instructions. They were pages worth of step by step instructions because I felt like my students needed that, and in some senses, they did need support. But I think that in retrospect, I was doing them a little bit of a disservice. They got used to that and they used it as a crutch, and then they couldn’t actually think about what they were doing and why they were doing it.
So, we kind of need to look at things from a different perspective here and having them more involved in that planning process, and that’s hard to do, like I said, because our students do often come to us, expecting--
Eugene: Mhm.
Matt: --intricate detailed step by step instructions, and if a step is missing, they don’t know what to do. So, how do we get them from here to there? It involves baby steps of involving them in little pieces of the process. It involves starting off maybe with giving them those step by step instructions and asking them to make sure that they can explain, “Why did you do this step? Why did you do this step?” And basically, provide a justification for every step that’s there that we can first start off doing together, and then later in the year, we might leave out a step and leave a space and say, “You’re going to need to figure out to get from here to here. There’s something that you need to do. What is it?” And then, you know, the next stage is to say, “Here are the materials that I’m going to give you, and you know, you’re ready. I think you can put these together and come up with a sequence.” And then the last stage is, “Hey, I got this question. What do you need to solve it?” And--
Eugene: Mhm.
Matt: --you know, this is the progression that goes from the beginning of the year to the end of the year. It goes from, you know, elementary school to high school. And so, we’re not having our students design PhD dissertation level research until their PhD.
Eugene: [laughs]
Matt: But, we want to get little steps along the way at every one of the grade levels.
Eugene: Yeah, that’s-- I remember early on developing an experiment probably, like, similar to your high school experience and I showed it to my colleague who’s a science learning specialist and she said, “Oh, I think it’s great. I would just take out all the instructions for how to do the lab.” And it was a really new notion for me that, and I think your idea of scaffolding this over time. Of course, we can’t expect your students immediately to be ready to be at that last step, but being mindful of that. And that’s, you know, this is why having that planning and carrying out investigations SEP is good because it makes us think, “Oh, this is part of a practice.”
Matt: Yeah, and I can actually give you a story from my college students, which is a little bit different world than our middle school students. But I used to have to do a food color imagery lab at the beginning of that class. On the very first day, we actually burned nuts and we might have even talked about that--
Eugene: Yeah, mhm.
Matt: --in a previous episode. But I used to have those intricate directions. It was in a Google form and they had to, you know, basically put in numbers of after each step, so we could check to see what was going on. And then a couple years ago, following my own NGSS advice--
Eugene: [laughs]
Matt: --I took those out, and I started off with basically no directions and had them working on this, and we would basically have just-- it took much longer to do, I’m going to be very honest with you. It took much longer to do, but we had people doing thing-- I let them do it wrong a whole bunch of times. And then we discover-- you know, somebody did it a little bit less wrong than somebody else and I would have different groups working and I’d have everybody, “Everybody! Hey everybody! Come over here and look at this on! Look at how this one’s different from yours.” And we basically were able to come to the correct procedure or-- not correct procedure, but a procedure that worked by doing it wrong a bunch of times and finally getting some usable results that we could have.
So that was a very exciting transformation for me to see that my students actually could do it and, you know, admittedly college students, but I did it cold turkey for them. That was day one, they’ve never met me before they walked into the classroom, and I said, “Just do this.” And, you know, they looked at me crazy at first. But over the course of several days, we were actually able to get them to do it.
Eugene: And that’s a great segue into this next question I had for you, which is about time. And, you know, there’s 20 performance expectations for each grade in middle school and, you know, as a curriculum publisher, we have to produce 180 days of instruction. We share that with teachers, they say, “Oh, I don’t have 180 days. How am I going to get through all this material?” And then this example that you just gave us, Matt. You said, “Oh, it takes a lot longer.” Like, it’d be a lot quicker if you just gave them the instructions. And so, from a teacher standpoint, what do you say to a teacher who’s saying, “I’m trying to get through the standards for my students, and when I start removing the instructions, everything takes longer”?
Matt: Yeah, and I hear that. And the first reply is yes, there’s no good solution to this. The answer is we need more time, but that’s obviously not practical. So what do I say is that despite the fact that we have this mantra of all students do all standards, we also have the mantra that depth is better than breath, and just touching superficially along the concept is not good preparation for the next level. So, we need to be thinking about making sure that we are providing the full breadth of the standards on all three dimensions, and sometimes that means spending a lot of time on, in this case, a science and engineering practice, planning an investigation, and going deep into that because we weren’t covering that very well before, if you will. We weren’t getting the full breadth of that part of the three dimensions. And it comes with the sacrifice of-- you know, I stopped covering… what did I cut out? I think I cut out sound from my class.
Eugene: Oh my gosh.
Matt: I know and it’s-- I cut out sound. We didn’t-- we don’t get to do it anymore and because we get some of the things that build up to it about our little particles and moving around things like that, but I don’t actually do the same things that I used to on sound. And it turns out actually those topics are not actually included in a lot of the NGSS. It’s one of the reasons why I felt comfortable is that they actually-- even though they’re 20 performance expectations, there actually are fewer disciplinary core ideas than there used to be in terms of science. So I feel like I make that choice to go deep on certain things. The only thing we want to make sure teachers are being a little bit mindful of is that they’re not being systematic in their bias to cut out--
Eugene: --the topics that they don’t like. [laughs]
Matt: The topics that they don’t like, or even certain populations being deprived of certain things. Like, we see this in high school that students are traditionally lower performers are losing physics. They just don’t ever get taught physics, and that is coming at the loss to our physics and electrical engineering and all of those fields that are cutting edge, are terribly underrepresented in terms of minorities and people from low income backgrounds, and it’s because we’ve been systematically depriving those people.
So you have to be a little bit careful about what you’re choosing to cut and why you’re choosing to cut it. But there’s definitely the understanding that you’re going to need to go deep on certain things and that’s going to come at a price.
Eugene: Yeah, and I do hear a lot of that from teachers but that’s, I think, a great way to think about it.
I think the last thing we’ll chat about today is about burritos. And early on, Matt and I discovered that one of the things we had in common is a love of burritos. And it may be surprising to some, but the subject of burritos isn’t so far away from science education and the environment. So let’s talk about burritos. Matt, some people say that burritos are often too big. Is that true and what can we do about that?
Matt and Eugene: [laughs]
Eugene: I love asking you any kind of question that I have, Matt.
Matt: I’m picturing-- actually one of the reasons I’m laughing, I’m picturing my daughter who’s now nine years old getting this monstrous thing, you know, delivered in front of her, and of course, her consuming the whole thing.
Matt and Eugene: [laughs]
Matt: You know, it’s too big for an average adult, but she’s downing at all. But I don’t know if they’re too big and I-- when you ask that, one of the things that first pops into my head is when we eat so much at once, what does her body do with it? Does it like realize, “Oh my god, there’s too much here,” and just basically not metabolize a bunch of it and dump it out and it goes out in our waste, or do we actually, you know, break all that stuff down and find a place to put it? And how does that work? And I have no idea. The thing that comes into my head as I think about, I think wolves.
Eugene: [laughs]
Matt: Don’t wolves eat-- they eat a whole bunch, they gorge and down a whole bunch of--
Eugene: Yeah. So do lions. Big cats do that.
Matt: Yes! So it’s possible that this is what it’s-- our body takes care of this and figures it out. But from a more practical perspective, I think we eat until we are full and maybe if you get one of those monstrous burritos, I think my basic plan -- and sometimes I do this -- is I cut it in half, put one part aside and save it for next time. Save it for lunchtime.
Eugene: And that sounds a reasonable idea as well. Well, I think that’s a good place to stop. Thanks for joining us at ‘Ask Matt’, where we explore NGSS, science education, and the environment with education expert and super nice guy, Matthew d’Alessio. Thanks for joining us today, Matt, and we’ll see you next time.
Matt: Bye bye, Eugene! Thank you!
[Outro music]
Eugene: Hello and welcome to ‘Ask Matt’. I’m Eugene Cordero, professor of meteorology and climate science at San Jose State, and Founder and Director of Green Ninja. Green Ninja is an educational initiative where students use science and engineering to solve real world problems.
I’m here with Matt d’Alessio, geology professor from Cal State Northridge and chief author of the 2016 California Science Framework. Matt has dedicated over 20 years of his life to science education and is a national expert on how to make science learning more effective and engaging. Matt was a high school science teacher, runs a sustainability education program at an elementary school, and spent a year as a stay-at-home dad. Matt cares deeply about the environment and he’s also recently received a Distinguished Teaching Award from Cal State Northridge.
I met Matt about three years ago and he agreed to help our team at Green Ninja with advice as we created our own NGSS curriculum. Today we are an approved science publisher in California, in large part because of Matt’s guidance and advice. We all learned a lot from our work with Matt, and I thought it would be great to share some of his wisdom and insights with others. So here we are.
The format is I ask Matt about a range of subjects from NGSS and science teaching to professional training and science content. And hopefully, we’ll learn some more about how to make this transition to NGSS easier and more rewarding for both teachers and students.
If you have any of your own questions, just send them to info@greenninja.org and I’ll share some of them with Matt in a future episode. So let’s get started. Thanks for joining us today, Matt.
Matt: Hey, good to be here again!
Eugene: So before we get started, do you have any special news to share with our listeners today about your wife’s recent participation in the special election for a seat on the city council?
Matt: We had an election here and it was a very exciting night a couple weeks ago on Tuesday. Lorraine was in second place for a while, and then after all the ballots came in, we eventually find out that she finished as the top finisher in the primary election, and so she’s moving on to the runoff!
Eugene: Wow, that’s amazing. Congratulations. So what’s the next step right now?
Matt: We are preparing to get her message out about a cleaner, more sustainable Los Angeles to the entire community for the election on August 13, which is going to be kind of in the middle of the summer. There’s nothing happening, except for it and it’s gonna be hard to get people out, but we hope that we can energize people because it’s a very important race.
Eugene: Wow, anyhow, that’s really amazing. Congratulations to her and to you and your whole family and all the folks supporting that work. Does she have any special position on education?
Matt: Well, the city council in Los Angeles doesn’t actually have much purview over education. She’s a very strong supporter of public education and has been working with our teachers, trying to support our teachers as they’ve struck recently in Los Angeles, and trying to get fair contracts and things like that. But, unfortunately, as a city council member, all she can do is create a city climate that is supportive of education, but has no actual authority over our public school system. So, no specific positions, but you can ask her about it.
Eugene: [laughs] Okay, maybe I will next time. So let’s turn our attention to education and NGSS, and I’d like to start off with the focus on assessment. So, what do you think about assessment and what do the California Framework say about how to assess science learning?
Matt: So, the purpose of assessment is to get information about how good a job we did at teaching things. The framework tells us, really, that we need to be using the results of assessment or there’s no point in doing assessment. And so, how do we use them? We have to have both the tools to interpret the assessment results, and we also need to take the time and effort to do that. And finally, when once we actually have looked at the assessment, we actually need to use those to inform and change our teaching.
And so, it’s really about looking at how well you are doing. You’re assessing yourself as much as you’re assessing the students and how you’re meeting each student’s needs. The only thing that gets that individual results from individual gets you is information about how well you’re reaching each of those different students and how you’re going to need to tailor your instructions to help those individual students. But from our perspective and writing the framework, we were really very teacher centered about this and saying, “These results are for you to use. For assessment of all types, you need to be actually taking advantage of that opportunity and using that to change your instruction in the future.”
Eugene: So, what are the different types of tools? I mean, we hear about formative and summative assessments. You know, are those kind of the primary tools that teachers are going to be using?
Matt: Well, if you remember from education literature and education classes, there’s this word called ‘formative assessment,’ where you’re sort of in the middle of your teaching, and ‘summative assessment’ is supposed to be at the end where you, at the end, sort of check to see how well you did in the end. And in an original draft of the framework, I actually had a heading labeled, “Formative versus Summative Assessment: A False Distinction,” and I talked--
Eugene: Oh, alright.
Matt: I talked in that about how, remember our purpose is to use the results of assessment, and the only difference between what is traditionally called formative assessment and summative assessment is the time scale over which you’re making the changes. So, if you are doing something that’s traditionally called a formative assessment in your classroom-- you ask something and your students reveal some sort of misconception. Then in the next five minutes, you might be modifying your teaching. If you give an end of unit test at the end of the unit, that might inform something about how well you did, so that next time you teach this particular unit, you might need to make some modifications on that time scale. And then at the end of the year, you get some picture from some statewide assessment and that gives you a big broad picture of how well your entire middle school program is doing at helping teach the three dimensions.
And so, you’re going to be making changes at a different scale. So it’s really our crosscutting concept of scale here. These different assessments are used on different time scales, but always the thread that ties them between is if you’re not using it to change what you do in your classroom for next time or for the next minute or for the next week, then there’s no point in actually performing that assessment task. It’s just a waste of time.
Eugene: So, it’s not the idea that you would assess your students, send the data to the students and maybe the parents, and then never kind of reflect on that.
Matt: That’s right. We need to make sure that we’re closing the loop is what it’s called in the assessment literature and that involves making changes to our instruction to improve. And many teachers do this intuitively and naturally on a minute by minute basis and we are trying to support that and give them some more formal structures to actually capture that formative assessment data so they can go even further with it.
Eugene: And in the framework, we see the notion of performance assessments, and is that different than traditional assessments?
Matt: Yeah. So, what you actually-- how you actually collect that information about how your students are doing and what they’ve gotten and not gotten and to what degree they’ve gotten things, that it should be thought of perhaps a little bit differently. Remember, our purpose in this NGSS is to help our students to think like scientists, to act like scientists, to know things like scientists do. And so it’s more authentic than we did in the past, which was sort of classroom learning, which is assessed by a classroom instrument like a test.
We want something more authentic and oftentimes that involves some sort of a performance task the students are doing where they’re actually creating something that is more authentic. They’re actually doing an investigation of something they’ve never seen before. And in the process of doing that investigation and analyzing their results, they’re actually applying the knowledge that they gained in the teaching that you did in the last couple of weeks, and we can actually see how well they do it, how well they’re actually able to put it into practice. And that’s the type of information we want. We don’t care whether they can spit it back on a test. We care, can they put it into practice? And that’s what performance task gets us.
Eugene: And how do teachers best assess performance activity?
Matt: Well, a lot of the time, we end up with these rubrics and the NRC actually put out a document called something like, “Assessing Three Dimensional Learning,” and it’s a very nice document. We use that to help write the assessment chapter of our framework because it’s really insightful. And one of the things that it emphasizes is that-- and I said it just a few moments ago-- that it’s not getting or not getting things. There’s actually a progression of getting things and you can see that within the NGSS from K to 12.
And so, we try and give a rubric to say how far along this progression are they? Do they understand the idea that matter is conserved? Yes or no? And then a more detailed thing is, can they actually track that matter and show you exactly where it’s gone? That’s a more sophisticated level than just knowing that matter should be conserved, but to actually follow us the next level up.
And so, we try and have these rubrics that help place people along this continuum of how much do they get it for every single disciplinary core idea, for every single science and engineering practice. Are they asking questions at the elementary school level or the middle school level or the high school level or the college level? Those are different level questions and we should be sort of scoring our students based upon where they are on that progression.
Eugene: You know, we’ve been doing a bit of professional training with teachers who are going to be piloting Green Ninja in the upcoming academic year, and I do get this question a lot from teachers who give an end of unit assessment, basically like an end of chapter quiz or exam. And how do you help when-- if a teacher says, “Oh, if there isn’t one of those, I feel really uncomfortable or the parents are going to, you know, might give us a hard time.” Do you have any suggestions for how to help teachers kind of think about, you know, how a performance assessment or we’re trying to assess where the students are thinking like scientists and how that might be a little bit different than the traditional multiple-choice and the chapter kind of instrument that had been used previously?
Matt: Yeah. Well, definitely one of the things that I see as a pitfall in our performance task that some of the teachers put together when they’re first starting off NGSS is they’re going to give this a try. If I convince them, you’re gonna have this culminating activity and they’re going to make a poster about things, and the scoring needs to be three dimensional as well. So, a lot of times they’ll sort of score based upon how the poster looks or something like that. But you actually want to be checking to see are there specific content, disciplinary core ideas that are properly conveyed in that poster and making sure that we identify that. And if it’s missing, that means that they didn’t-- they either don’t have it or they just didn’t know to put it on the posters.
This is one of the reasons why we-- during this performance task, we’re circulating around the room and looking at things and checking to see that the fact that it’s not on the poster is not-- is just an omission-- I guess, trying to distinguish between ‘do you not know it?’ or ‘do you just not put it on the poster because you didn’t get asked about it?’ And so we can give them that feedback as we’re walking around, so we can make sure that’s on there. And having those rubrics ahead of time saying, “I’m looking for this, this, this, and this,”--
Eugene: Mhm.
Matt: --can be really helpful. And when I say this, this, this, and this, what are the this is? Those are often revealed in our evidence statements that Achieve has put out for each NGSS performance expectation.
So we actually have a list of all the things that are core ideas that the student needs to have conveyed on that poster and those come to us through Achieve’s website. So that’s a nice tool to use as you’re trying to create those rubrics to really make sure that you’re not just having kids make a cute poster and having fun. Yeah, they’re gonna love doing it because it’s artistic--
Eugene: Mhm.
Matt: --it’s fun, but it needs to be rigorous, and so we need to actually be scoring all three of the dimensions.
Eugene: Now, I have obviously looked at those evidence statements, Matt, and I find them a little scary to be honest, that there’s a lot of stuff in there like-- you know, you’ll have the standard, the performance expectation, and then you have the different practices and crosscutting concepts and core ideas. And when you look at the evidence-- when you click on the evidence statements, there could be a lot more content in there.
Matt: There can be quite a bit. I mean, some of these performance expectations are, you know, covered in several weeks of instruction and so it’s like one sentence in the PE, but it’s really representing weeks worth. And I give the classic example from Earth and space science. I think it’s ESS2-1 in both high school and middle school, and it’s basically, “Understand everything about how the world works inside and out, and how--”
Eugene: [laughs]
Matt: “--it was constructed and [inaudible].” And I call it the Earth science performance expectation. It really is intended to encompass most [laughs] of what I consider Earth science.
Eugene: [laughs]
Matt: And that’s all in one PE. So, I hear you and I understand that, but remember, we’re sometimes only assessing a single PE after weeks worth of instruction--
Eugene: Yeah, yeah.
Matt: --to put in that context.
Eugene: And I know we talked about the California science test before, but it does come up in with talking to teachers a lot about, you know, is this approach aligned with the statewide testing? And you’ve-- I know you haven’t looked at it and haven’t been involved in the development of any of that testing, but in your experience, what would you say to a teacher?
Matt: Yeah. Well, the thing that the test does well is that it does include a little bit of context from these new problems that people have never seen before and that it probes that one little part of the-- if you were to imagine a performance task that you were doing in real life that might take a couple days, it has one little piece of that that the students are working on, and I think it’s-- considering the constraint of trying to assess in a matter of two hours, the entire middle school curriculum, I think it does a pretty neat job of doing that in many cases.
It’s a challenge and so we can actually take CAST-like items, and we can have our students practice some of those either as a-- you can have those as a test, but more usefully, we can put those in the middle of our units and use those sorts of things to-- as a formative assessment, sort of jump off and say, “Here’s this challenging question. Let’s see how we would address it.” And then, use that to go from there and actually follow it with more instruction, and I think they-- I find this in my own classroom. A lot of my best sort of items for assessment are much better as formative assessments because they’re-- the discussion that they generate. If they’re really good questions, they should generate like a discussion. You should be able to debate! Is it this one or this one? And there’s a little bit of truth in each of those multiple-choice responses. And we want to be able to have those discussions and you can’t do that on the actual test, unfortunately.
Eugene: Yeah, that’s right. And I think that-- I mean, I think that one of the hopes in NGSS is that, you know, students are going to be able to figure things out, and we’re giving them the tools, some experience, and practice in doing that. And it seemed like in looking at some of the CAST questions that students are given scenarios or situations or, like you said, a context and then they have to work with that, and they might not have studied that particular topic before that scenario. But hopefully, they’ve been gaining practice in figuring out, like, how to look at these two different graphs or whatever it is, and that by giving, you know, students practice and opportunities doing that, that will reflect hopefully well in the exam.
Matt: Yeah.
Eugene: We’d like to talk about shifts in instruction, and during the last couple of episodes, we talked about flipping the classroom upside down, which I thought was a nice way to think about how we think about instruction. I’d like to continue that, but now focusing on the idea of failure. And what I mean by that is, what happens when you’re a teacher and you run a demo and it doesn’t work? Or what if you have a student who’s creating something that designs an engineer design thing that it doesn’t do what it’s supposed to do? Is that a good thing?
Matt: Well, I’m actually running a summer institute right now in the other room and our teachers are asking that exact same question. It’s a very-- they’re really worried about their students giving up because they’re failing and they sort of turn off at some point when the failure-- when they get too much failure, if you will. And so we really have to be working very vigorously on changing our classroom culture and climate, and I think the inroad to doing this that’s both understandable for teachers and for students is with engineering design.
We talked about this a lot and our teachers seem to be very comfortable teaching it in an engineering design context because we often talk about the engineering design processes as sort of planning things, constructing them, and then the third pillar is revising. And, you know, the revision is a huge part of that engineering design process and people are very comfortable with that. Of course, it’s not perfect on the first try, but I want to bring everyone’s attention to the fact that, that revision process is actually in almost all of our practices that we have.
So, when we’re talking about developing and using models, we often have our students come up with an initial model, and then come back to it and revise it later, and that’s a huge part of it. And so, just creating an idea of instead of failure, it’s talking about we had a certain level of understanding and now we’re revising it now that we know more. And we really want to try and bring that into our explanations. We’re going to construct explanations, but we have first drafts and then we go from there. We want to go further with them and get them better. And so, talking about getting better, improving, revising, even our questions.
When I asked students to ask questions, we start with initial questions that are sort of good starting point questions, but then we refine our questions and make more detailed ones. And so, that’s the kind of thinking that I want teachers to get across to their students is, is that it’s all part of this progression from kindergarten through high school and you’re going to be making a lot of revisions along the way, and that’s how science works. And we’re going to have to get used to when things don’t work right or when we can’t, you know, when we didn’t find the answer the first time, we’re going to have to revise our thinking.
Eugene: And kind of following up on that-- and that’s, like, I think that’s a great point and I’m interested that your teachers are asking you very similar questions. As NGSS-- and if you go into the integrated model, you might be teaching subjects you haven’t taught before, for example, Earth science-- what happens when a student asks you a question that you don’t know the answer to?
Matt: This should happen all the time, if you’re doing NGSS right, in the sense that we are-- should be probing some really interesting and complicated problems that we don’t necessarily know the answer to. I think we talked about this in a previous episode where we were looking at a graph showing sea ice and it had some crazy ups and downs, and I asked you, “What’s causing those?” and you said, “I’m not entirely sure. It could be this or this, but I don’t know!” And that’s exactly the type of thing that we want our teachers to get into that experience of having them asking questions.
And so, I talked about the research on curiosity, and when a teacher expresses curiosity or asks questions about things and acts as the chief explorer in their classroom, that rubs off. The research shows that our students ask better questions when we are examples of explorers. And so, that’s the only thing I got to do is you got to be brave, just like the, you know, the great explorers of the earlier times. You are an explorer in the NGSS world and you just gotta go out there and say, “I’m going to give this a try and I want to learn,” because most of us became science teachers because we love learning new things!
Eugene: Mhm. Yep.
Matt: Sometimes, that means learning it right there with your students.
Eugene: That’s great. We do like to talk about the SEPs and CCCs -- science and engineering practices and crosscutting concepts -- and one of the SEPs is planning and carrying out investigations. So what’s that about?
Matt: Well, many of our teachers have been having their students do stuff hands-on and conducting investigations, but the big shift here I feel like from NGSS is the planning, the addition of the planning part where-- when I was teaching high school, I used to have these really, really detailed step by step instructions. They were pages worth of step by step instructions because I felt like my students needed that, and in some senses, they did need support. But I think that in retrospect, I was doing them a little bit of a disservice. They got used to that and they used it as a crutch, and then they couldn’t actually think about what they were doing and why they were doing it.
So, we kind of need to look at things from a different perspective here and having them more involved in that planning process, and that’s hard to do, like I said, because our students do often come to us, expecting--
Eugene: Mhm.
Matt: --intricate detailed step by step instructions, and if a step is missing, they don’t know what to do. So, how do we get them from here to there? It involves baby steps of involving them in little pieces of the process. It involves starting off maybe with giving them those step by step instructions and asking them to make sure that they can explain, “Why did you do this step? Why did you do this step?” And basically, provide a justification for every step that’s there that we can first start off doing together, and then later in the year, we might leave out a step and leave a space and say, “You’re going to need to figure out to get from here to here. There’s something that you need to do. What is it?” And then, you know, the next stage is to say, “Here are the materials that I’m going to give you, and you know, you’re ready. I think you can put these together and come up with a sequence.” And then the last stage is, “Hey, I got this question. What do you need to solve it?” And--
Eugene: Mhm.
Matt: --you know, this is the progression that goes from the beginning of the year to the end of the year. It goes from, you know, elementary school to high school. And so, we’re not having our students design PhD dissertation level research until their PhD.
Eugene: [laughs]
Matt: But, we want to get little steps along the way at every one of the grade levels.
Eugene: Yeah, that’s-- I remember early on developing an experiment probably, like, similar to your high school experience and I showed it to my colleague who’s a science learning specialist and she said, “Oh, I think it’s great. I would just take out all the instructions for how to do the lab.” And it was a really new notion for me that, and I think your idea of scaffolding this over time. Of course, we can’t expect your students immediately to be ready to be at that last step, but being mindful of that. And that’s, you know, this is why having that planning and carrying out investigations SEP is good because it makes us think, “Oh, this is part of a practice.”
Matt: Yeah, and I can actually give you a story from my college students, which is a little bit different world than our middle school students. But I used to have to do a food color imagery lab at the beginning of that class. On the very first day, we actually burned nuts and we might have even talked about that--
Eugene: Yeah, mhm.
Matt: --in a previous episode. But I used to have those intricate directions. It was in a Google form and they had to, you know, basically put in numbers of after each step, so we could check to see what was going on. And then a couple years ago, following my own NGSS advice--
Eugene: [laughs]
Matt: --I took those out, and I started off with basically no directions and had them working on this, and we would basically have just-- it took much longer to do, I’m going to be very honest with you. It took much longer to do, but we had people doing thing-- I let them do it wrong a whole bunch of times. And then we discover-- you know, somebody did it a little bit less wrong than somebody else and I would have different groups working and I’d have everybody, “Everybody! Hey everybody! Come over here and look at this on! Look at how this one’s different from yours.” And we basically were able to come to the correct procedure or-- not correct procedure, but a procedure that worked by doing it wrong a bunch of times and finally getting some usable results that we could have.
So that was a very exciting transformation for me to see that my students actually could do it and, you know, admittedly college students, but I did it cold turkey for them. That was day one, they’ve never met me before they walked into the classroom, and I said, “Just do this.” And, you know, they looked at me crazy at first. But over the course of several days, we were actually able to get them to do it.
Eugene: And that’s a great segue into this next question I had for you, which is about time. And, you know, there’s 20 performance expectations for each grade in middle school and, you know, as a curriculum publisher, we have to produce 180 days of instruction. We share that with teachers, they say, “Oh, I don’t have 180 days. How am I going to get through all this material?” And then this example that you just gave us, Matt. You said, “Oh, it takes a lot longer.” Like, it’d be a lot quicker if you just gave them the instructions. And so, from a teacher standpoint, what do you say to a teacher who’s saying, “I’m trying to get through the standards for my students, and when I start removing the instructions, everything takes longer”?
Matt: Yeah, and I hear that. And the first reply is yes, there’s no good solution to this. The answer is we need more time, but that’s obviously not practical. So what do I say is that despite the fact that we have this mantra of all students do all standards, we also have the mantra that depth is better than breath, and just touching superficially along the concept is not good preparation for the next level. So, we need to be thinking about making sure that we are providing the full breadth of the standards on all three dimensions, and sometimes that means spending a lot of time on, in this case, a science and engineering practice, planning an investigation, and going deep into that because we weren’t covering that very well before, if you will. We weren’t getting the full breadth of that part of the three dimensions. And it comes with the sacrifice of-- you know, I stopped covering… what did I cut out? I think I cut out sound from my class.
Eugene: Oh my gosh.
Matt: I know and it’s-- I cut out sound. We didn’t-- we don’t get to do it anymore and because we get some of the things that build up to it about our little particles and moving around things like that, but I don’t actually do the same things that I used to on sound. And it turns out actually those topics are not actually included in a lot of the NGSS. It’s one of the reasons why I felt comfortable is that they actually-- even though they’re 20 performance expectations, there actually are fewer disciplinary core ideas than there used to be in terms of science. So I feel like I make that choice to go deep on certain things. The only thing we want to make sure teachers are being a little bit mindful of is that they’re not being systematic in their bias to cut out--
Eugene: --the topics that they don’t like. [laughs]
Matt: The topics that they don’t like, or even certain populations being deprived of certain things. Like, we see this in high school that students are traditionally lower performers are losing physics. They just don’t ever get taught physics, and that is coming at the loss to our physics and electrical engineering and all of those fields that are cutting edge, are terribly underrepresented in terms of minorities and people from low income backgrounds, and it’s because we’ve been systematically depriving those people.
So you have to be a little bit careful about what you’re choosing to cut and why you’re choosing to cut it. But there’s definitely the understanding that you’re going to need to go deep on certain things and that’s going to come at a price.
Eugene: Yeah, and I do hear a lot of that from teachers but that’s, I think, a great way to think about it.
I think the last thing we’ll chat about today is about burritos. And early on, Matt and I discovered that one of the things we had in common is a love of burritos. And it may be surprising to some, but the subject of burritos isn’t so far away from science education and the environment. So let’s talk about burritos. Matt, some people say that burritos are often too big. Is that true and what can we do about that?
Matt and Eugene: [laughs]
Eugene: I love asking you any kind of question that I have, Matt.
Matt: I’m picturing-- actually one of the reasons I’m laughing, I’m picturing my daughter who’s now nine years old getting this monstrous thing, you know, delivered in front of her, and of course, her consuming the whole thing.
Matt and Eugene: [laughs]
Matt: You know, it’s too big for an average adult, but she’s downing at all. But I don’t know if they’re too big and I-- when you ask that, one of the things that first pops into my head is when we eat so much at once, what does her body do with it? Does it like realize, “Oh my god, there’s too much here,” and just basically not metabolize a bunch of it and dump it out and it goes out in our waste, or do we actually, you know, break all that stuff down and find a place to put it? And how does that work? And I have no idea. The thing that comes into my head as I think about, I think wolves.
Eugene: [laughs]
Matt: Don’t wolves eat-- they eat a whole bunch, they gorge and down a whole bunch of--
Eugene: Yeah. So do lions. Big cats do that.
Matt: Yes! So it’s possible that this is what it’s-- our body takes care of this and figures it out. But from a more practical perspective, I think we eat until we are full and maybe if you get one of those monstrous burritos, I think my basic plan -- and sometimes I do this -- is I cut it in half, put one part aside and save it for next time. Save it for lunchtime.
Eugene: And that sounds a reasonable idea as well. Well, I think that’s a good place to stop. Thanks for joining us at ‘Ask Matt’, where we explore NGSS, science education, and the environment with education expert and super nice guy, Matthew d’Alessio. Thanks for joining us today, Matt, and we’ll see you next time.
Matt: Bye bye, Eugene! Thank you!
[Outro music]