Keynote Panel: STEM - Delivering on the Promise

Moderator: Patricia OConnell Johnson
Panelists: Michael Lach, James Shelton

Good afternoon everybody. Can you hear me? Okay. It's a pleasure to be with you again. We're going to try to keep this sort of informal. We'd like this to be a conversation, as well as giving you an update on what's been going on at the Department of Education over the last year. As you know, it's been quite a year for education. And-- And, we'll be able to talk about some of the-- a lot of the work that's going on there. I'm going to give you a brief update on the Department of Education's MSP program. And then, I'm going to-- We're going to set this up by having me pose questions to my colleagues from the Department and have them answer them. And then, after that, we will be opening it up to the floor. So, I hope you all take advantage of this great opportunity to talk to two of the thought leaders at the Department of Education.

First, just a shout-out. How many people are here have a state-- Department of Education State MSP Grant? Two. Okay, look at that. Isn't that great? So, one of the things that we hoped for, when this program was first started, was that there would be that kind of collaboration that would be built at the state level. So, that's great. I know we also have a couple of projects that were originally state MSP projects that now have the larger MSP grants, right? Can you raise your hands if you have one of those? This is our Missouri folks. So, we're also seeing the smaller MSP projects being awarded the larger, more substantial NSF grants, to continue their work. So, I consider that all part of our success.

Real quickly, the state MSP program received its 2010 appropriation, and we got a slight increase. So, that program will be continuing. We have, in our last annual report, we were funding about 600 projects around the country, most of them between three or $400,000 dollars a year, but quite a big range. We provided intensive professional development to over 60,000 teachers. That professional development was the medium number of hours of professional development was about 105 hours in a 12-month period. So, very intensive for each individual teacher. Those teachers taught over two and a half million students. So, we have evidence from the information that we've collected from all of our projects that the large majority of the teachers made statistically significant gains in their knowledge-- as-- as measured statistically, in both math and science. And that the scores, the proficiency scores in math, as we could measure them in those teachers' schools, gained at a greater rate than schools and classrooms without those teachers. So, we have a very interesting set of-- of projects that are called MSP. The projects that you do, which are larger and more comprehensive. But, we also have a lot of work that's going on, that draws on a lot of the work that some of your projects have done, developing their own knowledge. So, there's a lot of work going on around the country to improve professional development for teachers, and helping students make great gains. So, that's it, in a nutshell, that we can talk about more about what's going on with the Department of Education's MSP, if you have questions.

But, I think the more interesting work that you may not know as much about will be described and talked about by my colleagues from the Department. So, I'm going to start by posing a couple of questions. Jim Shelton is the Director of our Office of Innovation. And, as many of you may know but may not-- some of you may not know--Part of the Era funding that we got included about $650 million dollars for innovation-- education innovation. So, I'm going to ask Jim to talk about that program. But also, since most of his career has been about sort of cutting edge of innovation in education, I'm also going to ask him to describe where he thinks what education will look like in ten years, if everything goes as-- as you imagine it should go, with this.

Alrighty. The second half caught me by surprise. Let me-- Let me buy myself some time by giving some context for the Investing in Innovation fund. The Investing in Innovation fund is $650 million dollars in total, and $10 billion dollars in competitive funding that was a part of the Recovery Act. The Recovery Act invested, as many of you know, $100 billion dollars in total education, basically $90 of which flowed out in formula funding over the first two years. And then, there's a bunch of competitive programs, everything from Race to the Top, which is state-based, which I know most of you are already aware of, the school improvement funds, additional funds for the data competitions. And then, there's this thing called the Investing in Innovation Fund.

The Investing in Innovation Fund was originally called, just as background, the-- the Invest in What Works and Innovation Fund. And, the reason that that's important is because literally, on my first two days, I got a call one day from a congressional office that said, "This is our big opportunity. We're so glad that you're there. This is the chance to go and find all the things that have really strong evidence, and take them to scale, so that we really are only tending to those things that we know really work for students." And then, the next day I got a call from someone else, and they all said, "This is a great opportunity. We're so glad that you're there. We're going to do things that are really out of the box, way out beyond the edge, the cutting edge, those things that are uncharted and untested." (laughter) So, in case you're wondering, like, if it seems like people are not sure that they're trying to do the same things when they put those laws together--

So, what we did is we said, "The reality is that one of the challenges in education that we know is that our continuum of innovation actually does break down." We are-- really have a lot of innovation that goes on in the field. But, if you said, "Is it structured in a way that actually we're clear it's tied to our very most important needs in the field? Is it tied to a researching agenda that actually is supporting it and feeding into it? Does it go from the early stages, once we find out that something is promising, it's something that has enough evidence to be validated at a greater scale? And then, ultimately, those things that prove out, are those the things that actually scale the most?" And, if you answer those questions about, by walking around the schools and districts in which you are-- in which many of you work, or probably not the ones that you work in, but go next door to the other districts, will find that actually, that's not the way it works. That, in fact, many things scale without really good evidence that they work. That, in fact, the field does not set great priorities around our research agenda.

We don't coordinate well across our research objectives across universities and programs and things of that nature. And so, we said, "How can we use this program to start to set the frame for that to happen?" And so, there's two-- actually, three things I want you to remember about the program. The first is that, what we tried to do is to use this as a way of setting the frame for bringing evidence
into making decisions about grant making, at least from the Department of Education. And, I think you'll actually start to see this frame make its way across the administration overall. And that is a little bit of money for a little bit of evidence, a pretty modest but meaningful amount of money for a moderate amount of evidence, and then big money only when you get to really strong evidence. And so, the investment innovation fund has three levels called Scale-Up, up to $50 million dollars, validation, up to $30 million dollars, and even the development grants are up to $5 million dollars over three to five years. They are targeting in the areas that I know Mike is going to talk about a little bit more in a second.

We have these things that we call the four assurances, the four priority areas of reform around standards in the-- high quality standards and assessments and human capital and data,and turning around the lowest-performing schools. So, people are going to propose solutions in those four areas that have those different levels of evidence. And they will design them so that they can answer several questions for us. One is, how do we know that this works? What is your level of confidence that it's going to work the way that you expect it to? And, what level of evidence do you bring to that question?

Second, what do we expect to learn from this project as you propose it? How are you putting it together and making sure that the way it's-- the evaluation is structured will provide us that we need--
-- the evidence that we need to move it forward? Third, is it sustainable beyond the funding that we're going to provide in this grant program? And, is it scaleable beyond your district and your partners to something that can actually serve not just hundreds, but thousands, if not millions of students? There's more to it than that, but that is the basic frame. There'll be a few competitive priorities for those of you who know how our competition works. It's basically bonus points. If you connect to an early learning strategy and have a very clear through line from early learning through to your K-12 project, that's one way to get bonus points. Same thing if you have a through line from your K-12 project out to college access to success. That's another way. If you have a disproportionate impact on students with disabilities or English language learners, that's another way to get bonus points. And the fourth is recognizing that there's a couple of components that make it a little more challenging for rural districts, including a matching component. We're giving a competitive preference priority to folks that are in districts, in rural districts. And, that's it. I'm happy to answer a lot more questions later.

PAT: Do you want to talk about the timeframe for the award--


PAT: -- competition?

JIM: So-- So, I've stopped naming dates. (laughter) We originally had planned to put this competition out last year but decided that it was more important to put all the state-based competitions together and then have the Investing in Innovation fund, which actually I didn't say-- this is really important. The Investing in Innovation fund is focused on LEAs and non-profits in consortiums of schools, not on states. So, LEAs can apply directly, as well as non-profits that work directly with LEAs to improve student outcomes. We expect, in the coming month, that being February, that this will be released, and expect that folks will have the normal-- about 60 days to get your applications back in. And then, because of the large numbers that we're expecting, we're probably going to take about the same amount of time before we get back to folks and tell them if they won or not. I'm punching on the-- simultaneous conversation

PAT: Yeah, I'll get back to you on that, I will. I'll give you more time to think about it. And, any of you that know about the Department of Education's process for making awards, what's required is-- has been spelled out in a federal register notice. So, if you go to the federal register system, you can find all the particulars. But, I thought it was really important to start with that because many of you have been gathering evidence, have been working for a number of years. And, you may be just the kind of organization or set of players that are-- are worthy candidates for this source of funds. The department's never had this much money available for innovation. So, it's a-- it's a-- it's a wonderful opportunity.

I'm going to turn to Michael, and I'm going to ask him-- The question is, what is a special advisor for science to the Secretary of Education? You may not have picked up, in his résumé, but he was a director of science in the Chicago Public Schools. So, it sort of makes clear, you know, why Arnie was very-- why Arnie knew about him and wanted him to come to work with us. But, we've never had a special advisor for science. So, I'm going to ask him to talk about how he sees his job. And then, I'm going to ask him to talk a little bit about the race to the top work, because he's been pretty involved in helping to-- to develop the strategies that we have for that program.

MICHAEL: Thanks. Happy to talk about a lot of that. I'm still feeling rather new to the Department, so I still get to define a lot of what this actually looks like. So, my role is to be the point person for the Department on science, mathematics, engineering and technology education, and try to synthesize our work in the Department, craft a comprehensive strategy that matches what the country needs and what the President wants, and then work with other agencies, both within the government and within the public and private sectors to get as many of us pointing in the same direction as possible to make those sorts of things happen. And, in the time I've been here, I think we've-- we've sort of come across a few ideas that are defining what the work really looks like.

So, the first is one that Jim alluded to. It's clear to us that we will not be able to get the results for kids in math and science and engineering and technology education that our country needs and our kids deserve, if we don't expand and really focus on the whole pre-K to 20 system. And, the Department has sort of set four assurances. These are sort of strategic directions or priorities that we use to get there. I'll talk a little bit about them, and we can flesh in there with some questions. So, we fit STEM education into this context. The first is the idea that we need clearer, common, higher standards that describe what kids know and are able to do. It's really clear, when you look at lots of data, that different states have different levels of expectation for what kids need to know. And, we have different ways to measure them and different support organizations and mechanisms all-- all around that work. And, we know that high standards make a difference for kids.

So, we have put a lot of our energy into listening to states as they say they want to make sure they are keeping their standards high. And, we now are hearing lots of states wanting to enter into partnerships with one another around fewer, clearer, higher standards. That's moving forward initially in reading, language arts and mathematics. Though, we hear there are states very interested in doing similarly in science and suspect that that work will follow afterwards. Connected to that is actually being able to measure and analyze the data that describes students' and teachers' ability to impact on those standards.

So, the second assurance focuses on data. And, this means everything from data systems that can track kids from pre-K through college, data that enables us to compare the performance of students and match that to a particular teacher or a particular principal, or teacher performance and being able to map that to the institute of higher education that provided the teacher preparation for that teacher. And, there's also a component around helping districts, schools, teachers make sense of what this data means, so that we can actually help kids help identify what's working and what's not working, and make the kind of changes that happen. So, investing in data systems, and all the tools and protocols and systems around that is our second-- second assurance.

The third deals with great teachers and leaders. And, this is this idea that fundamentally, education is about people. And, great people really make a difference. So, there's a lot of work to think about what we mean by an effective teacher. And, in our minds, an effective teacher is one who gets results for kids. And figure out ways to increase the number of those teachers and make sure that they are working where they're most needed. So, creating mechanisms and systems to make sure the best teachers and the most qualified teachers are teaching the kids who need that help and support the most. A lot of work on helping push on teacher evaluation systems, recruitment systems, preparation systems, is all sort of encapsulated into that assurance.

And the fourth is one that I know the secretary, you know, when you hear him talk, he gets-- he gets very passionate about the fourth assurance around turning around perpetually low-performing schools. He really made a name for himself doing this in Chicago. And, it's an incredibly painful and difficult thing to do. But, we also know that there's schools in our country that have been under-performing in the most egregious ways. And, they have been like that for years and years and years. And, what we're saying with this, is that there needs to be the courage amongst the leaders to say, if a school isn't working, you need to do something different. Because it's not about the adults there, it's about the kids. So, the fourth assurance is about turning around perpetually under-performing schools. And, there's a myriad of tools and mechanisms involved to do that.

So, in that context, my challenge is to figure out the science, mathematics, engineering and technology piece to that. And, we've, I think, established a few pieces that-- that have to be common around that. So, first is that, as the standards are changed, as they get higher, as they get clearer, we are going to need tools and supports for teachers so that they can teach to those standards. And so, we need to figure out a mechanism to incentivize and organize the education community broadly, to provide those kind of tools and supports for teachers, so that they can teach to these newer, higher standards. Similarly, we know that leadership matters, and it matters particularly in mathematics and science.

I'm-- I'm fond of recalling a citation from the Ed Weeks Quality Counts Report of a few years back that showed that most principals and superintendents sort of knew that there was a national problem on science and math education, but didn't think it was their problem and weren't doing anything about it. There's also, you know, pretty compelling evidence that schools organize differently around mathematics than they do around language arts. So, figuring out ways to bring that to bear on what we're doing. And third is sort of, because of that, we know that we have to be particularly focused on motivating and inspiring kids and adults around this work. It's not something that comes necessarily naturally to people. But, we have to find ways to get kids excited to do more science.

We have to find ways to get the adults in the system to-- to pay attention to these sorts of things more, in engaging ways. And, that can be through, you know, after-school programs and fun new curricula, and internship experiences, relationships with scientists and engineers. That kind of stuff is-- is very, very important. In fact, the President has asked all the scientists, engineers in the Federal Government to spend some time with students, and has-- has actually done a couple of pretty high profile events, just to sort of make that point. So, to get to your questions, rather long-winded response, I know. My job is to stitch all that stuff together into a compelling strategy that provides that kind of support. And, I'm not going to be able to do it without all of you helping.

PAT: One of the things that we keep hearing, and I know the National Science Foundation is pressing you all on, and that the Department has been talking about over the years, has to do with standards of evidence for education innovations, and what we need for better evidence, so that we can move this enterprise forward. So, I'm going to ask Jim to talk as the, sort of, director of this-- a lot of this work at the Department, to talk about this administration's notions about evidence and what-- what's useful evidence, and-- and how we can marshal that evidence for better educational outcomes.

JIM: Sure. I mean, the interesting thing about it is, that it's still evolving. We're still working through what it really means. It's really easy for us to start to plot points, you know, strong evidence. IES has had a definition of strong evidence, mostly tied to experimental designs, especially randomized control studies. The moderate evidence opens up the lens just a little bit, where we have not had really good definitions. And, one of the places where we're really excited about the opportunity to partner with the NSF is, what does rigor look like when you're actually in the developmental process? How do you actually do that in a way that starts to answer the questions about whether something is high potential and in what context, and what exactly about it makes it meaningful? I don't think I'm doing that. And so, one of the things that we're working on, we're working on those three parts for our definition hard inside the Department of Education.

As I mentioned, that's something that's going to cut across the administration. The thing that's interesting, though, as we've kept trying to push us, is right now, the Investing in Innovation fund has a higher evidence threshold than just about any other program in the Department. So, think about the paradox there. You're-- What, in any other sector would be your venture capital but nothing else, private equity, is got a higher standard of proof than the hundreds of millions, rather than billions of dollars that flow out with basically no requirement for evidence every day. So, that's, I think, the next frontier for us, is to say, "Sure. We can come up with these standards of evidence. But, what does it really mean in terms of what's required in the field in order to put it in front of our children, especially millions of them at a time. I don't know what else you want me to say. (laughter)

PAT: It's a work in progress. So, another thing that some of you have asked me about this afternoon has to do with the reauthorization of No Child Left Behind or the Elementary and Secondary Education Act. And, I know that the secretary has met-- made some statements lately and has said that it's a very high priority for the administration to get this legislation reauthorized. So, I'm going to ask, first Michael and then Jim, to just tell you whatever they can, legally, at this point, about what the Department's thinking is on ESEA and how math and science may be a part of that.

MICHAEL: I'll start, and you can sort of chime in, Jim, as we do this. So, I think-- I think what Pat said is exactly right. The Secretary is very interested in having the reauthorization of ESEA happen soon. We don't do the reauthorizing. That's what Congress does. We remember our civics and that. So, we're-- (laughter) So, we're somewhat at, you know, it's up to them to sort of figure out how this is-- how this is going to happen. But, I think there's, you know, over the past year, there's been built up an awful lot of momentum and consensus around the agenda that we've been driving. And, a lot of the key ideas that I talked about in the four assurances, we are optimistic will frame a lot of what happens in ESEA. For instance, the Secretary is very-- has said, many times, he wants to be very tight on the goals, so very clear about what's supposed to happen, but very, very loose on the means, letting schools, states, districts be free to innovate and try what happens, as long as the results are-- are coming in-- are coming in.

We also want to craft legislation that really gets at this idea that growth matters. And, there's lots of ways to measure what a-- an effective school or district is. So, I think you'll see a lot of work around ways to show what improvement is like, how to talk about that, and moving that-- moving that ahead. I think you'll also see, you know, we believe in-- we believe in accountability, and we believe that closing the achievement gap is a really important thing to do for the country. So, we want to make sure that we really put a spotlight on our most vulnerable kids all over the system, and make sure that people are working as hard as they can to do better by those kids.

What do you want to add, Jim?

JIM: I think there's probably three important things to add. On the last point, one of the most important shifts in our thinking is the shift to thinking about growth as a primary indicator of a school's value-add as opposed to just status. And so, making sure that, if you are a school that takes students from point A to point B, and that's a good long distance, then you should get credit for that, even if B is short of our ultimate goal. The second important thing is, and Mike alluded to this, is the shift to definitions of highly effective teachers and leaders away from high quality teachers and leaders. So, what impact are you actually having on students in their outcomes? And, what are the broader ways in which we actually define that?

And the third thing is, more an enabler than a-- than a shift, which is, this-- this enabling of better forms of assessment, as a part of our overall accountability system, so that we're answering questions that matter to us about what our students have learned and what they know and are able to do, as opposed to very narrow assessments of whether they happen to answer this particular type of question like we had last year well. Sorry. I didn't mean to be as judgmental as it sounded. (laughter)

The-- The-- The other-- The other things that I think are-- are important for us, and I'll use this to kind of segue back to that other question you asked me, is that Mike talked about the Secretary being clear, that, what we want to do is set the context for broad reform. And-- And, that means not just thinking about proficiency. It means, like, it's no mistake that our primary contest is now called "Race to the Top." Where, that's a theme that you can take in lots of different ways. It's "Race to the Top as Compared to Other Nations." "Race to the Top States Within the United States." But also, how do we actually create the incentives for not just getting to the minimum proficiency bar, but actually excelling? And, how do we push ourselves to actually think about how we build a system of continuous improvement so that the bottom is never acceptable again, whatever that bottom happens to be? So, this is where I'll come back to what education will look like in ten years.

Unfortunately, I think there's at least two paths we could go down that are really clear and primary. One is tied to a very simple state of the facts, which is that we're still in an economic crisis. In about two or three years, people are going to recognize that education revenues are a lagging indicator of-- of economic recovery. Like, the way most of our states calculate their education revenues are tied to things that take a long time to come back. What that's going to mean, unfortunately, for many of our states and districts, unless they do super-human things which, unfortunately, we have not shown the will to do, over time, is that most of our budgets are not going to be even what they are-- were in the last year once the recovery funds run out.

So, one path we can go down is that we kind of stare that in the face and fail to act until the cliff comes. And then, we make fairly draconian cuts without a real strategy for how we're going to actually produce better outcomes for children. That's the downside. (laughter) The other side is that we do three things. One is that we actually do think about a new and different ESEA, which is much more strategic and puts a lot more power into each of the resources that we put into our education from the federal level, but also helps alignment and leverage of resources between and among states as they move forward. I mean, there's lots of reasons that people could be for common and nationally benchmark college and career-ready standards. But, economies of scale and efficiency is-- is not the least of them, especially when you think about it, not only in terms of assessments but curriculum, professional development, resources, tools, etcetera, etcetera, etcetera.

The second thing that actually needs to happen is that we need to build a system, a learning system, and the infrastructure to support it, that allows us to dramatically improve the pace at which we learn and improve in education. So, as a system, we do not actually have the infrastructure that you would expect, just compared to almost any other sector, where whatever you're supposed to produce, you have really good instrumentation at the level at which you produce it. That goes back to the people who actually are supposed to be doing that work on a daily basis. There's a set of people who are behind them, looking at that data, at an even 'nother level of abstraction, making sure that they have the tools and resources that they need every day to make the best decision that they can in the moment.

Like, that and education would mean that teachers and students would have the resources and tools at their level, that tell them, on any given task, how a student is doing. And then, across students and teachers, it aggregates it to say, in what patterns are we seeing about, what tools and resources, curriculum, assessments we're using, that are actually working well or not working well, with which teachers, and which context. And then, at another level up, we would say, when we look across our entire system, we see that we have clear gaps in the curriculum assessments and other tools in professional development that we provide teachers and students. And, we need to be filling those gaps for very specific types of R & D. And, that's where our investment dollars need to be going, not anywhere they happen to-- someone might seem to write a nice proposal. With constrained resources, that's the kind of system that we're going to have to develop. And, we're going to have to do it at light speed, something that's unprecedented.

The good news is, like, we don't have to make it up. There's plenty of folks that have figured out how to do this before in other sectors and other countries, that have figured out how to apply many of those principles to education. And, if we do it right, we can skip a lot of steps and land in a place where we can actually move ahead very, very quickly. If we do that, then I think, in ten years, our education system looks incredible. And, we'll be able to provide students with incredible learning opportunities and experiences, where I talk about efficiency and improving productivity on the cost-saving side. Think about it in terms of the time savings, right. So, I know I'm talking for a long time.

PAT: Go ahead. That's good.

JIM: So, I'll give this one anecdote. Carnegie Mellon had this open learning initiative. A lot of what they focused on was the Gateway courses in math and science. One of their best courses was a course in statistics, where they did a complete course redesign. So, three things to remember about this: They redesigned the course. They did it with a grant of about $500,000 dollars, plus a lot of human investment on campus. This is a course, statistics course, which usually has a pass rate of about 42%, which went to a pass rate of about 98%. Usually a one-semester course. The students met for half the amount of time. And, they didn't meet every class. So, twice the pass rate, half the time. What were the three major contributors they say to this? One, students' portion of the redesigned course was online. So, when they go home after class, their coursework will be online. They get immediate feedback about what they'd mastered and what they hadn't mastered. Across students, the professor then gets feedback on what the class mastered. And then, it gives them some guidance and support on how to think about what the next class actually is. And, here's the third thing, where I'll tell you what actually happened. And then, I'll give my hypothesis. The third thing is, they actually redesigned the study groups around the data that they got as well.

So, my hypothesis from that one is, because they also found that-- We tend to think of top students and bottom students. But, on any given concept, it was changing who that was. So, there's one part that says, "Sure. You get the best student on a given concept to lead the study group. That's great." But also, what happened is, students who never led a study group had the opportunity to lead. And, we all know what the evidence is about what happens when you change self-efficacy. So, I talk about that and in different contexts. I say, it saves half the time, half the staffing. Think about all the things you could do with that. But also, think about what you could do in terms of new learning experiences with that additional time. Forget about the saving money and saving staff.

Think about what you could do, if you could cover the core content, and reduce times. And, you could create many more opportunities for students to do internships, to have relevant experiences in the field, to do and chase what they thought they were most interested in. What could we do in terms of re-engaging students then? That paints a very different picture of what education could look like in ten years. And, it solves a lot of the problem. Forget PSAs for STEM. All of a sudden, if I get to understand why I loved STEM when I was a little kid, because one thing I have to disagree with, most kids start off loving science. And somehow-- (laughter)-- we managed to steal that away. (laughter) So, if we could just learn how not to do that, (laughter) we'll be way ahead of the game. I'm sorry.