Transcripts

text | pdf |

---

SPEAKER: But still very cool out there. Highs only in the mid 30s today. Tonight, look for clear skies, cold. Lows around 19 tonight. On Wednesday, it will be quieter, sunny in the morning, partly cloudy in the afternoon, and a little bit warmer. Highs on Wednesday around 40 degrees, overnight lows dipping into the 20s. And then Thursday's looking partly cloudy, highs in the mid 40s. Friday, mostly clear with highs touching 50 degrees on Friday.

STEPHEN JOHN: This is MPR News Presents. Today, an APM Reports documentary called Under a Watchful Eye, How Colleges are Tracking Students to Boost Graduation. Many schools are using so-called big data to predict college success and help students complete a degree and pursue their ambitions. Last week, the documentary by APM Stephen Smith and Sasha Aslanian won a prestigious national headliners award, and you can hear it after news.

LAKSHMI SINGH: Live from MPR News, I'm Lakshmi Singh. If his tweet this morning is any indication, President Donald Trump seems to be itching for a fight with state governors over who has final say on lifting COVID-19 restrictions. He references Mutiny on the Bounty, writing, "A good old fashioned mutiny every now and then is exciting."

But New York Governor Andrew Cuomo says he's not going to fight the president at a time when everyone should be working together. Cuomo says the outbreak in his state appears to have peaked. He credits new Yorkers who have followed stay at home measures. Guided by coronavirus data, Cuomo and other governors on the East and West coasts are working on coordinated regional plans for reopening schools and businesses in phases and possibly over months.

Former President Barack Obama is speaking out about the need for steady leadership in the White House. And he says former Vice President Joe Biden is the person for the job. MPR's Juana Summers says Obama is now officially endorsing Biden for the Democratic Party's presidential nomination.

JUANA SUMMERS: Obama made his endorsement in a video released Tuesday after remaining neutral throughout the Democratic primary. In the video, he cast Biden as a steady leader, the kind the country needs to combat the coronavirus pandemic. He also made direct appeals to the scores of people who supported Vermont Senator Bernie Sanders, who endorsed Joe Biden on Monday.

Former President Obama remains incredibly influential among Democrats. His support could play a key role in energizing the party to come together and support Biden who still faces some skepticism from younger and more progressive voters. Juana Summers, MPR News.

LAKSHMI SINGH: Further underscoring all that is at stake during this pandemic, the International Monetary Fund is now Warning that the health crisis will likely cause the worst economic crisis since the Great Depression. IMF Chief Economist Gita Gopinath says cumulative loss to global GDP into next year could be around $9 trillion.

GITA GOPINATH: We are projecting global growth in 2020 to fall to minus 3%.

LAKSHMI SINGH: Gopinath says the IMF predicts global growth will partially rebound next year provided the pandemic fades later this year. While the death toll from COVID 19 could be 15% higher in England than the government had been reporting in its daily tally, here's the latest from MPR's Frank Langfitt.

FRANK LANGFITT: The UK's Office for National Statistics reported nearly 800 more deaths in England up to early April compared to the tally from the National Health Service during the same period. The NHS only reports deaths in hospitals, while the Office for National Statistics includes deaths in all settings, including nursing homes.

Meanwhile, Britain's Office of Budget Responsibility says economic growth could drop a record 35% if the lockdown lasts for three months. And unemployment would rise from 3.9% to 10% in the second quarter. Britain's department of Health says the COVID-19 death toll for all of the UK has now passed 12,000. Frank Langfitt, MPR News.

LAKSHMI SINGH: This is MPR.

SPEAKER: Support for MPR comes from NPR stations. Other contributors include Babble, teaching real life conversations in a new language like Spanish, French or German. Babbel's 10 to 15 minute lessons are available on a mobile app or online at babble.com.

STEPHEN CHAN: For MPR News in the Twin Cities, I'm Stephen Chan. Minnesota legislature's top Democrat says it's too soon to throttle back on efforts to control the coronavirus spread. House Speaker Melissa Hortman told reporters today that she has confidence in the strategy employed by fellow DFLer Governor Tim Walz that's says more Republicans are pushing to ease restrictions on movement and commerce. Hortman says it all depends on broader testing and health care system capacity.

MELISSA HORTMAN: Everybody is anxious for the stay-at-home order to end. Everybody is anxious for the economy to get up and running like normal. And everyone is heartbroken about the economic devastation, especially to our small businesses and especially to the hospitality industry.

STEPHEN CHAN: Hortman said legislative efforts to undo the peacetime emergency by Walz won't succeed. Meanwhile, state health officials today said nine new deaths raised the state's total to 79 since the outbreak began. Turnout in Wisconsin's chaotic spring election held during the coronavirus pandemic last week topped 34% of the voting age population, the highest for an April vote since 2016. Joe Biden was the winner in the presidential primary, and a liberal Dane County Circuit judge defeated incumbent conservative Supreme Court Justice Dane Kelly by nearly 11 points.

The Minnesota Orchestra has restructured the remainder of its current season due to the coronavirus. This involves postponing or canceling all concerts through this June. The Summer at Orchestra Hall season is now pushed back to 2021. The orchestra is adding five extra weeks of performances in August and early September this year to accommodate some concerts postponed from the current season. This is MPR News.

SPEAKER: Programming is supported by Minnesota Public Radio's educational sponsors. Our work together provides public radio service throughout the region. One of the sponsors we would like to thank is Michigan Technological University in Houghton, Michigan.

[MUSIC PLAYING]

STEPHEN JOHN: Welcome to MPR News Presents. I'm Stephen John. Colleges and universities have been turned upside down, as have so many other institutions in American life. Colleges closed for in-person learning, and students and faculty are trying to do what they can online. APM Reports produced a documentary about the ways colleges in the best of times use data to help prod students toward graduation. The documentary produced by Sasha Aslanian was awarded a prestigious first place national headliners award last week. Here's host, Stephen Smith.

STEPHEN SMITH: From American Public Media, this is an APM Reports documentary. Keenan Robinson always knew he wanted to study science. He went to a high school in a small town East of Atlanta, Georgia. Biology was his best subject.

KEENAN ROBINSON: My dad, my mom would always kind of call me like the king of trivia because I'd always have just random like science facts.

STEPHEN SMITH: Keenan's parents dropped out of college. They wanted their children to finish. When Keenan arrived on campus, he wrote down he intended to major in nursing.

KEENAN ROBINSON: Well, I always knew I had a passion for helping people.

STEPHEN SMITH: During his freshman year, Keenan earned a solid B average. But the University was tracking his data. And it knew from looking back at 10 years of previous student records that a freshman with a B average wasn't likely to make the cut for the nursing program. Keenan's advisor could see a warning on his computer screen. Keenan's nursing plan might be in jeopardy. His advisor Joshua Reeves says when that happens, it's his job to warn the student.

JOSHUA REEVES: Hey, you're not going to be competitive for the program.

STEPHEN SMITH: Reeves sometimes steers students like Keenan into another health care major that accepts students with lower grades.

JOSHUA REEVES: Once they know they have options, not, the end of the world you're still going to be successful, we're still going to get your degree that you're happy with, we're still going to get you in the field that you're going to enjoy, they're usually receptive to it. And like I say, they stay on track and we still get them to graduation.

STEPHEN SMITH: Keenan's advisor may have killed his nursing dream. But Keenan doesn't see it that way. His new major, respiratory therapy felt like the right choice to him.

KEENAN ROBINSON: I have asthma. I grew up having asthma and playing sports and everything. I know both of my grandparents and my mother's side had lung issues. And once it kind of clicked, it was something I wanted to do.

STEPHEN SMITH: From APM Reports, this is Under a Watchful Eye, How Colleges are Tracking Students to Boost Graduation. Keenan's decision to change his major wasn't just driven by the wisdom of an attentive advisor. Millions of data points and algorithms were lurking in the background, quietly nudging this change in Keenan's path.

It's called predictive analytics. Companies like Amazon and Netflix have been using predictive analytics for years to track our clicks and to steer us to buy or watch more of their stuff. That's how they make more money. Now colleges are using predictive analytics to try and keep you enrolled and collect your tuition dollars next semester.

Predictive analytics systems are taking hold far away from the nation's elite colleges where wealthy students have the luxury to engage in intellectual self-discovery. Instead, they're reshaping the college experience among students in less selective colleges, putting them on a narrow data-driven path to graduation with fewer dead ends and wrong turns.

APM Reports teamed up with Jill Barshay of the Hechinger Report. She's a longtime education and business reporter. Jill was curious about this data revolution on campuses. She spent more than a year talking with the people who are selling data packages, the colleges that are buying, and the students whose clicks are being tracked to find out whether mining student data can help those students get a diploma.

SPEAKER: Few colleges and universities in the country have come as far as fast as Georgia State.

JILL BARSHAY: The college where Keenan goes to school is Georgia State University in downtown Atlanta. It isn't a prestigious college like the Georgia Institute of Technology two miles away or the state's flagship, the University of Georgia in Athens. But it's the largest university in the state with more than 50,000 students. Construction cranes are building new dormitories. The Georgia State logo of a flame has spread across a lot of downtown real estate from office towers to the Atlanta Braves' old stadium.

It used to be an all-white commuter school. Black students weren't allowed to enroll until 1962. But by 2006, the school was majority minority. As the college accepted more first generation, less prepared students, graduation rates dropped. Only one and five Black men who enrolled in Georgia State graduated. For white students, it was somewhat better, one in three.

TIM RENICK: We had this challenge of we were getting more and more of the students we were less good and less skilled at supporting.

JILL BARSHAY: Tim Renick came to Georgia State as a religious studies professor in the 1980s. He moved into the administration, and it became his job to try to do something to improve Georgia State's pretty bleak graduation rates. He knew that low income, first generation students and students of color really lagged behind their peers.

Renick thought maybe the school could boost those rates if it did a better job of advising students about what classes to take and how to get through. But Georgia State had only one advisor for every 1,000 students. And the students who showed up in advisor's offices were either honor students who were managing just fine or failing students who were coming in too late. Renick needed to reach the students in between.

TIM RENICK: What would an advising system look like that targeted the students in the middle, that really served the B minus or C plus students who mostly are sailing under the radar screen at campuses like Georgia State but also dozens of other campuses. These are the students who don't raise their hand and answer every question. Oftentimes, they sit-in the back rows. They'll be quiet. They don't tend to come to office hours. They don't go out of their way to participate in campus life and so forth. But those are the students we realized who could move our graduation rate.

JILL BARSHAY: Renick had an idea that the answer was data. Perhaps data could help him point to which of these middle students could be saved, and when. He approached a company called EAB that was just starting to dabble in mining education data. They talked about using that data to hunt for patterns in student records from the past and then using those patterns to make predictions about which current students were at risk of dropping out. Renick signed a contract with EAB in 2011, and they started digging.

TIM RENICK: We used 10 years of data. We used 2 and 1/2 million Georgia State grades, 140,000 student records. We ran this big data set to try to figure out what were the advancing indicators that a student might drop out six months, 12 months, a year later.

JILL BARSHAY: And they found them in some surprising places like freshman English. Who would have thought that a B minus in introductory English freshman year would predict that a student wouldn't do well in the nursing program. But it does. The data found that students who graduated from the nursing program mostly had a B or better in introductory English.

A B minus, only a notch lower, might seem like a fine grade. But it's actually a giant warning bell for an aspiring nurse. You might not pass biology, anatomy, and other really tough requirements later on. Those with B minuses in introductory English often didn't make it in the nursing program and dropped out. That same B minus turns out to be just fine if you go on to major in accounting.

The historical data show that accounting majors graduated even if they got a C in intro English. It's not that an English grade is causing a student to succeed in accounting or fail in nursing. It's just a correlation in the historical data. A human alone would never be able to pick up on patterns this complex. But with the data, advisors can spot them.

At Georgia State, advisors have become the heart of the effort to boost graduation rates. These are the professionals who help students map out their college plans and navigate the university bureaucracy. Georgia State has tripled its advising staff.

SPEAKER: Lassiter, Margaret Lassiter.

JILL BARSHAY: There's a waiting room filled with dozens of students crowded on couches like a doctor's office.

SPEAKER: This is the countdown. If you've got earbuds in your ear and we call your name, you don't answer, you will be skipped, go to the next person.

SPEAKER: [? Glastola. ?]

JILL BARSHAY: When their names are called, an advisor walks them back to a small office for a one-on-one session. Keenan, the student we met earlier who switched his Major from nursing to respiratory therapy is talking with his advisor, Joshua Reeves.

JOSHUA REEVES: So you're going to have two lab science courses this semester. So be prepared to make sure you put enough time aside so you can focus on both of those courses as well as your two credit hour CPR course and your medical terminology course.

KEENAN ROBINSON: I figured I wouldn't be as much of a stress--

JILL BARSHAY: Reeves has two computer monitors up on his desk. One is turned toward Keenan so that he can follow along. The screen shows that Keenan is a solid B student. It also displays Keenan's probability of graduating. It's based on his grades and prerequisite classes, whether he's taking the right courses for his major, and whether he's gotten flagged for skipping class or flunking quizzes. The system sorts students into three risk categories and color-codes them like a traffic light, green, yellow, and red. Keenan can see his yellow light.

KEENAN ROBINSON: I already know what I have to work on. So it just kind of like reaffirms that I need to just keep up with everything. But basically, it doesn't really scare me.

JILL BARSHAY: Keenan remembers that he had a yellow light early on during his second semester of freshman year. That was a sign that his plan to go into nursing was risky. At Georgia State, you don't just declare a nursing major. You need to apply to the nursing program at the end of your sophomore year.

And Georgia's state had made an important discovery in its data analysis about students like Keenan who wanted to get into the nursing program. Many students who didn't get in ultimately dropped out of college because they had wasted their first two years taking prerequisites that didn't count toward another major.

It was critical to steer these students, many of them with above average grades, to something else sooner. It's up to each advisor whether to show these risk ratings to a student during an advising session. Reeves says he usually doesn't reveal a high or moderate risk warning to a student initially. Instead, he draws the student into a longer conversation.

JOSHUA REEVES: You tell them, we're going to probe. We're going to do a Q&A, OK, because you can sit here and tell-- because a lot of times those early alerts, what students are coming, how are your classes going? They're going well. OK. What about that US history class? It's going pretty good. It's OK. How'd you do on your first test? Oh, well, OK. Let me tell you. What had happened was--

And so you kind of got to pull it out of him. But then once they know, it's like, OK, he kind of knows. Then that's when they kind of open up. But for me in my advising style, I just don't usually come in and say, hey, you're here because you did poorly on your first exam. Let's kind of warm up to it. Let's talk about, I want you to open up to me, and then I'll help you along the way.

JILL BARSHAY: Running into academic problems is only one reason students drop out. Sometimes, it's money. And the data is pointing out those students too. Avery Vining is 20 years old and a sophomore. He's got good grades, no academic problems, even though he's taking some tough classes.

SPEAKER: [SPEAKING MANDARIN]

JILL BARSHAY: He's learning Mandarin Chinese for his major.

AVERY VINING: My major is international economics and modern language with a concentration in Chinese cultural studies.

JILL BARSHAY: In December of Avery's sophomore year, he was past due on a tuition bill for $1,400. He was about to be dropped.

AVERY VINING: I had considered taking a break, especially with mom.

JILL BARSHAY: Avery's mom had gotten sick and couldn't work. Avery was sending money home from his campus job to help pay her bills.

AVERY VINING: I was like, OK, this is a lot, especially now I was 19 at the time. So I was really nervous. I was like, OK, I don't want to hurt my future. I don't want to hurt anything. So if I can just put a pause on it, then OK.

JILL BARSHAY: Taking time off generally isn't wise. At Georgia State, 70% of the students who "stop out" as they call it, don't end up finishing their degrees. Georgia State's data system sent up a flare about Avery. Here was a strong student suddenly not re-enrolling. He didn't owe all that much money. As the final payment deadline approached, Avery checked his account online and was stunned. Georgia State had quietly paid his bill without telling him. Avery remembers calling his best friend.

AVERY VINING: Hey, you're not going to believe what just happened.

JILL BARSHAY: Avery was able to stay in school.

AVERY VINING: I was just like, OK, everything really is good. Everything can really be handled now. I can go back to being the normal Avery and being able to work, go to school, and focus on myself and my family.

JILL BARSHAY: Georgia State has issued more than 13,000 of these retention grants over the last six years.

TIM RENICK: We can't afford not to do it.

JILL BARSHAY: That's Tim Renick again. He's giving a PowerPoint presentation to a group of visitors. He does this a lot. Administrators from colleges around the country now flock to Georgia State, and Renick is happy to show them his charts and graphs about what predictive analytics has done for the school. He's explaining the financial payoff of giving money away to students.

TIM RENICK: Because the average grant of $900 comes immediately back to us, right? It fills a gap in what they owe us.

JILL BARSHAY: Any student Georgia State holds on to, it makes money on. That's because the student or the student's aid dollars are continuing to pay a big chunk of the tuition bill. Only a small part is forgiven by the university. Georgia State pays $150,000 a year to its data analytics vendor. That's a heavily discounted rate because they got in early. Renick says that investment pays off.

TIM RENICK: Every 1% we increase the graduation rate is worth about $3 million a year in added tuition and fee revenues. So we've raised our graduation rates 23 percentage points. So we're talking about $60 to $70 million of additional revenues that in the past was literally walking away. These were students who were paying customers who are walking away from the university who now are actually getting their degrees. So you talk about win-win situations.

JILL BARSHAY: The money has been good for the institution. But what's brought them fame and a lot of visitors from other colleges are their graduation rates.

TIM RENICK: There are no achievement gaps. So for each of the last four years at Georgia State, we're the only public University in the United States who can claim this, our Black, our Latino, our first generation, and our Pell students all graduated at or above the rate of the student body overall. No achievement gap.

So that thing, that kind of goal, which some see as impossible to attain was something that we've been able to do and do for four years in a row. And this institution in the shadow of the Martin Luther King district that was segregated well into the 1960s and that was grossly underperforming relative to our students of color as recently as a decade ago now is conferring more bachelor's degrees to African-Americans than any college or university in the United States.

JILL BARSHAY: I checked those numbers, and it's true. Georgia State is number one among nonprofit colleges in the number of degrees it awards African-Americans. Not only are more students graduating, they're getting through college quicker. The requirements haven't gotten easier, but fewer students are changing majors in the middle of their college careers. That means fewer wasted credits. Renick says students on average have shaved off a half a semester of courses to complete their bachelor's degrees.

[MUSIC PLAYING]

Georgia State's success is fueling an emerging industry. More than 30 companies are now selling these kinds of predictive analytics tools according to the consulting and research firm Eduventures. It's become a $500 million market. Already, 1,400 colleges around the country have bought predictive analytics tools. That's about a third of all higher ed institutions.

IAN ATKINSON: All righty, folks. I guess we'll crack on. Good afternoon. My name is--

JILL BARSHAY: At an education conference in Chicago, administrators from community colleges head off to sessions where predictive analytics companies are making sales pitches. Ian Atkinson is the pitch man for EAB. On Atkinson's demo screen, he shows the group how his company identifies the murky middle students who are at risk of dropping out.

IAN ATKINSON: I can now see, for example, who of my students at medium risk that have a 3.4 GPA. Guess what? You've probably got a lot of them, but you don't know who they are.

JILL BARSHAY: Down the hall, Brian Mikesell of Hobson's Starfish demonstrates his rival product.

BRIAN MIKESELL: So if you want to think about really moving the graduation needle, this is remarketing to students that have stopped out. And then sending them an email and telling them that, hey, Mary, we haven't seen you on campus since 2016. You were killing it here. You left. You're like at the 99 yard line of a football field. If you come back and finish four to six credits, we're going to award you a degree in any of these six courses.

JILL BARSHAY: A college typically pays $300,000 a year for these data dashboards. That's enough to pay the annual tuition bill for almost 100 community college students. Why are so many colleges buying this expensive data stuff? They face the same problem Georgia State did. Nationwide, roughly half the students who start college don't finish.

It wasn't always this way. The dropout problem got a lot worse in the 1990s when more people started going to college. Young adults who used to go straight from high school to the factory floor were suddenly on college campuses. Iris Palmer is a senior advisor at the think tank New America. Palmer says a lot of the new students couldn't cut it.

IRIS PALMER: There was a focus on getting students into college, but not a focus on getting them through college because it was sort of seen as their job as adults to be able to get through college. And that look to your left, look to your right, half of you won't be here was sort of the attitude.

JILL BARSHAY: For low income students, it was worse. They were taking on thousands of dollars of student loans to pay for their college educations. They were more likely to drop out with debt than get a degree.

MARK MILLIRON: You can't look at those data and not just be almost hit in the stomach by that.

JILL BARSHAY: Mark Milliron was working at the Bill and Melinda Gates Foundation in 2010. That year, President Barack Obama was taking colleges to task on their poor results.

BARRACK OBAMA: In a single generation, we've fallen from first place to 12th place in college graduation rates for young adults. Think about that. In one generation, we went from number one to number 12. Now, that's unacceptable, but it's not irreversible.

JILL BARSHAY: It was the middle of the recession. States were cutting funding to public universities. College leaders were under pressure to show better results for their students. Charitable foundations were weighing in. The head of the largest foundation wanted colleges to act more like businesses. In a 2009 speech, Bill Gates called for colleges and universities to publish their graduation rates.

BILL GATES: Colleges are not entitled to escape scrutiny at a time of plunging educational performance and permanent fiscal pressure.

JILL BARSHAY: Gates wanted colleges to use technology and data.

BILL GATES: Without measurement, there is no pressure for improvement.

JILL BARSHAY: Gates wanted more than transparency. He wanted to track student results. Through his foundation, Gates gave grants to colleges to buy data tools and software. Other foundations, Kresge and Lumina did too. Companies saw business opening to soak up these foundation grants. In fact, Mark Milliron left the Gates Foundation where he'd been giving out the grants and founded one of the first companies to sell predictive analytics to colleges. He says colleges needed outside help.

MARK MILLIRON: And part of the idea was, OK, they can't afford to hire their own data scientists. They can't afford to basically buy all the hardware to be able to run these data systems. Let's launch an initiative to figure out how we can impact this at scale and change the way people use data, really help them think about how they can use data to help students.

JILL BARSHAY: When million was pitching his idea to investors, he had to explain that he was going after the bottom half of the college market.

MARK MILLIRON: I still remember having the conversation when people are saying, yeah, we should go get Stanford and Duke. I remember basically saying, they're not going to be interested in this at all. They're basically a hospital who only took healthy patients. So of course, they're going to have good outcomes, right?

So they're not the ones we're going to focus on. You're really going to try to find the access universities and the kind of research universities and the community colleges who are really opening their doors to diverse students and want to help more of them be successful. That's the sector of education we are going after.

JILL BARSHAY: And that sector is buying. But the popularity of predictive analytics has raised some concerns. One problem is that race and ethnicity are closely intertwined with graduation rates. Historically, Black and Hispanic students have lower graduation rates than white students.

Georgia State doesn't enter a student's race or other demographic information into its algorithms. But even when schools don't do that, some observers are worried that the predictive analytics tools might be reinforcing inequities. New America's Iris Palmer wrote a user guide on predictive analytics for colleges and described the risks.

IRIS PALMER: So acknowledging that there is historic bias in higher education and all of our society. If we use that past data to predict how students are going to perform in the future, could we be baking some of that bias in?

JILL BARSHAY: Palmer says that the algorithms can unintentionally target Black and Hispanic students because they're hunting for patterns of dropping out of college such as low grades and missed assignments. Black and Hispanic students might have more of these dings in their records than white students. And they could disproportionately be flagged as high risk.

IRIS PALMER: And so what happen is they'll get discouraged like, why are they even trying, right? What that could end up doing is being a self-fulfilling prophecy for those particular students.

JILL BARSHAY: Those students might drop out or they might be shunted into easier majors. At Georgia State, Tim Renwick says he's heard these criticisms. But he says the reality is more students than ever are graduating in the toughest majors like STEM fields, Science, Technology, Engineering, and Math.

TIM RENWICK: We've increased by over 100% the number of STEM majors from Georgia State, not because we have a lot more students declaring stem as their major, but because we have fewer students declaring STEM majors and then wiping out after a couple semesters. So catching these problems early on has been beneficial to every student. But the students who have benefited the most disproportionately, well, are the students who are most underserved in the past, are low income, first generation, and so forth, and the students in some of our toughest majors.

JILL BARSHAY: But some STEM majors are tougher than others. And here's an example where race and graduation rates intersect again. Black students make up 40% of the student body at Georgia State. But they're slightly underrepresented in nursing, a very competitive program. Black students are slightly overrepresented in respiratory therapy, the program the data steered Keenan to. The difference matters. According to federal data, nurses make $11,000 more a year on average than respiratory therapists.

[MUSIC PLAYING]

Having all this data has also led Georgia State to take a hard look at individual departments. And one of the things the data showed is that a lot of students' ships were crashing on the rocky shoals of math. It's true at a lot of colleges students struggle to pass math classes. I went over to the math department and talked with Professor Valerie Miller. She's been on the math faculty for over 30 years.

VALLERIE MILLER: It's been over a year now. The dean asked us to try to raise our ABC rates to an 80%. And so, yeah. No, we feel the pressure. But we're not going to pass them just to pass them. That's ridiculous.

JILL BARSHAY: One thing the data showed was that many students kept failing and retaking math classes over and over.

MICHAEL STEWART: And I think the record was one student had taken a course 11 times.

JILL BARSHAY: Michael Stewart is the undergraduate director for the math department. He says looking at the data, the professors could see that a student who takes a second run at a math class has a chance of passing. But by the third, fourth, and fifth attempt, they probably weren't going to make it.

MICHAEL STEWART: When we saw what the data showed in terms of the success rates of students who had repeated multiple times, I think everyone sort of unanimously agreed that this was an appropriate thing to do to limit this.

JILL BARSHAY: The new rule says that a student cannot take a math course more than twice. Students would need to petition the bureaucracy to try a third time. If they're hitting a wall on this class, they might have to find a way around it like picking a new major that doesn't require that exact math class. Removing common pitfalls is a big theme here. The school steers students like Keenan away from majors where they're likely to stumble. It erases small, unpaid bills for students with good grades like Avery. Now it's limiting the number of times students can fail a course. It's a little like bowling with bumpers in the gutter lanes.

[MUSIC PLAYING]

STEPHEN SMITH: That's Jill Barshay of the Hechinger Report. This is Under a Watchful Eye, How Colleges are Tracking Students to Boost Graduation, a documentary from APM Reports. I'm Stephen Smith. And one program note, Jill mentioned the gates and Lumina foundations were both early funders of predictive analytics. They are both supporters of the Hechinger Report where Jill works. APM Reports receives funding from Lumina Foundation. We'll take a short break, and then we'll visit two colleges where predictive analytics have been less of a silver bullet.

The students we have here this semester, our data would predict that 42% of them won't be back in the fall term.

JILL BARSHAY: 42% will not be back.

42% will not be back in the fall term.

STEPHEN SMITH: We have more about this story on our website, APMReports.org. You can also explore our archive of education documentaries. You can subscribe to our podcast about k-12 and higher education. It's called Educate. Support for APM reports comes from Lumina Foundation and the Spencer Foundation. More in a moment. This is APM, American Public Media.

SPEAKER: This is MPR News Presents.

SPEAKER: We want to Thank our underwriters for their support. Their contributions help provide uninterrupted essential news and music on the MPR airwaves. Our sponsors are vital to our community. And when you give them your business, you help keep our communities strong, connected, and informed. Thank you.

STEPHEN SMITH: From American Public Media, this is an APM Reports documentary, Under a Watchful Eye, How Colleges are Tracking Students to Boost Graduation. I'm Stephen Smith. We began this hour at Georgia's largest university, Georgia State because it's kind of the poster child for the ways data can help colleges keep students on track.

Graduation rates at Georgia State jumped after the school started using predictive analytics, with Black students actually graduating at higher rates than white students. Low income students are now doing as well as everybody else. But many colleges who've tried predictive analytics haven't seen such great success. Part of the challenge is taking all that data predicting who's likely to drop out and deciding what to do about it.

And another problem is that at some colleges, the group of students at risk isn't the murky middle. It's pretty much the whole student body. Jill Barshay now takes us to two different campuses where the vast majority of students don't graduate. She went to a community college that struggled to produce results from monitoring student data. And then she went to a college where students mostly take their classes online and every keyboard click is tracked.

JILL BARSHAY: Montgomery County Community College outside Philadelphia is a two-year college that enrolls everyone who wants to come. It has more than 10,000 students.

SPEAKER: Oh, my gosh, what a great day.

CELESTE SCHWARTZ: Yeah.

JILL BARSHAY: The student radio station blasts music from a speaker on the campus quad as Celeste Schwartz gives me a tour.

CELESTE SCHWARTZ: You have to admit, it adds some character to the campus, right?

JILL BARSHAY: Schwartz first came to this community college as a student back in 1968. Now she's in her 50th year working for the college. She's vice president for information technology and chief digital officer.

CELESTE SCHWARTZ: People say, what are IT people doing in the middle of student success?

JILL BARSHAY: Schwartz oversees the effort to get more Montgomery students to graduate. It's a huge issue. In 2012, only one in seven students got an associate's degree in three years. A community college especially has a lot of students who might be first generation, older, like 30 and 40-year-old working adults, more low income and minority students, all groups that have lower graduation rates.

In 2013, Montgomery got a grant from the Gates Foundation and hired Civitas Learning to build them a predictive analytics tool. But they didn't use data quite the same way that Georgia State does. If they looked for patterns that predicted who wouldn't get a degree, they'd wind up flagging most of their students. The data could send out so many flares, it would be like putting a smoke detector on a smokestack.

So for Montgomery County Community College, the data scientists didn't model a student's likelihood of graduating, but something much less ambitious, the likelihood of returning next semester. A green light means that the student has a high likelihood of coming back. Red means they don't. That's a smaller pool of students to focus on.

The problem with focusing on retention is that students could be returning every semester and paying tuition, which is good for the college. But the college isn't measuring what really matters to students. Are they going to get a diploma? Still, coming back every semester is how you make progress. If you don't return, you're not going to graduate. So the college got predictions on who would not return. But then what?

CELESTE SCHWARTZ: I think the real challenge is, people don't know what to do with the information. They don't know what action to take. I know people who have dropped the tool. I know schools that have dropped the tool. The hardest part of this, once you get this thing set up, the hard part has nothing to do with the tool.

JILL BARSHAY: Schwartz and her colleagues published an article about their early missteps using predictive analytics. The article makes the point that predictive analytics isn't something you buy off a shelf and plug in and your graduation rates suddenly go up. Montgomery has tried different ways to help the students the data flagged. One experiment put the risk ratings in the hands of professors to see if they could hold on to students.

Monica D'Antonio is an English professor at the college. She specializes in teaching students who arrive at college without adequate reading and writing skills. It's called developmental ed, what used to be called remedial instruction. D'Antonio says she and her faculty colleagues were initially skeptical of predictive analytics.

MONICA D'ANTONIO: This is going to come back and bite faculty on the butt like we're passing too many students we shouldn't be or we're not passing enough or are we going to be pressured to increase graduation? Anytime numbers come out, that's what the fear becomes.

JILL BARSHAY: D'Antonio doubted that the data would have much to tell her.

MONICA D'ANTONIO: We feel like we have these intuitions about our students anyway. So like let us just do our job and don't put these numbers on us. And then you're always skeptical about how people are using the numbers.

JILL BARSHAY: But the computer models knew things about the students the faculty didn't know, 117 different things determine each student's risk of dropping out. Everything from how often they log into the computer system to how early they registered before the start of the semester. A student's race or ethnicity is in there too. But it's not very important. It ranks near the bottom as the 102nd most important variable driving the risk scores. In the faculty experiment, D'Antonio learned the risk ratings for each of the 16 students in one of her classes.

MONICA D'ANTONIO: There were four out of 16 were red. And only one actually came to the class. The other three never even showed. Like it was so right on, the predictive analytics. Not only did they not persist, but they had never even showed up from day one. So three of them, I never even met. So that's how right the analytics are.

JILL BARSHAY: D'Antonio put extra effort into trying to hang on to her one red light student who showed up.

MONICA D'ANTONIO: I did try some targeted interventions at him and then some of the yellow range students just in terms of maybe reaching out an extra email or, hey, you missed an assignment. Do you need more time? Even though I have a late work policy, things like that. So I did try to do some targeted interventions, but there really weren't many.

JILL BARSHAY: She contacted the red light students' other professors, but it wasn't easy to organize an intervention across academic departments.

MONICA D'ANTONIO: He did not stick out the semester. He stopped attending. Yeah. And he stopped attending all of the classes. So it wasn't just mine. He clearly like dropped out mid semester. So sometimes it works, and sometimes it doesn't. I mean, again, the analytics, all this stuff is great stuff, but sometimes people just-- their life gets in the way. I mean, no amount of data is going to stop whatever is happening in his life that's keeping him from coming here.

[MUSIC PLAYING]

JILL BARSHAY: The experiment was a bust. There just wasn't that much that individual faculty members could do to save one student at a time. But the college says it's had more success using data to identify support services that larger groups of students need. It's beefed up its advising office and tries to get new students to think about which majors they want to study sooner. It offers free online tutoring for all students.

It created an in-person orientation program for online students to start on a stronger foot. But administrators are coming to the conclusion that students aren't usually dropping out for academic reasons. Complicated lives get in the way of school. Montgomery recently opened a food pantry, and now it's thinking about bringing mental health counseling, drug rehab, and housing assistance to campus. The risk ratings don't signal who needs which services.

But a student whose risk score is rising maybe because she hasn't logged in to the computer system for two weeks is providing a clue that something else might be going on and that's an opening for an advisor or a professor to ask some questions and maybe steer the student to the right help. Results are slow at Montgomery. Graduation rates have improved by six percentage points. Now one in five students is graduating on time. Retention has gone up, but the data model still predicts that more than 40% of its students won't return after the summer.

SPEAKER: Raise your hand if you could use an education that's actually modern, actually affordable, and actually personal. If you're one of the 35 million Americans who hasn't finished your degree, well, Strayer is here to help. Welcome to the future of education.

JILL BARSHAY: Strayer is a for-profit university. It enrolls more than 50,000 students. Most of its classes are online. For the students who started in 2009, only one in 10 had graduated six years later. Joe Schaefer is the chief transformation officer at Strayer's parent company. He wondered if there were clues in student data that could help improve graduation rates. He began combing through student records to find what signaled a student would be likely to pass a course. Income, demographics, yes. But you can't change those things.

JOE SCHAEFER: What we identified very quickly is really that about 80% of what predicts student success and that is changeable over time is the student's own effort.

JILL BARSHAY: Schaefer measured effort by monitoring how often students logged in to course websites, how long they stayed logged in, when they completed an assignment, when they posted a comment on the class's discussion thread, how quickly a student read a teacher's feedback.

Along with Civitas Learning, Strayer added up all these clicks each day and gave each student an engagement score. Not only are the students in a fishbowl where administrators can monitor their data, so are the faculty. Schaefer noticed that some professors had more engaged students than others.

JOE SCHAEFER: One of the first faculty members who stuck out was Dr. Michael Curran.

JILL BARSHAY: Curran taught humanities. I found one of the videos he made for students on YouTube.

MICHAEL CURRAN: I thought I'd take a moment here at the beginning of the term and welcome everybody, let you know kind of--

JOE SCHAEFER: We could tell from the data that his students started off at higher engagement levels from day one in the course and then they continued to go up.

JILL BARSHAY: Schaefer studied what Curran was doing. Instead of waiting for the first day of class, he was sending videos to students weeks before the course even started.

MICHAEL CURRAN: A lot of video content. A lot of personal video content. So like I said, you get to know me. At some point, you'll probably see my kids in these videos as well.

JOE SCHAEFER: By the time Michael's students got to class, they'd already received multiple videos from him getting them excited about what they were going to learn in the class. They just came in more excited.

JILL BARSHAY: Curran was speedier than other instructors in grading assignments. And then he recorded personal feedback videos for each student.

MICHAEL CURRAN: Hello, Jonathan, and Thank you for your submission of assignment 2 here, your situational analysis. I'm sure you can see the grade over here, which lets you already nailed the assignment. And you can click on the rubric--

JILL BARSHAY: I asked Curran how he came up with his ideas.

MICHAEL CURRAN: It's kind of a YouTube world now. When people want to learn stuff, they don't really read it in books anymore. They just watch it.

JILL BARSHAY: Curran's instincts for how to engage his students proved correct in the data Strayer executives were measuring. Joe Schaefer.

JOE SCHAEFER: And all of those types of things taken together drive more motivation and a sense of caring the student feels is coming from the faculty member that drives them to give us more hours per week than certain other faculty members.

JILL BARSHAY: Strayer encouraged other instructors to copy what Curran was doing. Schaefer says Strayer wanted instructors to try to engage every student. And so the college doesn't tell instructors ahead of time which students are likely to fail even though Strayer has these predictions for every student in every course.

JOE SCHAEFER: We don't want them to focus on whether they're predicted or pass or not because it feels like it could be biasing. If someone was only 20% likely to pass and the faculty member has to decide where to spend their time, maybe they would feel like there's some threshold where I can't help this person.

JILL BARSHAY: But instructors can see the engagement scores. They're expected to monitor their student engagement scores in real time and boost them. Professor Curran tells students he's watching their online behavior in the class. He shows them the dashboards that color code each student's engagement level.

MICHAEL CURRAN: I'll tell students, you want to be a blue dot. Blue dots do great in my class. Blue dots pass. Blue dots get As. So you want to develop blue dot habits, engaged habits. I mean, the score itself is not-- they don't really need to know, oh, what is my exact score? I'm sure I don't even know the entire algorithm.

JILL BARSHAY: If professors don't know what data is leading to these predictions, I'd wondered what students know. And what do they make of all this tracking of their data? To get an answer to that question, I went back to Georgia State. Banners around campus tout that Georgia State is the second most innovative university in the nation. University leaders around the country gave them this distinction in large part because of predictive analytics. But the data tracking is not on students radar.

ADA WOOD: Most students are not aware that this is a thing and it's taking place.

JILL BARSHAY: Ada wood is an editor at Georgia State student newspaper, The Signal. The student journal is dug into what Georgia State was doing with their data and put out a special issue of the paper.

ADA WOOD: So what you see on the covers is a big building with an eye stuck on it. And we're trying to get this kind of almost like a big brother vibe, like somebody looking down onto the students who are beneath at the bottom of the page.

JILL BARSHAY: The reporters conducted an informal poll and found that most students didn't know Georgia State was using predictive analytics to measure their risk of dropping out. Ada says student reaction to learning that information was mixed.

ADA WOOD: Some said seem sketchy and some said seems very concerning. And then other students said it seemed exciting and efficient. And somebody else said it seems cool and very useful. So it looks like some students understand or think it will be beneficial to them to monitor their success, and others think it's an invasion of their privacy or their data is being exposed or some kind of almost like conspiracy they're afraid of their data being held.

JILL BARSHAY: Georgia State is bound by federal student privacy laws, which aim to safeguard grades and personal information from public view. But Georgia State can share and discuss student data with its predictive analytics vendor as long as it's in a legitimate educational interest. I read through the contract that Georgia State signed with EAB, its data vendor.

EAB stores a lot of confidential student information. But EAB promises not to disclose it to a third party. That means it cannot sell student data to marketers who want to tap into the college consumer market. There's a little wiggle room for EAB to disclose the data as long as students cannot be identified. EAB and other vendors use their customer data to report on national trends in college completion.

Recently, Georgia State has expanded the kind of data it collects on students. In 2018, the University began tracking how often each student connects with campus Wi-Fi and logs into the school's computer system. It doesn't track what the students are doing on their cell phones or which websites they're visiting, just how often they're connecting.

Georgia State calls it an electronic footprint. If a student's pattern suddenly changes, say, the student stops coming to campus or logging into their classes web pages, that can be an early alert that a student is stumbling. It might seem intrusive to track these digital habits, but some colleges are experimenting with taking it a step further. A professor at the University of Arizona looked back at the ID card swipes of first year students when they were new on campus. The local public radio station covered the experiment.

SPEAKER: --to enter offices, check out library books, access laboratories, buy a cappuccino and get back into the dorm. Each beep is recorded in the university's computer system--

JILL BARSHAY: A machine learning algorithm detected patterns of social engagement on campus and matched them to those who didn't come back sophomore year. The professor was able to predict with 85% to 90% accuracy who would drop out based on their first few weeks of activity on campus.

This was a one-off research experiment, but the results were tantalizing and show that personal data contains incredible clues of how students will do. Kyle Jones studies the ethics of predictive analytics in education. He's an assistant professor in the department of library and information science at Indiana University. Jones is concerned about the implications of the Arizona study.

KYLE JONES: They were doing this with good ends in mind. I mean, they did it to develop new retention models and figure out where they could potentially improve services to increase retention. That's a good thing. But at the expense of creating a pretty significant surveillance system.

JILL BARSHAY: For Jones, the ethical problem is that college surveillance or monitoring of students grades and computer clicks isn't being openly and clearly explained to students.

KYLE JONES: Students aren't aware. That's the most simple answer that I can provide. There's really no culture of informed consent when it comes to educational technologies in higher education. These decisions are often made by intermediaries like chief information officers, instructors, and advisors. And these decisions are made on behalf of students.

JILL BARSHAY: Even when students are shown their data, Jones says it's often unclear to them what's driving a red light, especially when the risk warning is calculated using artificial intelligence.

KYLE JONES: The educational technology is often a black box.

JILL BARSHAY: The algorithms can generate a lot of false positives, signaling risk for a student who will be fine or overlooking students who are actually at risk. The for-profit analytics companies don't publish their models and expose them to scientific scrutiny. Jones says that's a mistake.

KYLE JONES: They should require that these things be made available for analysis and that they should prove the efficacy of the models that they're selling.

JILL BARSHAY: And he provides a cautionary tale. One of the earliest uses of predictive analytics in colleges was at Purdue University in Indiana.

KYLE JONES: So researchers behind Purdue's what they called their course signals analytics program, the researchers argued widely and publicly that it boosts graduation by 21%. 21% is a huge claim. But it was a wrong claim.

JILL BARSHAY: Jones says 21% became a widely cited figure that encouraged a lot of universities to buy predictive analytics packages. But statisticians easily debunked it. Nevertheless, the exaggeration still circulates in the predictive analytics literature. Jones is currently conducting research on how predictive analytics is changing the student experience at college. He's skeptical that universities have students' best interests at heart.

KYLE JONES: It's any University's interest that they can demonstrate to stakeholders that students do not leave. And of course, this benefits the institution because it keeps students around to continue to pay their tuition and fees. So to these ends, predictive analytics help push students towards courses and programs in which they are predicted to succeed but are not necessarily well matched with their interests. So this is where we get into the values of higher education.

JILL BARSHAY: Jones asks, what's being lost when algorithms are steering students through college?

KYLE JONES: I always say that higher education is a time of exploration, a place where you start to figure out who you want to become for the later period of your life.

JILL BARSHAY: He argues that data can lead college staff to steer students too narrowly, limiting the mistakes that might have opened unexpected doors. Predictive analytics is probably going to increase on college campuses in the future. Driving the market is demographics. The college age population is predicted to fall by 15% after the year 2025.

That's because Americans stopped having so many babies after the 2008 recession, and the fertility rate hasn't recovered since. Ed Venit is managing director at EAB, one of the data vendors. He says colleges won't be able to rely on recruiting new students to replace all their dropouts.

ED VENIT: So you need students. Where are you going to find students in a market where the market is drying up? You're going to find them on your own campus.

[MUSIC PLAYING]

JILL BARSHAY: So is predictive analytics a good thing? In reporting this piece, I struggled with answering that question. After years of declines, college graduation rates began ticking back up in 2016. I don't know that predictive analytics caused those gains, but it could be a factor. Undeniably, Georgia State is helping more young adults get a college education.

There are other examples too. University of South Florida and Arizona State University are often cited as predictive analytics success stories. There's no way to quantify how many colleges are using data successfully and how many aren't managing to make the data pay off. Even when colleges are raising their graduation rates, it's impossible to know how much of this student success we should attribute to the data.

Georgia State also tripled the number of advisors. And maybe all the students needed was that extra personal attention. There's a risk that low income students will be systematically steered into easier majors when they make minor missteps early on simply because these kinds of students typically haven't had great graduation success in the historical data. And it's hard to get comfortable with the idea of colleges tracking so much seemingly private student behavior even if it is for their own benefit.

On the other hand, the current generation of students is used to having their behavior tracked for advertisers on Instagram and YouTube. Predictive analytics in college isn't so shocking for them. Isn't it better that these young adults move swiftly through college instead of stumbling on their own and dropping out? In the end, it seems that we're moving toward two different college experiences, one for students who go to elite schools and will have the freedom to fail and another for students at lower tier schools who can't afford to explore, stumble, and learn from their mistakes.

STEPHEN SMITH: That was Jill Barshay. She's a writer for the Hechinger Report. Jill's research was funded in part by the Spencer Education Journalism Fellowship. You've been listening to Under a Watchful Eye, How Colleges are Tracking Students to Boost Graduation, a documentary from APM Reports. It was produced by Sasha Aslanian and Jill Barshay with help from John Hernandez and Hina Srivastava.

The editor is Catherine Winter. Fact checker, Betsy Towner Levine. Web editors are Andy Cruz and Dave Mann. Mixing by Craig Thorson. Music help from Liz Lyon. Our theme music is by Gary Meister. The APM Reports team includes Alex Baumhardt, Shelly Langford, Emily Hanford, and Chris Julian. The editor-in-chief of APM reports is Chris Worthington. I'm Stephen Smith.

We'd like to know what you thought about this program. You can send us an email at contact@spmreports.org or you can find us on Facebook and Twitter @EducatePodcast. That's one word. You can find out more about this story on our website, APMReports.org. And there, you can also listen to many more of our documentaries. One is Students on the Move, which reports that the number of kids in the United States who are homeless is the largest it's ever been.

SPEAKER: I was kind of everywhere sleeping in cars, sleeping kind of wherever I could, hotel rooms.

STEPHEN SMITH: You can hear Students on the Move and all of our documentaries by subscribing to our podcast, Educate. Support for this program comes from Lumina Foundation and the Spencer Foundation. This is APM, American Public Media.

SPEAKER: This hour's documentary awarded prestigious first place national headliners award last week. The latest episode in the APM educate podcast takes a look at the upheaval on college campuses caused by the coronavirus pandemic.