In this December 2021 episode of the ‘Meet The Education Researcher’ podcast, Neil Selwyn spoke with Mark Warschauer (UC Irvine) about his work exploring the intersections between digital technology and language learning, alongside issues of social inclusion and digital education in countries such as India, Egypt and Peru. (nb. this text has been lightly edited for clarity)
*
[Neil]
It is always interesting to reflect on how our academic research careers turn out. So, given that you could have chosen to research just about any aspect of education, what is it about the topic of digital technology and education that has kept your attention for such a long time?
*
[Mark]
Well, it all started when I went to Hawaii for graduate school. And I was walking down the road and I saw a sign on campus it said ‘Free surfing workshop’. So I thought, ‘Hmm, I always wanted to learn how to surf’. And I went there, and it turned out to be a web surfing workshop. So that was back in the ’90s and we were just getting started on all this. But, all kidding aside, I think what I love about digital learning is how it’s always changing. I’m a pretty intuitive person, and I’m a person who likes to do lots of different things. And I like the fact that the digital learning landscape is always changing. I like staying on top of things, I like trying to figure out where it’s going, what’s new, what’s interesting … and what’s not going anywhere. So, it fits my personality well, I think it keeps me young and it certainly never gets boring. So I’ve really enjoyed it.
*
[Neil]
I find that as well. The biggest secret of digital learning research is that you can actually research any topic you like.
*
[Mark]
I know! If I was stuck studying a series of poems by Shakespeare from several hundred years ago, and just having to dig deeper and deeper into the same texts for my whole life, I think I’d die.
*
[Neil]
Ha! But that said, the fact that technology is always changing means there’s always a continuous hype cycle as well. So, I’m really interested to look back to when you first started … what was the hype around technology and learning in the 1990s? And how have those initial areas of hope and hype played out over the subsequent 30 years?
*
[Mark]
I think the hype has been pretty consistent, which is that technology is a ‘game changer’ … Once you bring in technology, everything you knew about language/ children/ society/ learning gets thrown out because technology is the ‘magic bullet’! Put those computers in the classroom, stand back and watch the game change. [Pause] But, you know, it just doesn’t work that way! Just to share some of my experiences over the years … when I was at University of Hawaii in the ’90s – I was writing some edited books collecting experiences of what people were doing with computers. And I met some university French teachers from Stanford and they told me that digital media creation was transforming how they were teaching French. So, I went and observed their classroom. I saw a first semester French class and, yes, students were sitting around excitedly developing a PowerPoint-type presentation. It was a big thing at the time to be able to actually create a multimedia presentation. But the fact was that for most of the time the students were figuring out how to use the software and talking to each other in English. They were spending several hours talking to each other in English to create a software that maybe had two sentences or French in it. This was instead of what people should be doing in a first semester French class which is immersion, listening and starting to use the language and develop communication skills. So the programming itself became more salient than the language learning. I think that that’s a big problem with the ‘game changing’ phenomenon. What exactly are you changing about the game?
*
[Neil]
… and a lot of this hype also relates to the idea of social inclusion, social equality and widening opportunities … you’ve done a lot of research around these issues. Can you tell us about some of those projects?
*
[Mark]
Later on, I did a small research project on the ‘Hole-in-the-Wall’ in India, which was supposed to support ‘minimally invasive education’. They put computers in the wall and these poor slum kids were supposed to come up and teach themselves from it. And the researchers wrote lots of papers, and got lots of money and awards. But when I went there, the computers were broken-down, the internet wasn’t working … the computers that were working were dominated by a few boys playing really simplistic games – certainly not the kind of games that you learn anything from. And I interviewed the parents who were complaining that their kids had stopped doing their schoolwork because they were just playing games there after school. And this goes on and on. And, of course, later, I carried out work on the ‘One Laptop per Child’ program, which was a lot of the same thing. I think the hype has always been that you can solve complex educational social problems by just throwing computers at them … and now it’s iPhones, tablets, VR, or whatever the new technology is. And that approach has never worked.
*
[Neil]
So, why are we so still willing to accept these claims of technology being a game changer? As you say, there are so many lessons that we should have learned over the past three decades … why do people still fall for this?
*
[Mark]
Well, that’s a good question. Morgan Ames – up at Berkeley – wrote a really good book on the ‘One Laptop per Child’ program. She analyses a lot of the people who have been behind this program, and they tend to be upper-middle class white men who grew up in very privileged conditions mostly in the United States, but probably a few of them also in Australia, England and Canada. And, you know, these people thought they had been able to teach themselves with computers, but they actually started with a lot of literacy skills, a lot of math skills, a lot of science skills, a father who was an engineer, a neighbour who was a computer scientist, and, you know, opportunities to go down to Hewlett Packard around the corner to take some free workshops. So, then these people project all this on to the rest of the world. And they don’t understand what it’s like to be a kid in Peru who doesn’t have electricity in his house, and is poor and illiterate, and his family is illiterate. And what that kid can do when you throw a computer in his lap is a lot different.
*
[Neil]
So it’s all about context. I think when Bill Gates said that ‘content is king’, he should have said ‘context is king’.
*
[Mark]
Exactly. Context is king.
*
[Neil]
But … a lot of people remain keen to talk about the fact that technology inherently improves learning. So, what is your take is on that assumption. Is this even a question that’s worth answering or researching?
*
[Mark]
No! … I think the best response to that was given by Chris Dede from Harvard, who said that computers aren’t like a fire that generates warmth. So you don’t just sit around a computer and ‘learn’ similar to the way you might sit around a fire and get warm. Computers are more like clothes – they have to fit right, they have to be the right material, they have to be appropriate for what you’re doing with them. You know, sometimes journals contact me and ask me to review work by somebody who has done a meta-analysis of technology and learning. I mean, all technology! All learning! All ages! So, I say no – I’m not going to review that paper – it’s worthless. If we really open up ‘technology’ to include books, pencils, whiteboards, paper, libraries … everything that’s really a technology, then it becomes clear how absurd asking that question is.
*
[Neil]
Absolutely … but you have done a lot of research on technology and language learning. What insights have you actually gained from your research there? What have you been able to find out from studying language education about the ways in which technology comes to interact with what is a very complex form of learning?
*
[Mark]
Thanks for asking that. That’s a great question. I started this research the early 1990s. And there was a notion from the field of Computer Assisted Language Learning that language was a stable thing and that the computer was something that instructed kids. But, from the very beginning, the students I interviewed in a lot of different contexts made me realise that it wasn’t so much that technology was a way to learn this stable thing ‘language’. It was the fact that technology and language were intertwined together in ways that were really important for people’s lives. So, technology was actually changing the way that people were usinglanguage. And it was really important for their life’s purposes and goals to learn those new ways. So, this suggested a whole different understanding of how technology and literacy were intertwined. That’s been an important thread that’s kept with me through my professional life
*
[Neil]
Yes – languages are obviously a really important part of culture … and technologies are a very important part of culture. And as you say, you can’t really separate the two apart. But technology has moved on a lot since ‘Computer Aided Instruction’ and those first language packages that you were looking at. You’ve subsequently done some really interesting recent projects on the topic of conversational agents. This is an area of technology that fascinates me. So first of all, for the uninitiated, can you explain what a ‘conversational agent’ is and how this software is being used to support learning?
*
[Mark]
Sure, I think most people listening to this have probably used Siri by Apple, or Alexa by Amazon, or Google Assistant. You know, ‘Hey, Google’. So those are all conversational agents. And some of the technology for real-time chatting has existed a long time … but only in textual form. So, the ability to support real-time chat with speech is what makes these technologies ‘Conversational Agents’. This is where the breakthroughs have occurred over the past few years. For a long time, I was working on things related to writing. But, as we discussed at the beginning of this conversation, technology changes. And that’s one thing that’s exciting about it. All these developments in Conversational Agents have created a lot of new possibilities related to speech.
*
[Neil]
So going back to this idea of hype, what actual future potential is there for Conversational Agents? What do you see as the potential benefits and what are the clear limits? To what extent should we be allowing ourselves to get excited about Conversational Agents?
*
[Mark]
Well, let me tell you about some of research we’ve done – I think that that might help answer the question. One of the things about Conversational Agents is that it’s easier to work with them in constrained environments rather than unconstrained environments. So, we know that young children learn best from reading when it’s shared reading with an adult. And it’s kind of the same thing with kids watching television – they learn more from watching a science television show if they’re co-viewing with an adult who’s asking them questions. So, we took advantage of that – programming Conversational Agents to simulate the role of a reading partner or co-viewing partner. Kids will be looking at the pages of a book and the smart speaker will narrate the book to them, but then stop and ask them questions. And then it might dialogue a little bit with them on their responses.
We are working with PBS Kids – which is a big public television network here in the United States – and they create a lot of the really popular kids science TV shows. So, often in these shows the characters – like Dora the Explorer, Mickey Mouse, or whoever – will ask the kids a question to spark their interest. We’re programming the technology so that you watch it on an iPad or laptop, and the character will ask a question … but we’ll understand the kid’s response and then dialogue a little bit. So, because it’s a constrained environment, the Conversational Agent is initiating the conversation. And then we can anticipate what kinds of responses will come up, and then program the dialogue that will flow out of that.
We find that kids do learn about as much from these episodes with a conversational agent as they do with a human. But there’s some interesting differences in the conversation. If it’s a simple question then children give short answers to both humans and the computer. If it’s a complex question, they do tend to give longer answers to the human … But this is with scripted conversations. If the conversation was unscripted it’s certainly the case that the human would be able to follow up on the child’s response, take them in different directions. There would certainly be a great deal more flexibility.
The other thing that was interesting from this research was …well, who do you think children spoke more clearly to – the human or the conversational agent?
*
[Neil]
… I imagine the conversational agent …
*
[Mark]
Right! Somehow, the children were able to understand the fact that they had to speak more carefully to the computer, and so they spoke more clearly.
*
[Neil]
A lot of people would see that as an example of the ‘reduced bandwidth’ and the ‘dehumanization’ of tech-based education. In your experience, do you think there’s a notable difference between talking with a Conversational Agent as opposed to talking with a human? And, if so, is any loss offset by the benefits – particularly in terms of equality of opportunity?
*
[Mark]
You know, the gold standard is the human. In our research we had a very carefully scripted thread – but a good human partner can lead much richer conversations. So it’s like automated writing evaluation (which I also did research on before), which I also think is useful. In fact, we had a paper with one of my graduate students which was called ‘Automated Writing Evaluation: the Utility of a Fallible Tool’ – meaning, of course, that the technology is very fallible but it can still be useful. And I think a Conversational Agent compared to a human is very fallible, but it can be useful. I think it’s useful in a couple of ways. One of the things, at least in the United States, is not all kids have equal access to rich reading experiences with well-trained adults. Kids from low income or working-class families might have parents who are too busy, or they don’t know about joint reading, or they’re illiterate themselves.
But, we’re not trying to replace humans! So, our next stage of this research is to focus on going from dialogue to trialogue. How can we use Conversation Agents in situations where there’s a child and a parent? On the one hand, we will ask questions of the child and hopefully model good questioning that the parent can look from learn from. But, after we’ve done that, the technology will then provide some conversational starters for the family. Okay, you talked about what Rosita was cooking in this story – what do you guys like to cook together? What was the last thing you cooked together? What’s your favourite recipe? So, we’re trying to offer the best of all worlds – not to replace humans, but to complement them and stimulate them.
*
[Neil]
… and there’s a lot of smart pedagogical design going on in the background. So we need to get away from the idea that technology is somehow doing this for itself.
*
[Mark]
Sure. That’s part of what we were talking about earlier – it’s all about the context. In the classroom it’s the context of the teacher and the student. One of my favourite mottos from my big research project on laptops in schools was ‘Laptops will make a good school better, but they won’t make a bad school good’. And I think you could also say the same thing about instructional designers, technology designers, and software designers. I think we are having some good results with this Conversational Agent project because we know what we’re doing. We have experts on science, learning, literacy learning, etc., who are designing the conversational flows. That’s the challenge! Most of the companies who are going to be producing this stuff are not motivated by principles of good learning – you know, they’re commercially oriented. So we want to get involved in the educational AI game, and try to steer it in a positive direction.
*
[Neil]
So there is still a role for expert educationalists – this is nice to hear! Moving on to the present day, then, my attention was taken by one of your new projects on ‘Elementary Computing for All’ – looking to explore ways of integrating computer science into literacy instruction. What is the goal here, and how are you setting about it?
*
[Mark]
The goal is to get all kids involved with computational thinking and coding computer science. A lot of people bring up the economic argument about coding, which is true. I mean, computer science is becoming central to many fields of work. But it goes well beyond the economic and all the way to citizenship. Having a basic understanding of how computers work is really valuable for being a full productive citizen in today’s world. But computer science one of the most divided areas of education. Typically, computer science classes are offered at high school. And by that time, kids have already been self-sorted into the ‘techies that can do this stuff’ and the rest who think ‘we don’t know anything about this’. And this stems from kids’ experiences of expensive computer camps, robotics kits, their parents, their neighbours …. and especially for English language learners.
There’s correctly a lot of focus in the United States on developing language and literacy skills. But those skills are often pitted against other things – you know, the idea that ‘they can’t do computer science, because they’re not experts in English yet’. But what is computer science? What is coding? It’s a language! And what our kids in the project do is use the visual programming language Scratch. So Scratch was designed by people at the MIT Media Lab for young children where they don’t have to type in the words themselves, but they drag in blocks, drag in pictures and other things. And what do kids do with it? Kids tell stories – they tell stories about their lives that are about cats, about music, and what they’re interested in.
There used to be a belief that people could only learn one language at a time. And if people were Spanish speakers then we had to push away Spanish so that they could learn English. But students soon realize that when they are learning one language they are also learning general concepts of language. Getting strong in one language helps with other languages. And actually, there’s advantages of learning multiple languages … and coding is just another language. So, that’s what our curriculum is all about – basically looking at the language aspects of coding – having kids program stories in the same kind of genres that they’re also learning to write about. We’ve been teaching them the discourse of computer science, which gives them a lot of opportunities to talk to each other. We’re having them read stories about diverse computer scientists, and having in-depth PD for teachers, so that any elementary school teacher could do it. So coding is no longer just for the privileged kids or just the ‘Hour of Code’ that kids do for an hour a year. This is something that we can get into all elementary schools, and we’re excited about it.
*
[Neil]
And this project is bringing together all the big previous threads in your work that we’ve just talked about … literacy, language, social inclusion, learning …
*
[Mark]
Yeah – it’s fun. That’s been the fun thing about this career. You go through a lot of twists and turns and iterations. It’s really fun to see how it all comes together in different ways and indifferent places.
*
[Neil]
A couple of more general questions to finish off. First off, you head up a very successful ‘Digital Learning Lab’. The ‘lab’ model isn’t as common in Faculties of Education as it is in i-Schools … how does the lab model work, and how difficult is it to keep everything going? It seems like an awful amount of work for the Director!
*
[Mark]
Well, yes and no. I mean, one of my greatest skills is finding the right people and getting out of their way! Now that’s kind of an exaggeration, because obviously I’m not getting entirely out of their way. But it is a case of finding the right people, and giving them space to work together and to learn from each other. So, to begin with started things off with our undergraduates. It’s very easy at UCI to recruit undergraduates who want to get involved in research. And you typically don’t require any money or expenses to do that – you can set up directed research labs, or independent studies, or credit experiences. And that’s a great source. Of course, I have a good PhD program and having graduate students really helps when you start getting more money – then you can hire postdocs. But all this was a progression – it started small and it grew bigger and bigger.
But I think we have enabled it to grow bigger by delegating to other people, respecting their initiative, having very open practices, not feeling everything needs to be closed down. Instead, we welcome collaborations, sharing data, sharing authorship, and so on. And you know, this is actually the way that most STEM fields are going and have already gone for many years. In education – as you point out – thing are not typically done that way, but I think that it is the way of the future.
You know, sometimes it comes up when we’re reviewing CVs in Education – where’s the first-authored work of this person? Or where’s the sole-authored work? How can this person be a serious researcher without sole authored work? In chemistry or physics, if you’re going up for tenure and you have sole-authored papers then people would look like you are crazy! Who’s checking your work? Where’s the team that’s holding you accountable? So I think Education research has to move away from the idea of the brilliant individual, and instead value the ability to work in teams. I think that’s the way of the future.
*
[Neil]
Well, my final question relates to that. What you’ve just described promotes a deliberate open ethos, and I’m really interested this idea of open practice. You’re the editor of AERA Open, which is a bold venture for established organizations like AERA and the publishers Sage. How’s the journal been going so far? And what future do you see for education publishing and research in this era of open access, post-publication review, and everything else?
*
[Mark]
Well, we’re just stepping down after six years, so it’s actually a good time to take stock of where we’ve been. We’ve been pretty successful – we’ve published about 450 articles over the past six years. We’ve got good metrics in terms of Impact Factor and all that. We were selected as the best new journal in the social sciences by an organization here in the US. But it’s the ‘open’ parts of the journal that I’m most proud of. Our articles were downloaded nearly a half-a-million times in 2020, which for something that’s trying to have an impact on education is really valuable. We’ve really pushed hard on this idea of data sharing. For a lot of people this is not part of the ethos in education – and sometimes for good reason. I mean, there’s a lot of privacy issues – we’re collecting data from schools which may not be excited about having the data shared. But if you de-identify data, if you anonymize data, if you set it up early on in terms of the ways that you get permissions, then it can be done. And so we have set up resources for people on how to share their data. We’ve been communicating the desirability of that, although we haven’t yet required it. But just by strongly encouraging it we have had three-quarters of all our authors publishing their data – which I think is a great breakthrough.
The other thing is that we’re online only – not in print. I go to the AERA journal meetings, and I see a lot of the other journals being obsessed with, you know, ‘We have we have so many articles in each issue … so, we can only publish so many articles a year.’ So they’re looking to take only articles that have positive effects, or only articles with significant effects, or only articles on exciting topics for the reader. Our journal is trying to break that mold, and make more use of criterion reference – is this rigorous educational research? If so, it will be published. Now, where do we go from here? You know, we’re only one-tenth of the way there. Our author fees are low, but we still have author fees which I regret. How do we find a way to break the stranglehold of the big publishers that have open access without depending on the authors? The review process is getting more and more time-consuming and laden and complex. You mentioned the possibility of post-publication review. Imagine that the whole journal system that we have right now didn’t exist, and we were starting from scratch. I don’t think we would come up with what we have now. So I don’t have all the answers regarding where it goes and how we get there. But I look forward to being part of those discussions.
*
[Neil]
Well, massive congratulations for getting it all off the ground. It’s a brilliant venture …so, Mark, thanks ever so much for taking time to talk about your work. It’s all genuinely fascinating and it’s been great to meet you face-to-face … in a way! I look forward to reading much more of your work in the future. Thanks.
*
[Mark]
Thank you. Thank you so much.
Critical Studies of EDUCATION & TECHNOLOGY 