August 18, 2021

Citizen Science and the science of gamification

By Conor McKechnie and Dodi Axelson

Citizen Science and the science of gamification

What's the science behind apps like Duolingo™ and Yousician™? Gamification! And what happens when you apply this science to ... scientific discoveries?

Join Dodi and Conor and their guests, Zoran Popovic, University of Washington and Helen Spiers, Zooniverse.


DODI: Hey Conor, you know what one of the most amazing things that happened in the scientific community after the COVID pandemic was?

CONOR: We all have to stay away from each other.

DODI: Yeah. And we didn't have to hug people we don't really like.

CONOR: Exactly. Like all of us introverts could be introverts without feeling bad. Yeah.

DODI: That's right. But seriously, some scientists actually put-up puzzles of the Coronavirus spike protein online, and they gamified a scientific puzzle. This game was called Foldit. And players all over the world could design a protein that binds to the spike.

CONOR: And that's how they were able to what solve the problem of getting to a vaccine so quickly?

DODI: Partly, it helped scientists understand how the Coronavirus spikes work. And it made it easier to set up the right kind of lab experiments. So, for today's episode, I went back to a computer scientist from my alma mater, the University of Washington. Go Huskies. His name is Zoran Popović, and he told us about Foldit and what games mean for science.

CONOR: Brilliant. And I love the fact that like getting people working on puzzles or games together makes you solve problems faster. It's that element of citizen science, isn't it? And that's what Helen Spiers uses, and I spoke to her. She's the biomedical research lead at Zooniverse, which is one of the largest platforms for people powered research in the world. And she and I talked about the different projects that they've got going.

DODI: So, a lot of people can think that scientists are all working, no play. But some researchers are using gamification to solve scientific problems, interacting, getting instant feedback. That's something that we all respond to very well.

CONOR: So how citizens, 'average-Joes', like me and you and games can help change science, is that it?

DODI: That is what matters in this episode of Discovery Matters.

CONOR: So, Dodi told me about the folding of the protein again, and the Coronavirus spike.

DODI: Well how about Zoran tells us about that.

ZORAN POPOVIĆ: We issued a number of different puzzles where we were posing the particular proteins that could have a direct impact on how COVID can be neutralized. My name is Zoran Popović. I'm a Professor of Computer Science at the University of Washington.

DODI: So, what the University of Washington did was really creative and fun. They uploaded different COVID puzzles, and people spent hundreds of 1000s of hours solving these puzzles. And then these users proposed new structures for the protein spikes, which were then tested in a laboratory to confirm if they actually worked.

ZORAN POPOVIĆ: So, in many ways Foldit is serving as the initial starting ground for a lot of the lab experiments, which of course, are a lot more expensive. So, if you can figure out how to explore a smaller space, rather than everything, that's a big win for a laboratory. And that's exactly what Foldit is trying to do.

CONOR: That's brilliant. It's like a win win for scientists and gamers. So why proteins specifically?

ZORAN POPOVIĆ: If you can predict the way in which the protein folds, what kind of shape it assumes, you practically know the secret of life, because proteins are the doers of the cell. Everything that happens in the cell happens with proteins.

DODI: And six years ago, there was an HIV protein that lived on the surface of an HIV virus, scientists were trying to predict the shape of that protein. They'd been working on it for 13 years.

ZORAN POPOVIĆ: They said, 'Okay, well, we're not quite succeeding, let's just see if by any chance, people that Foldit could do something'. 10 days later, there was a brand-new structure for this HIV virus protein that was confirmed in the laboratory. So, you know, we had 13 years versus 10 days acceleration, by way of what happened with this protein.

CONOR: Ten days, that is amazing. The power of gaming. I feel like I wasted too much time playing Doom, and Call of Duty™, and stuff and I should have been playing this instead while I was at university, there's a wasted degree for you.

DODI: Well, and there's more to it. Zoran described another game they've created at the University of Washington called Mozak, the BRAINBUILDER.

ZORAN POPOVIĆ: Some of the exciting things for neuroscience now is that we're looking into a number of different neuropathies. So basically, figuring out you know Alzheimer disease, and many other kinds of diseases that have to do with the deterioration of neural structures.

DODI: In Mozak you build models of brain cells, which in turn helps neuroscientists understand different neural structures in the brain. So, the players trace and classify different neurons in a three-dimensional space.

ZORAN POPOVIĆ: We're basically applying Mozak towards really understanding what is the reason, cause, and structure of those kinds of diseases. And so not just trying to reconstruct neurons, but really help, sort of understanding what it means for an ageing brain to sort of deteriorate and how that can be addressed, once we understand the mechanisms of it.

CONOR: So Foldit and discovering the protein structure in like 10 days, instead of 13 years, that relied on collaboration does Mozak rely on collaboration as well?

DODI: Absolutely. It's like a massive multiplayer online game, you know, like World of Warcraft™ or Fortnite™, which I never played, but people talk about that. But these, you know, in these games, your interaction with other gamers is teaching you different ways of playing...

ZORAN POPOVIĆ: Allowing people to collaborate and review each other's work so that there is kind of an interplay between the neuron reconstructors from all over the world, and research scientists who basically in real time get feedback of 'this part looks reasonable, this part is a little bit suspicious', something that the scientists should see is a reconstruction that many people agree on. So there has to be some kind of consensus built into the game.

DODI: And I think this makes an awful lot of common sense. The reason Zoran wanted to get into gamification is because he believes it will change the way we teach and engage with science. You know, there's this idea that you have to spend 10,000 hours doing something before you can learn it really well.

CONOR: Yeah, the jury's out on that though, there's some data that says it might not quite be true, despite Malcolm Gladwell being you know, a really marvelous read. But if you do practice something for a long time you become an expert. I wonder whether it isn't more about the gaming mentality of wanting to beat the final boss. So, every time you fail, you just go again, that's how gamers think.

ZORAN POPOVIĆ: Well, where did this idea come from? If I was practicing to play violin, right, and I practice with a master, back in a 14th century, that probably makes sense, you know, 10 years, 10,000 hours. But what if I had a computer system that understands exactly what drives me? What are my strengths? What are my weaknesses, and constructs the experiences over time, in such a way that every new experience makes me significantly better than what I was just five minutes ago? In that particular structure, I could become an expert, hopefully a lot sooner.

DODI: So, the way we teach has been the same for so long that teacher-student relationship and so on, that has been going on for centuries. But you're right, something happens in the brain when we play games. Zoran says that is something intrinsic in human structure in nature.

ZORAN POPOVIĆ: To sort of be compelled to do things and to become better through rapid feedback and getting these slight jolts of endorphin every once in a while, where you're feeling good about partial accomplishment. In many ways, the game is a perfect vehicle for sort of this kind of lab rapid learning. So obviously, you know, if you look at younger kids, they learn through smaller games, whether it's, you know, the toddler dropping the plate on the floor, and seeing whether it breaks or not, or whatever. Everything is a little mini game, sometimes with more costly consequences than others. But in many ways, we're always having this rapid feedback of, 'Hey, I'm going to try a whole bunch of these things and the reaction of what happens from it is going to give me some sense of pleasure'. And if I keep doing this, over a period of time, I'm sort of built to learn in that way.

CONOR: Ah, so it's about dopamine bumps every time your kind of like beat one of the bosses. You get a dopamine bump.

DODI: We are so happy when we play games.

CONOR: So, when you're playing Zelda™, and you've gone through the whole temple fighting monsters, and you finally win, there's a treasure chest and you're like you're flooded with endorphins. I know the feeling. And then you've gotten like a new shield or hearts or something cool like that.

DODI: Exactly. I spend way too much time on Words with Friends™, and they have recently redone the reward system so that every time I play like I'm getting some sort of reward, it's incredibly gratifying and addicting. Zoran says that the current educational system isn't taking advantage enough of this kind of gamification mindset for learning.

ZORAN POPOVIĆ: Right, we have this sort of theoretical lectures where you're supposed to ingest knowledge by hearing somebody, and then all of a sudden understand it and be able to apply it to solve creative problems. That doesn't happen. That's not really how we learn. So, the key point of games then is to go from this glacial speed, the feedback of lecture, do some homework and maybe get better over it to something within seconds.

CONOR: And actually, I see my kids doing this already. They gamifying their own learning, as they review for examinations, right? They build themselves little games.

DODI: Mine too. So, the more frequently you get feedback, rewards, those feelings of progress the faster you're going to learn. So, Zoran and his colleagues are trying to make that feedback loop go faster.

CONOR: So, learning by doing and learning by gaming, what's not to love?

DODI: Yeah, exactly. And it turns out, these games have been really successful in teaching completely new people who tried to do protein folding or reconstruction of neural brain structures for the first time. It's like that person he described who wants to learn how to play the violin...

ZORAN POPOVIĆ: ...And in a matter of months, not only become very good at it, but publish discoveries in Nature and Science, you know, the most eminent scientific journals in the world, with discoveries that didn't exist before, you know, so what better example that it is actually possible to go from ground zero not knowing anything to be able to contribute, and to break the boundaries of science within months.

CONOR: Brilliant, just breaking the boundaries of like doing science by doing science in a different way and of course, making it more fun and teaching people to engage with science, the citizens, don't we know that we need that.

DODI: Making it accessible.

CONOR: Yeah.

DODI: That's right.

CONOR: So, this is a little bit like Zooniverse, which doesn't gamify, but it is another platform for collaboration in citizen science and one of their projects, which I love is called Etch-A-Cell.

DODI: Oh, that sounds cool.

CONOR: I know. It makes me think of that kind of etch-a-sketch game. Yeah, a digital drawing on the screen. So, what do you know about Endoplasmic Reticulum, Dodi?

DODI: Oh, my goodness. So that sounds like the gooey stuff that ghosts are made out of in GhostBusters™.

CONOR: 'He slimed me!'

DODI: Yeh, exactly.

CONOR: Ectoplasm, yeah, brilliant. No, it's not ectoplasm. We're not looking for ghosts. It's Endoplasmic Reticulum, and I'm going to send you a link, can you just open this?

DODI: Let me try. Okay, I have opened the link and I am looking at a black and white image of something that looks, I don't know, like the surface of another planet.

CONOR: It’s not actually another planet. It's an electron microscope image of a cell, and your job as a Zooniverse citizen science person is to find the worms or little blobs in the image, trace around them with the pen. And those worms are what the scientists are really looking to plot here, the Endoplasmic Reticulum or the ER. They're the largest structures in our cells. So, while you do all the colouring in, why don't we listen to Helen Spiers, tell us what is going on in this project.

HELEN SPIERS: It's a huge challenge in terms of how rapidly technology is progressing within the domain of volume electron microscopy.

CONOR: This is Helen Spiers.

HELEN SPIERS: I am a biomedical research leader for the Zooniverse citizen science platform, and I also lead the developments of the Etch-A-Cell project at The Francis Crick Institute.

CONOR: So, Helen and her colleagues at the Francis Crick in London, focus on using these highly sophisticated microscopes using volume electron microscopy to zoom in on the cells at nanometer scale and visualize them in 3D.

DODI: Ever since we talked to Molly Stevens, in our Discovery Makers miniseries, I have been fascinated by this tiny, tiny little world.

CONOR: You notice so many big things happening in such tiny, tiny, tiny places. I love that too.

HELEN SPIERS: So, we're taking an organelle-by-organelle approach, the first organelle that they focused on was the nuclear envelope. That project we've now managed to bring to the point where we've published a preprint that came out last year.

CONOR: So, they take the data that all the volunteers like you are doing right now, doodling by drawing around the organelles or segmenting them. And that data is then used to train machine learning algorithms. So, the computer can then learn to do this by itself in the future.

HELEN SPIERS: A task that would take members of the research team about 30 hours to manually segment the structure within a single cell, it now takes the trained model about a minute to make a prediction on this structure. About a minute we'll make a prediction on the structure.

DODI: I think I finished drawing the slime.

CONOR: The endoplasmic reticulum, Dodi! Not the slime.

DODI: Right, not the slime!

CONOR: So, you finished with the image hit done.

DODI: Done.

CONOR: Now you're a scientist bona fide.

DODI: Finally, finally.

CONOR: So, look, maybe that's not all it takes to become a scientist, but at least you've contributed to finding out what these ER look like in cells. And you and 1000 other volunteers in the Zooniverse have helped researchers to train their computers to do this segmentation automatically, which then can help them understand and study disease.

DODI: So that feels really gratifying. I've just played a game and done something smart.

CONOR: Let's just look at the numbers of what this enables. Right? They've got 50 different citizen science projects going on 1.6 million registered users. And together they've done more than 584,000 classifications in all of these different projects.

HELEN SPIERS: So, it started with the first project, which was Galaxy Zoo, back in 2007.

CONOR: So funny that we very often look at pictures in the life sciences, and we think that they look like space, right? And that was the first generation of Galaxy Zoo. It started with images from galaxies that had been taken by the Sloan Digital Sky Survey.

HELEN SPIERS: The research team that was analyzing/categorizing those galaxies were having to do it by eye. So, processing that data and describing the features that they were interested in, the morphology of galaxies.

CONOR: And Galaxy Zoo was this massive success on the day after launch, they were getting like 70,000 classifications an hour.

HELEN SPIERS: The result was that they produced the largest ever census of galaxies that have been produced.

CONOR: Zooniverse projects aren't just about medicine and physics. They have a history, social sciences...

HELEN SPIERS: ...nature observations, bird observations or species...

CONOR: ...biology, zoology and so on. There's even a project about transcribing handwritten pages by Shakespeare. I mean, can you? So, I'm going to send you a link. Can you read what it says here?

DODI: I am clicking on it. Shakespeare's world, Shakespeare's world talk. Oh, my goodness, the handwriting is so beautiful. We don't write like this anymore. It's really swirly. And let me see if I can pick up some words 'hereof', 'John'. That's about as far as I can get, this is difficult!

CONOR: Its madness, isn't it? It’s like a mixture between sort of Arabic and hieroglyphics.

DODI: And it's beautiful!

CONOR: People actually derive meaning from that, and we hardly write handwriting anymore at all right? What a loss.

DODI: So, this would be hard to figure out on your own. But maybe between the two of us, we can start to understand it.

CONOR: And they have talk forums where volunteers can ask questions about classifications and interact with the research team. But there's one more reason for having forums.

HELEN SPIERS: We've also had a lot of discoveries that have emerged from volunteers asking questions about unusual images that they've encountered, and they've seen something odd. Having that opportunity for collaboration and dialogue between different volunteers and research teams means that those serendipitous discoveries can bubbled to the top.

DODI: Serendipity, here it comes again!

CONOR: Exactly. So much seems to be focused on that. Helen told me that it goes well beyond people's brain and computing power, that they're tapping into their creativity and their ability to ask questions about the data.

HELEN SPIERS: One of the really unique and wonderful things about working with people on these sorts of tasks, you wouldn’t get that necessarily with a random stranger. This is interesting.

DODI: Can you imagine a computer saying, 'this is fascinating'?

CONOR: Yeah, exactly. So, you know, it’s the wonder that maybe isn't quite replaceable? Just yet.

DODI: You know, and it's like Zoran, from the University of Washington again. He said, computers are good at predicting the shapes of proteins, but when it comes to drug design, that task needs human creativity. Now they're using Mozak to completely reconstruct an entire mouse brain. Doing that means they have to train the machine learning algorithms to do just those small tasks.

ZORAN POPOVIĆ: How to reconstruct as many neurons in as much as accurately as possible, so that the algorithms can start doing more of this stuff, and let people do the most exciting most complicated things, while computers do the rest? And that's our only hope, of reconstructing hundreds of millions of neurons at the same time.

CONOR: Humans and machines living side by side, one day will invent the last machine we need to, and they'll do it all for us, right?

DODI: Using game. Yeah, that's right.

CONOR: We thought we were playing games, but no, we brought on the singularity. Oops!

DODI: Well, we are in a game.

CONOR: Are we really going to go there now? Are we going to have the simulation conversation now?

DODI: Yes, we are. No, we're not.

CONOR: No, we're not. So, this isn't quite gamification. Zooniverse doesn't use the game mechanisms. Helen told me about a paper that looked at whether gamification was always a great idea in citizen science.

HELEN SPIERS: They interviewed volunteers who had been contributing to a project called “Old Weather”, who had used some gamification in their project. I think it was, if you were the person who had contributed the most classifications associated with a particular ship in the project. I think it was weather logs associated with ships. Then you got to be the captain of that ship. Regardless of the gamification, the long and short of it is, they interviewed volunteers. And while some people had been motivated by the gamification, other people found it off-putting. There was a psychology, where some people found that it was demotivating, because they would come to a project – say if they were a volunteer that hadn’t contributed previously – and they would see these leaderboards and feel that they could never rise to the top of it.

DODI: So maybe gamification might not always be brilliant in science, but it can make a difference when trying to learn new things for all students, for all children.

ZORAN POPOVIĆ: Not necessarily, you know, disrupt the entire school system by just throwing away classrooms, everything else, but really figure out how these kinds of mechanisms of rapid feedback can be used in standard way of learning such that any child anywhere, no matter what their socio economic situation may be, can become really excited or interested in science, because instead of being told that they're not good at this, or because their classroom or structure is not conducive to their learning, with this rapid feedback, realize, 'hey, I can do this, hey, I'm really good at this, hey, I want to know more'. And before you know it, they're not just learning about how to multiply numbers, but they're actually actively participated in the scientific discovery project. If you can increase the base, if hundreds of million kids that basically are learning every day, and of course, there's more kids come here every day, there are more kids born every day, if we can have them learn whatever they want, as a result, with this new kind of learning new kind of discovery, we will have a completely different scientific force over the next five to 10 years.

CONOR: So, gamification, and citizen science, for me are really important because it gets people more engaged in science and the more people that are engaged in science, the more people understand the output of it. And we, you know, we fight against these pernicious forces of anti-science thinking, conspiracy theories, and so on. So, it's really, really important. And you got to etch your first cell.

DODI: I did! Every day's a school day, I have done something new and learned an awful lot and I had fun and I got that reward. You know? Here's the reward.

CONOR: Absolutely. So, a little bit of a dopamine bump. What can we say? So why don't you give us a little bit of a dopamine bump? Go and rate us wherever you are? Wherever you cast your pods or pod your casts. Give us a rating. We'll see you next time.

DODI: Thank you for listening. Our executive producer is Andrea Kilin. Discovery Matters is produced in collaboration with Soundtelling. Production: Tanvir Mansur. Our theme song was written by Thomas Henley and additional music is from Epidemic Sound.

Listen to more podcast episodes.