[Damien Blenkinsopp]: Hey Yoni, thanks so much for coming on the show.

[Yoni Donner]: Thanks so much for inviting me. It’s a pleasure.

[Damien Blenkinsopp]: Great. So, we before we get into it, I wanted to hear a little bit about your first got involved in your interest area. How did the Quantified Mind come about? Was this a mini project? Was it something you were interested in, at first, doing for yourself?

[Yoni Donner]: Yes. It started with a pretty different context. I always wanted to cure aging since I was pretty young.

[Damien Blenkinsopp]: Nice.

[Yoni Donner]: And I still do.

[Damien Blenkinsopp]: Yeah, I’m there.

[Yoni Donner]: Turned out to be a somewhat more difficult problem. But no, the original thought was that, perhaps, the most accessible part of aging that we can start working on is the aging of the brain. Especially since that sort of leads everything else and there were all of these reports of these things that might be useful.

And since I had lots of friends who were interested in this, and they often annoyed me when they massively consumed blueberries because they thought they would help, I really looking for some scientific backing of all these potential interventions. And it turned out there isn’t actually much. Most of the stuff that goes into newspapers is completely not validated, and we wanted to do it ourselves.

So, I just looked at “what can science tell us about how to measure cognitive performance,” and it turns out that they’ve been very good at measuring the differences between people, but they’ve done almost nothing to measure the inter-individual variation.

So, it’s very hard to compare a person to themselves under many different conditions for several reasons. So, the tests were not built for this purpose at all so I kind of have to adapt them to make them test the person several times. And also they’re very inefficient so sometimes you take a long time to get the data or you need a psychology student to get the data for you.

And it was a disappointing finding that I realized, if I really want it done, I have to build it myself. Especially since I’ve never really enjoyed building websites or anything of that kind. I really just like analyzing data and writing over convoluted algorithms.

But with some help from some friends in the first stages, we got this going, and since then it has actually been mostly used for other purposes. Namely for people to test more acute interventions rather than long term processes. And that’s fine. Whatever is useful is great.

[Damien Blenkinsopp]: So, you’re using it for the long – are you still using it for the aging? You want to maintain, or you want to improve over time, your brain responses?

[Yoni Donner]: There are several stages. The first one is simply to validate that we can accurately track the process of cognitive decline. This would be the control.

Once that is established, we want to start looking at the most promising interventions. So whether they be physical exercise or even jogs. Although I’m not generally a huge fan of jogs, simply because they seem to have more side effects than positive effects. But yeah, so we did make some progress.

Actually, only the last year, I finally managed to get some collaborators to recruit subjects and do a longitudinal study within person aging. So that is all very new now. Now, this is at this stage of trying to get more researchers for a slightly longer pilot study because this was sort of to establish the methods and verify that we get clean data and so on.

What I can say is we definitely have seen, on the data that’s been collected randomly with people who opted in to provide their age, that the classical effects of aging are very, very clearly seen. So, at least in that sense, all of the stuff that we’ve known has been replicated.

But now I mostly rely on working with researchers who can actually do the interventions. So it would be great, for example, to do a caloric restriction study in humans.

[Damien Blenkinsopp]: Right, and see if that has an impact. But I’m guessing you’re going to have to do this over five years or something to see anything significant?

[Yoni Donner]: So, that is a great question. So the whole point was to not wait five years. So Quantified Mind was built to be so precise that we could actually see effects given a reasonable sample size with a much shorter amount of time. Because, obviously, no single individual will reliably show decline over two months even if they’re over 70 or over 80 years old.

But it’s an average effect that is quite strong. So there is some effect size that you could say over a month of your life there is an average decline. It’s probably very, very small compared to, for example, random daily variation. That’s why, if you have a sufficiently large number of measurements, you could actually see that.

And I have a lot of data by now about the accuracy of this – how well [unclear 10:06] measures the actual observability. So, the reliability of the test is actually very, very high. And you got almost zero noise in the measurement itself so you’re just fighting against the noise in the actual cognitive function. So really people would vary more day to day more than they would on average over a month. But that’s fine since we average over that eventually with enough data.

[Damien Blenkinsopp]: Right. Well this is really cool because I didn’t know that this was the original purpose of it, and actually, this is what I’ve been interested in lately. I’ve been interested in mild cognitive impairment and I talk about cognitive decline.

And I’ve actually had some scans showing some nasty structural changes due to something I went through a couple of years ago. So I want to kind of repair that and get it back up to speed. So, it’s one of the reasons I’ve – and when you’re say anti-aging, you really think about the brain.

I’ve also thought, for me, the two areas that seem most important to me are energy and the brain. Brain so that we can carry on thinking and walking around. And if you don’t have energy you can’t really get stuff done either. Because once your productivity is gone you can’t work on any of this stuff.

[Yoni Donner]: Yeah, that’s true. Actually, what you said now is almost an exact quote from the first quote I gave in proposing this project in an aging conference four or five years ago.

[Damien Blenkinsopp]: Oh wow. Cool.

[Yoni Donner]: So yeah, completely agreed on this.

[Damien Blenkinsopp]: Well you’ll have to tell me afterwards about the anti-aging conferences. Are there any good ones that you like or prefer? Because I’m sure the audience is interested in this stuff too.

[Yoni Donner]: I don’t if it’s still going on. I have seen anything about this in a while, but it was called the Personalized Life Extension Conference. It was run by Christine Peterson who now does Foresight though. Maybe she doesn’t even do that anymore.

It was very cool. I got a slot there to just propose this project that was not even close to existing yet. It was just an idea, but I put on a spiel on all of my belief in why we need a new tool and what’s the problem with existing measurements.

And I have to say, at least I got one thing in my life right, that I did build it exactly the way that I proposed it. But it was nice. I got to talk to Peter Thiel about this right after my talk and –

[Damien Blenkinsopp]: Oh awesome.

[Yoni Donner]: He set me up with one of his people to actually discuss funding for the project. Of course that never materialized because I’m not a business person. So I never follow up on business talks.

[Damien Blenkinsopp]: Oh well it’s a great intro to Peter Thiel as that’s quite a big name. Might come in useful to you later maybe with the tool. So I’d like to talk about what future plans that you have for the tool later. But for now, could we take a step back? Because you’ve said that this tool is quite different to a lot of the others out there. And some of the ones, when I contacted you, I was thinking about, is things like the – we had Adrian Owen who developed the Cambridge Brain Sciences set of tests, if you know those. We had him on the podcast a little while ago.

[Yoni Donner]: They got really famous from publishing in [unclear 12:54].

[Damien Blenkinsopp]: Publishing, and with the study which showed the brain training wasn’t effective. Was it that one, or something else?

[Yoni Donner]: That may have been that. There was one study that was really famous that I think Adrian Owen was the first author on.

[Damien Blenkinsopp]: Yeah. It might have been that one. Anyway so, and in this CANTAB, which is supposedly the best validated tool, is the reason it’s not relevant to you as CANTAB is because it’s looking at the differences between people rather than a person in time, as you explained earlier?

[Yoni Donner]: Well each of the existing tools has some substandard features and some that are not. So, I should be more clear. There have been mainly three axes of scaling that the Quantified Mind is supposed to provide that have not been existing altogether in existing tools.

So one of them is just something that Lasuis Claviger [check 13:41] cross experiments. It’s very easy for a researcher to just set up a new experiment and get a very validated and standardized set of tests and tools for analysis.

So, part of this is, for example, providing easy access to the entire raw data through APIs, but in doing things like randomizing subjects into groups or very easily controlling how the experiment is applied. And they also provide all kinds of algorithms for making data analysis easier and distinct outliers very reliably. So this helps scale across experiments.

And then there’s the [unclear 14:13] in person component, which simply was optimized to begin with in this test. So, all the tests have been adapted to be very, very efficient and to be completely repeatable. When I say completely, you could even take the test one billion times if you wanted to, and it would still be effective. Not only that, it would be more effective because the practice effects get weaker over time.

[Damien Blenkinsopp]: That’s something we should highlight for people. A lot of these tests you can get better at them over time. So this is what you call the practice effect or the training effect.

[Yoni Donner]: Right.

[Damien Blenkinsopp]: And in this case, because you want to see if there’s decline or improvement, you wanted to eliminate that. So you’re saying in these tests there’s not very much of that. As I understand it, after you’ve played it a few times, you’ve done it a few times, there’s not much change in terms of practice or training effect.

[Yoni Donner]: Yeah, that’s generally true. I also have more precise on all of the tests of exactly the practice magnitude. In the worst case it takes about five sessions to get it down to manageable levels. By that I mean that it’s smaller than most of the hypothetical effect sizes that we’re interested in measuring. So just time of day variation and so on.

But I do have exactly the compositions of the variance for all the tests. I’ve got all of this data for my thesis. So, a lot of tests don’t even have practice effects at all. For example, reaction time tests are almost zero. And one way is if you can extrapolate the practice effect and eliminate it.

But I think it is generally better not to make any model based assumptions and not to fit additional parameters. So if it’s possible for any experiments to start with just a few practice sessions just to get this out of the way.

[Damien Blenkinsopp]: Right. Yeah so, in practice someone should do those tests a few times a few days in a row and then they could consider that in their baseline.

[Yoni Donner]: Right. Yeah. So that’s the protocol that we follow on most experiments now. Four for five practice sessions before.

[Damien Blenkinsopp]: Does it have to be the same day or could you just spread those out over a week or something?

[Yoni Donner]: Generally, I would say spreading them is a little bit better, but it’s kind of insignificant next to just the value of doing them at all. So often with this kind of thing, I think it’s better to make sure that just people do them and not be too strict about rules because that will just result in losing subjects and –

[Damien Blenkinsopp]: Right.

[Yoni Donner]: Or they wouldn’t do it at all.

[Damien Blenkinsopp]: Excellent. And then I just wanted to bring up the other tools and see how it compares in your mind. There’s Lumosity and there’s BrainHQ Posit Science. How does your tool compare to those?

[Yoni Donner]: Yeah. So Lumosity, we actually became good friends when I built Quantified Mind and we talked about this a little bit, and we really like each other because we are completely complimentary and non-competing at all because they are focused on brain training and also on user acquisition.

Obviously, the whole thing is a big game and they’re huge and they make something that appeals to everyone. We joked about this the other day. Almost everyone’s mother plays Lumosity.

That is very different. They don’t do a very precise measurement of the instruments, but they do try to make arguments about brain training, and they’re very, very good about the gamification effect and user retention.

Quantified Mind is completely focused on being a precise measurement tool and, even more so, a research instrument. So that’s a very different focus. But I should say, I just published a paper with Lumosity last month that used their data since they still have a lot more of it about human learning dynamic.

So, that was a lot of fun. And mostly I did the whole research and they provided their amazing data. But it was a great experience just collaborating with the VP of R&D there – is a great guy.

[Damien Blenkinsopp]: Yeah. Cool. And I as I understand BrainHQ Posit Science is pretty much the same as Lumosity, but just smaller.

[Yoni Donner]: Yeah. I think there are subtle differences. Posit does stay more focused on some pathologies or specific kinds of improvement. Lumosity, more appeal to the general public.

[Damien Blenkinsopp]: Great. And what’s your opinion on the whole brain training area? Like I said, we had Adrian Owen on and he talked about his study where they tested a lot of people in the UK and they found no effects at all after they’d been doing some training for a while with the Cambridge Brain Sciences test. Do you have an opinion on that?

[Yoni Donner]: I think it’s very hard to just put a binary result or to say for sure brain training does not work at all. But I think if we consider what work should mean to a reasonable person, we can kind of conclude this question anyway.

Because for something to work, it needs to have, not just a nonzero effect size, but also an effect size that’s big enough to be worth the effort. And even if you do something – I do a [unlcear 18:36] every day. And let’s suppose you even improve your working memory in the general transferrable sense, all the results that we have so far – even the most of the domestic ones, even the ones that say it works – show a pretty small effect size.

And it still takes a lot of your time and a lot of your energy. For example, people told me on Quantified Mind there is a single [unclear 18:58] test, which is not even as horribly annoying as to do an the Quantified Mind, and people still said they fight with their significant others after doing this. [Laughter]

So, it drains a lot of willpower, it drains a lot of energy to do those things, and you get a very tiny effect in the end. So, if you are dedicated enough to do something good for your brain, there is nothing in the literature right now that comes even close to physical exercise, and this has been documented so many times.

So I’d say do some effort to clean up your diet. I don’t know, ditch the caffeine addictions. But if you drink it, in small amounts. Sleep well and exercise a lot. It will be much better use of your time than brain training.

Of course brain training is just appealing to people who like playing video games. So it seems like if you like doing these engaging things with your computer, then you just don’t suffer that much. But it’s still very time consuming and the effects are quite small even according to the most optimistic results.

[Damien Blenkinsopp]: Right. I went through a phase of using Lumosity and I came to the conclusion that it was just a huge sink of my time and it wasn’t really providing any benefits. But what I did notice was, when I was sick, I would get a huge crash – [Laughter]

[Yoni Donner]: Oh yeah. That’s really true.

[Damien Blenkinsopp]: in my data. So that was interesting. But the amount of time to play the games is a lot when you’ve been playing it for a while, the games tend to start taking a long time. Some of them were taking – I felt like it was ten minutes, and that was just too much time at the beginning of the day, I was doing it for this.

So, that’s one of the things that attracted me to Quantified Mind because you said you had a focus on keeping it efficient and minimal. Before we get into that I just wanted to point out something you just said. You were saying the willpower. It drains the willpower. It sounds like we’ve both been thinking about willpower quite a lot and how that impacts. Could you explain what you meant in a bit more detail when you’re saying that doing these kinds of tests could drain willpower and that could have impacts on the rest of your life, right?

[Yoni Donner]: Yeah. Well so I don’t want to say things that are too conclusive in a field that I’m not an expert in, but I actually did – there was a master student who did his thesis project with Quantified Mind on exactly the thing called ego depletion, which is highly related to willpower.

So ego depletion is, for example, when I give you a stroop test and then you’re more likely to eat a cookie after. Because some hypothesis could be that it drains your willpower. And it definitely looks this way with the Quantified Mind test.

So, we actually did an interesting experiment where we gave people a 20 minute long stroop test, which is really torture. And you can definitely see that people just cannot, even within the test itself – within those 20 minutes – they cannot maintain their ability to answer the difficult trials.

So you can divide the test to easy, moderate, and hard trials to pretty rough division, and you could definitely see that they keep getting the easy ones correct. But there are points where they just collapse and they start messing up the hard ones.

[Damien Blenkinsopp]: That’s a matter of endurance – the longer you’ve been doing it?

[Yoni Donner]: So, we didn’t get longitudinal data on this, unfortunately, so we don’t know if the same people got better the second time they did the 20.

[Damien Blenkinsopp]: I meant in terms of if you’ve been doing the test for ten minutes versus one minute. Was there an endurance effect in terms of willpower potentially?

[Yoni Donner]: Yeah. So that was the main hypothesis. It turned that it’s a bit more complicated like everything in life. So then, in general, yes. You do get less likely over time if you average other people. But there are also times where people seem to sort of get it back together. Actually, towards the end. It seems like they suddenly notice they’re close to the end so they have another bout of energy.

[Damien Blenkinsopp]: Right.

[Yoni Donner]: But yeah, and we also had these results when people just report as qualitatively. They feel drained after doing a long [unclear 22:33] and it’s fine. I do personally believe that there is also a lot of this classical “willpower is not that important if you don’t believe that it is.”

[Damien Blenkinsopp]: You think there’s merit to that?

[Yoni Donner]: Well it seems to be partly true and partly there is something that does drain. And it also seems to be trainable to some degree. It’s just one other reason not to spend too long doing things that don’t give you much benefit. But there are many other reasons to not spend too long doing things that don’t give you much benefit.

[Damien Blenkinsopp]: Great. The cookie test you brought up – the experiment of, I think it was, Roy Baumeister?

[Yoni Donner]: Yeah. And actually one of the things really is the stroop test by the way. So this is direct evidence.

[Damien Blenkinsopp]: I understand. Could you explain what areas of Quantified Mind what areas of the brain are you testing or what cognitive capabilities are you looking at?

[Yoni Donner]: There are some tests that looking at reaction times and speed directly. So there are few reaction time tests and motor speed test. There’s some visual and spatial abilities. Which I’m going with a somewhat thematic order because these things are a little close to reaction times.

There’s a lot of executive function and working memory stuff. So there’s always the argument of whether working memory and shorter memory are the same thing or not. So there are things for both. There are some verbal learning stuff. So more long term learning. This is the main emphasis.

There have been other tests that people have put in who have collaborated with me on studies. Actually emotion regulation and decision-making, but we didn’t ever get a lot of data for those so I didn’t get to analyze their psychometric properties. But yes, so mainly those things. I’d say the most rough division –

[Damien Blenkinsopp]: Yeah. How do these relate to the people at home? If they’re thinking about working area and the executive function area, how is that going to impact their daily life?

[Yoni Donner]: So these are probably the most important ones. Actually more important than speed. Probably speed would correlate to what you’d think of as alertness or even energy – weakly correlate. And executive functions would more correspond to what people would think of as focus or attention, or really getting things done, or even flow, I would dare say.

[Damien Blenkinsopp]: Getting into flow. The ability to get into flow. Okay.

[Yoni Donner]: And working memory is the best correlator out of these things to intelligence in general. So to be efficiency of work. But again, these are all very rough generalizations in making things interpretable.

[Damien Blenkinsopp]: Yeah, right. So to connect to the day to day is a little bit difficult. But working memory is basically, roughly how many things you can keep in your mind.

So, I always thought if you’re solving some kind of puzzle or you’re trying to make some kind of decision, if you can have ten variables in your mind – I think it’s seven the amount typical for working memory. But if you can basically play around with those things more in working memory similar to a computer has ram, then it’s easier to make decisions and more complex decisions.

[Yoni Donner]: Yeah, exactly. And of course every kind of work in modern life is full of these multiple things that come at you and if you have too many of them. Even writing code, is one of the most obvious things that depend on working memory, right?

[Damien Blenkinsopp]: Well I think pretty much everything I can think of. When you’re problem solving, I think that’s very dependent on working memory. Is that correct?

For most people, if they’re going about their jobs. Most jobs these days, it’s troubleshooting, it’s problem-solving, it’s planning. Would you fit those into the working memory area?

[Yoni Donner]: Yeah. If we were take into just a generalization and not perfect [unclear 26:05], yeah, totally.

[Damien Blenkinsopp]: I like that you put the caveat out there. It’s good. [Laughter]

[Yoni Donner]: Yeah. Sorry, I just got my PhD so have to be an uberous careful scientific person. [Laughter]

[Damien Blenkinsopp]: No, that’s great. So in terms of the scientific validation between your tests that you’ve put up there, how strong is it?

[Yoni Donner]: It’s quite good. So, to begin with, everything is building on tests that have been very, very extensively used in the literature. That’s how I selected them to begin with.

But I also did many independent types of validation. First of all, reliability is extremely high. It was higher than I ever hope to achieve. The basic result on reliability in this data shows that a one minute test for almost all tests is sufficient to measure almost perfectly the skill which is being measured. Which is great.

And for validation, I looked at internal structure and external structure. So these mean how the tests relate to each other, and that behavior supports completely psychometric theory and psychological theory. So that suggested they are measuring the right thing.

And the external variables also look exactly correct, so the same we’d expect. There are extensive results on this. I am slowly working on a paper that, hopefully, will eventually bring all of these results out. But for now it looks really good.

So everything that we would predict does seem to behave correctly, and then, of course, there are new results that we had no predictions about so this gives us some confidence to believe in them.

[Damien Blenkinsopp]: So, when did this launch and how many users have you had using the system so far?

[Yoni Donner]: This was early 2012, and it grew fairly linearly. I never tried to get users, but it just happened so that’s nice. It grew fairly linearly. We have now over 40,000. I think most people, of course, don’t take many tests.

[Damien Blenkinsopp]: Right. They do a few and then disappear.

[Yoni Donner]: There was several jumps in the middle where some event happened and then a ton of people signed up overnight.

[Damien Blenkinsopp]: Great. Do you know how many tests have been taken to date?

[Yoni Donner]: Yes. It would be something like half a million tests. I think I even computed the number of individual trials and it was something like 660 in a million or –

[Damien Blenkinsopp]: Wow. I’m guessing you look at that. Do you sometimes look at that data to see if there’s anything interesting that comes out on the averages or -?

[Yoni Donner]: So looking at the aggregates, definitely that’s the way to do psychometric analysis. So without looking at who’s doing what and names at experiments, you can still look at relationships between tests and practice effects and complexity effects. These are real interesting things – and even time of day, which is always there.

I don’t just look at people’s data because that’s a big invasion of privacy. But there are cases where researchers are designing experiments that are run on the platform. Then I give them some access rights. Then everyone who signs to that experiment specifically opts in to having the data available only to that researcher, and often we do the data analysis together. So I do get to see a lot of cool stuff.

And yeah, it looks like these results are pretty encouraging. We definitely knew, to begin with, that the effect sizes to be expected with in-person variation would be rather small, but it’s very nice to actually find them.

[Damien Blenkinsopp]: Right. That’s cool. One of the things I’d like to make clear about this tool is basically that, in order to do experiments, you’ve added the ability to add variables. And there’s some basic variables you’ve already added in yourself, like “have I had coffee today, have I had chocolate today” that you have tested for. So it enables us to control for different things and see if they’re having an impact on us, on our cognition and at the different test areas.

[Yoni Donner]: Correct.

[Damien Blenkinsopp]: So, do a lot of people make use of that function and you can see the differences between say coffee and no coffee?

And you’re saying also the time of day. Do you track that with location? So, I’m in London right now, for instance.

[Yoni Donner]: Yes. People who are moving around and don’t update it in the time zone, I would lose that data. I do not take into account all the data from people who did not demonstrate that they were aware that a time zone field exists and needed to be updated. So, I only include people who explicitly change their time zone at least once to make sure that at least that part will be roughly –

[Damien Blenkinsopp]: So, I’m gone.

[Yoni Donner]: Yeah. There were cool results with the time of day because I think no one has looked at the time of day effects simultaneously across a wide variety of tests, and I’ve done exactly of this. Only a few weeks ago finally got to play around with this. It was cool.

So you can’t actually look at a uniform this time of day because different people have different chronotypes. So there would be owls and larks and all these other names. So instead of uniformly averaging across everyone, I used a nonparametric clustering algorithm to find, automatically from the data, what are the clusters that we can see. It was really cool because you could definitely see that almost all the people are worse at night.

But definitely some people get this afternoon dip and some do not. And some people peak in the late morning where as some others actually slowly improve throughout the day and only collapse late night. It was really cool to see this emerge from the data itself with no prior assumptions.

[Damien Blenkinsopp]: That’s interesting because I can make assumptions about those cases. You could probably too. It’s speculation, but a lot of people get slight adrenal fatigue so they could be more tired in the afternoon. I could imagine that it would. It’s said to affect cognitive.

So, it’s funny that your data has pulled out those scenarios, which would be very interesting. So what would be the minimal test? Because we’re talking about efficiency here and we talked about how doing a lot of testing might reduce our willpower and have some impact on our self-control during the day and some other impacts.

So, in terms of someone who wants to do some tests and basically see where they are – track some that are not having cognitive decline or potentially looking at days they drink coffee versus not or some other test – what would be the minimal test you could do once per day to track while experimenting like that?

[Yoni Donner]: Yeah. I’d say even just one minute and pick one test or maybe two minutes and two tests, and that’s it. So, two back or three back or a good test, cover a wide variety. It’s mostly working memory, but you’d also get a component of speed in that.

So if you do that I’d say you don’t really need to measure speed separately or choice reaction time or something. And it’s also valuable to put stroop or what is called sorting on the Quantified Mind.

But definitely one to two minutes and no more. There’s no need it and it just reduces the likelihood of doing this many, many times, which is far more valuable than adding more tests to a single measurement.

[Damien Blenkinsopp]: Right. So you could basically do this test once per day, control the variables and it’s basically a minute of your time to get potentially something useful.

[Yoni Donner]: Yeah. Or multiple times per day if you’re looking at something that changes across the day or the effect of coffee obviously are not constant across the day.

[Damien Blenkinsopp]: Right. Exactly. I know that you did a lot of tests in the past. I’m not sure if you’re still doing a lot of tests with this on yourself. What kind of discoveries have you made about yourself?

[Yoni Donner]: That’s interesting. So, I definitely agree with what you said before about getting sick. Fortunately – this is famous last words – that did not happen in quite a long time. So I didn’t get that data point. But last time I measured this the effect was huge. It was unbelievable. It was somewhat that I could compare this to, for myself, something like five days of pretty severe sleep deprivation.

[Damien Blenkinsopp]: Right. Exactly.

[Yoni Donner]: Awful.

[Damien Blenkinsopp]: Did you see kind of like a crash and then a slow recovery?

[Yoni Donner]: Even when getting better?

[Damien Blenkinsopp]: Yeah, right. Well for me, personally, it took it five days, seven days to get back to baseline after the initial day when you fall sick. Of course it depends on the sickness, so it’s a bit variable.

[Yoni Donner]: I think the recovery was a bit faster, but still we probably had different things going on. But yeah, it was a very strong effect. I actually did a [unclear 33:51]. One of the strangest things I’ve done on myself.

So I like doing things that they also keep me engaged by their own right. So an experiment which is not boring because it gets you to keep doing it. We had this funny discussion about a little odd result in the literature. You may know this one. Glycogen depletion followed by glycogen overcompensation resulted in a significant improvement in cognitive functions.

They did this on rats and this was a long, long time ago. I may not get the details right away. I think they basically let them run to exhaustion until the poor rat just collapsed. And then they fed them a ton of sugar, more than they needed to refill the glycogen. And then killed them and witnessed abnormally huge amount of glycogen in their range.

And then they hypothesized that this would also translate to a behavioral output. So I did this on my poor human self for like several days.

[Damien Blenkinsopp]: Okay. So could you explain how you did that to yourself?

[Yoni Donner]: Yeah. So without the killing part. So a lot of exercise. I think that was also a good excuse to get myself back into exercising. So, it was a lot of cardio and then some weight lifting.

I read what you’re supposed to do to do some glycogen depletion. So a lot of many, many, many sets with relatively low weights. That seemed, at least by subjective experience of wanting to diet, seemed to do the job. And then trying to do some calculations of what exactly would be the glycogen overcompensation, and then take cognitive tests about several hours later.

I don’t remember the exact numbers. It was a few years ago. But it was a hypothesis based on a minimal amount of existing data. That seemed to work great. It’s totally not worth it. It’s like brain training. But definitely I got some of the highest scores I’ve ever seen in my life.

[Damien Blenkinsopp]: Cool. That sounded like a huge effort actually. That sounded like it was a couple of days to do that.

[Yoni Donner]: Yeah. So I only did like five times, but it was pretty significant. This is definitely not something that I want to do in life, and it’s not worth it to get this benefit. But it was just interesting.

Again, this is the kind of thing you want to know or you or you want to test. And if there’s some open question, why not just settle it with science?

[Damien Blenkinsopp]: Right. And who knows? If you had a really, really difficult decision to make or some kind of planning session or something, you might to do that as a one off to solve that life decision that you have or –

[Yoni Donner]: It’s true. Or if you have an exam at 3:00 PM or something and you decide to waste your time on doing this crazy stuff instead of studying. You might at least know that you’ll be quite pumped up when you get there.

[Damien Blenkinsopp]: What other experiments have you done that have had some significant impact? Are there any you feel you’ve integrated into your life because they’re worthwhile because the time expense for actually doing these things isn’t too much to get some kind of benefit.

[Yoni Donner]: The biggest impact one is time of day and I think that would apply to a lot of people because you don’t really need to do strange manipulations or interventions for this. So just design the activities that you do such as they fit with your natural rhythms.

I’m currently at the point where I’m way more productive in the early part of the day. So, I leave all of the stuff that doesn’t require too much mental power to the later parts of the day. But that’s a very easy one.

And an interesting one that we’ve seen that’s not on me, but an actual study that was done quite recently was that the effects of temperature are not what people would expect. Which is also a little consistent with existing literature, but there’s not that much existing literature on this.

This was a great study where several hundred subjects and everything was perfectly controlled in terms of temperature and humidity. They did all the practice stuff perfectly so the data was very, very clean and very, very good, and it showed that people do not actually predict correctly when they function at their best.

So most people would report being more comfortable at a slightly lower temperature than the temperature which their brains worked the best behaviorally. This seems like a little counterintuitive, but I looked into this and it turns out that this is actually consistent with results that were previously known. But I think we brought a better resolution to this question.

[Damien Blenkinsopp]: It’s always interesting when it goes against our own sense of wellbeing. As you’re saying it’s a little less comfortable, a little bit hotter than maybe we feel is comfortable is when we’re working best.

I was also wondering if you think a lot of people might try to guess their rhythm during the day. I’ve always been a morning person and I’ve always told people I’m a morning person. Over time, I think I’ve got some more stamina now so I can work for longer periods. I can work maybe 10 to 12 hours and I don’t feel so bad if I’m not doing it too many times a week.

So basically, people should kind of test this kind of thing with the test. But would you expect them in [unclear 38:36] you mean speaking to people about their time stamp test and does it kind of reflect what they thought?

[Yoni Donner]: That’s interesting. I think most people have a good sense of this, especially who are workaholics because they actually try to use their brain at all times of the day. So they really discover what it’s like.

Of course you mix in all effects. Obviously you get a little tired just by doing mentally exerting work. So it’s not a perfectly controlled study because then you would have to be lazy the whole day and see if you still function not quite as good in the night.

But most people don’t do this and they are accurate. There aren’t that many people who have shared their results or given me access to their data explicitly.

But there’s at least one person who said he actually did change his routine based on these results. Because he used to do a lot of intellectually engaging work for the evening and it turns out his results were very strong biased towards morning strength.

So he moved everything around and moved, for example, his physical workouts to the evening because he found that he didn’t need much willpower to get started. So he would do the workout and didn’t feel that it matters too much if he lifts a little bit less weight because that also changes during the day and it might not be correlated too.

So that was a cool example of changing things accordingly. And that’s really, again, the easiest experiment to do. You don’t need an intervention at all. You just get the data.

[Damien Blenkinsopp]: Yeah. Great. Are there academic studies that you can talk about that have been done with The Quantified Mind? Because I noticed some of them are restricted.

[Yoni Donner]: Yeah. It’s true. So there are a lot going on. There’s always someone somewhere doing yet another kind of coffee study. Which is funny, but it’s nice because it always works.

[Damien Blenkinsopp]: Let’s talk about coffee because I think – did you do a bulletproof coffee study? Was that where it was done because -?

[Yoni Donner]: That’s not really academic, but it was a study. Yeah. It had problems. It was definitely not blind and there was a selection effect by the [unclear 40:35] class.

[Damien Blenkinsopp]: Just for the people at home, that means that the people had opted in big as like they like bulletproof coffee basically.

[Yoni Donner]: Yeah. They were actually recruited through the Bulletproof Coffee website.

[Damien Blenkinsopp]: Instead of being randomly given Bulletproof Coffee.

[Yoni Donner]: Yeah. And this in combination with not being blind means that the placebo effect would be huge. Because you’re exactly telling the people who would believe that this would work on them that they are currently under the condition that would work for them.

Having said that, the results were more interesting than just a pure monotonic improvement. So Bulletproof Coffee has this as a component of the coffee and the butter. So even in our data, butter had zero effect, but coffee had a large effect.

This means coffee, like Bulletproof compared to Starbucks, not Bulletproof compared to no coffee, which would obviously have an effect.

[Damien Blenkinsopp]: Okay. And the quality of the coffee basically or something about the coffee that was better. It’s interesting –

[Yoni Donner]: Or the placebo of the coffee, yes.

[Damien Blenkinsopp]: The facts that people were using the Bulletproof versus the other. Yeah, that’s cool. Well in coffee, in general, because everyone thinks of course that we’re performing better when we’re on coffee. You certainly feel good.

I’ve had an interesting – because when I started using this tool I thought coffee would make a difference. I’m drinking Bulletproof Coffee in the mornings because I feel like it gives me a ton of energy. So I’ve been doing that for a long time and I’v found that it makes no difference to my score so far.

[Yoni Donner]: Oh interesting. Even when you are slightly sleep deprived?

[Damien Blenkinsopp]: I really make an effort to sleep well. So I wouldn’t say that I’m – today maybe is the only day in a long time that I didn’t sleep great, and I know why that is. It’s because I took a few things yesterday. But that doesn’t really happen to me that I’m sleep deprived. So is that the one situation where coffee has the biggest impact? You’ve seen that?

[Yoni Donner]: Yeah. The literature definitely shows this that the effects of coffee are – I think there is still evidence to suggest they exist in non-sleep deprived individuals. But those are actually very rare in modern society.

It’s great that you take care of yourself, and I wish everyone did the same. But definitely the literature shows that the more sleep deprived you are, the more difference coffee would make.

There’s this famous study with Marines or Navy Seals or something where it showed that even the effect of the dose keeps improving after 300 milligrams, which seems like a gigantic dose to me. But that was still better than 200.

[Damien Blenkinsopp]: Wow. So, for the people at home, how many coffees is that?

[Yoni Donner]: That’d be like five espressos.

[Damien Blenkinsopp]: Wow.

[Yoni Donner]: I’m not a huge coffee expert really, but I think that sounds right. So yeah, this was still better than 200.

[Damien Blenkinsopp]: Okay. Yeah.

[Yoni Donner]: So I think it depends. But in general, whenever you use something chemical, even if coffee and it’s safe and it’s not that bad and it feels nice, you’re still playing with stuff that we don’t fully understand. I do firmly believe that, if you can first improve your life by sleeping better, then this is a better approach to do than optimizing coffee intake.

[Damien Blenkinsopp]: Right. I think one of the interesting things about this is when you feel good, you do more work. I find personally you tend to take on more bigger tasks and things like that.

And what I would say is I think coffee – I don’t know if I’m an addict – coffee makes me feel good. And so, I think I do more, but that wouldn’t necessarily show up in a cognitive test that I know of. Is that correct?

[Yoni Donner]: It might show in an indirect way if only by showing that you did more tests because this is a big effect when you’re not actually in a controlled study. You could just decide you don’t feel like taking the test now, and that would usually show that you’re a little weak on willpower or something.

[Damien Blenkinsopp]: You were saying it would show up for sleep deprived people. So, have you seen that coffee makes quite a big impact on people in general for the aggregate later? So, basically then you would be saying like a lot of people are sleep deprived who are taking a test to some degree.

[Yoni Donner]: Well so, first of all, you can never know. You can know if you ask every single person and if you believe all their answers, but that’s definitely not the case in an internet-based testing environment.

Coffee definitely seems to have a real effect in aggregate. It is not huge. I think it is the order of magnitude or time of day variations and then, of course, this is correlation that most people would consume coffee at this specific time of day. So these are confounded.

[Damien Blenkinsopp]: Right. A lot of people have it in the morning and, as you were saying, a lot of people, because of the time, that’s when they’re going to be performing better.

[Yoni Donner]: But we are now actually doing – this is happening this last week and this week. There’s a collaborator at Harvard who’s doing an in lab coffee study with Quantified Mind. So we’ve done a million coffee studies, but this is probably the most rigorous one.

So, they actually bring everyone to the lab and they randomize them. They don’t tell them what coffee they are getting, and they did the practice before and they have a crossover design. So hopefully we’ll see something.

[Damien Blenkinsopp]: Great. So when these studies are published, you list them on the site or you put references somewhere?

[Yoni Donner]: I would. I don’t think anything has been published in a journal yet because all of this stuff is fairly new in academic publishing takes years and years. There have been things I didn’t list that have been published informally. Like a student did a project and submitted it, and it was approved and they have a PDF somewhere. I guess I should look into putting some more of those.

[Damien Blenkinsopp]: Great. So are there other tactics that you seen, whether it’s the number of hours slept or anything else that you’ve seen, that people could think about testing?

If you had a priority list of tests worthwhile doing for people, experiments to see if it helps improve their cognitive capacities a bit, which ones would you list? If you were starting from scratch, which would be the top five you would start with having seen what you’ve seen that might have some potential uplift?

[Yoni Donner]: Right. I think this should be very individualized and people should start with their biggest suspect for, not what might make them better, but rather what might make them worse because these things have a much larger effect in practice. For example, someone who is good and sensitive would see a much stronger decline by consuming fruit than someone who just takes pure [check 46:28] to try to be a little bit better.

So, in general, effects of improving over the healthy well functioning baseline are much smaller than effects of fixing something that’s broken. I guess a lot of people do have somethings that they are sensitive to or that are broken in some weak sense. Like someone who is chronically sleep deprived, you could say that something is broken in their lifestyle.

And of course, this is not a moral judgement, but it just says that they might be able to see a larger improvement by fixing this and if they know that there is something they suspect. If someone suspects all kinds of food sensitivities, or even allergies, pathogens, anything that they feel hurts them, it will be useful to try to test exactly what it is using factorial designs and cognitive testing to fix this.

The other thing, I find it cool that a lot people are using this to really track their aging process. So this takes a lot of discipline to repeatedly take a test once a month, for example, for years and years. But this is admirable when people are doing this and it can maybe give you serious suggestions to when your brain is no longer that of a very young person and you should start taking more care of yourself because when you’re young you can get away with much more. So that seems also useful.

[Damien Blenkinsopp]: And I think sleep’s one of the ones that most people are guilty of. I’ve been guilty of it for a very long time. Especially driven workaholic type people. We just don’t want to sleep.

Have you seen any number of hours? A lot of those tests as you for the – well the standard ones I’ve been doing ask you for the number of hours you slept the night before. I’ve certainly noticed straight away – for instance, today, because I slept six and a half hours, it was lower. So for me, that’s low. Normally, I’m seven and a half hours.

Have you noticed anything in terms of the number of hours slept that you see some dips and dives or anything?

[Yoni Donner]: Yeah. So again, there is not that much data, and again, I don’t look into people’s data. So this is only people who ask me to look at their data and conversations with that and so on, which filters a lot.

But the most interesting result I can remember is that it seemed we actually did see a skip one day effect where the strongest effect would be with a delta of one day. Which was also interesting because one person also shared their result with me that suggested that it was the same with drug use.

[Damien Blenkinsopp]: So you’re saying there’s a lag effect,so it hits you the day after?

[Yoni Donner]: Yes. It’s a lag effect, yeah. It seems to be and it’s hard to say if it’s exactly two days, but it seems that the effect, when skipping a day, was stronger than the next.

[Damien Blenkinsopp]: Okay. Of course I have to ask you about Nootropics because it’s one of the biggest topics right now. You mentioned Piracetam and of course it is many others.

Have you got any anecdotal effects? Anecdotally have you gotten any information from it all about people taking Nootropics and getting any benefits based on the test results?

[Yoni Donner]: Yeah, a little bit. So people seem to like Modafinil, which is a strong one.

[Damien Blenkinsopp]: It is, yeah.

[Yoni Donner]: And you never know. You’ve seen the one where Dave goes on Nightline and talks about Modafinil?

Yeah. That was quite a while ago, right?

[Yoni Donner]: Yeah. So there’s like a minute there when they talk about Quantified Mind and the experiment he was doing. So I trust him when he says he got a strong result. But of course, one should always be careful.

For example, here’s a study design that I don’t like. You take Modafinil. You do cognitive testing, you get your scores. Then you stop taking Modafinal for a week and get back on it and you see that your scores during that week were much worse. This is controlling for practice so that’s fine.

But the problem here is that you might as well just show that you became addicted to Modafinil. You don’t actually know if this performance on Modafinil is better than what you had before you started. You just know that now it hurts you to get off it. So this, I don’t think they built control for.

[Damien Blenkinsopp]: Right. The other thing is people talk about Modafinil kind of feels like you’re running high speed gear or something and you get a lot done that day. But I wonder if maybe the days after you could pay for it with slightly lower cognitive capacities as in, when you come off of it, you’re basically trying to catch up or something.

[Yoni Donner]: Right. This is the borrowing from your future self.

[Damien Blenkinsopp]: Right. Basically, it is that thing you’d have to control for too.

[Yoni Donner]: Yeah. It would be great to have some other more controlled study or some more centralized resource of good practices when designing those studies. It is definitely something to look into and might be, and I asked before about, points for improvement for the future.

But definitely, self-explanatory limited by the fact that it is complicated and not complicated in an intractable way. But complicated in way which just in practice most people don’t take into account because it takes a lot of experience and thinking about these things.

[Damien Blenkinsopp]: You mean to set up a proper experiment?

[Yoni Donner]: Yeah. So, for example, the effects you just described are these lag effects and –

[Damien Blenkinsopp]: Great. Do you have any future plans to expand the functionality of Quantified Mind? It sounds like there’s quite a few academic projects starting to run with it.

[Yoni Donner]: Yeah. So this is great and I got a count quite recently since I put it on some presentation. It was around 30 that are either being done or have been concluded, and they’re either a stage of planning a follow up or writing up or just kept for internal use or something.

So I definitely want to figure out what gives researchers the most value and how we can improve that and provide that. I do want to also make users who are not researchers happier, but that’s just a slightly lower priority simply because it has a lower impact on the overall progress of science, which sounds pompous.

But I hate saying things like this, but it is in the end about impact and I do think that it will help more in the longer run if we have more general human knowledge about, even individual variations, but the effect of things we just don’t even know right now. And when people just learn something and they are the only ones who gain knowledge, it’s a small impact.

[Damien Blenkinsopp]: All right, I understand. If people want to learn more about this, where should someone look first? If they wanted to learn a little bit more about – we’re talking lament terms here. I don’t know if you’d have good references, like presentations or books or anything like that where people could learn more about cognitive, basically, testing and assessment and basically the tests that you have on the Quantified Mind.

[Yoni Donner]: That is a great question. You’re right, in predicting that I don’t often think about how to present information in lament terms. We still have the science page on Quantified Mind, which is kind of readable. Maybe even too readable because people might not get enough information.

This is a little silly, but there is my thesis which has two entire chapters about this and it’s probably also not very readable. It’s also not published yet. So this science page would be a good place to start and for specific questions, everyone’s always encouraged to write to me and it’s always fun to talk about cognitive testing and so on.

[Damien Blenkinsopp]: Great. What would be the best ways for people to connect with you? Are you on Twitter – Quantified Mind? Where are you most active?

[Yoni Donner]: The contact page on Quantified Mind is a good way. I’m a not a social media person.

[Damien Blenkinsopp]: Is there anyone besides yourself that you’d recommend to learn more about cognitive testing or running experiments for cognitive testing?

[Yoni Donner]: That is a good question. I know some people who are local and you meet them in all kinds of conferences. I don’t know if there’s one resource. Definitely Lumosity is not bad because, even though the main tool is the commercial game product, they actually have a large group of people who are more into the data analysis and resources.

They publish all the results in a human readable form as well on their blog, which is a good practice. So that’s nice to look at. There’s the Human Cognition Project which is to give more researchers access to this data and to generate more results. And then, of course, all the papers in science and nature also have popular interpretations.

[Damien Blenkinsopp]: All right, great. Thank you for those. Now just a little bit about you and what you’re doing these days. Are you tracking any of your metrics or biomarkers, like blood or cognitive tests or anything, on a routine basis?

[Yoni Donner]: Not really. My habits that are I start tracking something when I think there is something to learn. And I insist on not tracking it anymore when I’m not learning anymore. This is nice because sometimes you discover that you can predict.

You just know things that you didn’t know before. You know your heart rate. You don’t need to measure it. I’m sure you’ve experienced this after years of doing this stuff. And so it seems to be true for a lot of measurements.

I wouldn’t say that I can predict cognitive performance because that has that funny property, which is why we need this. As your brain changes, its ability to predict itself also changes accordingly so that’s why you have these wrong conceptions. So this I would still use, but I want an important question to have when doing this.

[Damien Blenkinsopp]: Right, yeah. It sounds like you basically do little projects on something you’re interested in and then you kind of move on. You’re just saying that basically builds self-awareness by doing these things with each one so you can kind of tell yourself where you’re at also.

[Yoni Donner]: Yeah. That’s exactly right. There’s a time cost and there’s an effort cost in tracking anything. So you’ll never track everything so might as well make sure that you track the things that count.

[Damien Blenkinsopp]: And what are the biggest changes you’ve made in your behavior over the years with experiments if any?

[Yoni Donner]: Yeah. I probably realized that I’m not actually built for short term rewards. All things that normal people call fun I don’t find them fun, and I don’t find them useful.

And for my longer term happiness, the things that make me the most satisfied are creating value. Being productive, learning, developing. So with time, I guess I put much more of an emphasis on really building my future self and not so much doing satisfying things in the short term.

[Damien Blenkinsopp]: Great. Because you find that satisfying as well.

[Yoni Donner]: Yeah. So there are two things.

[Damien Blenkinsopp]: It sounds like you find – the same way it makes you happy, the same way.

[Yoni Donner]: Yeah. And data can help you reach that conclusion in two ways. One is when you do this not very precise. But I try to make those subjective measures as precise as possible by just breaking them down to so many individual categories that can be scored and then getting numbers and doing something.

But also just, in the sense that if you are tracking things that matter to you in the short term, and you’re witnessing their anti-correlation with things that are supposed to be short term rewarding as opposed to long term benefits, then this is a good way of actually making change. Because you can’t deny from yourself a change that you can see in the numbers.

This was a very abstract thing so I’ll try to get more concrete. If you are tracking your fitness and you noticed that you are not paying that much attention to your stoop [check 57:15}, it goes down. This hurts.

If you know I lifted this much weights a few months ago and I can’t reach that anymore, it really sucks and this makes you change some things. And to me those health things, those performance related metrics have a huge impact in making me change behavior.

[Damien Blenkinsopp]: All right. So last question. What would be your number question – we ask this of everyone – to someone trying to use data to make better decisions to improve any aspect of their health, performance, or longevity? Just something about themselves.

[Yoni Donner]: So there are many aspects of using data. So definitely one of them is make sure that they are collecting the right kind of data, and in a consistent way, which makes for a valid experiment. But also not try to overdo it.

I’ve seen so many people just fail because they’ve tried to do too much and got lost in the details or in the process itself. Focus on the highest value thing at a time and do it properly and win, really win, and then move on.

[Damien Blenkinsopp]: All right. With the minimal effort, right?

[Yoni Donner]: Yeah. Passive tracking is great. Some people do a lot into tracking and getting data, but then don’t do the minimal effort in just learning data analysis. Some people still don’t actually feel that comfortable playing with raw data. I feel it’s worth learning for almost everyone because it’s not that complicated.

[Damien Blenkinsopp]: Thanks. That’s a great point. And do you got a tip? Where would you start? Would it being using Excel statistical correlation or what would be the first thing someone could try that would add a lot of value?

[Yoni Donner]: I think that is not bad. But if you’re at the point where you get weird files from tools then Excel won’t help you with this and you can’t always load giant data into Excel.

So, I would suggest try to learn Python OR. These are not so complicated, and they’re extremely powerful. And of course, this day, there are so many tutorials. All of this stuff is very easy with some reasonable effort. But people who can learn to play guitar can also learn to us Python.

[Damien Blenkinsopp]: Right. It seems really, really complicated when you look at programming languages. But I’ve done a bit of programming in my time and I’m not a programmer at all. I’m a business guy. But yeah, it looks much worse than it is. That’s what you’re saying, right?

[Yoni Donner]: Yeah. And also, trying to answer a very concrete question about your data is very different than building Gmail or something. You’re not building it. It’s not programming in the sense of building giant tools.

It’s really doing something very, very specific. There would be 50 lines of code maybe, if it’s complicated and involves reading files. But you just want to generate your graphs the way you like them.

And then throw out some bad data points and maybe combine data sources. It’s very hard to use some general tools that shield you away from programming. And often you get to a level of complexity, which is akin to actual scripting.

[Damien Blenkinsopp]: Well Yoni, thank you so much for your time today. Really enjoyed talking to you about all of these and setting some of the science straight on what we can’t really look at and decide that it’s going to work or not. So thanks for your time.

[Yoni Donner]: Yeah. Thank you! This was great and I admire what you’re doing. So good luck with all of that.