How to Assess the Strength of Claims in Your Zettelkasten

• Aug 6th, 2022
• by Sascha

We all are guilty of the following: We think we know something in spite of very weak evidence. Our Zettelkasten will reflect this behavior with no mercy. This will become a problem because the game of evidence is a game of trust. This trust game in the arena of creative knowledge work will be played between your present self and two other selves: Your past and your future self.

When you browse your Zettelkasten, you will judge its trustworthiness by the behavior of your past self. Your trust will be high when you write your notes cleanly and transparently.

1. Clean means that you write in plain language and not in the gibberish that is sometimes – or, in the case of humanities, quite often – used to impress professors and colleagues. If you allow yourself to follow such a habit you will achieve one thing very reliably: You will disguise the information from your future self. Your present self will ask your past self: “Why the duck did you write such garbage?” Using big words is a method to disguise and pretend. Even if you try to fool others with big words, at least don’t do it yourself.
2. Transparent means that you make very clear what you came up, and how you came up with it. Let’s imagine you have read something interesting in some magazine and want to process it. If you cite what’s in the magazine and add a footnote for it with a link to the article, then you are not transparent. You can’t trust the so-called journalist. It is not a modern phenomenon. Now it is just more obvious that journalism is not reliable.¹ By relying on something non-reliable you will fall on your face. Either you trust your past self and make major gaffes, or you begin to distrust yourself which will deteriorate your creative knowledge work.

So, what is the actual problem we have to deal with? It is the problem of trust. How can we trust in evidence and how can we trust ourselves? The trust in ourselves is a matter of real trustworthiness. You can’t really trick yourself. All you can do is to make yourself negligently naive and pay the price. We don’t want that.

The solution to that problem is to understand how evidence works and when to trust evidence, no matter if you provided the evidence yourself or want to evaluate others’ evidence. To deal with this problem I will give you two techniques that are part of a broader method I outline in one of my books, “The Low Information Diet”. They are practical applications that are based on the work of Nassim Taleb and philosophy of knowledge.

Technique 1: The Lindy-Filter

The Lindy-Effect is a term I picked up from Nassim Taleb. It basically says that the longer something exists, the longer it will continue to exist if it is a non-perishable entity. The reasoning is that something that proved itself in the test of time means it proved itself for the future. This is the reason why we can have access to timeless principles. The idea is very prevalent in books like Principles by Ray Dalio and 7 Habits of Highly Effective People by Covey (Amazon affiliate links). Often these books are based on the personal lessons a very successful person has experienced repeatedly over his or her lifetime. But in addition they often refer to nature with advice like “Look to nature to learn how reality works”.² If you refer to nature in the way it unfolds itself, you base your argument on aspects of reality that are billions of years old. If you refer to religion, you base your argument on aspects of a collective learning process that is at least many millennia old.

That leads to another aspect of the Lindy-Filter, in a way I came up with on my own: The Consilience of Knowledge. This is a term I learned from Edward O. Wilson, but I learned the principles beforehand from authors like Bruno Latour and the position of coherentism. The principle is very easy to understand: The more perspectives come together when proving something and/or failing to falsify it, the better. Imagine you read something about self-confidence in an obscure self-help book but then find similar statements in Buddhist texts. You dig a bit deeper and find the same statements in psychoanalyst and behaviorist books. And even more: You read something about confidence and “trust” in games theory that further supports the initial claim. Now you’ve got more than just multiple claims of the same sort. They are from very different methods and refer to different time spans. Buddhist texts are very old and quite time-tested. Some other texts are not that old but are more rigid – hopefully, based on scientific rigor.

You can see that accumulating evidence from different perspectives can add trust in a non-linear way. If you are only reading buddhist texts and base your claims just on these, you are a fundamentalist. If you add some psychoanalyst literature you can still be subjected to selective reading. After all, the psychoanalysts allow for much room of interpretation. But if your throw games theory into the mix you get something very rigorous. You will never have flawless evidence but at that stage your claims are very solid. The non-linearity comes from the probabilistic nature of the multiple layers. Let’s just assume that a claim has initially a chance of 0.5 (5 in 10) to be true or not. You find some evidence from one perspective and scale the probability up to 0.4 (40% chance of being wrong). Because one perspective is only improving this one chance, you couldn’t get very high in confidence. But if you apply another perspective you can multiply. The probability of some claim being true from two perspectives is 0.5 times 0.5: 0.25! (25% chance of being wrong). Adding another perspective and you get your confidence up since the probability is down to 0.125 (12,5% chance of being wrong). Five different perspectives and you have only a chance of 3.1% of being wrong. Gaining another perspective is the way to go. (The numbers are not accurate but for illustrative purposes)

Now, let’s bring those two aspects together: Time and multiple perspectives. For my work on the human condition I only accept claims that are valid from the following perspectives:

1. Evolutionary biology and psychology. That means something needs to be true for more primitive lifeforms (e.g. lobsters), mammals, primates, hunter-gatherers, and has to have some continuity over all evolutionary stages.
2. Religion. That means that the claim has to be validated by at least one religious body of text. (I personally concentrate on Christianity and Buddhism.)
3. Classical texts. The claim has to be made in a classical text that stood the test of time (for example the stoic literature).
4. Empirical evidence. This should be self-explanatory.
5. Structural evidence. An example is games theory. Another would be an analytical claim.

Example: Self-worth.

Claim: An authentic sense of self-worth is accompanied by low reactivity (neuroticism). It has a component of appropriateness. Too much a reactivity is evidence for pathological low self-worth. Too little reactivity is evidence for arrogance or narcissism (e.g. not willing to change your opinion when challenged is a typical sign).

1. There is something I call the serotonin-status connection that underlies confident behavior. It can be verified in humans, primates and even back to crustaceans. High serotonin and status is accompanied with low reaction to stressors. The reactivity is an adaption to the actual risk of the situation. Being high in status is a lower-risk situation than being low in status (There is an exception for the alpha position btw).
2. Buddhist texts advice for a behavior that could be called “non-reactivity”. This is a trait of high self-worth. Low self-worth comes with neurotic behavior which can be characterized as too much of reactivity.
3. Stoic texts refer to the same thing as Buddhism: Non-reactivity.
4. Empirical evidence shows quite neatly that high sense of status comes along with non-reactivity.
5. In a setting of low risk of damage a confident behavior is more probable. An example for this is the confidence trick: You create a situation with the illusion of a low-risk environment. Confident behavior is the result.

Application in my own archive: When I make a claim I have a copypasta prepared to pre-structure the note. I then paste it in the note I am making the claim to encourage my future self to supply this claim with a multi-layered justification as described above.

Technique 2: The ZKM Evidence Scale™ (ZES)

If you have a note with a claim you can use the following scale:

ZES-0. You only have anecdotal evidence. Some blogger wrote something. Some journalist wrote something. Everything that came out of press and media. Everything anyone told you. Example: Some guy told you a story on how he dealt with depression. Or some Op-Ed.

ZES-1. You found it in secondary literature. That means that claims are not original but the text refers to external sources of evidence. Examples: Quoting Hari’s Lost Connections on the topic of depression. Or any review or meta-analysis (I deviate from the current axiom in science here).

ZES-2. You found it in primary literature. That means the claims are from the original source. Example: A randomized clinical trial of alprazolam versus progressive muscle relaxation in cancer patients with anxiety and depressive symptoms by Holland et al.³

ZES-3. You not only read secondary and primary sources. You checked the relevant cross-references as well. Here the combination of the diligent studying of primary literature and meta-analysis is highly valuable and the strengths of the meta-analysis is actually effective.

There is quite a resemblance to the layers of evidence. No coincidence. :) There is general structure of this “thing” named evidence. For some added mind-fuck: The concept of “evidence” in my Zettelkasten is subjected to both the Lindy-Filter and the ZES.

Application in my own archive: I applied it for a while. But for me it is not necessary. I do this for years now and can assess the ZES without any special tag or other marks. But when I used it, it was a tag, e.g. #ZES1.

Conclusion

In the end there are many ways to increase trust in your system of creative knowledge work. Part of the Zettelkasten Method are principles, techniques and recipes that build trust.

An important reason why systems of knowledge work fail is lack of trust. It is trust that ensures that you and your system will still be in love in the future. The two techniques above can be trust-building tools to ensure a long and happy future with your Zettelkasten.

Christian’s Comment: I remember how badly I wanted to have a clear definition of Sascha’s ZES™ rating back in the day. Were my notes any good?! – I have little to no clue when it comes to rating the quality of studies, for example. Being able to gauge the quality of my notes gives the subjective rating of “this note doesn’t feel very useful for the long term” an objective spin and turns the fuzzy feeling into a problem that is technical and solvable: get more points of view to support or, even better, refute the claim you so badly want to keep.