Pitfalls of Decipherment

I am barely a cipher amateur. Nonetheless, I’ve read and studied about many historical cipher attempts, both successes and failures, and over the last few years, and I’ve been privy to a great many decipherment attempts on the Voynich, both in public and in private. And so even if I have not made any significant decipherments of my own, the successes and failures of others have still taught me many useful signs of error in the process of decipherment. The failure of many of these attempts may be obvious to me and others, but is often not at all apparent to their proponents. This should not be so, as there is available a very simple two-step test one can use to determine a the correctness of any proposed solution. These tests are derived from several sources, but most notably the works of Elizebeth and William Friedman. Both of these conditions MUST be met in order for a claimed decipherment to be deemed correct:

1) Repeatability: If one can explain the system used, to a second person, and then they derive the same cipher text as the solution’s proponent, then part one has been passed.

2) Meaning: If the system is repeatable, as above, then the results must have meaning.

But the problem is that, in a very many instances, a decipherer believes they have met the above two-step criteria, and passed the test, when they have not. As I asked, why? It usually is because they have been trapped by various pitfalls, in which “ways around” the tests… although invalid… seem to obviate a need to meet those basic requirements, or convince them that they have met them. I’d call these “pitfalls” then, and are very dangerous. By not recognizing them in one’s own work, by stepping over or around them, one can become victim to spending huge amounts of life and effort continuing to work on what is a failed scheme.

List of potential pitfalls to decipherment:
1) Defending subjective input: It is normal, in many cases, for a decipherer to insert speculative plain text characters, words or phrases in order to test cipher schemes: Such as using the name of an item in an illustration, or a word which might make sense in the context of the work in general. But then a problem arises if the system is not seen as flawed when it needs to alter itself in order to allow for new results from new cipher text. That is, if the scheme needs to be altered to fit the new, speculative plain text, then this should be seen as a test of the scheme, and that the scheme has failed. It this “red flag” is missed, there is no limit to the complexities that that a decipherment scheme/system can grow to, in order to continue to adapt to speculative, desired, plain text. But we know the solution is wrong, because no cipher system needs to adapt for individual words. No matter how complex, a proper cipher will work consistently to decipher the text without needing to adapt as it progresses.

The signs to look for are if the decipherer will not try new text, and only sticks to small section of selected text. Another is if the system is not shared, often said to be “too complicated” for others, or if there is no expressed, relatable system at all. Sometimes a system is shared, but cannot be used to create meaningful text by a second party, and then this shows the solution is wrong.

2) Multiple Plain Text Choices: If at any point in the decipherment process, choices of multiple possible plaintext letters or words are needed, the number of outcomes quickly rises. The level of subjectivity in such decipherment schemes can be so high that many different translations of meaning, or near meaning, can be derived from the same cipher text. What to look for are charts with columns or rows of alternate “translations” for one cipher character or word.

Often the pitfall is that the translator has a concept of what they think the content is, or should be, so then the choices they make for the output don’t seem subjective to them, but the only logical possibility from the many variations conceivable during the process. In these cases, a proposer may believe the “repeatability” requirement must be bent, as there needs to be a mutual understanding of what choices should be made in the process, and that the original users of the cipher would possess that understanding. Another excuse is that the original creator alone possessed the necessary understanding, as they never intended anyone else to read the plain text. They then feel that, possessing this understanding, they alone come to the proper results. This is a dangerous pitfall, as there is really no way to convince oneself, or others, that this is unlikely to be the case. But it is  historically unknown as a concept, if that helps.

Another good test is falsification, as if many other results can be derived by using other choices of characters, then at the very least it should be apparent that any particular results cannot be known if correct or if in error. That is, any such results are virtually indistinguishable from guesswork, and therefore, the solution can be assumed incorrect.

3) Anagrams: Similar to the above, if any string of plain text results needs to be reordered to derive meaning, the chance are the derived meaning is purely speculative on the part of the decipherer. It is true that anagrams have historically been used to hide information, but rarely used to hide it in a way that another person could readily derive the meaning without help. This is a common misconception about various historic uses of anagrams, such as those by Roger Bacon and Galileo. They were using anagrams to insert a “watermark” of sorts in the test, so that they could later reveal that they were privy to some knowledge, so that they could later claim precedence to that knowledge, but without revealing it to unwanted eyes. But the purpose was not for another party to discern the meaning on their own, as it needed help from the creator to find it.

But if for whatever reason anagrams might be suspected, after only a few characters the possible translations quickly rises beyond any sensible use of hiding plain text, since many alternate plain texts can be derived from even short strings of plaintext characters. This means it becomes purely subjective, and almost from the start of the process. This was one of the pitfalls that William Romaine Newbold fell into, when attempting to decipher the Voynich Manuscript cipher text.  He derived long strings of characters, from which he, or really anyone, could assemble some resemblance of meaningful text. Newbold was assuming Roger Bacon content, however, and so he manipulated his anagrams until he found it.

4) Small Set of Input/Output: If a scheme seems to work for a select few words, usually under 20 (and rarely approaching it), and then the decipherer stops attempting their scheme on new words, it then becomes a pitfall. There are several such claims of translation, some of which have made it into mainstream media. In order to avoid this, one must make certain that their scheme continues to work on a larger set of cipher words, and that they do not stop at early, perceived, “successes”. Likewise, for those attempting to determine the validity of such small set solutions, they should first insist that the proponent apply their method to a larger set (editors and producers take note). Until they do so, the solution should not be taken seriously.

5) Lack of plain text meaning: This is only a pitfall if it is not seen as a failure of a decipherment, as per the two step test at the top of this post.

It is of course easy to translate the Voynich text, or any cipher I suppose, in a way that produces “something”. But if that something has no discernible meaning, it is wrong. The pitfall comes when one does not accept that this lack of meaning exists, or that it is important. For the former, one may think the encipherer must have had a meaning in mind for the resultant ramblings; for the latter, that it is simply not a problem that there is no meaning- that is, they simply do not address it, to themselves or to others. But if one wants to self test their scheme, or the scheme of others, then lack of meaning is a sign of an incorrect solution.

Nonetheless, there are many claimed solutions which produce meaningless text… in the case of the Voynich, this almost always involves long strings of repetitive words and phrases, as the cipher text of the Voynich has much repetition. It is often claimed that these meaningless solutions must be either song, chants or poetry we don’t understand, or lists of recipe ingredients or formulas, which we simply do not know how to use due to our modern viewpoint. In reality, they are simply gibberish, and not the solution.

6) False Patterns: It is human nature to seek, and then find, patterns in randomness. But the ability can become a pitfall when left unchecked in the practice of decipherment. This pitfall not only arises in seeing patterns in the text, but in illustrations, also. And a very good self-test is, again, if the results have meaning. There has to be a greater context for the pattern, or it is probably purely subjective. If one does not make an attempt to find that greater context, or diminishes the importance of doing so, they may never see the error in the scheme. But also, like many schemes which are vulnerable to subjective errors, a pattern may be “seen” which matches a preconception for meaning and context, and this then is mistakenly thought to be validation of the pattern. Another good check in that case is falsification… that is, seeing if other patterns can be also found, with other meanings, in the same text or illustration… by oneself, or better yet, by others. If they can, then there is no way of knowing which one, if any, may be correct, and the solution is in error.

7) Skipping the process: This is rare, but there are cases of decipherers simply inserting plain text for cipher characters or words, purely speculatively (or maybe loosely based on some belief of what the Voynich may be about): This can be food sources, if one thinks it a cookbook; or chemical names, if one believes it a formula book; or maybe herbal and plant names, if one believes it a pharmacopeia. Great lists of meanings for characters or words may be offered. The tests as usual are repeatability and resultant meaning, but these are often avoided in this case. One case I’ve seen is that the proponent wrongly claims repeatability because of anyone using the list will arrive at the same exact results as they do. But the point being missed in this case is that the list is the decipherment, and the list must therefore be repeatable, and is not. And so, repeatability has actually failed. Another danger is that one may simply surmise there is a missing code book that was originally used to derive the lists of meanings; and that the results are (again) simply not understandable to a modern mind. But if one is objective, and realizes that any list of words can be substituted for one’s own (for any speculative code book), and that any results can be claimed meaningful, as in #5, they will see the error, and so can avoid this pitfall.

Conclusion: There are many historical, and even contemporary, instances of decipherment attempts which have consumed large portions of their proponent’s life and effort. In the case of the many failed Baconian theories, several individuals spent decades in a fruitless pursuit of hidden meanings in Shakespeare’s texts. But there are many other cases like this, and unfortunately even in our time. The Voynich cipher has engendered many dozens, if not hundreds, of its own instances of this unfortunate effect. I’ve personally witnessed several cases in which very brilliant and sincere people have fallen into the traps I relate above, and so are expending their precious life energy, year after year, to baseless chimeras so easily avoided- if but a small amount of careful introspection would be applied. I offer my observations, above, as a well-meant and helpful warning to them.