Homophonic substitution

[doranchak]

I agree with you and no one can deny the many odd things in z340 (pivots, even/odd, prime phobia and so on). Like others I spent a lot of time to find out which cipher methods can produce these characteristics. But there is still a chance that some (or even all) of them could be coincidences (doubt it).

Yes, there is still the chance they could be coincidences. I wish I had access to a qualified statistician who could help us rank the improbability of the various phenomena. Some of them seem improbable, but if you look for 1,000,000 improbable sequences or patterns, you are likely to find one. One sequence by itself may be improbable, but the fact that you searched for a million different ones makes it more likely that you'll find one.

Another way to approach this might be to keep looking at Z408 to see if any other interesting and improbable phenomena can be found there, but truly are resulting from chance alone.

I like to listen a german Podcast called "Alpha Centauri" which is all about astrophysics. An episode about the search of dark matter begins with a story about a man who is searching for something in a dark night below a street lamp. A policeman comes around and asks the man what he is searching for. The man answers: "I am searching my car keys". The policeman starts to help him in his search for the keys. Half an hour later he asks: "Are you sure that you have lost your keys right here?". The man answers: "No, I think I have lost them somewhere else but this is the only illuminated place".

My approach is "what would I have done if I created a cipher which has been cracked in a couple of days?". Which strategies could lead to a harder but still solvable cipher? Well, this approach will become totally useless if he never intended to produce a crackable cipher.

I think that is a very good approach. I worry that in his laziness, he made the cipher harder but in his haste, screwed up and made the plaintext forever inaccessible. It is one of the reasons I want to more thoroughly rule out classical, known cipher schemes, since highly specific encipherment errors on unknown schemes is a much larger search space! (Once I can see for sure the car keys aren't illuminated by the lamp, I can start wandering around in the dark).

[smokie treats]

I did have one idea on the way home from work today. I am not aggressively working on the 340 right now, but am starting to learn JavaScript for a break.

Anyway, one original variation of a classical cipher would be playfair, but instead of encrypting two plaintext at a time that are next to each other, you could encrypt two plaintext at a time that are not next to each other.

If you encrypt side by side plaintext, then the result is high diffusion of period 1 repeats. But if you encrypted plaintext in pairs that are at a distance from each other, I am thinking that there would not be as much diffusion. You could encrypt plaintext 1 and 20 together, 2 and 21 together, 3 and 23 together, all the way to 19 and 38 together. Then start again with 39 and 58, 40 and 59, 41 and 60, etc.

Maybe someday I will take a closer look at that to see if the result would resemble the 340 in some ways.

[smokie treats]

I spent some time on the 340 today. I have had some nagging issues about transposition, and one of them is whether a few transcription nulls, or transcription skips for the mirrored 340, could produce so many period 29 repeats and the pivots.

Here is the 340 mirrored, with period 29 repeat positions colored. 75 of the 170 positions in the bottom half are colored, which makes 44.1%. If the count of period 15 repeats is boson higgs significant, and the count of period 29 repeats astronomical, across the entire message, then what about just the bottom half? The pivots for the mirrored version are period 29.

period.29.mystery.1.png

I made a spreadsheet suite to make period 15 messages, with an inscription rectangle of 22 columns x 15 rows and ten leftover, not transcribed plaintext at the end. Then I encoded with homophonic cycles, making about 25% of symbol selection random within cycle groups. Then I compared the messages with messages that had transcription skips introduced at random positions, with varying counts of transcription skips. For many of the messages, I limited the skips to the bottom half, and kept the count of skips to 4 or less. But I also tried a lot of variations.

For many of the messages with just a few skips, the count of period 15 repeats would drop slightly, and the count of period 29 repeats would increase slightly. But with no variation was I able to replicate a count of 65 period 29 repeats. Not even close. I could get 50 period 29 repeats with 3 skips, but getting more than that was difficult.

I am going to say that the null / skip idea for causing the pivots is probably not correct, and go back to the drawing board. I am still thinking that the message is some sort of transposition, but maybe he changed transcription direction for the bottom half, or did something else. There must be some sort of scheme that would make so many period 29 repeats, with 44.1% of the positions in the bottom half colored, and that must be what created the pivots.

[doranchak]

Thanks for the post, smokie - very interesting analysis despite the negative result on nulls.

Here is the 340 mirrored, with period 29 repeat positions colored. 75 of the 170 positions in the bottom half are colored, which makes 44.1%. If the count of period 15 repeats is boson higgs significant, and the count of period 29 repeats astronomical, across the entire message, then what about just the bottom half? The pivots for the mirrored version are period 29.

That is a very interesting question. I've never before thought about looking at the regional biases of the periodic ngrams. I'm going to add that to my ever-growing todo list. The link between the period 29 mirrored pivots and the bottom half bias is an interesting connection.

I am going to say that the null / skip idea for causing the pivots is probably not correct, and go back to the drawing board. I am still thinking that the message is some sort of transposition, but maybe he changed transcription direction for the bottom half, or did something else. There must be some sort of scheme that would make so many period 29 repeats, with 44.1% of the positions in the bottom half colored, and that must be what created the pivots.

Interesting you should mention that. I chatted with Nick Pelling recently about the Zodiac ciphers. He is strongly in favor of a "Z340 was made in two separate halves" hypothesis. You may be on to something by investigating in that direction. Nick goes into more detail about his hypothesis here: http://ciphermysteries.com/2015/06/14/t ... ng-opinion

[Mr lowe]

stnotmuchfilescal
lmostsschneremshe
sayertrinetostact
ledontoutintalent
sasionshuinsuchhe
ttionayanhinstmod
srelinaaretyllsme
trandburnbesitdom
sfeescarytteornar
diedoubluateandim
buriamandsliusiea
tthirtemartimemtl
rformantrianchind
stutorecubaidusmi
faninethoralpreda
romecoutistsloint
didearnisuponresc
alienceofcoisedem
mersaluestedsubsa
ccleasasuitairssa
Hi Smokie and Doranchak hope all is good... can you please explain why the first coloumn down does not hold some importance SLSLSTST to me it is so sequential that it can not be random. could it be filler or something like HAHAHAHa (maybe the whole thing needs more analysis) just got back from india and that sequence has been nagging me for the last few weeks.

[smokie treats]

... can you please explain why the first coloumn down does not hold some importance SLSLSTST to me it is so sequential that it can not be random. could it be filler or something like HAHAHAHa (maybe the whole thing needs more analysis) just got back from india and that sequence has been nagging me for the last few weeks.

I don't know. This is the 340?

[Mr lowe]

yes it is smokie.. its a columnar scy.. did it two months ago..

this is it before it was run through a solver

ABCDEFGHIJKLMNHOL
PQDNBRSHTUVWMQSIX
AOYZaEbKcdBDeBfHB
PghDCBiGBjUEkLXlE
SmnKoUeIpqUSrHIIX
BsqoUfYktIjlSBQiu
SvVPwUxmWMEYLLNFV
BWOtyz0v12ZeKByoF
nJddeHk3YBBZD4Ukb
yjMyDr2LrOBVm1uwF
z0WwxFOcyePjGRKZO
BB5KWBZQk4BKQgQBL
vJ64FmlBaKfUHTqCy
ABGBia7HpzOjurnFj
JmljUXs5DvmP84Vhm
4iFXHopsjSERPDjUB
uKh9k3lqSr8DtbdSH
xLqdC!XDJ!DKN9ydF
FZ4RkL0MABXhNG2NO
HHL7xeknrjskjbSex

[doranchak]

My guess is that, even if you shuffled that candidate plaintext, many shuffles would contain similar sequences, but not necessarily in the same places. For example, you might have "EBEBEAEA" in some other column, starting on some other row. So I think you have to consider that any letters appearing to repeat that way would attract your attention. But, I don't yet know exactly how probable those sequences are, without doing the shuffle test first. I can try to do it some time.

[doranchak]

Here is a quick estimate of the probability, based on the assumption that the candidate plaintext is supposed to represent a message in English, and thus would have an English frequency distribution for letters from its alphabet (13% for E, 9% for T, 8% for A, on so on).

The pattern you are observing is: SLSLSTST

So that's 4 S's, two L's and two T's. We can break that down into three events:

1) We happened to select four identical letters. For example, it could be four S's, four T's, or four U's, and so on, for every letter of the alphabet.
2) We happened to select two identical letters. For example, it would be two L's, two M's, or two N's, and so on, for every letter of the alphabet.
3) We did #2 again (two T's, two U's, or two V's, and so on, for every letter of the alphabet).

Or, say it like this:

1) We picked four letters that are the same.
2) We picked two letters that are the same.
3) We again picked two letters that are the same.

Based on English letter frequencies, the probability of picking the same letter twice works out to 0.065 (6.5%, or about 1 in 15.3). The probability of picking the same letter four times works out to 0.00050 (0.05%, or about 1 in 2003).

Thus, the probability of all three of the above events occurring is: p1 * p2 * p3 = 0.00050 * 0.065 * 0.065 = 0.0000021125 (about 1 in 470000).

That's the estimated probability of that interesting pattern forming vertically at row 1, column 1. Seems rare, right? But we need to also count the chance of the pattern showing up at other rows and columns. The pattern can happen anywhere except the last 7 lines, otherwise the pattern would fall off the end. So that means there are 17 columns and 20-7=13 rows available to pick from. That's 17*13 = 221 possible starting points for the pattern. Which means we need to count the above probability 221 times. 221*0.0000021125 = 0.000467 (about 1 in 2142).

But we're not quite done yet. What if we saw the same pattern going horizontally instead of vertically? It turns out that there are 10*20 = 200 spots the horizontal pattern could appear. The odds of that are: 200*0.0000021125 = 0.0004225 (about 1 in 2367).

Thus, the odds of the pattern appearing horizontally or vertically are: 0.000467+0.0004225 = 0.0008895 (about 1 in 1124).

This is a very rough estimate and doesn't account for other similar kinds of patterns that we would notice. I would put this in the category "worth looking at further."

What scytale width/period did you use? Can you show the transcription of the cipher prior to the scytale transposition step? I'm really curious now about the sequentially repeating bigrams (APAP, SBSB).

[doranchak]

Here's what your scytale looks like when rotated and flipped, so the repeating bigrams read from left to right:

Screen Shot 2016-08-15 at 3.47.23 PM.png

The other things I noticed in there besides your APAPSBSB pattern:

The "BL" bigram appears very close to its repeat.

The "0k" bigram appears in the same row as its repeat.

The "FO" bigram appears in the same row as its repeat.