Page 11 of 13

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 3:37 am
by Jarlve
smokie treats wrote:Here are the new smokie66ABC messages. The irony is that if I untranspose at period 20 they will solve easily because I only added some symbols to the keys for the top and bottom rows to get the regional biases and high P20 repeats. If you use different keys, then no P20 repeats as we have determined.

Thank you for your ciphers smokie. Are you sure that 2 keys sufficiently diffuse bigrams?

Plaintext 1 to 100, 2 individual keys top/bottom totalling 10000 samples. Ioc mean: 2240.99. Bigrams mean: 34.51.

Code: Select all
Ioc:

- Mean: 2240.9998
- Variance: 8288.891199959988
- Standard deviation: 91.04334791713224
- Sigma of 2236: -0.05491669753347465

Code: Select all
Bigrams:

- Mean: 34.5131
- Variance: 30.86042839000066
- Standard deviation: 5.555216322520722
- Sigma of 37: 0.4476693355609145

Code: Select all
AZdecrypt solve substitution score (1000000 iterations, 10 restarts):

- Mean: 20323.37121536038
- Variance: 1782381.007860658
- Standard deviation: 1335.058428631743
- Sigma of 20351: 0.02069481308614695

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 3:47 am
by Jarlve
borkky wrote:I won't judge who was first with the idea. Are you ok if i write "credit Jarlve & smokie treats"?, since he detailed the separation more?

If Z340 ever gets solved, I am sure it will be a collective effort, built on all the work made by many users here & elsewhere.

Perhaps you could just credit the cipher section of this forum? That would embrace the collective effort spirit, in which I believe too.

There is a double entry in the wiki:

If you highlight all occurrences of certain symbols, they seem to avoid the middle of the cipher text. (Source: Jarlve)
Some symbols are exclusive to the first 6 and last 6 rows. An unusual number of symbols are missing from the middle 8 rows. (Source: smokie treats)

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 5:07 am
by Largo
I had some thoughts this morning. Sorry if any of this has already been mentioned. In this case simply ignore it =)

I am still irritated by the frequent occurrence of the "+"symbol. If more than one key was used, there are few explanations:
- The "+"symbol maps to the same plaintext letters in both keys.
- The "+"symbol maps in both keys to a frequently occurring plaintext letter (E, T, A, O, I, N etc.).
- It is actually a polyphone (but why should you do that in this case? That seems unrealistic to me)

z340 was written on a rather large sheet that did not fit in the envelope. It must have been folded. If you look at the high-resolution scan, you can see where it was folded. Eventually, Zodiac might have intended this to display regions. I'll take a closer look at that on occasion.


If different keys were used alternately, this could be the cause of incomplete but not completely absent cycling. Has anyone ever done some tests before? For example, rows 1-6 and 15-20 with key A, then rows 7-14 encoded with key B and check the cycling? If not, I'd like to put that on my todo list. I now simply call this type of encryption "cake encryption" because, as with a cake, there are different "layers". So I don't always have to refer to rows.

When I needed a new key for testing, I made it relatively easy and used a trick. It also works very well if you work with paper and pencil. That's how it works:

opposed_key.png


As you can see, I left the order of the cipher symbols unchanged. Only the plain text letters were swapped in their order. Of course, this has an effect on the efficiency of the key, since the result is no longer "smooth". It would be interesting to know how this affects the cycles of a cake cipher. And the frequent occurrence of the "+"symbol could also be explained by this, since a correlation could take place depending on the key.

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 5:15 am
by borkky
Jarlve wrote:Perhaps you could just credit the cipher section of this forum? That would embrace the collective effort spirit, in which I believe too.


Sounds good, fixed now!

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 5:16 am
by Largo
Okay...check this out:

Rows 1-6, followed by rows 15-20:

Code: Select all
HERabcdVPeIfLTGgh
Nb+BjkOlDWYmnoKpq
BrstM+UZGWjqLkuHJ
SbbvdcwoVxbO++RKg
yzM+u12hI7FP+34e5
bwRdFcO-ohCeFagDj
rBXfos4zCEaVUZ7-+
ItmxuBKjObdmpMQGg
RtT+Lf#Cn+FcWBIqL
++qWCuWtPOSHT5jqb
IFehWnv1ByYOBo-Ct
aMDHNbeSuZOwAIK8+


Perfect 5-Cycles:
Code: Select all
VPDrsVPDrsVPD (72)
PDrsUPDrsUPD (56)
PDrsxPDrsxPD (56)
PDrszPDrszPD (56)
PDrs7PDrs7PD (56)
PDrUxPDrUxPD (56)
PDrU7PDrU7PD (56)
PDrz7PDrz7PD (56)
PDsUxPDsUxPD (56)
PDsU7PDsU7PD (56)
PDsz7PDsz7PD (56)
PrsUxPrsUxP (42)
PrsU7PrsU7P (42)
Prsz7Prsz7P (42)
GkrsUGkrsUG (42)
GDrsUGDrsUGD (56)
DrsUvDrsUvD (42)
DrsUxDrsUxD (42)
DrsUyDrsUyD (42)
DrsU1DrsU1D (42)
DrsU7DrsU7D (42)
DrsU5DrsU5D (42)
DrsvyDrsvyD (42)
Drsv1Drsv1D (42)
DrsxyDrsxyD (42)
Drsx1Drsx1D (42)
Drsx5Drsx5D (42)
Drsz1Drsz1D (42)
Drsz7Drsz7D (42)
Drsz5Drsz5D (42)
Drs75Drs75D (42)
Drs45Drs45D (42)
DrUvyDrUvyD (42)
DrUv1DrUv1D (42)
DrUxyDrUxyD (42)
DrUx1DrUx1D (42)
DrUx5DrUx5D (42)
DrU75DrU75D (42)
Drz75Drz75D (42)
DsUvyDsUvyD (42)
DsUv1DsUv1D (42)
DsUxyDsUxyD (42)
DsUx1DsUx1D (42)
DsUx5DsUx5D (42)
DsU75DsU75D (42)
Dsz75Dsz75D (42)
rsUvyrsUvy (30)
rsUv1rsUv1 (30)
rsUxyrsUxy (30)
rsUx1rsUx1 (30)
rsUx5rsUx5 (30)
rsU75rsU75 (30)
rsz75rsz75 (30)


Perfect 6-Cycles:
Code: Select all
PDrsUxPDrsUxPD (72)
PDrsU7PDrsU7PD (72)
PDrsz7PDrsz7PD (72)
DrsUvyDrsUvyD (56)
DrsUv1DrsUv1D (56)
DrsUxyDrsUxyD (56)
DrsUx1DrsUx1D (56)
DrsUx5DrsUx5D (56)
DrsU75DrsU75D (56)
Drsz75Drsz75D (56)


I haven't quite understood the scoring yet, but compared to the unmodified z340 it seems amazing to me

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 5:52 am
by doranchak
Jarlve wrote:There is a double entry in the wiki:

If you highlight all occurrences of certain symbols, they seem to avoid the middle of the cipher text. (Source: Jarlve)
Some symbols are exclusive to the first 6 and last 6 rows. An unusual number of symbols are missing from the middle 8 rows. (Source: smokie treats)

Oops! I seem to be losing track of all the observations. I went ahead and merged them together.

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 6:47 am
by borkky
Largo wrote:Okay...check this out:

Rows 1-6, followed by rows 15-20:

Code: Select all
HERabcdVPeIfLTGgh
Nb+BjkOlDWYmnoKpq
BrstM+UZGWjqLkuHJ
SbbvdcwoVxbO++RKg
yzM+u12hI7FP+34e5
bwRdFcO-ohCeFagDj
rBXfos4zCEaVUZ7-+
ItmxuBKjObdmpMQGg
RtT+Lf#Cn+FcWBIqL
++qWCuWtPOSHT5jqb
IFehWnv1ByYOBo-Ct
aMDHNbeSuZOwAIK8+


Perfect 5-Cycles:
Code: Select all
VPDrsVPDrsVPD (72)
PDrsUPDrsUPD (56)
PDrsxPDrsxPD (56)
PDrszPDrszPD (56)
PDrs7PDrs7PD (56)
PDrUxPDrUxPD (56)
PDrU7PDrU7PD (56)
PDrz7PDrz7PD (56)
PDsUxPDsUxPD (56)
PDsU7PDsU7PD (56)
PDsz7PDsz7PD (56)
PrsUxPrsUxP (42)
PrsU7PrsU7P (42)
Prsz7Prsz7P (42)
GkrsUGkrsUG (42)
GDrsUGDrsUGD (56)
DrsUvDrsUvD (42)
DrsUxDrsUxD (42)
DrsUyDrsUyD (42)
DrsU1DrsU1D (42)
DrsU7DrsU7D (42)
DrsU5DrsU5D (42)
DrsvyDrsvyD (42)
Drsv1Drsv1D (42)
DrsxyDrsxyD (42)
Drsx1Drsx1D (42)
Drsx5Drsx5D (42)
Drsz1Drsz1D (42)
Drsz7Drsz7D (42)
Drsz5Drsz5D (42)
Drs75Drs75D (42)
Drs45Drs45D (42)
DrUvyDrUvyD (42)
DrUv1DrUv1D (42)
DrUxyDrUxyD (42)
DrUx1DrUx1D (42)
DrUx5DrUx5D (42)
DrU75DrU75D (42)
Drz75Drz75D (42)
DsUvyDsUvyD (42)
DsUv1DsUv1D (42)
DsUxyDsUxyD (42)
DsUx1DsUx1D (42)
DsUx5DsUx5D (42)
DsU75DsU75D (42)
Dsz75Dsz75D (42)
rsUvyrsUvy (30)
rsUv1rsUv1 (30)
rsUxyrsUxy (30)
rsUx1rsUx1 (30)
rsUx5rsUx5 (30)
rsU75rsU75 (30)
rsz75rsz75 (30)


Perfect 6-Cycles:
Code: Select all
PDrsUxPDrsUxPD (72)
PDrsU7PDrsU7PD (72)
PDrsz7PDrsz7PD (72)
DrsUvyDrsUvyD (56)
DrsUv1DrsUv1D (56)
DrsUxyDrsUxyD (56)
DrsUx1DrsUx1D (56)
DrsUx5DrsUx5D (56)
DrsU75DrsU75D (56)
Drsz75Drsz75D (56)


I haven't quite understood the scoring yet, but compared to the unmodified z340 it seems amazing to me


Excuse my ignorance, but what is a perfect 5-cycle? The patterns looks highly fascinating but i grasp to understand what you did there :-)

Is it this stuff? viewtopic.php?f=81&t=3616 (Cycle types thread)

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 7:10 am
by smokie treats
Jarlve wrote:Thank you for your ciphers smokie. Are you sure that 2 keys sufficiently diffuse bigrams?


That's what I came up with. I made smokie 64 last month, and had to select a chunk of the book Brave New World that would do it.

Largo wrote:When I needed a new key for testing, I made it relatively easy and used a trick. It also works very well if you work with paper and pencil. That's how it works:

opposed_key.png


As you can see, I left the order of the cipher symbols unchanged. Only the plain text letters were swapped in their order. Of course, this has an effect on the efficiency of the key, since the result is no longer "smooth". It would be interesting to know how this affects the cycles of a cake cipher. And the frequent occurrence of the "+"symbol could also be explained by this, since a correlation could take place depending on the key.


I did something similar last month where I transposed and encoded 768 messages from the book Brave New World. First I transposed at period 20, then I used a tabula recta to encode the top / bottom and a different shift for the middle. I even checked all of the shift differentials to find out how to come up with a middle section of 8 rows that had only four unique symbol positions. EDIT: Then I encoded homophonic. There is a very detailed explanation about the experiment starting at the top post here:

viewtopic.php?f=81&t=3196&start=750

It seems like so much longer than a month ago.

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 7:28 am
by doranchak
borkky wrote:Excuse my ignorance, but what is a perfect 5-cycle? The patterns looks highly fascinating but i grasp to understand what you did there :-)

Here is an illustration. Largo's modified cipher:

Image

Highlight all occurrences of the symbols V, P, D, r and s:

Image

Read them out in order, and you get: VPDrs VPDrs VPD
It is a perfectly alternating sequence, potentially of homophones for a single plaintext letter. Z408 was filled with such patterns.

Re: Unigram distance curiosity

PostPosted: Mon Dec 11, 2017 7:49 am
by Largo
borkky:
You can find this feature in Jalve's "AZDecrypt" at menu "Stats 2".