Cycle types

Re: Cycle types

Postby Jarlve » Thu Nov 23, 2017 2:49 pm

doranchak wrote:
smokie treats wrote:I have some ideas. Maybe instead of adding to the cycle type list, I think that, since we can find all of the consecutive alternations like ABABAB and score them, then we can also find, count and score other arrangements, like AABAAB or ABAABA as well.

I implemented some code to do that a while ago. It takes a set of symbols then looks for the longest repeating sequences involving those symbols.

Here are the raw results:

Good idea guys, I will add it to the list:

The pattern cycle, which repeats a pattern that includes at least one repeat of a substitution, 122333 - 122333 - 122333, 1233455 - 1233455 - 1233455. (doranchak, smokie treats)

EDIT: definition
Last edited by Jarlve on Sat Nov 25, 2017 5:47 am, edited 1 time in total.
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Cycle types

Postby Jarlve » Thu Nov 23, 2017 2:58 pm

Quicktrader wrote:Jarlve,

I agree with you regarding the different cycle types representing the sequences of homophones used. Some homophones have been used frequently in one part of the cipher, others only in other parts of the cipher. This indicates some sort of regional or semi-regional sequence type as the cycles had been broken up by purpose to complicate the cipher even more. It could also be the reason why the ZDK tool alone can't handle the cipher.

I currently do some runs with my (Python) FCCP model. In fact it works fine and due to its focus on certain regions of the ciper it works independently of any homophone cycles, too. Depending on the basic configuration, one run does between 200m and 2,000m variations on one region only, with approximately 500,000 to a million 'results' (e.g. words found of length >4). Most of those are duplicates, only a few hundred (200-300) words remain (plaintext). First I'll finish the runs on one sequence, a later goal will be to cross-check the results of different strings to each other. It's a good way to completely avoid the cycles..

QT

Hey Quicktrader,

To clarify, the use of regional or semi-regional cycles in the 340 is a hypothesis by moonrock. Currently, I do not know where to stand on this matter. Though I find it interesting in relation to the symbols that only appear in the top and bottom 6 rows. Does your FCCP model work on the 408?
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Cycle types

Postby smokie treats » Fri Nov 24, 2017 5:03 am

Jarlve wrote:
doranchak wrote:
smokie treats wrote:I have some ideas. Maybe instead of adding to the cycle type list, I think that, since we can find all of the consecutive alternations like ABABAB and score them, then we can also find, count and score other arrangements, like AABAAB or ABAABA as well.

I implemented some code to do that a while ago. It takes a set of symbols then looks for the longest repeating sequences involving those symbols.

Here are the raw results:

Good idea guys, I will add it to the list:

The pattern cycle, which repeats a pattern that includes at least more than one repeat of a substitution, 122333 - 122333 - 122333, 1233455 - 1233455 - 1233455. (doranchak, smokie treats)


I have decided to start working on it. Break down all two symbol "cycles" into chunks of different sizes, just A and B, and look for patterns. Patterns could be reversed too. Like AAABABABAB BABABABAAA. Something like that may be more like we are looking for.

Or an improbable concentration of a certain pattern by region. Maybe more ABAB in the middle 8 for example.

Then there are "cycles" with odd numbers of symbols and even numbers of symbols. With odd numbers, there might have to be an option for removing the middle symbol. Like AAABABABABABABABAAA. Remove the middle B first, then make comparison AAABABABA ABABABAAA.
User avatar
smokie treats
 
Posts: 1620
Joined: Thu Feb 19, 2015 1:34 pm
Location: Lawrence, Kansas

Re: Cycle types

Postby Jarlve » Fri Nov 24, 2017 11:30 am

smokie treats wrote:I have decided to start working on it. Break down all two symbol "cycles" into chunks of different sizes, just A and B, and look for patterns. Patterns could be reversed too. Like AAABABABAB BABABABAAA. Something like that may be more like we are looking for.

Or an improbable concentration of a certain pattern by region. Maybe more ABAB in the middle 8 for example.

Then there are "cycles" with odd numbers of symbols and even numbers of symbols. With odd numbers, there might have to be an option for removing the middle symbol. Like AAABABABABABABABAAA. Remove the middle B first, then make comparison AAABABABA ABABABAAA.

I am working on it to and hope to share observations in some time. It is funny that you mention cycles with odd and even number of symbols since that is also what I have been doing for palindromic cycles and I also let the middle symbol be ignored.

Here is an example of a 3-symbol cycle that appears in the 340 that could be classified as an imperfect shortened cycle. Though shortened and lengthened cycles as a full blown cycle scheme are ruled out. More on that later.

Code: Select all
Cycle: OMZOMOOMZOMOMZOMOOMZO
----------------------------
OMZ
OM
O
OMZ
OM
OMZ
OM
O
OMZ
O
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Cycle types

Postby Largo » Fri Nov 24, 2017 12:33 pm

Jarlve wrote:Different homophonic substitution keys for odd and even positions?

Wow, I'm really impressed how quickly you found out. However, I still wonder how to solve such an encryption completely. Have you succeeded in doing so? Do you have the plain text?

Jarlve wrote:If yes, then you underestimate the work that has been done so far.

Obviously, I underestimated that. If we once meet in Germany, I would be happy to donate as much Orangensaft as you like: D
Largo
 
Posts: 455
Joined: Tue Jun 14, 2016 4:38 am
Location: Frankfurt, Germany

Re: Cycle types

Postby smokie treats » Fri Nov 24, 2017 4:32 pm

Here is my first observation. I reduced all L=2 cycles to AB and counted and sorted them by count for both the first three hundred and forty symbols of the 408 and the 340.

408. We know that he most perfectly cycled, and here are the top 20 by count and the consecutive alternations were detected against all other possible arrangements. Left column is by count, right column is the arrangement.

12 ABABABABABA
8 ABABABABA

7 AAABAAAA
6 AAABAAA
6 ABAABABAA
6 ABABABBABBAB
6 ABABBABABBAB
5 ABABAABAAA
5 ABABAABABABAB
5 ABABABABAA
5 ABABABABABAA
5 ABABABABABABB
5 ABABABABBA
5 ABABABABBAB
4 AAABAA
4 AABAA
4 AABAAA
4 AABAAAABBA
4 AABAABABA
4 AABABABBABA

340. There are a lot more arrangements with consecutive alternations, but the number of consecutive alternations is much lower ( see red ). Considering that detection worked well with the 408, I am wondering about ABAABA for starters ( see blue ). And then AABAAB ( see green ).

21 ABABA
19 ABABAA
18 ABAABA
18 ABAB
15 ABAABAA
14 ABABABA
14 ABBAB
10 AABAABAAB
10 ABAAAAB
10 ABABBABA
10 ABBAAB
9 AABAAB
9 AABAABAA
9 AABBABA
9 ABABAAB
9 ABABAB
8 ABAAAB
8 ABAAABA
8 ABAAABAA
8 ABABAABAAB

Maybe he just followed some different patterns besides just ABABAB. EDIT: Maybe he used different patterns for different letters.
User avatar
smokie treats
 
Posts: 1620
Joined: Thu Feb 19, 2015 1:34 pm
Location: Lawrence, Kansas

Re: Cycle types

Postby Largo » Fri Nov 24, 2017 6:12 pm

smokie treats wrote:Then there are "cycles" with odd numbers of symbols and even numbers of symbols. With odd numbers, there might have to be an option for removing the middle symbol. Like AAABABABABABABABAAA. Remove the middle B first, then make comparison AAABABABA ABABABAAA.

Jarlve wrote:I am working on it to and hope to share observations in some time. It is funny that you mention cycles with odd and even number of symbols since that is also what I have been doing for palindromic cycles and I also let the middle symbol be ignored.



I don't know (again) whether we're talking about the same thing. Do you mean that the symbols of z340 should be sorted by even/odd and then examining the cycles? This improves the perfect 4- and 5 cycles found by azdecrypt.

Sort z340 even before odd positions:
Perfect 4-cycles:
Code: Select all
5lX25lX25 (30)
5#8X5#8X5#8 (56)
5#825#825#8 (56)
5#X25#X25# (42)
58X258X258 (42)
5xX25xX25 (30)
PlX2PlX2P (30)
P3xXP3xXP (30)
P3x2P3x2P (30)
P3X2P3X2P (30)
P#8XP#8XP#8 (56)
P#82P#82P#8 (56)
P#X2P#X2P# (42)
P8X2P8X2P8 (42)
PxX2PxX2P (30)
3xX23xX2 (20)
#8X2#8X2#8 (42)


Perfect 5-cycles:
Code: Select all
5#8X25#8X25#8 (72)
P3xX2P3xX2P (42)
P#8X2P#8X2P#8 (72)


Just to be on the safe side: Are you talking about the fact that Zodiac may have written the plaintext horizotally left to right, but encoded it in a different direction or something like that (odd before even or from top to bottom)? This would ensure that a plain text written in the usual direction does not contain any traitorous ngrams after substitution. At the same time, the result would look as if the encrypted text had been transposed instead of the plaintext. This technique would also be easy to realize with paper and pencil.
But that wouldn't fit in with the fact that many of the lines don't have any repetitions...... oh, never mind, I'm tired and should not post. I'm just trying to find ways to avoid ngrams and cycles without having to use complicated procedures.
Maybe I should deal with the topic a little longer before I post something again.
Largo
 
Posts: 455
Joined: Tue Jun 14, 2016 4:38 am
Location: Frankfurt, Germany

Re: Cycle types

Postby smokie treats » Sat Nov 25, 2017 1:30 am

Largo:

Zodiac cycled his homophones when he encoded the 408, and there is evidence of some cycling in the 340 but not as much. I want to know why. I figure he may have used some other patterns besides perfect cycles, but still perfect. Or regional encoding, or perhaps some pattern that causes regional encoding. With 63 symbols I have 1,953 possible combinations of symbols. I use numbers for symbols because it is easier for me. For each of the 1,953 combinations, I delete all symbols in the array that are not the two symbols that I am looking at, and collapse the array. Then convert the symbols to either A's or B's.

The 408 is on the left, the 340 on the right. X axis is count of that pattern found out of all possible symbol combinations. For the 408 the top two were long sequences of consecutive alternations, or perfect cycles. The 408 has more of the longer pattern than any other pattern. The 340 has more combinations with ABABA than anything else, it is a shorter pattern but there a lot of them. Some are not true, some are false. I am wondering about ABAABA and ABAABAA. Since the method worked so well for the 408, I wonder if some of these are actually true patterns, actual letters.

cycle chunks 1.png

To cause regional bias, or the symbols that appear exclusively in the top 6 and bottom 6 rows, I hypothesize that he did encode with regional or semi regional cycles. Moonrock's idea. Something like this: AAABABABABABABABAAA. The A's at the beginning and end would cause the regional bias. I can't divide into equal chunks because there is an odd number of symbols, so I have to take out the middle symbol so I can compare. AAABABABA ABABABAAA. Except that this exact cycle wouldn't cause the A to avoid the middle 8 rows.

Hypothesis

Maybe he did something like A B C A B C B C B C B C A B C A B C. That would cause the regional bias for A. If he did the exact same or very similar thing like this with more than one symbol, then maybe we can detect them and find out if they are statistically improbable or not. I am saying that maybe he did this to avoid the long perfect cycles, because with long perfect cycles a person could maybe identify them and use frequency attack. If you applied frequencies, then maybe you could see that the message was transposed. He could have hidden the homophone groups better by just randomly selecting homophones from their groups, but he liked to cycle and use patterns.
You do not have the required permissions to view the files attached to this post.
User avatar
smokie treats
 
Posts: 1620
Joined: Thu Feb 19, 2015 1:34 pm
Location: Lawrence, Kansas

Re: Cycle types

Postby Jarlve » Sat Nov 25, 2017 3:32 am

Largo wrote:Wow, I'm really impressed how quickly you found out. However, I still wonder how to solve such an encryption completely. Have you succeeded in doing so? Do you have the plain text?

It is quite difficult.

A straightforward way is to go into AZdecrypt and go to Functions, Manipulation, select Raise periodic and enter From: 1, To: 340, Step: 2. This will create new symbols for every odd symbol that is not unique to the set of even symbols. In the case of your cipher it raises the amount of symbols to 127, a multiplicity of 0.373. That is certainly within the possibilities of AZdecrypt with 6-grams or higher but not a given. Though it is in fact much harder because 2 sets of unique symbols are interlaced with eachother. I have noted great difficulties in trying to solve such ciphers. Probably because of diminished internal structure and higher degree of freedom because of the interlacing.

I have not succeeded, but have the above running for half a day or so and may keep it running for a couple of days. If that fails I could still try some other ways. If this would be the 340 I feel we could solve it if we put our heads and efforts together.

Largo wrote:Obviously, I underestimated that. If we once meet in Germany, I would be happy to donate as much Orangensaft as you like: D

Thank you for your hospitality!
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Cycle types

Postby Jarlve » Sat Nov 25, 2017 4:11 am

Largo wrote:Do you mean that the symbols of z340 should be sorted by even/odd and then examining the cycles? This improves the perfect 4- and 5 cycles found by azdecrypt.

This is not so unexpected. Because of the randomization in the 340, longer pieces of ciphertext decrease the odds of perfect cycles to appear.

Here are the periodic perfect 3-symbols cycles scores for the 340 by rows and columns. Period 2 by columns would mean odd/even, and by rows it means first half and second half of the ciphertext. These stats are not in the current AZdecrypt release but will be included for the next. Notice that the 340 through periods 2 to 5 scores much better by rows, this is also indicated by the positive percentage numbers (over 100%).

Code: Select all
AZdecrypt periodic perfect 3-symbol cycles stats for: 340.txt
--------------------------------------------------------
Period 1:
- Row/column 1: 1060, 1060 (100%)
Period 2:
- Row/column 1: 3908, 3018
- Row/column 2: 2948, 1428 (154.20%)
Period 3:
- Row/column 1: 2516, 1306
- Row/column 2: 1908, 428
- Row/column 3: 1888, 1528 (193.50%)
Period 4:
- Row/column 1: 2232, 208
- Row/column 2: 2014, 176
- Row/column 3: 344, 204
- Row/column 4: 304, 276 (566.43%)
Period 5:
- Row/column 1: 804, 320
- Row/column 2: 328, 408
- Row/column 3: 2858, 436
- Row/column 4: 612, 96
- Row/column 5: 252, 284 (314.37%)

And now your latest cipher, which has individual sequential homophonic substitutions for odd and even positions. Notice the crazy numbers at period 2 by columns, a giveaway.

Code: Select all
AZdecrypt periodic perfect 3-symbol cycles stats for: largo_oddeven.txt
--------------------------------------------------------
Period 1:
- Row/column 1: 1194, 1194 (100%)
Period 2:
- Row/column 1: 3856, 12686
- Row/column 2: 4806, 23112 (24.19%)
Period 3:
- Row/column 1: 1642, 2312
- Row/column 2: 1810, 392
- Row/column 3: 808, 2452 (82.62%)
Period 4:
- Row/column 1: 2276, 390
- Row/column 2: 1800, 554
- Row/column 3: 852, 1490
- Row/column 4: 44, 1766 (118.38%)
Period 5:
- Row/column 1: 984, 84
- Row/column 2: 736, 404
- Row/column 3: 96, 72
- Row/column 4: 180, 84
- Row/column 5: 32, 396 (195%)
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

PreviousNext

Return to Zodiac Cipher Mailings & Discussion

Who is online

Users browsing this forum: Chaucer, Goodkidmaadtoschi, Shawn, tGkTcy2W9B4p60o and 43 guests

cron