Page 1 of 1

Comparison of uniques between the 340 and others

PostPosted: Sat Dec 20, 2014 9:28 am
by Jarlve
Hey all,

This is sort of a continuation of my thoughts in The411's thread: viewtopic.php?f=81&t=2097

Considering all 340 symbols as a new starting point, it is counted how long this line continues in a specific direction without repeats.

In the following images you see the graphs of this data for the 340 and others. The red graph is normal, considering the cipher from left-to-right, top-to-bottom and the green graph shows the cipher horizontally mirrored. With corresponding numbers on the right side of the screen. Interestingly enough the 340 peaks at 26 counts for series of 17 uniques. There also seems to be a bigger difference between the red and green graphs with the 340 than with most of the other test ciphers. I was able to reproduce this by taking the top 340 part of the 408 plaintext through my homophonic substitution routine, which specializes in flattening the symbol frequencies. But with my same routine and random cycle this is much less visible.

Why does the 340 peak at 17? Is this just a coincidence that happens from the relations between the plaintext, quality of the homophonic substitution and symbol cycle or is there more going on?

And where does the red/green graph (visually observed) difference come from? Maybe the cipher occupies a state that is farther away from random so that any changes made to it have more significance.

Image

340: https://www.dropbox.com/s/vnfcv1hhfo6ko ... 0.png?dl=0
408 top 340 chars part: https://www.dropbox.com/s/bely02n661o45 ... s.png?dl=0
408 plaintext top 340 chars part, my routine, perfect cycle: https://www.dropbox.com/s/lsft2y5mzp849 ... e.png?dl=0
408 plaintext top 340 chars part, my routine, random cycle: https://www.dropbox.com/s/ctne9yrffqkha ... e.png?dl=0
340.ccactus.example.txt: https://www.dropbox.com/s/guek9y8ug0quj ... e.png?dl=0
340.gardi.example.txt: https://www.dropbox.com/s/si61ehl3mxn3j ... e.png?dl=0
340.mikec.example.txt: https://www.dropbox.com/s/ctlkhhfge2lwe ... e.png?dl=0
340.mikee.example.txt: https://www.dropbox.com/s/ix63uytb5p6c0 ... e.png?dl=0
340.tonyb.example.txt: https://www.dropbox.com/s/crlek9siwyta1 ... e.png?dl=0
340.tonyb2.example.txt: https://www.dropbox.com/s/ydzzn0ezc04e3 ... e.png?dl=0

Re: Comparison of uniques between the 340 and others

PostPosted: Sun Dec 21, 2014 5:04 am
by Jarlve
I did some testing with swapping rows randomly. The data gathered could indicate that no or not much transposition of rows is current in the 340.

Red graphs are horizontal, green graphs are vertical.

Image
Image

340 original: https://www.dropbox.com/s/eo43doc4k0q45 ... l.png?dl=0
340 1 row swapped: https://www.dropbox.com/s/q33edrcg83ksv ... 1.png?dl=0
340 2 rows swapped: https://www.dropbox.com/s/4jhkoc1g6q4m9 ... 2.png?dl=0
340 3 rows swapped: https://www.dropbox.com/s/td8wea5s5tgyi ... 3.png?dl=0
340 4 rows swapped: https://www.dropbox.com/s/csnlcik3dhjuz ... 4.png?dl=0
340 5 rows swapped: https://www.dropbox.com/s/9nxg1tn85yyjw ... 5.png?dl=0
340 6 rows swapped: https://www.dropbox.com/s/jq8ie4p5qewls ... 6.png?dl=0
340 7 rows swapped: https://www.dropbox.com/s/hm89gpq3xbyue ... 7.png?dl=0

Re: Comparison of uniques between the 340 and others

PostPosted: Sun Dec 21, 2014 3:02 pm
by Jarlve
To answer one of my own questions: the differences in the graphs - normal vs mirrored - occur from breaking up of the homophonic cycle (duh). This typically right-shifts the red graph versus the darker one. Or in the very first image of this topic the green vs red. I added other colors, they represent the different directions, a darker version of a color represents the mirrored counterpart.

340 updated:

Image
https://www.dropbox.com/s/fuhtvst74qlpr ... 0.png?dl=0

The following image shows the average graphs for 20 different plaintexts encrypted with my homophonic algorithm, this clearly shows the difference between the red and darker red graph, a big indication of homophonic substitution in that direction:

Image
https://www.dropbox.com/s/dj3e8d1myenot ... l.png?dl=0

EDIT: Updated the above 2 images. There was an error with the white/grey graphs and numbers.

Re: Comparison of uniques between the 340 and others

PostPosted: Tue Dec 23, 2014 12:12 pm
by Jarlve
I created 100 17x20 "340-like" plaintexts. All were handselected. About a third is from Guttenberg, another third from the internet about various topics, and another third were from a book excerpt website. For all the plaintexts also a random counterpart was made, matching letter frequencies exactly. The plaintexts start with "p", the randomized counterparts with "r".

Download: https://www.dropbox.com/s/y0erqtwa02v1a ... 0.zip?dl=0

From these I generated averages for the "uniques" graphs to see what I could find. Randoms generate shorter uniques.

Normal:
Image
https://www.dropbox.com/s/4lgrmxo5mywqo ... 0.png?dl=0

Random:
Image
https://www.dropbox.com/s/zawnzn6w6y6xg ... 0.png?dl=0

The next image shows the 17x20 grid with on each position a number representing the left-to-right, bottom-to-top unique count and instances of 17 colored orange. If you are wondering about the shades of blue, it is a modulo 10 scheme.

Image
https://www.dropbox.com/s/axj1cse5iypfm ... 1.png?dl=0

And next colored from blue (1) to green (29).

Image
https://www.dropbox.com/s/gk8bhh3htwy7g ... 2.png?dl=0

Re: Comparison of uniques between the 340 and others

PostPosted: Thu Dec 25, 2014 5:21 am
by Jarlve
Merry christmass everyone!

The following images showing a comparison between the normal 340 and the uniplus version, where every "+" symbol is replaced with a new unique symbol raising the symbol count to 86 (62+24). I've been playing around with the idea of the 340 being some kind of double encryption cipher, and that for some reason, the "+" symbol is a leftover from the first encryption scheme and that it was not included in the secondary encryption (probably homophonic substitution).

The "+" symbol is very strange, it doesn't seem to fit the frequencies typical for homophonic substitution. It only falls once on a prime number. It creates 8 doubles, all directions considered, skewing the repeat information. I'm really getting the impression that it is possibly a raw leftover from a previous encryption scheme. Maybe some tests can be done with different encryption schemes to see if there are some that have a tendency for the prime phobia effect to occur.

I would say that in terms of uniques/repeats the uniplus cipher appears to look a bit more normal compared to what I have seen/expect from homophonic with 63 symbols. For instance, counts for 1 unique (double symbols) is lower, which is expected with the "+" symbol creating 8 doubles as stated earlier. Difference between red and darker red jumped from 104% to 109% (horizontal direction). Strangely the difference between white and darker white persists from 113% to 114% (diagonal, top to bottom-right, east-south arrow). I guess it is possible that some information was encoded in this direction or that the cipher was written in this direction. Word search?

Image
https://www.dropbox.com/s/xzekythmeo73j ... d.png?dl=0

340 normal:
Image
https://www.dropbox.com/s/k2o12pszi4ju2 ... l.png?dl=0

340 uniplus:
Image
https://www.dropbox.com/s/rsf0jhv8jzcj0 ... s.png?dl=0

Re: Comparison of uniques between the 340 and others

PostPosted: Fri Dec 26, 2014 12:23 pm
by Jarlve
My plaintext library is updated, it is in my signature and now includes the directions and some Vigenere encryptions for the plaintexts 1 to 100. More to come.

I generated "prime phobia" graphs for all the normal, random and Vigenere versions of my plaintexts to see if anything. The horizontal axis is the number of symbols, for instance in the 340 the count for the "+" symbol is 24. The vertical axis is the frequency for the colors, and the colors represent the amount of times a symbol landed on a prime number.

To give a clear example, in the image for the normal Vigenere, the green graph has a point at (horizontal 24, vertical 3). Which is also in the green list at the right side of the screen. This means that in all the 100 normal Vigenere encrypted versions of the plaintexts, 3 sets of symbols were found that counted 24 symbols of which only 1 of them lands on a prime number position per set.

I am currently unconcluded about the findings. Just that if the "+" is a left-over thing (that was not included in the homophonic substitution) it would seem that it is not that rare to occur for a set of symbols of 24 to only land once on a prime number position.

Prime phobia frequencies for normal plaintexts:
Image
https://www.dropbox.com/s/ljzais8q5rmkv ... 0.png?dl=0

Prime phobia frequencies for randomized plaintexts:
Image
https://www.dropbox.com/s/vnp73aj81zevi ... 0.png?dl=0

Prime phobia frequencies for normal Vigenere (keyword Zodiac) plaintexts:
Image
https://www.dropbox.com/s/zokjwp4vh846x ... 0.png?dl=0

Prime phobia frequencies for reversed Vigenere (keyword Zodiac) plaintexts:
Image
https://www.dropbox.com/s/67situqcye1zk ... 0.png?dl=0

Re: Comparison of uniques between the 340 and others

PostPosted: Sun Dec 28, 2014 8:16 am
by Jarlve
Possible bleed through of plaintext writing direction after homophonic substitution.

The first image shows the average for 100 plaintexts with the text in normal (horizontal) left-to-right, top-to-bottom direction, the second one is with the text in vertical direction. This is to show that in terms of the uniques, writing direction can easily be deduced. There is a right-shift of the red graphs in the first image which represents the horizontal direction and in the second image there is a right-shift of the green graphs which represent the vertical direction.

1) horizontal: https://www.dropbox.com/s/60jfwuc9v0rlp ... 0.png?dl=0
2) vertical: https://www.dropbox.com/s/ff8egxjsnkwvt ... 0.png?dl=0

The following 2 images show that this information bleeds through after applying homophonic substitution. As you can see the effect is minor but the algorithm I wrote for homophonic substitution is really excellent at distributing this information thus I expect to see a little bit more of this for the 340 and 408. The main problem is that I don't have averages for the 340.

1) horizontal with homophonic: https://www.dropbox.com/s/x3u6on13kvilv ... 0.png?dl=0
2) vertical with homophonic: https://www.dropbox.com/s/g7okjh9gly6dk ... 0.png?dl=0

As I said before I am considering that the 340 plaintext could be a word search. I'm thinking of modifying my decrypting program to score in all directions but I don't know beforehand if it will be possible to recover such plaintexts so some testing will be needed.

Image

Re: Comparison of uniques between the 340 and others

PostPosted: Tue Dec 30, 2014 7:45 am
by masootz
word search is a great idea and would fit zodiac's MO of making things more complicated without necessarily being smarter about it.

Re: Comparison of uniques between the 340 and others

PostPosted: Tue Dec 30, 2014 2:52 pm
by Jarlve
Word search as plaintext could explain the pivots, or at least increase the chances of such things happening. And maybe the "+" symbol serving as filler in the word search. Possibly hinting that it is a very dense word search, which I guess would have costed considerable effort and/or expertise. Just thinking aloud... But, it's just, would a "simple" word search remain unsolved for 45 years... I'm not sure.

I currently believe that the "+" symbol is either, not part of the homophonic substitution or the cycle used for the symbols as it appears is close to random for some reason.

Re: Comparison of uniques between the 340 and others

PostPosted: Tue Dec 30, 2014 3:28 pm
by Jarlve
Jarlve wrote:I currently believe that the "+" symbol is either, not part of the homophonic substitution or the cycle used for the symbols as it appears is close to random for some reason.


p1 plaintext, homophonic substitution, random cycle:
https://www.dropbox.com/s/kj2bc4yby6wfu ... e.png?dl=0
Not much difference between red graphs and others.

p1 plaintext, homophonic substitution, perfect cycle:
https://www.dropbox.com/s/py446yru9ui36 ... 1.png?dl=0
Big difference (x2+) between red graphs and others, red (horizontal) is 129% that of darker red (horizontally mirrored).

340:
https://www.dropbox.com/s/j1mdvz0o3mn6t ... 0.png?dl=0
Good difference between red graphs and others. 104% between reds.

340 with "+" symbol removed and the remaining symbols on the 19th line also removed to make the cipher fit into a 17*18 grid (306):
https://www.dropbox.com/s/h9l37h3j843mi ... 6.png?dl=0
Red scores higher than in the original 340 with 34 symbols removed! Difference between reds jumps from 104% (original) to 115%.