Unigram distance curiosity

Re: Unigram distance curiosity

Postby Largo » Sun Dec 10, 2017 9:08 am

Jarlve wrote:Convert the cipher to numbered by appearance. You have 63 symbols numbered from 1 to 63, numerically add 63 to the symbols of the middle 8 rows to make these distinct from the other rows/key. Then transpose. You can do it with AZdecrypt, under functions, manipulation, raise periodic, from: 103, to: 238. 6-grams should do the job.


I didn't think of that! Clever!
Do you think it is possible to extend the solver for encryptions with two keys? The solver splits the numerical version of the encryption into several parts (rectangles) and increases the numbers within one of the rectangles. Basically the same as you do for region-based transpositions.

smokie treats wrote:Again, the font is so awesome and thanks Largo.


You're welcome! I'm glad it's useful to you. :)
Largo
 
Posts: 454
Joined: Tue Jun 14, 2016 4:38 am
Location: Frankfurt, Germany

Re: Unigram distance curiosity

Postby Jarlve » Sun Dec 10, 2017 11:48 am

Jarlve wrote:
borkky wrote:It is plausible the center piece is using one encoding, and the top+bottom uses another.

This would definitably be a solvable hypothesis, something that we can check. Though, the question is, how many different arrangements/combinations are we looking at or do we want to consider.

I am working on this and wrote a hill climber that should solve a 2 keyed cipher among any distribution of rows. Example, key 1 occupies rows 1 to 6 and 15 to 20 and key 2 occupies rows 7 to 14. Another example, key 1 occupies the even rows and key 2 occupies the uneven rows (such as smokie5). It worked very well on smokie5 and and the 340 is running now. I would be happy if someone could make a few more test ciphers that are in line with borkky's hypothesis:

borkky wrote:He splits the clear text in two parts, and encodes them separately.
First half with one cipher, the other half with the other cipher, adding filler symbols at the end to produce a neat rectangle.
Then he makes two horizontal cuts, and rearranges the pieces.
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Unigram distance curiosity

Postby smokie treats » Sun Dec 10, 2017 1:00 pm

Smokie 66ABC are all P20. I just used a +/-53 symbol key for the middle rows, and the same key but about 10 extra symbols for the top and bottom rows. That's so I could get a high count of P20 repeats. Cycling is 25% random. There is no homophone, or + symbol, so RAW IOC will be a bit lower than the 340.

A

20 42 49 8 1 6 2 28 51 10 43 9 44 3 29 57 19
11 53 19 57 12 56 24 8 39 21 4 18 36 28 19 25 22
45 61 47 7 42 6 18 38 14 28 11 43 8 15 1 2 16
7 61 31 61 3 32 17 55 48 33 14 53 39 41 54 58 13
16 20 29 60 39 46 19 47 30 32 11 21 32 25 40 22 18
44 19 30 39 33 21 12 5 40 8 21 47 38 24 52 42 3
9 39 36 43 20 15 21 8 18 10 11 12 25 13 32 10 4
39 23 11 20 1 16 25 33 44 12 43 6 40 34 4 19 28
12 9 2 29 37 45 12 30 11 42 20 5 51 21 41 31 3
44 32 14 13 23 12 41 16 18 22 10 38 28 29 5 33 1
23 30 28 17 10 41 7 26 12 16 42 38 15 2 17 3 9
47 24 13 10 16 52 25 20 37 31 29 21 32 1 14 10 33
6 12 41 7 47 39 20 23 6 43 13 50 20 15 28 32 32
19 7 40 19 10 37 11 28 12 33 36 6 49 9 26 29 31
39 13 44 13 11 39 30 13 8 2 14 35 21 3 32 28 37
31 20 59 10 32 36 60 21 33 50 11 16 33 53 7 36 24
53 12 7 44 13 15 16 14 39 46 15 31 38 59 61 14 1
34 54 40 42 10 3 35 52 30 32 40 16 33 25 58 31 26
32 57 16 18 54 12 18 36 3 4 22 13 43 57 9 13 27
56 10 13 12 29 44 60 8 61 45 15 30 24 57 55 53 17

EDIT: B

20 21 23 22 33 6 57 14 38 20 1 10 20 39 43 22 25
40 44 50 26 11 27 12 13 45 18 33 24 16 38 30 56 26
17 57 20 10 15 37 23 25 37 4 21 7 28 31 8 39 2
17 26 17 38 62 16 14 62 19 22 17 11 12 46 41 3 27
35 28 32 55 59 18 22 30 60 42 51 39 32 58 61 28 20
19 21 15 12 1 15 21 44 56 2 40 10 45 56 57 36 47
34 44 45 50 10 42 9 33 19 43 6 19 24 5 1 4 8
21 2 40 22 30 16 18 46 44 27 45 41 31 43 7 11 19
25 3 12 51 42 32 47 30 44 1 40 2 13 33 10 38 34
18 45 26 39 18 9 29 47 15 20 31 6 35 28 29 28 38
10 39 43 41 21 3 44 12 13 50 10 27 21 11 28 20 33
12 13 43 38 25 20 51 37 26 27 28 22 20 25 10 5 11
32 41 18 36 43 47 39 44 21 30 26 27 34 40 17 53 29
45 18 1 25 43 52 12 35 22 20 13 21 2 22 7 10 44
29 56 41 45 42 58 10 11 4 54 16 14 22 31 8 23 12
48 60 46 5 13 61 46 57 29 17 32 16 3 34 9 38 59
56 4 55 30 35 10 40 50 35 24 41 31 11 32 18 43 20
35 1 49 25 26 27 53 29 46 47 58 34 12 21 49 22 44
39 31 5 13 2 25 42 26 56 55 55 60 27 39 49 58 10
45 57 1 11 12 6 18 20 54 21 48 32 6 53 22 62 54

C

9 6 22 37 40 38 10 10 17 30 39 12 27 13 56 28 25
53 33 1 2 34 9 63 10 24 17 37 26 3 31 55 18 35
11 62 23 33 65 59 16 54 1 17 45 38 32 39 23 41 53
7 54 63 19 29 47 37 37 60 2 56 45 16 32 24 3 53
43 14 28 44 11 47 31 42 32 2 40 8 50 20 15 25 38
39 21 28 37 30 38 13 36 9 38 24 43 9 27 43 11 18
46 3 9 35 47 10 19 31 12 11 22 32 54 17 16 9 43
53 21 20 54 19 33 30 28 45 25 19 8 34 26 48 14 43
17 21 5 10 8 49 48 46 24 11 20 39 7 28 25 38 1
18 41 51 2 39 5 3 1 2 41 3 37 29 1 44 2 3
1 9 22 45 10 11 9 38 53 26 10 40 39 31 47 43 37
41 52 54 16 53 39 23 8 53 32 4 39 44 37 11 26 26
28 30 28 9 49 10 21 31 12 42 2 15 45 3 40 1 14
39 26 11 35 32 39 3 17 27 33 19 20 22 28 37 29 30
13 31 2 1 39 35 35 30 9 43 32 33 38 9 34 18 46
31 23 32 24 47 7 11 46 2 12 16 40 44 47 37 15 9
21 17 35 41 40 63 41 19 13 42 56 44 11 45 46 64 10
2 16 23 14 2 20 40 1 11 41 16 42 50 60 16 37 5
38 62 59 65 2 27 56 39 31 23 63 43 44 33 45 9 43
32 37 61 19 3 30 12 18 51 31 58 58 63 22 28 10 23

Let's see if anyone can determine how many rows in the top and bottom, which are the same, have the key with extra symbols.
User avatar
smokie treats
 
Posts: 1619
Joined: Thu Feb 19, 2015 1:34 pm
Location: Lawrence, Kansas

Re: Unigram distance curiosity

Postby Largo » Sun Dec 10, 2017 1:41 pm

Jarlve wrote:I would be happy if someone could make a few more test ciphers that are in line with borkky's hypothesis


Here you are:

Largo_Sing_Along
Code: Select all
8z0Oap:sI7n;WGkRN
2XvArxKhPlzMS8QVO
nL9:sJ4t3iZNabfBo
uVUx;AF38isL14qyN
lwuMk2erx8o07sVLd
:=r2vpy8e+aCGxTY0
a5K-p7:qPaRdw0PHM
Vit1o3ZGyTFmSIsji
RmbX=uYvZwBp+T;Lj
Ohn:cNS3;yPFQWfA-
0EkZJWUePFxF9T34J
b;z+Q:VA0LFwCyNBP
0UhtDUePFOMonXlaa
zq2U0pmvaeuuVKDx9
awSVgLiucj:37QyCP
L0rd5pmGw8iVHFtp=
Fox3G1jvMWq9SLPY4
FbI+QwCh57OZVpR0:
VRveZTxkuV;3rIXz5
EB=SoF4WfYb1uuOmw
Largo
 
Posts: 454
Joined: Tue Jun 14, 2016 4:38 am
Location: Frankfurt, Germany

Re: Unigram distance curiosity

Postby smokie treats » Sun Dec 10, 2017 5:02 pm

smokie treats wrote:Smokie 66ABC are all P20. I just used a +/-53 symbol key for the middle rows, and the same key but about 10 extra symbols for the top and bottom rows. That's so I could get a high count of P20 repeats. Cycling is 25% random. There is no homophone, or + symbol, so RAW IOC will be a bit lower than the 340.

A

20 42 49 8 1 6 2 28 51 10 43 9 44 3 29 57 19
11 53 19 57 12 56 24 8 39 21 4 18 36 28 19 25 22
45 61 47 7 42 6 18 38 14 28 11 43 8 15 1 2 16
7 61 31 61 3 32 17 55 48 33 14 53 39 41 54 58 13
16 20 29 60 39 46 19 47 30 32 11 21 32 25 40 22 18
44 19 30 39 33 21 12 5 40 8 21 47 38 24 52 42 3
9 39 36 43 20 15 21 8 18 10 11 12 25 13 32 10 4
39 23 11 20 1 16 25 33 44 12 43 6 40 34 4 19 28
12 9 2 29 37 45 12 30 11 42 20 5 51 21 41 31 3
44 32 14 13 23 12 41 16 18 22 10 38 28 29 5 33 1
23 30 28 17 10 41 7 26 12 16 42 38 15 2 17 3 9
47 24 13 10 16 52 25 20 37 31 29 21 32 1 14 10 33
6 12 41 7 47 39 20 23 6 43 13 50 20 15 28 32 32
19 7 40 19 10 37 11 28 12 33 36 6 49 9 26 29 31
39 13 44 13 11 39 30 13 8 2 14 35 21 3 32 28 37
31 20 59 10 32 36 60 21 33 50 11 16 33 53 7 36 24
53 12 7 44 13 15 16 14 39 46 15 31 38 59 61 14 1
34 54 40 42 10 3 35 52 30 32 40 16 33 25 58 31 26
32 57 16 18 54 12 18 36 3 4 22 13 43 57 9 13 27
56 10 13 12 29 44 60 8 61 45 15 30 24 57 55 53 17

EDIT: B

20 21 23 22 33 6 57 14 38 20 1 10 20 39 43 22 25
40 44 50 26 11 27 12 13 45 18 33 24 16 38 30 56 26
17 57 20 10 15 37 23 25 37 4 21 7 28 31 8 39 2
17 26 17 38 62 16 14 62 19 22 17 11 12 46 41 3 27
35 28 32 55 59 18 22 30 60 42 51 39 32 58 61 28 20
19 21 15 12 1 15 21 44 56 2 40 10 45 56 57 36 47
34 44 45 50 10 42 9 33 19 43 6 19 24 5 1 4 8
21 2 40 22 30 16 18 46 44 27 45 41 31 43 7 11 19
25 3 12 51 42 32 47 30 44 1 40 2 13 33 10 38 34
18 45 26 39 18 9 29 47 15 20 31 6 35 28 29 28 38
10 39 43 41 21 3 44 12 13 50 10 27 21 11 28 20 33
12 13 43 38 25 20 51 37 26 27 28 22 20 25 10 5 11
32 41 18 36 43 47 39 44 21 30 26 27 34 40 17 53 29
45 18 1 25 43 52 12 35 22 20 13 21 2 22 7 10 44
29 56 41 45 42 58 10 11 4 54 16 14 22 31 8 23 12
48 60 46 5 13 61 46 57 29 17 32 16 3 34 9 38 59
56 4 55 30 35 10 40 50 35 24 41 31 11 32 18 43 20
35 1 49 25 26 27 53 29 46 47 58 34 12 21 49 22 44
39 31 5 13 2 25 42 26 56 55 55 60 27 39 49 58 10
45 57 1 11 12 6 18 20 54 21 48 32 6 53 22 62 54

C

9 6 22 37 40 38 10 10 17 30 39 12 27 13 56 28 25
53 33 1 2 34 9 63 10 24 17 37 26 3 31 55 18 35
11 62 23 33 65 59 16 54 1 17 45 38 32 39 23 41 53
7 54 63 19 29 47 37 37 60 2 56 45 16 32 24 3 53
43 14 28 44 11 47 31 42 32 2 40 8 50 20 15 25 38
39 21 28 37 30 38 13 36 9 38 24 43 9 27 43 11 18
46 3 9 35 47 10 19 31 12 11 22 32 54 17 16 9 43
53 21 20 54 19 33 30 28 45 25 19 8 34 26 48 14 43
17 21 5 10 8 49 48 46 24 11 20 39 7 28 25 38 1
18 41 51 2 39 5 3 1 2 41 3 37 29 1 44 2 3
1 9 22 45 10 11 9 38 53 26 10 40 39 31 47 43 37
41 52 54 16 53 39 23 8 53 32 4 39 44 37 11 26 26
28 30 28 9 49 10 21 31 12 42 2 15 45 3 40 1 14
39 26 11 35 32 39 3 17 27 33 19 20 22 28 37 29 30
13 31 2 1 39 35 35 30 9 43 32 33 38 9 34 18 46
31 23 32 24 47 7 11 46 2 12 16 40 44 47 37 15 9
21 17 35 41 40 63 41 19 13 42 56 44 11 45 46 64 10
2 16 23 14 2 20 40 1 11 41 16 42 50 60 16 37 5
38 62 59 65 2 27 56 39 31 23 63 43 44 33 45 9 43
32 37 61 19 3 30 12 18 51 31 58 58 63 22 28 10 23

Let's see if anyone can determine how many rows in the top and bottom, which are the same, have the key with extra symbols.


Please forgive me, I made a mistake and use the wrong transposition matrix.
User avatar
smokie treats
 
Posts: 1619
Joined: Thu Feb 19, 2015 1:34 pm
Location: Lawrence, Kansas

Re: Unigram distance curiosity

Postby smokie treats » Sun Dec 10, 2017 5:15 pm

Here are the new smokie66ABC messages. The irony is that if I untranspose at period 20 they will solve easily because I only added some symbols to the keys for the top and bottom rows to get the regional biases and high P20 repeats. If you use different keys, then no P20 repeats as we have determined.

A

47 33 38 43 30 19 35 1 59 53 3 27 31 6 20 3 43
49 32 25 17 36 39 20 13 40 21 48 47 37 57 52 14 11
18 29 20 29 4 5 1 56 21 30 55 59 27 21 31 35 16
38 41 38 32 21 56 32 46 12 42 33 28 29 2 17 7 30
44 31 10 1 18 40 23 28 29 26 21 19 11 12 16 41 42
10 21 25 24 37 15 27 11 32 8 38 2 12 30 28 46 35
21 17 45 14 33 31 8 43 11 50 19 28 20 41 27 3 45
16 11 26 42 1 36 44 23 13 49 32 28 10 29 43 41 2
9 27 39 42 37 32 43 41 10 28 11 21 12 31 10 48 9
18 42 29 19 40 22 17 5 8 18 43 20 9 30 27 42 13
35 34 44 11 30 6 28 31 21 12 12 11 12 17 40 3 36
8 5 14 1 24 7 6 4 10 9 14 29 14 43 32 33 10
45 25 15 44 12 10 24 1 43 32 11 31 45 41 8 19 18
13 36 7 27 12 10 6 3 35 20 28 19 29 19 14 7 17
13 36 11 12 32 30 20 27 28 42 16 43 51 1 46 41 42
21 43 10 2 39 26 48 34 44 14 41 11 19 37 18 1 42
10 57 58 29 20 25 36 42 31 33 32 36 56 52 37 30 11
20 43 54 34 52 12 9 31 32 13 19 38 57 39 40 37 24
2 55 52 58 20 56 36 37 24 45 27 38 57 28 59 12 21
10 26 33 24 54 41 23 56 29 31 50 14 6 5 41 32 9

B

44 17 18 39 50 1 2 28 30 57 3 15 19 33 54 20 10
45 6 1 22 2 57 46 20 7 36 37 16 39 53 3 11 59
23 63 34 55 54 41 28 49 42 34 44 56 45 1 45 25 55
26 51 11 55 17 63 40 44 43 33 21 42 13 19 2 3 32
19 12 38 41 20 34 41 56 17 55 57 48 25 59 60 44 57
36 20 21 12 1 41 33 31 14 45 10 11 8 32 39 10 15
25 17 46 26 32 10 11 9 44 45 16 50 38 38 40 1 15
2 34 46 8 10 28 11 42 42 52 11 18 51 44 19 48 45
3 27 4 31 32 17 22 37 3 46 2 3 1 39 10 35 33
22 6 2 41 34 34 7 44 47 10 1 9 13 30 25 29 45
31 11 50 6 24 26 42 48 26 12 18 43 38 39 11 21 9
25 36 19 33 1 2 17 45 34 18 9 8 3 11 5 41 43
19 40 44 14 26 12 4 16 38 5 17 32 11 7 30 45 39
11 46 27 21 43 18 35 11 43 41 27 9 33 40 43 13 38
10 34 11 41 35 12 28 46 12 26 34 34 11 12 42 39 35
2 27 53 41 31 14 37 34 53 42 17 57 58 14 34 39 43
45 11 47 32 12 20 62 6 35 34 23 2 12 21 40 34 46
63 49 35 17 3 15 8 50 59 1 10 61 23 36 30 25 41
7 50 19 56 20 31 11 42 17 45 48 9 32 62 12 55 2
8 14 7 60 22 11 43 59 45 29 30 51 18 46 20 9 19

C

48 4 9 41 35 10 13 1 6 36 2 1 6 23 30 19 38
31 24 29 20 11 37 17 54 32 12 7 56 58 39 37 30 5
50 55 42 8 9 31 28 46 26 10 23 14 35 7 7 43 61
60 29 32 56 43 37 44 60 49 11 12 55 36 6 15 26 19
38 18 6 20 46 17 7 21 1 6 30 57 56 59 28 13 18
36 9 42 11 11 30 28 12 31 55 12 20 10 13 46 39 2
41 35 10 1 27 19 28 36 29 42 37 24 8 38 27 7 48
2 3 16 11 45 25 33 42 32 53 28 39 17 41 1 20 32
40 21 19 37 29 12 8 20 21 9 49 27 51 44 45 10 38
29 13 14 30 13 29 21 3 26 15 42 5 40 20 31 25 43
26 15 41 7 14 15 13 13 12 3 23 2 35 40 2 24 42
13 12 37 9 35 18 6 10 36 52 11 43 41 6 8 1 42
24 12 17 38 39 19 40 1 9 2 37 3 10 35 18 19 32
11 34 17 4 43 30 44 48 12 41 33 25 36 9 34 1 9
55 30 18 14 9 10 54 60 45 16 1 57 17 11 56 4 27
9 40 28 49 27 30 35 39 57 24 61 41 54 42 43 55 40
27 32 61 60 41 42 9 38 23 30 30 3 54 18 43 36 9
39 40 1 61 28 47 1 60 38 23 37 19 26 59 41 29 11
17 39 35 51 44 57 31 7 3 27 33 42 23 12 6 14 31
45 5 26 55 54 58 10 16 8 1 20 36 7 37 52 22 40
User avatar
smokie treats
 
Posts: 1619
Joined: Thu Feb 19, 2015 1:34 pm
Location: Lawrence, Kansas

Re: Unigram distance curiosity

Postby borkky » Sun Dec 10, 2017 7:32 pm

Tonight I extended a little on the widget, added a info box about the "center" area.

https://martinlindhe.github.io/zodiac-widget/
My Z340 widget for the two-ciphers hypothesis: https://martinlindhe.github.io/zodiac-widget/
borkky
 
Posts: 28
Joined: Sat Dec 02, 2017 1:31 pm

Re: Unigram distance curiosity

Postby Jarlve » Mon Dec 11, 2017 3:15 am

borkky wrote:Tonight I extended a little on the widget, added a info box about the "center" area.

https://martinlindhe.github.io/zodiac-widget/

This widget investigates the observation that an unusual number of symbols are missing from the middle rows, as noted by smokie treats

Sorry to bring this up borkky, I can be a little pedantic. If memory serves well then I believe it was me who first made this observation. At the main post of this thread and almost a year earlier in the Route Transposition and Phenomenon thread:

by Jarlve » Thu Dec 29, 2016 6:02 am

Thanks doranchak,

I've been looking at the distances between symbols and that these may be related to the encoding. While I was trying to distinguish between palindromic and randomization I stumbled upon another weird thing in the 340. If you square all the distances between the symbols and total it you will find that the 340 scores very highly compared to other ciphers. Here is why.

Image

On average, less frequent symbols will show a greater distance between each other than more frequent occurring symbols, but why is this such a default thing in the 340 with top-bottom symmetry to boot?
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Unigram distance curiosity

Postby Jarlve » Mon Dec 11, 2017 3:18 am

Largo wrote:
Jarlve wrote:I would be happy if someone could make a few more test ciphers that are in line with borkky's hypothesis


Here you are:

Largo_Sing_Along
Code: Select all
8z0Oap:sI7n;WGkRN
2XvArxKhPlzMS8QVO
nL9:sJ4t3iZNabfBo
uVUx;AF38isL14qyN
lwuMk2erx8o07sVLd
:=r2vpy8e+aCGxTY0
a5K-p7:qPaRdw0PHM
Vit1o3ZGyTFmSIsji
RmbX=uYvZwBp+T;Lj
Ohn:cNS3;yPFQWfA-
0EkZJWUePFxF9T34J
b;z+Q:VA0LFwCyNBP
0UhtDUePFOMonXlaa
zq2U0pmvaeuuVKDx9
awSVgLiucj:37QyCP
L0rd5pmGw8iVHFtp=
Fox3G1jvMWq9SLPY4
FbI+QwCh57OZVpR0:
VRveZTxkuV;3rIXz5
EB=SoF4WfYb1uuOmw

Thank you, no periodic transposition right? It is running now and may take a while.
User avatar
Jarlve
 
Posts: 2544
Joined: Sun Sep 07, 2014 9:51 am
Location: Belgium

Re: Unigram distance curiosity

Postby borkky » Mon Dec 11, 2017 3:30 am

Jarlve wrote:Sorry to bring this up borkky, I can be a little pedantic. If memory serves well then I believe it was me who first made this observation. At the main post of this thread and almost a year earlier in the Route Transposition and Phenomenon thread:

by Jarlve » Thu Dec 29, 2016 6:02 am

Thanks doranchak,

I've been looking at the distances between symbols and that these may be related to the encoding. While I was trying to distinguish between palindromic and randomization I stumbled upon another weird thing in the 340. If you square all the distances between the symbols and total it you will find that the 340 scores very highly compared to other ciphers. Here is why.

Image

On average, less frequent symbols will show a greater distance between each other than more frequent occurring symbols, but why is this such a default thing in the 340 with top-bottom symmetry to boot?


Of course credit should be right!

As I'm new here, I found the info from the wiki http://zodiackillerciphers.com/wiki/ind ... tions#Z340 under "Unigrams":

Some symbols are exclusive to the first 6 and last 6 rows. An unusual number of symbols are missing from the middle 8 rows. (Source: smokie treats)


I won't judge who was first with the idea. Are you ok if i write "credit Jarlve & smokie treats"?, since he detailed the separation more?
Last edited by borkky on Mon Dec 11, 2017 3:48 am, edited 1 time in total.
My Z340 widget for the two-ciphers hypothesis: https://martinlindhe.github.io/zodiac-widget/
borkky
 
Posts: 28
Joined: Sat Dec 02, 2017 1:31 pm

PreviousNext

Return to Zodiac Cipher Mailings & Discussion

Who is online

Users browsing this forum: Shawn, versaceversace and 30 guests

cron