A diagonal shift?

[Jarlve]

Hey everyone,

I have possible indentified a correlation between different sets of information that on first glance seems very related to the work of traveller1st trying to explain the pivots in: viewtopic.php?f=81&t=964&hilit=diagonal

The correlation seems so strong that I believe that the first person who can make sense of the data has a chance of solving the 340. I will try to explain it in a way that everybody can understand.

First of all the 340 is believed to be a homophonic substitution cipher similar to the 408. Various statistics back this up, more specifically it very likely seems to be cyclic. Homophonic substitution means that every letter could have more than one symbol substitution attached to it. Then there are 2 variants cyclic and random, cyclic means that for each letter, the symbol map is followed in order. And for random a dice is rolled to determine the symbol. Both have very different statistical signatures. I will from now on refer to cyclic homophonic substitution as CHS and random as RHS.

The letter to symbol map for the 408.



In a sense, how I see it, homophonic substitution simply blurs out the plaintext and information of the cipher.



Now some time ago I developed a system that uses the information of the non-repeats, I don't know if anyone else uses this data but I have found it to be extremly powerful. This measures, using every symbol in the cipher as a starting point, the length of each unique string. For instance, the string akin to CHS "ABCABCABC", has 7 unique sub-strings with a length of 3. For RHS the string is more likely to be random and will have shorter sub-strings of non-repeats. But measuring this for any cipher will give you a total equal to the cipher's length so you have to multiplicate the counts by the length of the string. This system is so powerful that it can easily distuinguish between CHS and RHS or between a plaintext, a random plaintext and vigenere. It can also be used to determine writing direction.

I took the information of the non-repeats to the next level and measure this data for 96 orientations. With orientation I mean, we write text in a right-to-left, top-to-bottom manner. Notice that you have a primary direction and a secondary direction. I figured our writing system is 2 dimensional and that using the common wind directions you have 16 different orientations. But the actual system that I developed for this uses an input and output direction, so following this you have already 240 (16x15) variations. I simplified this to either doing an orientation or undoing it, which relates to the input and the output. For each of these 32 orientations I also added the option to alternate the primary direction, starting even or uneven. So there you have 96.

Is it needed to gather the non-repeat data for 96 different orientations? How could this possibly relate to anything?

To answer these questions I have to spill quite some data. And I will use some ciphers to compare, each cipher will be capped to 340 characters. Now following various data for various ciphers including that of the non-repeats and other systems for reference later on, you don't have to interpret it right now. Just scroll down now.

CHS ciphers:

408:

Code: Select all

system 1: 3538, 11317
system 2: 5714
system 3: 3351

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 4692, 4247, 4466.
w-s: 4338, 4466, 4247.
e-n: 4338, 4466, 4247.
w-n: 4692, 4247, 4466.
rotations ------------->
s-e: 3062, 2907, 2997.
n-e: 3135, 2997, 2907.
s-w: 3135, 2997, 2907.
n-w: 3062, 2907, 2997.
diagonals 1 ----------->
ne-se: 2757, 2952, 3045.
sw-se: 3052, 3045, 2952.
ne-nw: 3052, 3045, 2952.
sw-nw: 2757, 2952, 3045.
diagonals 2 ----------->
se-sw: 3055, 3115, 2992.
nw-sw: 2802, 2992, 3115.
se-ne: 2802, 2992, 3115.
nw-ne: 3055, 3115, 2992.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 4692, 4247, 4466.
w-s: 4338, 4466, 4247.
e-n: 4338, 4247, 4466.
w-n: 4692, 4466, 4247.
rotations ------------->
s-e: 3218, 3326, 3474.
n-e: 3315, 3474, 3326.
s-w: 3315, 3474, 3326.
n-w: 3218, 3326, 3474.
diagonals 1 ----------->
ne-se: 2578, 2531, 2828.
sw-se: 2890, 2828, 2531.
ne-nw: 2890, 2531, 2828.
sw-nw: 2578, 2828, 2531.
diagonals 2 ----------->
se-sw: 2979, 3007, 2882.
nw-sw: 3001, 2882, 3007.
se-ne: 3001, 3007, 2882.
nw-ne: 2979, 2882, 3007.
------------------------


ray_n:

Code: Select all

system 1: 3488, 11518
system 2: 5629
system 3: 3530

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 5046, 4823, 4664.
w-s: 4452, 4664, 4823.
e-n: 4452, 4664, 4823.
w-n: 5046, 4823, 4664.
rotations ------------->
s-e: 3090, 3063, 3359.
n-e: 3535, 3359, 3063.
s-w: 3535, 3359, 3063.
n-w: 3090, 3063, 3359.
diagonals 1 ----------->
ne-se: 2934, 3296, 3279.
sw-se: 3038, 3279, 3296.
ne-nw: 3038, 3279, 3296.
sw-nw: 2934, 3296, 3279.
diagonals 2 ----------->
se-sw: 3057, 3117, 3355.
nw-sw: 3264, 3355, 3117.
se-ne: 3264, 3355, 3117.
nw-ne: 3057, 3117, 3355.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 5046, 4823, 4664.
w-s: 4452, 4664, 4823.
e-n: 4452, 4823, 4664.
w-n: 5046, 4664, 4823.
rotations ------------->
s-e: 3325, 3414, 3457.
n-e: 3210, 3457, 3414.
s-w: 3210, 3457, 3414.
n-w: 3325, 3414, 3457.
diagonals 1 ----------->
ne-se: 3098, 3069, 3062.
sw-se: 2987, 3062, 3069.
ne-nw: 2987, 3069, 3062.
sw-nw: 3098, 3062, 3069.
diagonals 2 ----------->
se-sw: 2650, 2640, 2695.
nw-sw: 2703, 2695, 2640.
se-ne: 2703, 2640, 2695.
nw-ne: 2650, 2695, 2640.
------------------------


408 redone with my routine:

Code: Select all

system 1: 3410, 13284
system 2: 6034
system 3: 4272

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 7384, 6538, 6549.
w-s: 5704, 6549, 6538.
e-n: 5704, 6549, 6538.
w-n: 7384, 6538, 6549.
rotations ------------->
s-e: 3343, 3089, 3134.
n-e: 3430, 3134, 3089.
s-w: 3430, 3134, 3089.
n-w: 3343, 3089, 3134.
diagonals 1 ----------->
ne-se: 3301, 3669, 3721.
sw-se: 3883, 3721, 3669.
ne-nw: 3883, 3721, 3669.
sw-nw: 3301, 3669, 3721.
diagonals 2 ----------->
se-sw: 3566, 3489, 3651.
nw-sw: 3517, 3651, 3489.
se-ne: 3517, 3651, 3489.
nw-ne: 3566, 3489, 3651.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 7384, 6538, 6549.
w-s: 5704, 6549, 6538.
e-n: 5704, 6538, 6549.
w-n: 7384, 6549, 6538.
rotations ------------->
s-e: 3371, 3322, 3388.
n-e: 3285, 3388, 3322.
s-w: 3285, 3388, 3322.
n-w: 3371, 3322, 3388.
diagonals 1 ----------->
ne-se: 3551, 3845, 4216.
sw-se: 3734, 4216, 3845.
ne-nw: 3734, 3845, 4216.
sw-nw: 3551, 4216, 3845.
diagonals 2 ----------->
se-sw: 3518, 3363, 3433.
nw-sw: 3403, 3433, 3363.
se-ne: 3403, 3363, 3433.
nw-ne: 3518, 3433, 3363.
------------------------


340 (suspected CHS):

Code: Select all

system 1: 4033, 13344
system 2: 5475
system 3: 3418

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 4462, 3997, 4304.
w-s: 4260, 4304, 3997.
e-n: 4260, 4304, 3997.
w-n: 4462, 3997, 4304.
rotations ------------->
s-e: 2977, 3234, 3380.
n-e: 3111, 3380, 3234.
s-w: 3111, 3380, 3234.
n-w: 2977, 3234, 3380.
diagonals 1 ----------->
ne-se: 3092, 3669, 2887.
sw-se: 2893, 2887, 3669.
ne-nw: 2893, 2887, 3669.
sw-nw: 3092, 3669, 2887.
diagonals 2 ----------->
se-sw: 2845, 3041, 3347.
nw-sw: 3377, 3347, 3041.
se-ne: 3377, 3347, 3041.
nw-ne: 2845, 3041, 3347.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 4462, 3997, 4304.
w-s: 4260, 4304, 3997.
e-n: 4260, 3997, 4304.
w-n: 4462, 4304, 3997.
rotations ------------->
s-e: 3092, 2969, 3046.
n-e: 3125, 3046, 2969.
s-w: 3125, 3046, 2969.
n-w: 3092, 2969, 3046.
diagonals 1 ----------->
ne-se: 2998, 2969, 2930.
sw-se: 2881, 2930, 2969.
ne-nw: 2881, 2969, 2930.
sw-nw: 2998, 2930, 2969.
diagonals 2 ----------->
se-sw: 3592, 3161, 3502.
nw-sw: 3161, 3502, 3161.
se-ne: 3161, 3161, 3502.
nw-ne: 3592, 3502, 3161.
------------------------


RHS cipher:

mike_c:

Code: Select all

system 1: 5263, 15604
system 2: 5769
system 3: 3224

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 3297, 3449, 3404.
w-s: 3398, 3404, 3449.
e-n: 3398, 3404, 3449.
w-n: 3297, 3449, 3404.
rotations ------------->
s-e: 3533, 3093, 3059.
n-e: 3211, 3059, 3093.
s-w: 3211, 3059, 3093.
n-w: 3533, 3093, 3059.
diagonals 1 ----------->
ne-se: 3344, 2978, 3184.
sw-se: 3415, 3184, 2978.
ne-nw: 3415, 3184, 2978.
sw-nw: 3344, 2978, 3184.
diagonals 2 ----------->
se-sw: 3410, 3160, 3447.
nw-sw: 3439, 3447, 3160.
se-ne: 3439, 3447, 3160.
nw-ne: 3410, 3160, 3447.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 3297, 3449, 3404.
w-s: 3398, 3404, 3449.
e-n: 3398, 3449, 3404.
w-n: 3297, 3404, 3449.
rotations ------------->
s-e: 3022, 3129, 2993.
n-e: 3135, 2993, 3129.
s-w: 3135, 2993, 3129.
n-w: 3022, 3129, 2993.
diagonals 1 ----------->
ne-se: 3006, 3318, 2991.
sw-se: 3322, 2991, 3318.
ne-nw: 3322, 3318, 2991.
sw-nw: 3006, 2991, 3318.
diagonals 2 ----------->
se-sw: 2904, 3046, 3164.
nw-sw: 3061, 3164, 3046.
se-ne: 3061, 3046, 3164.
nw-ne: 2904, 3164, 3046.
------------------------


Enough with the code boxes already. For CHS there is a significant increase for the non-repeat number in which direction it was encoded also scaling with the amount of symbols, the quality of the mapping and application. Note that for the 408, which has some rows of the 3rd part and has some erros, this number sits at 4692 for a 54 symbol cipher. I generally compare it to the mirrored version of this direction to estabilish the direction in which the CHS was done. For ray_n's cipher this sits at 5046 for a 55 symbol cipher. 7384 for my 408 redone with 63 symbols which uses a perfect symbol distribution over the letters. And for mike_c a RHS cipher 3297. You can see easily see the difference between CHS and RHS now.

Now the 340 comes in at 4462. This is quite low for a 63 CHS symbol cipher as you can see, and for some time I have known that this indicates something. But what? CHS and RHS mixed? Transposition? A two part cipher? A random plaintext?

CHS and RHS mixed is possible but the cipher does not solve.

Some objections versus a straightforward two part cipher, let's compare the IoC of the top and bottom 10 rows of our test ciphers. The IoC relates to the symbol frequencies.

408: 0.0177, 0.0167.
ray_n: 0.0165, 0.0172.
408 redone: 0.0119, 0.0116.
340: 0.0180, 0.0181.
mike_c: 0.0173, 0.0159.

The 340 has almost identical values which is possible, but maybe unlikely, more so, in a two part cipher. My 408 redone is also very close but it is mathematically perfect because it is computer generated CHS using my algorithm. This I would say could be seen as an objection but also something that I will refer back to later on. Another objection is that nether parts on their own have been solved throughout the years. And I have more to back this up.

There is little difference between a CHS or RHS cipher with a random or normal plaintext, there is some bleedthrough going which I have shown in other threads but it is rather small. I could post more tables to show this but I don't want to bloat the topic with this so take my word for it. The reason can be found in the "blurring" analogy.

It will become evident that transposition is likely the cause. But you can do transposition before CHS and after. And since the non-repeat data (4462) is so low for the 340 I'm going under Assumption A: transposition was done after CHS symbol assignment.

Now what I actually only realised just yesterday, is that there is a significant "bump" for some of the non-repeat data for the 340. More specifically these numbers:

in: (do)
diagonals 1 ----------->
ne-se: 3092, 3669, 2887.
sw-se: 2893, 2887, 3669.
ne-nw: 2893, 2887, 3669.
sw-nw: 3092, 3669, 2887.

out: (undo)
diagonals 2 ----------->
se-sw: 3592, 3161, 3502.
nw-sw: 3161, 3502, 3161.
se-ne: 3161, 3161, 3502.
nw-ne: 3592, 3502, 3161.

What does this mean, what could possibly be the relevance of diagonal non-repeat data in suspected right-to-left, top-to-bottom cipher? Look at this new table for the 408 which has the top 10 rows encoded in a nw-ne direction.

Code: Select all

system 1: 3953, 12387
system 2: 5605
system 3: 3281

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 4052, 3871, 3783.
w-s: 3988, 3783, 3871.
e-n: 3988, 3783, 3871.
w-n: 4052, 3871, 3783.
rotations ------------->
s-e: 3123, 2926, 2757.
n-e: 2999, 2757, 2926.
s-w: 2999, 2757, 2926.
n-w: 3123, 2926, 2757.
diagonals 1 ----------->
ne-se: 2900, 3119, 3077.
sw-se: 2648, 3077, 3119.
ne-nw: 2648, 3077, 3119.
sw-nw: 2900, 3119, 3077.
diagonals 2 ----------->
se-sw: 2696, 3223, 3293.
nw-sw: 2789, 3293, 3223.
se-ne: 2789, 3293, 3223.
nw-ne: 2696, 3223, 3293.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 4052, 3871, 3783.
w-s: 3988, 3783, 3871.
e-n: 3988, 3871, 3783.
w-n: 4052, 3783, 3871.
rotations ------------->
s-e: 3228, 2980, 3200.
n-e: 3127, 3200, 2980.
s-w: 3127, 3200, 2980.
n-w: 3228, 2980, 3200.
diagonals 1 ----------->
ne-se: 2842, 2776, 3046.
sw-se: 2818, 3046, 2776.
ne-nw: 2818, 2776, 3046.
sw-nw: 2842, 3046, 2776.
diagonals 2 ----------->
se-sw: 3284, 3587, 3469.
nw-sw: 3429, 3469, 3587.
se-ne: 3429, 3587, 3469.
nw-ne: 3284, 3469, 3587.
------------------------


Notice that the suspected primary direction has changed from 4692 to 4052 and there is now a significant "bump" in the diagonal data. Especially in the input (undo) section, since that undoes the transposition.

diagonals 2 ----------->
se-sw: 3284, 3587, 3469.
nw-sw: 3429, 3469, 3587.
se-ne: 3429, 3587, 3469.
nw-ne: 3284, 3469, 3587.

Okay, I will post another example but this time for a RHS cipher:

Code: Select all

system 1: 6645, 17701
system 2: 5757
system 3: 3205

primary direction, secondary direction: normal, primary direction alternated starting even, uneven.

output: (do)
------------------------
normal directions ----->
e-s: 3319, 3164, 3576.
w-s: 3263, 3576, 3164.
e-n: 3263, 3576, 3164.
w-n: 3319, 3164, 3576.
rotations ------------->
s-e: 4062, 2933, 2994.
n-e: 3789, 2994, 2933.
s-w: 3789, 2994, 2933.
n-w: 4062, 2933, 2994.
diagonals 1 ----------->
ne-se: 3327, 2861, 3244.
sw-se: 2914, 3244, 2861.
ne-nw: 2914, 3244, 2861.
sw-nw: 3327, 2861, 3244.
diagonals 2 ----------->
se-sw: 2778, 3150, 3076.
nw-sw: 3169, 3076, 3150.
se-ne: 3169, 3076, 3150.
nw-ne: 2778, 3150, 3076.
------------------------

input: (undo)
------------------------
normal directions ----->
e-s: 3319, 3164, 3576.
w-s: 3263, 3576, 3164.
e-n: 3263, 3164, 3576.
w-n: 3319, 3576, 3164.
rotations ------------->
s-e: 3089, 3282, 3254.
n-e: 3191, 3254, 3282.
s-w: 3191, 3254, 3282.
n-w: 3089, 3282, 3254.
diagonals 1 ----------->
ne-se: 3153, 3286, 3148.
sw-se: 3308, 3148, 3286.
ne-nw: 3308, 3286, 3148.
sw-nw: 3153, 3148, 3286.
diagonals 2 ----------->
se-sw: 2910, 3245, 2977.
nw-sw: 3124, 2977, 3245.
se-ne: 3124, 3245, 2977.
nw-ne: 2910, 2977, 3245.
------------------------


As you can see this cipher has a significant "bump" in the rotation section 4062 versus 3319 in the right-to-left, top-to-bottom direction. So what does this mean? I will show you an example what rotation in this direction does to a cipher.

Rotation s-e, as output:



The different colors indicate the distance from its original position and you can visually see a diagonal shift going on. This actually very much relates to the transposition scheme (after RHS) used in this cipher, which simply swaps symbols: string "ABABABAB" becomes "BABABABA", but since you have 17 columns this cause a diagonal shift of information which represents itself in the data of the non-repeats!

Now let's compare the first top and bottom 10 rows of each test cipher in terms of non-repeats for the right-to-left, top-to-bottom direction and it's mirrored counterpart.

CHS:
408: top: 2275, 2158, bottom: 2101, 2075.
ray_n: top: 2620, 2136, bottom: 2321, 2211.
408 redone: top: 3965, 3053, bottom: 2952, 2550.
340: top: 2084, 2250, bottom: 2273, 1879.

RHS:
mike_c: top: 1566, 1464, bottom: 1699, 1901.

The first number is supposed to be bigger with CHS and the 340 actually manages the other way around for the first part. This strange cipher just keeps on giving. Note that the difference for the second part of the 408 is quite low due to having a couple of rows of the third part which uses RHS.

In some of the code tables I have posted you will see system 1, 2, 3 etc. System 1 measures the distribution of symbols over the grid, more even distributions score lower. Which are expected for CHS. System 2 measures the distance information of the difference of what comes before and after a symbol on average. A smaller difference indicates a more random distribution. System 3 is just the total of the non-repeats divided by the orientations, higher is more in line with CHS.

So how do they compare?

408:
system 1: 3538, 11317
system 2: 5714
system 3: 3351

ray_n:
system 1: 3488, 11518
system 2: 5629
system 3: 3530

408 redone:
system 1: 3410, 13284
system 2: 6034
system 3: 4272

340:
system 1: 4033, 13344
system 2: 5475
system 3: 3418

All 3 ciphers but the 340 have a number of around 3500 for system 1, the 4033 in the 340 is significant and indicates a higher degree of randomness.
System 2 comes up with a lower number for the 340 also indicating a higher degree of randomness.
System 3 for the 340 does not correlate well with the expected numbers of the non-repeats in the right-to-left, top-to-bottom direction.

To some degree the aforementioned "bump" in the diagonal non-repeat data of the 340 also could indicate transposition rather than a mix of RHS and CHS. Okay, now yesterday I also have discovered something really strange which seems to have a strong correlation with the "bump".

Comparing test cipher data for the non-repeat data for the following sets: symbols that sit on an even position, symbols that sit on an uneven position.

408: even: 2319, 2005, uneven: 1837, 1871.
ray_n: even: 2467, 1780, uneven: 1957, 1758.
408 redone: even: 3014, 2491, uneven: 2949, 2504.
340: even: 1706, 1795, uneven: 2129, 2129.

For CHS the first number should be bigger. Now, it has to noted that the uneven numbers for the 408 are not what they should be and it persists throughout the cipher. But for the 340 the data is even stranger, for even the second number is bigger and for uneven they are equal. But in total the 340 is: 4462, 4260. Now consider the following example for which I leave out every symbol that falls on a multiple of 3 (every 3rd symbol).

340: 2804, 2334.

Where does this big difference comes from all of a sudden? Can the cipher get any stranger?

Look how it possibly relates to the pivots (discovered by doranchak) in the next image, suggested by travellerst1st. but I made the same discovery in relation to the data yesterday w/o being aware of his work on this: viewtopic.php?f=81&t=964&hilit=diagonal



Now 1 + 1 being 2 this could very well explain the "bump" in the diagonals of the non-repeats. But how to make sense of the data? If in terms of the non-repeat data the symbols on even positions seems flat, not have a clear indication of direction, and the symbols at uneven positions as well. Then you could suggest that 2 data streams are running through eachother, one normal and one reversed, both streams spread out evenly between even and uneven positions. But that does not explain the clear indication of direction becoming apparant after removing symbols that fall on every third character!

Furthermore, is the uncanny similarity in IoC for the top and bottom 10 rows of the 340 additional indication of 2 seperate streams, or some mixing going on? The diagonal "bump", the non-repeat data for symbols at positions, even, uneven and multiples of 3, and the similarity of the IoC between the first and second part. I feel like it is all connected somehow.

[Jarlve]

The test ciphers used in this write up:

408:

Code: Select all

¼ºP/Z/uBºËOR¥ÐX¥B
WV+ÅGyF°¼HP¹K‚ÑyÅ
MJy^uI˽ÑTÔNQyDµ£
S¢/¼·BPORAuºÆRÌÑE
Ë^LMZJÄÒ\ÐFHVWžy
¹+ÑGD¼KI£°ÑX¾µ¤S¢
RNÔIyEÌO¾ÑGBTQS·B
LÄ/P·B¹XÑEHMu^RRË
ÃZKÑÐI£WÑ‚¾µLMÒ¼·
BPDR+Ê¥°\N¢ÅEuHËF
ZÃÐOVWIµ+ÔL£Ì^R°H
I¼DR¸TyÒ\ÄÅ/¹XJQA
PµM¾RuÔºL£NVEKH¥G
ÒI‚J˵¼¾LMÌNA£Z¢P
¤uÐËA¼·BVW\+VTÔOP
^¥SÒÌÆuÅ°½D¤GººIM
NË£SÃE/¼ººZÆAP·BV
ÐÅXÑWѸF·¾Ã+¹¼A¼B
ºOTµRuÃ+¸ÄyѸ^SÑW
VZÅGyKE¸TyA¼º·LÔ¸


ray_n:

Code: Select all

R¤„¸YW¾·½“ZZµÊ+³M
™B°V„TSQA±šÔѲ¥£‰
XWžJY¾Š¸ZT¾·RŽAº¤
±¥®´²™Â’M¹ÌºÃY˽¤
YZWT¸„AÊ·N´B”±’E¥
²¤SZQ³+µMʸËRV„Q
JY·´ŽZ¥+šMVžJTW¾¼
¸£ÔÃÎBŠÐN¼·™AW°V„
¹£SEÑ”±ËR¥X½™+’µ¸
DRË+´V„̳£Ë‰ÂÊžQ²
™Y“·Ã´šV“Z¥TŽA£™N
M¾J±°VB²£®¹SWE’ŠR
¤Ë¸YR+PµMZ·“½™DTÊ
AºÑM±³X²V´£„ž”YDÂ
ËWš™+ZRË+ZV°£„¾QT
JA½¥BΤS¹W³Ñ±ÌWžŠ
²WMRº¤DY“ôZTWƾW
´´£ËÌAÔ‰®P±“B¸²¤Z
S·°ÆPY+N’M¹º¤š™ËÔ
E¼ÌµVºWÑ”ZR¥žQ´¸‰


408 redone:

Code: Select all

071<=<2893EIK>LK:
?O@QSWYA4\5Z[MaXR
cfWF6]0YbNgBfXG^d
TH<1;72EJj38kIUaC
4F_eDfLhi=Z`P>QVW
[?bSG5Y\c@aMZ]kTH
JAgNX[UE6bV9LfS:;
^M<078YNaB_d1FIJ2
lCZbD`e=aL[\]ch39
:4GI>mK?i@HRA5^6Y
BlCEOD_`=g\dTFJ>]
^0GInMWhiNQ<ZLffj
1_eUJ2g;`c?P@[\KV
h]Mf3^4Y_dS`jeAH5
k6B0j178OCiDPNgE2
FKThUk3R=VGkS9:\c
>4dTl?<5;7@kj689O
AQLbBanZ:UlC[0j1;
7HM]I2lDnNXbnEVa=
P>RSWY?nLXj389^in


340:

Code: Select all

HER>ÐÌ^VPËI²LTG±Ä
NÐ+B¢·OºDWy•<»KÆ£
BŸ„ÃM+uZGW¢£L·¤HJ
SÐн^̾»V´ÐO++RK±
¸¼M+¤ÔÊÄIµFP+ˆ³Ë/
оR^FÌO-»ÄCËF>±D¢
·µ+KѺƒ±uÃXGV•¤LI
¢G±JÆÊ·O+¸Ny¤+¹L¼
Ä<M+Â+ZR±FBßA°³K
-¤ÌuV+^J+Oн<FBŸ-
u+R/µÔEIDyBÐÂTMKO
±<ÃÌRJI»µT³M•+ˆBF
¤°¼SŸ·+NIµFB⃾R
ÌGFN^Ƶ±³Â•ÃV³Ô++
ŸBX²»„³¼CE>VuZµ-+
IÕ´¤BK¢OÐ^•ÆMÑG±
RÃT+L²°C<+FÌWBI£L
++£WC¤WÃPOSHT/¢£Ð
IFËÄW<½ÔB¸yOB»-CÃ
>MDHNÐËS¤ZO¾AIKƒ+


mike_c:

Code: Select all

I4X91iZg;`GJ=b=7[
2j2fKG8OPEUH6B:dX
FXLfXmaJJ0QdWATXP
kKZUHkl@ZHnCK4[TJ
A0Qnnd[gnOSK6CM\7
[U0VL32G415Y4Y9[7
KT[J=?dM9;`O5=kM8
bK4_CfDT=m5G5IkY4
VDFcWZB`TEFcXb67f
KTJ9IQ25iVJ0?diZQ
]hDLQ;A^ToMkK9[]X
QgHX4THmJ]D1a4_K8
J;10V_3Ua45onZG=5
d^IVgW5iAhBACdWL;
LD]GZQAa42GcPY[UR
pH:;AF[72dWBD2?D7
65i8KAO^K1cECdM6J
eH?nI4JPD1jLbO0V^
KN^kWeU0XVB:gdK0V
LDgR=?fK92jN7kYAU


[AK Wilks]

Jarlve - I have been following your very innovative, precise and interesting work for some time. I have not posted on any of your threads because I had no specific questions, criticisms or helpful suggestions to offer. Until now...maybe.

I am very interested in your ideas about possible words in the 340 encoded diagonally. I have had a specific interest in this partially because I have some interest in the Raw Graysmith as recreated by Ed, Claston and others, which I have also worked on and feel may have some moderate merit as partially correct in spots as a first stage solve which needs one or more other steps. In that Raw Graysmith 69 words appear vertically and diagonally, as is, correct spelling, no anagrams. And 11 of those words are related to crime generally or Zodiac specifically. For example, going from a vertical correct spelling no anagrams DUEL, next to the D is an L in a diagonal north east direction is correct spelling no anagrams LIST, which goes to the T. With the B adjacent going into a diagonal north east direction as a correct spelling no anagrams BOMBS. See viewtopic.php?f=81&t=260 .

Aside from that particular reason, for which further study may show that I ultimately may be right (it was likely intended by Z) or perhaps wrong on (it is likely chance), I also think Z's desire to increase difficulty of solution after the 408 and the failure to decode the 340 lead me to think he may have encoded some or a lot of the 340 message vertically and diagonally.

Following up on the good work you have shown here, aided by Trav and others, you should be aware of the work of Professor Kevin Knight of USC. He is an expert in computers, langauges and codes. A few years back he and his USC team made headlines when they devised a computer program that successfully broke the centuries unsolved Copiale Cipher. See https://www.youtube.com/watch?v=Eam0Tk-1FyI . They established German as the language, and broke the code and revealed it to be the guidebook and manual for a Masonic secret society that promoted free thought and civil liberties. Ideas which at that time could land you in prison. Hence the secrecy and the encoding of the manual. Anyway the good news is he and his team have been working on the Voynich Manuscript and the Zodiac codes.

And as Doranchak noted at http://www.zodiackillerciphers.com/?p=448, towards the middle of that page, a 2013 study by Professor Knight at USC showed that unlike the 408 which shows a strong tendency towards traditional left to right repeat bigrams, the 340 shows no particular bias, in fact there are slightly more repeat bigrams in a diagonal north east read. Which I think perhaps matches in part what you are finding. Anyway I just wanted to call it to your attention and regardless say thanks for your efforts and keep up the good work. I think you are making some real discoveries.



Doranchak attended a lecture by Prof. Knight and stated:

[Knight]also talked about the unsolved Zodiac 340, and how it has no obvious reading order bias based on analyzing bigrams in several directions. [In the chart above] you can see the 408 has a preference for left/right order, since it results in a spike of repeating bigrams. So, the 340 “could be nonsense, or maybe bigrams are smoothed out via more careful substitutions.”

AK Wilks: Or could it perhaps indicate that some parts of the 340 message were encoded to have a vertical or NE diagonal read?

[Jarlve]

Hey AK Wilks,

I appreciate your interest and thank you for the information you have posted. It is interesting that the lack of direction is also represented in the count of the bigrams. It's safe to say that at least 50% or even more of the cipher was encoded horizontally, but the direction seems unclear. Maybe it has to do with the spike in the non-repeats at the length of 17, which is another very strange thing about the 340. Come to think of it I think the FBI also had difficulty estabilishing a direction back in the late 60's and therefore went under the assumption of an oxcart scheme.

Let's go through it more thoroughly.

340, horizontal vs mirrored (17x20) non-repeats while removing symbols at interval:

Even: 2129, 2129. 100.0%
Uneven: 1706, 1795. 95.0%
Every 3rd: 2804, 2334. 120.1%
Every 4th: 3101, 3306. 93.7%
Every 5th: 3539, 2939. 120.4%
Every 6th: 3774, 3559. 106.0%
Every 7th: 3918, 3973. 98.6%
Every 8th: 3751, 3318. 113.0%
Every 9th: 4124, 3768. 109.4%
Every 10th: 3975, 3650. 108.9%
Every 11th: 4018, 3717. 108.0%
Every 12th: 3964, 3657. 108.3%
Every 13th: 3848, 3873. 99.3%
Every 14th: 4195, 3628. 115.6%
Every 15th: 4191, 3543. 118.2%
Every 16th: 4085, 3584. 113.9%
Every 17th: 4096, 3820. 108.2%
Every 18th: 4467, 3898. 114.5%
Every 19th: 4207, 3780. 111.2%
Every 20th: 4221, 3721. 113.4%
None: 4462, 4260. 104.7%

Average: 108.7%

408, horizontal vs mirrored (17x20) non-repeats while removing symbols at interval:

Even: 1837, 1871. 98.1%
Uneven: 2322, 2095. 108.1%
Every 3rd: 2994, 2629. 113.8%
Every 4th: 3393, 2713. 125.0%
Every 5th: 3985, 3312. 120.3%
Every 6th: 4142, 3597. 115.1%
Every 7th: 3742, 2939. 127.3%
Every 8th: 3884, 3051. 127.3%
Every 9th: 4085, 3485. 117.2%
Every 10th: 4383, 3813. 114.9%
Every 11th: 4149, 4086. 117.3%
Every 12th: 4507, 3560. 126.6%
Every 13th: 4631, 3685. 125.6%
Every 14th: 4213, 3301. 127.6%
Every 15th: 4726, 3773. 125.2%
Every 16th: 4282, 3535. 121.4%
Every 17th: 4220, 3782. 111.5%
Every 18th: 4409, 3765. 117.1%
Every 19th: 4489, 3551. 126.4%
Every 20th: 4709, 3650. 129.0%
None: 4692, 4338. 108.1%

Average 119.7%

In a way I'm quite happy with these results, as it does show a direction for the 340!

The average for the 340 is 108.7% while the 408 has 119.7%, since the 340 has 9 symbols more than the 408 I would expect it to come in higher. Current thoughts are that something is running through the cipher, disturbing its data. Something which probably also causes a diagonal shift of the cipher. I have been looking into swaps at a short distance but it doesn't really disturb the data in the manner I'm looking for. Columnar transpostion also doesn't cause what I'm looking for. I will try to isolate the positions for the "thing" that is running through the cipher.

AK Wilks: Or could it perhaps indicate that some parts of the 340 message were encoded to have a vertical or NE diagonal read?

Maybe it will become appearant if I manage to isolate the disturbance.

[Jarlve]

I ran a test removing a random number of symbols at random positions for a large number of iterations scoring and averaging the numbers of the non-repeats. Darker areas have a more negative impact on the non-repeats, so these symbols are causing the disruption. It is much more apparant in the 340. About half of these fall on positions "every 3rd and every 5th" so that's why these values are higher. It also shows that the non-repeat data is much easier to climb for the 340. But why? Transposition? A more random cycle? Polyalphabetic? More digging that's for sure.

Every 3rd: 2804, 2334. 120.1% <---
Every 4th: 3101, 3306. 93.7%
Every 5th: 3539, 2939. 120.4% <---

Also notice for the 340 that lines 1,2,3,11,12,13 and 20 are least affected because they have no repeat per line. (the FBI theory?)

340:


https://www.dropbox.com/s/6szywgrr5tly2 ... t.png?dl=0

408:


https://www.dropbox.com/s/lzm1vtx5isrgh ... t.png?dl=0

[Jarlve]

One way to make sense of seemingly contradicting data is moving into the direction that a large part of the cipher is random. Or what Dan Olson suggested so I will take a look at that.

I will repeat the CHS direction test for the 340 with the first 3 lines of the 1st and 2nd part removed, lines 1,2,3 and 11,12,13 to be exact. To see if this chunck has a direction. There should be none if it is random.

340, horizontal vs mirrored (17x14) non-repeats while removing symbols at interval:

Even: 1122, 1156. 97.0%
Uneven: 1534, 1613. 95.1%
Every 3rd: 1834, 1477. 124.1%
Every 4th: 2073, 2016. 102.8%
Every 5th: 2075, 1893. 109.6%
Every 6th: 2257, 2066. 109.2%
Every 7th: 2682, 2364. 113.4%
Every 8th: 2264, 2255. 100.3%
Every 9th: 2510, 2248. 116.5%
Every 10th: 2406, 2091. 115.0%
Every 11th: 2464, 2085. 118.1%
Every 12th: 2680, 2109. 127.0%
Every 13th: 2704, 2313. 116.9%
Every 14th: 2530, 2232. 113.3%
Every 15th: 2748, 2380. 115.4%
Every 16th: 2548, 2392. 102.7%
Every 17th: 2519, 2488. 101.2%
Every 18th: 2677, 2575. 103.9%
Every 19th: 2565, 2394. 107.1%
Every 20th: 2586, 2269. 113.9%
None: 2757, 2690. 102.4%

Average: 110.1%

There is now a stronger indication of direction without these 6 lines. The average is higher than that of the normal 340. Very strange because these 6 lines on their own are: 1735, 1430. 121.3%

Unbelievable.

[Jarlve]

Following up on AK Wilks post on the bigrams. I decided to redo the test for all combinations of primary and secondary wind directions, with the primary direction either, not alternated, alternated starting even or alternated starting uneven. I believe my results will be more thorough.

Bigram counts bigger than 1 percentages for the test ciphers, all capped to 17x20.

CHS:

408:
Horizontals: 39.0%
Verticals: 17.8%
Diagonals NE and SW: 21.8%
Diagonals SE and NW: 21.2%

ray_n:
Horizontals: 39.7%
Verticals: 22.0%
Diagonals NE and SW: 20.0%
Diagonals SE and NW: 18.2%

408 redone:
Horizontals: 44.6%
Verticals: 14.1%
Diagonals NE and SW: 20.0%
Diagonals SE and NW: 21.0%

340:
Horizontals: 29.8%
Verticals: 20.4%
Diagonals NE and SW: 26.9%
Diagonals SE and NW: 22.7%

RHS:

mike_c:
Horizontals: 39.3%
Verticals: 20.3%
Diagonals NE and SW: 20.7%
Diagonals SE and NW: 19.4%

408 redone random cycle:
Edit: corrections made for this data.
Horizontals: 38.5%
Verticals: 17.9%
Diagonals NE and SW: 19.8%
Diagonals SE and NW: 23.6%

408 CHS mimick algorithm (viewtopic.php?f=81&t=2218):
Horizontals: 43.1%
Verticals: 17.9%
Diagonals NE and SW: 17.2%
Diagonals SE and NW: 21.5%

Summarizing, the bigram counts seem to be a good indication of direction and make for another powerful tool in our arsenal. There is little difference for the counts between CHS and RHS. Results for the 340 score highest for horizontals and diagonals but it differs strongly from the other ciphers. Let's test the relevance of the bigrams by encoding the 408 vertically.

408 plaintext:

Code: Select all

ilikekillingpeopl
ebecauseitissomuc
hfunitismorefunth
ankillingwildgame
intheforrestbecau
semanisthemostdan
gerousanimalofall
tokillsomethinggi
vesmethemostthril
lingexperenceitis
evenbetterthanget
tingyourrocksoffw
ithagirlthebestpa
rtofitisthatwheni
dieiwillbereborni
nparadiceandallth
eihavekilledwillb
ecomemyslavesiwil
lnotgiveyoumyname
becauseyouwilltry


Bigram counts percentages:
Horizontals: 30.1%
Verticals: 22.3%
Diagonals NE and SW: 24.0%
Diagonals SE and NW: 23.4%

408 vertical S-E plaintext:

Code: Select all

iciwbaoeetoaedlea
lasienktrhfriilsm
iumlcgiheaftwceie
ksodaelennwoiedwb
eergurlmcgiflawie
kieasosoeetilnilc
itfmeuosithtbdlla
liuemsmtttaiealnu
lsniaaetiigsrlbos
istnnnthsnitelete
nohtiihregrhbtcgy
gmahsmiivylaohoio
punetanleottremvu
eckfhlglnuhwnieew
ohioeogibrehihmyi
pflrmfinerbenayol
lulroavgtoenpvsul
eniesleetcsiaelmt
binstlsxektdrkayr
etgtdtmprspiaivny

Bigram counts percentages:
Horizontals: 23.7%
Verticals: 28.6%
Diagonals NE and SW: 23.9%
Diagonals SE and NW: 23.6%

408 vertical plaintext CHS encoded 63 symbols:

Code: Select all

0719:<@CDKA=EPQF>
R?V2GY]L^ad_34SWe
5gfT8i6bH<dM97I0J
]XBP=CUDZ[9@1EP9;
FG`jh^Qe8i2dR>93H
]4I?VAWBJCN5S\6T7
0OdfDg@X1KcL:PUQ<
R2hEeVfMNO=3F>SYg
TWZ4?<GK56jX_U;AV
0WL[\YMaXZ1NHQIOJ
[BbK23c`Ci^a:L8jk
ie=bVf45lkR>@cA6B
mh\DM?YSE@NO_Felg
G7]daTjUZhb9[0HI9
Ac1BJ@i2;`Ca3bfk4
mdQ^ed5\D_:EY<kAR
SgT`B=ljK@FZmlWhU
G[6HXQIJL8V0>CRfM
;1\WNSXnD]OP^]?k_
EKiLPMem`Vm2<3lYk


Bigram counts percentages vertical versus horizontal:
Horizontals: 25.2% / 44.6%
Verticals: 29.0% / 14.1%
Diagonals NE and SW: 21.8% / 20.0%
Diagonals SE and NW: 23.8% / 21.0%

Bigram test is very relevant but what could this mean for the 340? Maybe its plaintext is random? Let's test a cipher with a randomized plaintext version of the 408.

408 randomized plaintext:

Code: Select all

heinsmndogcgihngu
aietlodhaaitlalcc
metikvilltixsifil
psfnnalsrftestnvn
narteinifittedetm
ebnerigtelneciurl
elnfesieaoiiurblo
eimfoohobwekerwio
tagctironktnanibo
csohielukelhersad
wovmoeslhsbudsvrl
tasttwtaeohtceyea
ertuwtdrippyeclsu
lsitlsenaoingealt
sebneevwarmgplyyl
giiemuslheelhniea
ioekeinimhigribhl
taieimihltytnkgas
gsuetiaeoyiafbmep
lrtmlehwrmelomiuo


Bigram counts percentages:
Horizontals: 24.5%
Verticals: 24.8%
Diagonals NE and SW: 25.2%
Diagonals SE and NW: 25.4%

408 randomized plaintext CHS encoded:

Code: Select all

03;BFICKLOQP<1DOS
U=4Y^MK2VW>Z_X`RQ
J5[?cd@ab\AeG;f<^
gHfEBU_Fhf]6GYCdD
EViZ7=B>f?[\8K9]I
:kC3j@PY4`D5RATha
6bEf7H;8WN<=Sil^L
9>JfMN0Lkm:c3jm?M
ZXOQ[@hNBc\CUDAlL
RFM1;4_Tc5`26iGVK
mNdIL7Ha0FkSKGdjb
]WHYZm[X8M1\Q9n:U
3h]TmYKi<ggn4R^FS
_G=Z`H5EVN>BP6Wa[
F7lC89dmXjJOgbnn^
P?@:ITG_234`0DA5U
;L6c7<E=J1>Oh?k2a
\V@8AI;0b]nYBcPWH
OFS9Z<X:Mn=UflJ3g
^i[I_41mjJ5`NI>TL


Bigram counts percentages:
Horizontals: 28.5%
Verticals: 19.2%
Diagonals NE and SW: 24.5%
Diagonals SE and NW: 27.7%

Well, this seems to correlate pretty wel with the 340 doesn't it. But when looking at the data of the spread of the symbols all my CHS test ciphers sit at around 3500. The 340 comes in at 4000, which could mean 2 things. CHS and RHS mixed to a far greater extent than what is seen in the 408 and the plaintext may be random. Or transposition, more specifically transposition which moved information up and down in the cipher, which should have moved horizontally. Either in a subtle manner all through the grid or in one concentrated area probably not bigger than 10 rows. This could have been diagonal and or vertical. Given some of these subtle discrepancies in the 340 I do think there is a full grid solution available.

[Jarlve]

An idea that I came up with is that it may be possible to localize an area of higher disturbance, if such an area exists in the 340. A horizontal "sweep" will be performed on the cipher for a block of 10 rows. So the first set of rows will be 1 to 10, then 2 to 11, then 3 to 12, up to 11 to 20. I tried to create a test cipher to compare the 340 to but it is so unique that I cannot come up with anything similar. I tried mixing CHS with RHS and a randomized plaintext but that gave an all time low on horizontal bigram counts, the only cipher that "compares" is the one in my previous post but that uses perfect CHS which is a really big difference with the 340.

Example image showing the sweep (red rows are excluded):



Bigram percentages:

1-10:
Horizontals: 30.8%
Verticals: 24.4%
Diagonals NE and SW: 23.1%
Diagonals SE and NW: 21.5%

2-11:
Horizontals: 28.5%
Verticals: 24.2%
Diagonals NE and SW: 22.6%
Diagonals SE and NW: 24.5%

3-12:
Horizontals: 29.9%
Verticals: 23.6%
Diagonals NE and SW: 22.7%
Diagonals SE and NW: 23.6%

4-13:
Horizontals: 35.9%
Verticals: 26.7%
Diagonals NE and SW: 16.6%
Diagonals SE and NW: 20.6%

5-14:
Horizontals: 30%
Verticals: 22.8%
Diagonals NE and SW: 22.8%
Diagonals SE and NW: 24.2%

6-15:
Horizontals: 23.6%
Verticals: 24.9%
Diagonals NE and SW: 23.9%
Diagonals SE and NW: 27.5%

7-16:
Horizontals: 28.9%
Verticals: 22.9%
Diagonals NE and SW: 24.5%
Diagonals SE and NW: 23.5%

8-17:
Horizontals: 27.4%
Verticals: 26.1%
Diagonals NE and SW: 22.4%
Diagonals SE and NW: 23.9%

9-18:
Horizontals: 27.1%
Verticals: 23.7%
Diagonals NE and SW: 28.7%
Diagonals SE and NW: 20.3%

10-19:
Horizontals: 23.9%
Verticals: 26.2%
Diagonals NE and SW: 30.8%
Diagonals SE and NW: 19%

11-20:
Horizontals: 17.1%
Verticals: 33.7%
Diagonals NE and SW: 28.5%
Diagonals SE and NW: 20.5%

Let's split them up and see if there are some tendencies.

Horizontals:
30.8%
28.5%
29.9%
35.9%
30%
23.6%
28.9%
27.4%
27.1%
23.9%
17.1%

Verticals:
24.4%
24.2%
23.6%
26.7%
22.8%
24.9%
22.9%
26.1%
23.7%
26.2%
33.7%

Diagonals NE and SW:
23.1%
22.6%
22.7%
16.6%
22.8%
23.9%
24.5%
22.4%
28.7%
30.8%
28.5%

Diagonals SE and NW:
21.5%
24.5%
23.6%
20.6%
24.2%
27.5%
23.5%
23.9%
20.3%
19%
20.5%

Averages:
Horizontals: 30.01%
Verticals: 27.92%
Diagonals NE and SW: 26,66%
Diagonals SE and NW: 24.91%

Thoughts, it is noteable that section 6 to 15 seems to be most unaware of its direction:

6-15:
Horizontals: 23.6%
Verticals: 24.9%
Diagonals NE and SW: 23.9%
Diagonals SE and NW: 27.5%

Which agrees with some of my work in: viewtopic.php?f=81&t=2158&start=20 Where I suspect rows 6 to 15 to have a specific transposition independent of bigram data.

Maybe it makes sense that in a word search, the center area will have the most directions going in. I have pondered the word search idea before and it would make sense with having certain symbols as empty filler "+, q" etc. Which I believe not to be part of the homophonic substitution, since it are these symbols specifcally that disturb the non-repeat data. Also if you add up all the positions of the "+" symbols and divide that by its count you have 171 (only 1 difference from center of cipher). This seemingly even distribution I think, could also be a little hint towards the word search direction. The pivots are also more likely.

This is a good example of how it could be that also correlates to some extent with the data posted in this thread. Uncertain direction in the middle, more verticals at the bottom.

[AK Wilks]

Really outstanding work. Thank you for doing this! I know many are following with interest including me.

Keep up the good work. If and whenever you get the chance please consider these two questions:

1. You mentioned the analysis by FBI RRCA (Code) Unit Chief Olson. He thought the 340 might be divided into two parts with the second part starting IIRC at line 11. You said in your study IIRC lines 1-3 & 11-13 stood out as different. Could that be consistent with those lines being the start of a new message and a new coding scheme?

2. You mentioned lines 6-15 as showing least amount of directional bias? Would you tend to think a word search type message with possible diagonal and/or vertical words would be more likely to occur in lines 6-15? Or roughly 1-10 horizontal 11-16 heavy diagonal 14-20 heavy vertical?

Thanks again.

[Jarlve]

AK Wilks, thank you for your interest.

It is very hard to answer such questions but I will try to do so and explain my reasoning behind it.

1) No. In the 340 you have 9 rows (1,2,3,7,11,12,13,15,20) that have no repeats. Rows (1,2,3,11,12,13) seem consistent with what is expected for rows that have no repeats. In a 63 symbol cipher it is very normal to have a large amount of rows that have no repeats. And these rows could appear anywhere. The rather small observed difference, I think, was just from comparing rows with no repeats to rows with repeats. A straightforward two part message, 170 characters each, with for instance 2 different encodings is out of the question here. I have done testing and such schemes upset the positional and bigram data of the cipher to a very large degree. There is however a "significant enough" difference between the positional data of the 340 and other CHS ciphers including the 408 which indicates that something else is going on. Likely information travelling up and down somehow when it should have travelled horizontally and/or symbols not being part of the encoding.

When symbols are not part of the encoding a few things happen, they disturb the homophonic cycle and cause a shift of information against what is normally expected. They also cause, in case with a fixed column count of 17 a vertical stretch of information. Which can explain the difference in the positional data by inflation of irregularities.

It is possible that a 2 part message/encoding was "masked" by mixing it more evenly between the rows. For instance, message 1 from rows 1 to 5 and 11 to 15 and message 2 from row 6 to 10 and 16 to 20. But that would still have resulted in a very significant reduction in bigram counts which is not present for the 340. Which is in general another very strong objection versus a 2 part message.

2) Can't say yet. The next step for me is to make a word search and encode it in a few different ways and see how well the bigram counts correlate with what is actually there. Also the bigram test that I did was not well suited to the typical word search, because I wasn't testing for that specifically. And the relevance of a bigram test on a 170 character string can also be questioned. I need to remain skeptical, but I also need to do something in order to make progress.

At the moment the word search looks like a promising direction for which I will do more work.