Page 2 of 9

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 7:32 am
by smokie treats
Jarlve wrote:I'm mindblown and need some time with this, it looks very promising. I applied your transposition to a plaintext and then mirrored it and a few things stood out, bigram peaks at period 15 and 31. It managed to shift the period from 30 to 31. Can you do it the other way around? Reading from your post it seems that you have not found strong evidence of a column like this in the 340 but could you see how it looks if you add columns from position 1 to 18 instead of removing? And also for rows?

Here's your transposition matrix with spaces added for easier visual reference.


I am glad that you liked smokie38. However it did not work to create 340 bigram repeat stats because the center column caused too much distortion to the transposition structure. It may have been more successful if only in the bottom half, but I felt that would have been much more arbitrary than one full gibberish column.

You should have gotten a spike at period 29 when mirroring, so maybe try some other plaintexts. They are not all the same. I used the plaintext with the highest count of period 2 bigram repeats as compared to period 1 repeats, and I was still unable to reproduce 340 period 29 stats.

I was very pleased with the shuffle test. I can shuffle a region of any size or shape, and the test works well to identify gibberish rows and columns. I have not tried it on other types of regions. It would probably be more reliable when taking a slice through the transposition structure. I did not find any evidence that column 9 of the 340 is independent of the assumed transposition structure.

Not sure what you mean by adding columns from 1 to 18.

One approach to the pivots would be to try to find a transposition scheme that would naturally create them, whether single or in pairs, AND the period 15 / 19 repeats. I think that more extensive work in that area would help us to decide what the pivots really are. Natural result of transposition, intentional visual clue installed before or after drafting the balance of the message, or statistical anomaly.

I am working on that and considering, among other things, multiple inscription or transcription rectangles, incomplete rectangles, and alternating columns or rows during inscription or transcription.

EDIT: I really like the tracing map, or "matrix" idea because it allows us to easily share transposition ideas. I saw "matrix" in your new program. Is that what it does?

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 9:27 am
by Jarlve
smokie treats wrote:I am glad that you liked smokie38. However it did not work to create 340 bigram repeat stats because the center column caused too much distortion to the transposition structure. It may have been more successful if only in the bottom half, but I felt that would have been much more arbitrary than one full gibberish column.

I agree that it causes to much distortion. A misalignment caused by either addition or omission of characters is probably be going to minimal unless it somehow chimes in with the period itself. Given the results from the main post I'd say there's a good chance that it exists. Though, a minimal misalignment would probably not prevent us from solving the cipher straight away. So I feel there's something else also. Which could be another misalignment, but not in the form of addition or omission.

An example could be the following strange observation (I have this shared before). Right shift all the columns of the 340 by 1 and measure period 19 and you'll now find 45 bigrams. Going from 37 to 45 with such a minimal step blows my mind, the statistical difference between 37 and 45 is huge and there is no combination of both column and row offsets in either the normal/mirror/flipped/reversed period 15/19 variations that scores higher. And it is just one step away. I need to do more research on this if it could be a step in the process or not.

Code: Select all
dHER>pl^VPk|1LTG2
)Np+B(#O%DWY.<*Kf
JBy:cM+UZGW()L#zH
2Spp7^l8*V3pO++RK
/_9M+ztjd|5FP+&4k
(p8R^FlO-*dCkF>2D
|#5+Kq%;2UcXGV.zL
9(G2Jfj#O+_NYz+@L
Kd<M+b+ZR2FBcyA64
--zlUV+^J+Op7<FBy
OU+R/5tE|DYBpbTMK
F2<clRJ|*5T4M.+&B
Rz69Sy#+N|5FBc(;8
+lGFN^f524b.cV4t+
+yBX1*:49CE>VUZ5-
2|c.3zBK(Op^.fMqG
LRcT+L16C<+FlWB|)
p++)WCzWcPOSHT/()
c|FkdW<7tB_YOB*-C
+>MDHNpkSzZO8A|K;

17 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
33 18 5 19 20 21 22 23 24 25 26 27 28 29 30 31 32
41 20 34 35 36 37 19 38 39 15 26 21 33 13 22 40 1
16 42 5 5 43 7 6 44 30 8 45 5 23 19 19 3 31
54 46 47 37 19 40 48 49 17 11 50 51 9 19 52 53 10
21 5 44 3 7 51 6 23 55 30 17 56 10 51 4 16 25
11 22 50 19 31 57 24 58 16 38 36 59 15 8 28 40 13
47 21 15 16 41 32 49 22 23 19 46 18 27 40 19 60 13
31 17 29 37 19 61 19 39 3 16 51 20 36 34 62 63 53
55 55 40 6 38 8 19 7 41 19 23 5 43 29 51 20 34
23 38 19 3 54 50 48 2 11 25 27 20 5 61 14 37 31
51 16 29 36 6 3 41 11 30 50 14 53 37 28 19 52 20
3 40 63 47 42 34 22 19 18 11 50 51 20 36 21 58 44
19 6 15 51 18 7 32 50 16 53 61 28 36 8 53 48 19
19 34 20 59 12 30 35 53 47 56 2 4 8 38 39 50 55
16 11 36 28 45 40 20 31 21 23 5 7 28 32 37 57 15
13 3 36 14 19 13 12 63 56 29 19 51 6 26 20 11 33
5 19 19 33 26 56 40 26 36 9 23 42 1 14 54 21 33
36 11 51 10 17 26 29 43 48 20 46 27 23 20 30 55 56
19 4 37 25 1 18 5 10 42 40 39 23 44 62 11 31 58

smokie treats wrote:You should have gotten a spike at period 29 when mirroring, so maybe try some other plaintexts. They are not all the same. I used the plaintext with the highest count of period 2 bigram repeats as compared to period 1 repeats, and I was still unable to reproduce 340 period 29 stats.

I noticed the variability, something to take in mind with the 340.

smokie treats wrote:Not sure what you mean by adding columns from 1 to 18.

A misunderstanding on my part, you shuffled the symbols of these columns.

smokie treats wrote:EDIT: I really like the tracing map, or "matrix" idea because it allows us to easily share transposition ideas. I saw "matrix" in your new program. Is that what it does?

Yes. Put your cipher, be it numerical or characters in the input window and put a transposition matrix into the output window and then go to functions -> transposition and click on "use transposition matrix" and then transpose/untranspose. One way to generate these from scratch in AZdecrypt is to go to text filters and click on generate numbers and fill in the from# and to# fields and click on process. After that you can use dimension and transposition to define it.

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 10:43 am
by Jarlve
This is a follow up to the main post. Now, 2 new unique symbols were added with the mirored 340 as a starting point. The x-axis are the positions ranging from 1 to 341 were the first symbol was added and the y-axis the second symbol from 1 to 342. The resulting variation was then untransposed with period 15 and lastly normal and reversed strings are generated. The square on the left is the normal string and the square on the right is the reversed. Thus, each pixel is a solve and its color intensity is the AZdecrypt score with 10 restarts.

Image

Code: Select all
LBEHERERSTOOBREAC
URSUEINSHOWADOWAS
STEFORNOHITSETTOA
LLEHALARARONEARDI
NREGIONEDINCINCAR
SASOUSISACUITCOLL
UCINGSOOTITUPRESE
TISSGUESHIHAREATH
LEASINTHANDOFASAI
NBRETHETERIAPOFTH
ESOFOURDOUTOFGALS
INCOMSFIRESPECTAN
DASOUTHEDIAINTSIN
COASFRCHNONTHATWA
NISTHINGITHSETOAM
USINTHENHERWIFEAS
PEARTSETHNARCHCLO
THEREDSSARINGARSI
NSONATHADINALLITS
OFFICEHHAROTHEGRO
DW

Code: Select all
Combinations processed: 233244/233244
Measurements:
- Summed: 4698243026.503229
- Average: 20143.03916286476
- Lowest: 19476.80753592638 (Add character(74), Add character(130), Period(UTP,15), Reverse)
- Highest: 21341.11905763229 (Add character(200), Add character(331), Period(UTP,15), Normal)

Observations,
- There is a cross of increased scores, which sits roughly at x: 200, y: 200. In line with previous observation. If more characters were omitted, why aren't they showing up as much?
- The area of increased scores is hardly visible with the reversed string, indicating that one direction is better than another (plaintext phenomenon).
- Highest scoring variation: 21341.11905763229 (Add character(200), Add character(331), Period(UTP,15), Normal). The highest 340 score I've seen. Though 2 unique symbols have been added raising the multiplicity slightly and 233244 combinations were processed.
- Some interesting fragments in the resulting plaintext "INALLITSOFFICE" and "COLLUCING".

A good hypothesis is that the cipher once existed in a 23 by 15 grid and then was taken off by columns and transcribed in a 17 by 20 grid bottom-to-top or right-to-left and 5 characters (or less?) had to be left out (going from 345 to 340). It seems hard to reconcile leaving out 5 characters with bigram findings though. But it makes allot of sense for everything else. I made a cipher as such but had to raise the raw ioc (repeat potential) significantly above that of the 340 to match 41 bigram repeats for mirrored period 15. I left out 5 characters. Can you solve it?

Code: Select all
%H:G)9?;AWQ^8VT+"
?9=8-;,=;5=Y=A94P
@,'>.+F(OG1ST$H7=
U]8/9*LZPT=Q^Q4=C
%,^VJ9I"A)7VR;GA!
)'JV;.2>TX=H*CSK5
N;H;HV9PTOT,G:A]Q
S,A<D7VUR_E@QVF+G
:.%^TVI_EV>";?=18
RLV:;A,GH8,]PW2G9
J5(A4GT)VPH]=9$;H
,6+G?R5BO#FA9<T=D
!J5=TV]Q'0/*(XH8;
A;2KGS&7_R%"9UI@E
9'^=1@>K)AQVKVVVH
B)TVPU:<;O=H;HC,\
]GH.LG"F9;^FH8PA4
2R@>9A;/4$V8:V2+M
4PVK3:!9#AS^;FDU7
I[^ETJC"F,7A%Q8?H

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
7 6 18 13 19 8 20 18 8 21 18 22 18 9 6 23 24
25 20 26 27 28 16 29 30 31 4 32 33 15 34 2 35 18
36 37 13 38 6 39 40 41 24 15 18 11 12 11 23 18 42
1 20 12 14 43 6 44 17 9 5 35 14 45 8 4 9 46
5 26 43 14 8 28 47 27 15 48 18 2 39 42 33 49 21
50 8 2 8 2 14 6 24 15 31 15 20 4 3 9 37 11
33 20 9 51 52 35 14 36 45 53 54 25 11 14 29 16 4
3 28 1 12 15 14 44 53 54 14 27 17 8 7 18 32 13
45 40 14 3 8 9 20 4 2 13 20 37 24 10 47 4 6
43 21 30 9 23 4 15 5 14 24 2 37 18 6 34 8 2
20 55 16 4 7 45 21 56 31 57 29 9 6 51 15 18 52
46 43 21 18 15 14 37 11 26 58 38 39 30 48 2 13 8
9 8 47 49 4 33 59 35 53 45 1 17 6 36 44 25 54
6 26 12 18 32 25 27 49 5 9 11 14 49 14 14 14 2
56 5 15 14 24 36 3 51 8 31 18 2 8 2 42 20 60
37 4 2 28 40 4 17 29 6 8 12 29 2 13 24 9 23
47 45 25 27 6 9 8 38 23 34 14 13 3 14 47 16 61
23 24 14 49 62 3 46 6 57 9 33 12 8 29 52 36 35
44 63 12 54 15 43 42 17 29 20 35 9 1 11 13 7 2

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 11:03 am
by doranchak
Jarlve wrote:I belief to have found evidence of transposition misalignment in the 340.

Here's a graph that shows it. The x-axis are the positions from 1 to 341 at which a new unique symbol was added. And the y-axis is the AZdecrypt score with 100 restarts. Prior to the other operations the 340 was mirrored. Highest score is slightly over 21000 with a symbol added at position 200.

That's very interesting. It is similar to the spikes I found a while back when I was playing around with different transposition schemes:

viewtopic.php?p=43702#p43702

The noteworthy result from those experiments was that several high scoring AZDecrypt runs were due to operations involving "PeriodColumn(2) Period(18)" operations, which produced more repeating bigrams. Generally, AZdecrypt seems to score higher whenever repeating bigrams are high. Many combinations of operations are able to produce high numbers of repeating bigrams, so it's hard to figure out which combination is correct.

To recap those operations, PeriodColumn(2) simply re-arranges the columns 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17 as 1,3,5,7,9,11,13,15,17,2,4,6,8,10,12,14,16. Then, Period(18) simply untransposes the entire cipher with period 18.

I am wondering if the various ways of getting AZDecrypt spikes at around 21000 are indications of being close to a real untransposition, or if those operations are truly false positives. If I recall correctly, an AZdecrypt spike of around 21000 is only about 1 sigma (1 standard deviation) away from the average of AZdecrypt runs on random shuffles of Z340.

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 11:18 am
by doranchak
Jarlve wrote:
smokie treats wrote:An example could be the following strange observation (I have this shared before). Right shift all the columns of the 340 by 1 and measure period 19 and you'll now find 45 bigrams. Going from 37 to 45 with such a minimal step blows my mind, the statistical difference between 37 and 45 is huge and there is no combination of both column and row offsets in either the normal/mirror/flipped/reversed period 15/19 variations that scores higher. And it is just one step away. I need to do more research on this if it could be a step in the process or not.

As a point of reference, the "PeriodColumn(2) Period(18)" operation I mentioned produces 44 bigrams. It's interesting that we can find simple steps that yield significant increases in ngrams, with what seems to be a wide variety of steps.

Simple modifications allow more bigrams to be produced:

"PeriodColumn(2) Period(18) Swap(5, 105, 6, 1) Swap(101, 225, 3, 1)" produces 53 bigrams. Swap(5,105,6,1) exchanges a block of height 6 width 1 at position 5 with the same sized block at position 105. Swap(101,225,3,1) exchanges a block of height 3 width 1 at position 101 with the same sized block at position 225.

"PeriodColumn(2) Period(18) Swap(4, 104, 10, 1) SwapLinear(101, 238, 55)" produces 52 bigrams. SwapLinear(101, 238, 55) exchanges a substring of length 55 at position 101 with a substring of the same length at position 238.

"PeriodColumn(2) Period(18) RectangularSelection(7, 0, 7, 3) Period(10)" produces 52 bigrams. Period(10) is only applied to the RectangularSelection(7,0,7,3) which means a rectangular selection with upper left corner at row 7 column 0, and having height 7 and width 3.

There are several other variations of this type but many seem to have the "PeriodColumn(2) Period(18)" in common.

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 3:37 pm
by Jarlve
doranchak wrote:That's very interesting. It is similar to the spikes I found a while back when I was playing around with different transposition schemes:

viewtopic.php?p=43702#p43702

The noteworthy result from those experiments was that several high scoring AZDecrypt runs were due to operations involving "PeriodColumn(2) Period(18)" operations, which produced more repeating bigrams. Generally, AZdecrypt seems to score higher whenever repeating bigrams are high. Many combinations of operations are able to produce high numbers of repeating bigrams, so it's hard to figure out which combination is correct.

I'll add "PeriodColumn(2) Period(18)" to my list of things I need to check.

For clarification, there exists no bias in the code that makes AZdecrypt score higher when more bigrams are present in the cipher, I don't think you meant to say that either. It's still a bit of a guess as to why and to what extent it is happening but I believe it may have to do with bigrams being a plaintext property.

doranchak wrote:I am wondering if the various ways of getting AZDecrypt spikes at around 21000 are indications of being close to a real untransposition, or if those operations are truly false positives. If I recall correctly, an AZdecrypt spike of around 21000 is only about 1 sigma (1 standard deviation) away from the average of AZdecrypt runs on random shuffles of Z340.

It depends on how you get there. The problem is that some operations allow the cipher to be molded, especially with hill climbing or iterative processes. What I did in the follow up to main post of this thread was also include the reversed string, which always have the same bigram count. Observe, that the right side square does not have bright spots such as the left side square. So that seems to indicate direction. That's a trick which can add weight.

doranchak wrote:Simple modifications allow more bigrams to be produced:

"PeriodColumn(2) Period(18) Swap(5, 105, 6, 1) Swap(101, 225, 3, 1)" produces 53 bigrams. Swap(5,105,6,1) exchanges a block of height 6 width 1 at position 5 with the same sized block at position 105. Swap(101,225,3,1) exchanges a block of height 3 width 1 at position 101 with the same sized block at position 225.

"PeriodColumn(2) Period(18) Swap(4, 104, 10, 1) SwapLinear(101, 238, 55)" produces 52 bigrams. SwapLinear(101, 238, 55) exchanges a substring of length 55 at position 101 with a substring of the same length at position 238.

"PeriodColumn(2) Period(18) RectangularSelection(7, 0, 7, 3) Period(10)" produces 52 bigrams. Period(10) is only applied to the RectangularSelection(7,0,7,3) which means a rectangular selection with upper left corner at row 7 column 0, and having height 7 and width 3.

I disagree that these are simple modifications. Given the arguments, the swap and rectangular selection allow for a truly immense amount of variations and your hill climber is stacking them. This basicly allows molding to some degree. The 340 can be configured up to 265 bigrams so you can increase all you want. Judging from the 408 I expect the 340 to come out at anywhere between 45 to 50 bigrams when correctly untranposed.

From a classical cryptology point of view I'm exploring a viable transposition hypothesis. The plaintext existed in a 23 by 15 grid and then was taken off by columns and transcribed into a 17 by 20 grid following either right-to-left, top-to-bottom or left-to-right, bottom-to-top. In that process some symbols were left out. I like it because it is a minimal hypothesis that can explain rare observations in the 340 such as pivots and periodical repeats (I understand that this hypothesis could be wrong and I would like to find out).

Or do you find "PeriodColumn(2) Period(18) Swap(5, 105, 6, 1) Swap(101, 225, 3, 1)" more viable?

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 5:02 pm
by doranchak
Jarlve wrote:I'll add "PeriodColumn(2) Period(18)" to my list of things I need to check.

For clarification, there exists no bias in the code that makes AZdecrypt score higher when more bigrams are present in the cipher, I don't think you meant to say that either. It's still a bit of a guess as to why and to what extent it is happening but I believe it may have to do with bigrams being a plaintext property.

From my transposition experiments I produced the following plot of azdecrypt score vs # of repeating bigrams. It shows a cluster of scores that seems to drift upwards as the # of repeating bigrams increases.
plot-azdecrypt-vs-bigrams.jpg

Not sure how to interpret that - the scores still have a lot of variance, but the upwards drift is still detectable.

Also, "PeriodColumn(2) Swap(3, 156, 3, 3)" seems to greatly increase the L2 cycles (see item 5 here: http://zodiackillersite.com/viewtopic.php?p=43702)
Jarlve wrote:It depends on how you get there. The problem is that some operations allow the cipher to be molded, especially with hill climbing or iterative processes. What I did in the follow up to main post of this thread was also include the reversed string, which always have the same bigram count. Observe, that the right side square does not have bright spots such as the left side square. So that seems to indicate direction. That's a trick which can add weight.

That's a really nice plot. I like how you are using azdecrypt scores as a tool to prioritize candidates.
Jarlve wrote:I disagree that these are simple modifications. Given the arguments, the swap and rectangular selection allow for a truly immense amount of variations and your hill climber is stacking them. This basicly allows molding to some degree. The 340 can be configured up to 265 bigrams so you can increase all you want. Judging from the 408 I expect the 340 to come out at anywhere between 45 to 50 bigrams when correctly untranposed.

From a classical cryptology point of view I'm exploring a viable transposition hypothesis. The plaintext existed in a 23 by 15 grid and then was taken off by columns and transcribed into a 17 by 20 grid following either right-to-left, top-to-bottom or left-to-right, bottom-to-top. In that process some symbols were left out. I like it because it is a minimal hypothesis that can explain rare observations in the 340 such as pivots and periodical repeats (I understand that this hypothesis could be wrong and I would like to find out).

Or do you find "PeriodColumn(2) Period(18) Swap(5, 105, 6, 1) Swap(101, 225, 3, 1)" more viable?

Not particularly; the goal of my transposition explorer was to discover transpositions that resulted in increased azdecrypt scores and increased bigram counts, while minimizing the number of transposition steps. I only report the results as a matter of reference rather than indicating a belief in a particular scheme. I agree that molding the cipher can give many paths to high bigram counts. But I think it's compelling that "PeriodColumn(2) Period(18)" is all that's needed to bring the bigram count up to 44. Is it a feature, or a phantom?

In the end, I got frustrated with that project, because there are too many possible schemes to rule out, without careful controlling it with test ciphers. I think you are right to focus on classical cryptology rather than strange homemade schemes, at least until all the classical schemes have been ruled out.

Re: Evidence of transposition misalignment in the 340

PostPosted: Wed Dec 07, 2016 8:11 pm
by smokie treats
Jarlve, I did some work on this idea. See the third post down here: viewtopic.php?f=81&t=2617&start=1330

I spent some time on the 340 today. I have had some nagging issues about transposition, and one of them is whether a few transcription nulls, or transcription skips for the mirrored 340, could produce so many period 29 repeats and the pivots.

Here is the 340 mirrored, with period 29 repeat positions colored. 75 of the 170 positions in the bottom half are colored, which makes 44.1%. If the count of period 15 repeats is boson higgs significant, and the count of period 29 repeats astronomical, across the entire message, then what about just the bottom half? The pivots for the mirrored version are period 29.

period.29.mystery.1.png
period.29.mystery.1.png (34.65 KiB) Viewed 21 times

I made a spreadsheet suite to make period 15 messages, with an inscription rectangle of 22 columns x 15 rows and ten leftover, not transcribed plaintext at the end. Then I encoded with homophonic cycles, making about 25% of symbol selection random within cycle groups. Then I compared the messages with messages that had transcription skips introduced at random positions, with varying counts of transcription skips. For many of the messages, I limited the skips to the bottom half, and kept the count of skips to 4 or less. But I also tried a lot of variations.

For many of the messages with just a few skips, the count of period 15 repeats would drop slightly, and the count of period 29 repeats would increase slightly. But with no variation was I able to replicate a count of 65 period 29 repeats. Not even close. I could get 50 period 29 repeats with 3 skips, but getting more than that was difficult.

I am going to say that the null / skip idea for causing the pivots is probably not correct, and go back to the drawing board. I am still thinking that the message is some sort of transposition, but maybe he changed transcription direction for the bottom half, or did something else. There must be some sort of scheme that would make so many period 29 repeats, with 44.1% of the positions in the bottom half colored, and that must be what created the pivots.


It is not conclusive and I did not show all of my work. But with varying counts of randomly placed skips, some concentrated in the area of the pivots, I was not able to reproduce mirrored 340 period 29 stats.

Re: Evidence of transposition misalignment in the 340

PostPosted: Thu Dec 08, 2016 6:35 am
by Jarlve
doranchak wrote:Not sure how to interpret that - the scores still have a lot of variance, but the upwards drift is still detectable.

Also, "PeriodColumn(2) Swap(3, 156, 3, 3)" seems to greatly increase the L2 cycles (see item 5 here: http://zodiackillersite.com/viewtopic.php?p=43702)

Yes, I remember these items. I'm incredibly skeptic that anything special happened during or after the encoding. My hypothesis is that Zodiac made a key, just like with the 408 but did not actively cycle the symbols as he did with the 408. Instead he took greater care not to repeat symbols in a given view window, which could have been the rows. It lines up with everything we know about the encoding. In that hypothesis increases you find to the cycles must be phantoms.

doranchak wrote:Not particularly; the goal of my transposition explorer was to discover transpositions that resulted in increased azdecrypt scores and increased bigram counts, while minimizing the number of transposition steps. I only report the results as a matter of reference rather than indicating a belief in a particular scheme. I agree that molding the cipher can give many paths to high bigram counts. But I think it's compelling that "PeriodColumn(2) Period(18)" is all that's needed to bring the bigram count up to 44. Is it a feature, or a phantom?

Given the increase I think it is more likely to be a feature.

doranchak wrote:In the end, I got frustrated with that project, because there are too many possible schemes to rule out, without careful controlling it with test ciphers. I think you are right to focus on classical cryptology rather than strange homemade schemes, at least until all the classical schemes have been ruled out.

You could use it to help us with the schemes we are exploring.

Re: Evidence of transposition misalignment in the 340

PostPosted: Thu Dec 08, 2016 7:11 am
by Jarlve
smokie treats wrote:Jarlve, I did some work on this idea. See the third post down here: viewtopic.php?f=81&t=2617&start=1330

Okay, thanks for pointing that out. I'm going to take a look at it also. The basic question is if 1 to 5 nulls during transposition can be a likely cause for period 30 to shift to 29. Perhaps you are right and we need to move on to a more general question.

Does there exist some structure (any combination of symbols at any positions) in any step of the transposition process which causes period 29 bigrams repeats?

We have a couple of observations that link together:
- There are 2 pivotal structures in the 340 that are made up out of bigram 29 repeats.
- After mirroring the 340 there are 41 period 15 bigrams repeats and 34 period 29 repeats. A period shift from 30 to 29 is a real possibility due to some misalignment.
- A sharp increases of AZdecrypt scores (21000+) while adding a single new symbol in a contiguous region of about 15 characters wide (including other transposition steps). This contiguous region sits around one pivot.
- Period 30 bigrams repeats also increase while adding a single (or more) new symbol in the region around the pivot (position 200 give or take).

About the contiguous region and why it is important. If for instance a plaintext is transposed in a 23 by 15 grid. And the letters are taken off by columns and at one column one letter would be left out. Then you would find that to correct the misalignment, it would be valid to add a character over an entire region of about 15 characters wide. Because you just need to restore the length of the column (which is laid out horizontally after transposition) from 14 to 15. Another reason why I gave the original observation of this thread more weight.