Page 4 of 9

Re: Evidence of transposition misalignment in the 340

PostPosted: Fri Dec 09, 2016 7:19 am
by smokie treats
Doranchak, somewhere you discussed at length the probability of finding two plaintext pivots in a 17 x 20 rectangle, and then the probability of encoding with a 63 symbol homophonic key and still getting pivots. I think that is very important and would like to put a link to that discussion in my Transposition and Phenomenon thread.

Wouldn't the probability of the plaintext aligning into pivots, no transposition involved, be exactly the same as the probability of pre - route transposition plaintext being in exactly the right positions to align themselves into a pivot after route transposition ( one big inscription rectangle, one big transcription rectangle )? Wouldn't the probability be roughly the same?

Re: Evidence of transposition misalignment in the 340

PostPosted: Fri Dec 09, 2016 7:41 am
by up2something
smokie treats wrote:Wouldn't the probability of the plaintext aligning into pivots, no transposition involved, be exactly the same as the probability of pre - route transposition plaintext being in exactly the right positions to align themselves into a pivot after route transposition ( one big inscription rectangle, one big transcription rectangle )? Wouldn't the probability be roughly the same?


Here's his reference page (with links to ZKS discussions).

http://zodiackillerciphers.com/wiki/index.php?title=Pivots

And I agree that the probability is the same.

Re: Evidence of transposition misalignment in the 340

PostPosted: Fri Dec 09, 2016 9:00 am
by smokie treats
up2something wrote:
smokie treats wrote:Wouldn't the probability of the plaintext aligning into pivots, no transposition involved, be exactly the same as the probability of pre - route transposition plaintext being in exactly the right positions to align themselves into a pivot after route transposition ( one big inscription rectangle, one big transcription rectangle )? Wouldn't the probability be roughly the same?


Here's his reference page (with links to ZKS discussions).

http://zodiackillerciphers.com/wiki/index.php?title=Pivots

And I agree that the probability is the same.


Thank you!

Jarlve, here is one that you might like. Inscribe into rectangles 4 x 15. Lift the plaintext from the inscription rectangles top down left right. Then transcribe into 17 x 20 rectangle.

75% of the period 1 bigrams become period 15, while 25% of the period 1 bigram become reverse period 44. Half of the period 2 bigrams become period 29, and half of the period 2 bigrams become period 30.

conversation.december.9.b.png

Re: Evidence of transposition misalignment in the 340

PostPosted: Fri Dec 09, 2016 9:56 am
by Jarlve
I'm taking a closer look at the pivot phenomenon. 4 different 90 degree pivots are possible in a grid. And they can occur in any length. If you look at the pivot table below then the pivots that appear in the 340 are type 4 and have a length of 3. Unequal lengths of the horizontal and vertical part of the pivots are also a possibility as well as gaps.

Code: Select all
1ABC  CBA2
A        A
B        B
C        C

C        C
B        B
A        A
3ABC  CBA4

Pivot probability versus 340:
Pivot type 4, length 2: 0.100755% for 1 pivot to occur per randomization
Pivot type 4, length 3: 0.001649% for 1 pivot to occur per randomization

We would expect to see 2 to 3 samples per 1 million randomizations that have 2 pivots of the same type. One idea is then to explore different transposition combinations to see the probabilities they yield.

I tried it on the 408 and in general the chances improve when adding periodical operations. I found that combining period 13 transposition along with removal of 2 characters yields a pivot in about the same region as the 340. And the chance for one pivot to occur increases to 0.08495575221238938. Probably because the pivots are unmirrored period 39 repeats and 13 is a factor.

The x-axis is removal of the first character 1 to 340 and the y-axis is removal of the second character 1 to 339. See the resemblance with where I found increases in AZdecrypt scores while adding characters earlier.

Image

Image

Re: Evidence of transposition misalignment in the 340

PostPosted: Fri Dec 09, 2016 10:09 am
by Jarlve
Here is one such variation of the 408. I was not able to preserve the original ASCII overlay for this sample. It has a type 4 length 3 pivot at row 15, column 15.

Code: Select all
(M",7;8BO4".F1$<*
8>QF(QER(NT4/8QHL
3C4,'6(U#@9S:K=J-
!-#,UA%M%E3-.',81
-!O="@5<7K4?I5N48
*?F-0T+K-T-*.NMF.
-F&F5)EK)H$SP<@,2
=VSH,>T+$"#78M0:@
G5"%U!8K/N$<P9G<@
A4>S-ST-!RS9:I5E@
*:L6A46I01,JC@QRU
E'&"AMN>-7-'D#1@)
*?*T)LJ(L"JDB'M<B
+U$+<&M#MT)"%!DTG
SVU(FF,IV&31MD?L7
F#?4,R0J$I(+>!)M5
=GO-TBD/02DF,D98*
(L+%G?8$P:E%H=:U?
/(KLQ0PB+8/8:2$AB
MOHSKI7K!KTI.H'

1 2 3 4 5 6 7 8 9 10 3 11 12 13 14 15 16
7 17 18 12 1 18 19 20 1 21 22 10 23 7 18 24 25
26 27 10 4 28 29 1 30 31 32 33 34 35 36 37 38 39
40 39 31 4 30 41 42 2 42 19 26 39 11 28 4 7 13
39 40 9 37 3 32 43 15 5 36 10 44 45 43 21 10 7
16 44 12 39 46 22 47 36 39 22 39 16 11 21 2 12 11
39 12 48 12 43 49 19 36 49 24 14 34 50 15 32 4 51
37 52 34 24 4 17 22 47 14 3 31 5 7 2 46 35 32
53 43 3 42 30 40 7 36 23 21 14 15 50 33 53 15 32
41 10 17 34 39 34 22 39 40 20 34 33 35 45 43 19 32
16 35 25 29 41 10 29 45 46 13 4 38 27 32 18 20 30
19 28 48 3 41 2 21 17 39 5 39 28 54 31 13 32 49
16 44 16 22 49 25 38 1 25 3 38 54 8 28 2 15 8
47 30 14 47 15 48 2 31 2 22 49 3 42 40 54 22 53
34 52 30 1 12 12 4 45 52 48 26 13 2 54 44 25 5
12 31 44 10 4 20 46 38 14 45 1 47 17 40 49 2 43
37 53 9 39 22 8 54 23 46 51 54 12 4 54 33 7 16
1 25 47 42 53 44 7 14 50 35 19 42 24 37 35 30 44
23 1 36 25 18 46 50 8 47 7 23 7 35 51 14 41 8
2 9 24 34 36 45 5 36 40 36 22 45 11 24 28

Re: Evidence of transposition misalignment in the 340

PostPosted: Sat Dec 10, 2016 5:02 am
by Jarlve
Jarlve wrote:I found that combining period 13 transposition along with removal of 2 characters yields a pivot in about the same region as the 340. And the chance for one pivot to occur increases to 0.08495575221238938. Probably because the pivots are unmirrored period 39 repeats and 13 is a factor.

The x-axis is removal of the first character 1 to 340 and the y-axis is removal of the second character 1 to 339. See the resemblance with where I found increases in AZdecrypt scores while adding characters earlier.

It's just the same pivot that appeared in all variations so the chance number inflated. And the resemblance is unworthy of note, it is just that the pivot appeared in about the same position.

Re: Evidence of transposition misalignment in the 340

PostPosted: Sat Dec 10, 2016 5:48 am
by Jarlve
smokie treats wrote:Jarlve, here is one that you might like. Inscribe into rectangles 4 x 15. Lift the plaintext from the inscription rectangles top down left right. Then transcribe into 17 x 20 rectangle.

75% of the period 1 bigrams become period 15, while 25% of the period 1 bigram become reverse period 44. Half of the period 2 bigrams become period 29, and half of the period 2 bigrams become period 30.

Hey smokie,

I tried your scheme but was unable to preserve many bigrams after encoding. Then reduced it to 2 rectangles with a few variations but it just does not come out right. It is so hard to reconcile period 15 with period 29. With mixing of periods, the shift from one period to another is usually gradual.

Re: Evidence of transposition misalignment in the 340

PostPosted: Sat Dec 10, 2016 9:34 am
by smokie treats
Jarlve wrote:It is so hard to reconcile period 15 with period 29.


Best chance for a transposition that has a lot of period 15 and 29 repeats would be so that period 1 becomes period 15 and period 2 becomes period 29. Or maybe vice versa. But I have not been able to find one yet.

It could be a matter of interpretation. draft the message into a rectangle with different dimensions, count the cells in another direction, and they are not period 15 and 29. Maybe there is a transposition that could easily explain a different interpretation of the two periods. But then you wouldn't have pivots.

On my to do list is to try to make a three rectangle transposition where the plaintext is transcribed into the final 17 x 20 rectangle vertically. What we are currently interpreting as period 29 repeats derived from period 1. Forget about trying to include the period 15 repeats, just see if converting period 1 to interpreted period 29 could create a lot of pivots.

Check out this, starting at the 9th post down and definitely go the next page. Multiple consecutive incomplete rectangles can sometimes make for very interesting results. Sometimes period 1 becomes period 1 again or period 2 becomes period 2 again in certain areas of a 4th rectangle, whether horizontal or vertical. That could explain the pivots maybe, but I haven't found the right combination yet.

viewtopic.php?f=81&t=2617&start=1380

You might enjoy making matrices from different combinations of rectangle sizes and shapes, whether complete or incomplete, and shade the cells conditioned on original position. One idea would be to generate thousands of these, not encoded, and compare to the cells of the 340 that are "true" for period 15 / 19 and period 29 / 39. See if you can find a close match, or something that creates pivots.

Another idea: Make matrices and then mirror them. Watch what happens to spikes when that happens because mirroring is reversing individual rows, not the entire message.

Another idea: Find a transposition that creates period 19 and period 5 repeats. Reading left right top bottom, there is also a spike at period 5. Several of these inhabit the last couple of columns on the right and first couple of columns on the left. When the message is mirrored, they become period 29.

At least without transposition, the pivots are astronomically improbable, and with transposition maybe a little more probable.

?

Re: Evidence of transposition misalignment in the 340

PostPosted: Sun Dec 11, 2016 5:23 am
by Jarlve
smokie treats wrote:Best chance for a transposition that has a lot of period 15 and 29 repeats would be so that period 1 becomes period 15 and period 2 becomes period 29. Or maybe vice versa. But I have not been able to find one yet.

I think that your idea with transcribing around a column is in the right direction. Because when you add or remove a row in the 340, the period does not change, but it does with columns. There are other pointers that something with columns is going on, right-shift the entire cipher by one column and bigrams increase to 45. doranchak's column period 2 increases bigrams to 44. Add one column at the right side of the cipher and period 5 bigrams increase substantially. I figure that these features could be linked to the a misalignment that was introduced vertically.

smokie treats wrote:It could be a matter of interpretation. draft the message into a rectangle with different dimensions, count the cells in another direction, and they are not period 15 and 29. Maybe there is a transposition that could easily explain a different interpretation of the two periods. But then you wouldn't have pivots.

If you wish I could so some sort of scan for that, to your specifications.

I've just scanned the full range of, directional * dimension * directional, and here are the highest returning bigram results:

Mirror, Dimension(14,25), Diagonal(UTP,3): 42
Mirror, Dimension(14,25), Diagonal(UTP,6): 42
Flip, Dimension(16,22), Diagonal(UTP,1): 42
Flip, Dimension(16,22), Diagonal(UTP,8): 42

Code: Select all
Combinations processed: 265776/265776
Measurements:
- Summed: 5510230
- Average: 20.7326094154476
- Lowest: 7 (Mirror, Dimension(98,4), Diagonal(UTP,3))
- Highest: 42 (Mirror, Dimension(14,25), Diagonal(UTP,3))

smokie treats wrote:On my to do list is to try to make a three rectangle transposition where the plaintext is transcribed into the final 17 x 20 rectangle vertically. What we are currently interpreting as period 29 repeats derived from period 1. Forget about trying to include the period 15 repeats, just see if converting period 1 to interpreted period 29 could create a lot of pivots.

Check out this, starting at the 9th post down and definitely go the next page. Multiple consecutive incomplete rectangles can sometimes make for very interesting results. Sometimes period 1 becomes period 1 again or period 2 becomes period 2 again in certain areas of a 4th rectangle, whether horizontal or vertical. That could explain the pivots maybe, but I haven't found the right combination yet.

viewtopic.php?f=81&t=2617&start=1380

Well, it seems that multiple inscription rectangles are hard to reconcile with bigram repeats in general. So it is something that does not really catch my interest at the moment.

smokie treats wrote:You might enjoy making matrices from different combinations of rectangle sizes and shapes, whether complete or incomplete, and shade the cells conditioned on original position. One idea would be to generate thousands of these, not encoded, and compare to the cells of the 340 that are "true" for period 15 / 19 and period 29 / 39. See if you can find a close match, or something that creates pivots.

I get the general idea and was thinking to do that somewhere next year.

smokie treats wrote:Another idea: Make matrices and then mirror them. Watch what happens to spikes when that happens because mirroring is reversing individual rows, not the entire message.

Another idea: Find a transposition that creates period 19 and period 5 repeats. Reading left right top bottom, there is also a spike at period 5. Several of these inhabit the last couple of columns on the right and first couple of columns on the left. When the message is mirrored, they become period 29.

It's funny but we have the same ideas floating around in our head. Correlating interesting transposition schemes with the 340 is my usual modus of operandi. Further automatization of looking for particular features is on my to do list.

smokie treats wrote:At least without transposition, the pivots are astronomically improbable, and with transposition maybe a little more probable.

Again. I've thought the same thing. Though I don't think they randomly occured from transposition. They must be a result from transposition misalignment. That misalignment caused period 29 repeats to spike and that gave birth to the pivots. In this case, it would still be a very rare phenomena but I don't see any other way. The 340 is not supernatural.

Re: Evidence of transposition misalignment in the 340

PostPosted: Sun Dec 11, 2016 8:50 am
by Jarlve
Here's another plot, x-axis is a single removed character from position 1 to 340 and y-axis is a single added character from position 1 to 340. The color brightness is the AZdecrypt score. Highest return scored 21100. The 340 cipher was mirrored prior to any operation and the final operation was period 15 untranspose. The diagonal line is an artifact where the add and remove character overlap.

Image

Code: Select all
Remove character(47), Add character(199), Period(UTP,15)

SUEHEADOURSALIFON
SOHALTSCHOISSCHAS
INANOTCARDIACTTHE
MICANTANDARAGEONS
TREGUILTOBOUTINGH
OOTITNEERDETASAGU
ESHIHAREATHSTRICS
THISLOFRAINSURETH
EGASARTOFTHESOFOU
NDENTOFCERNINREAS
FIRESTERTINDANDBE
HELIGNSTHISREASFR
THOONTHATERMISTHI
NGHTHCITRENTSINTH
ETHEREAFIRSTERIAN
THSANDHDBOTHCREDS
SATSCADRENSHENATH
ICANDBORNTOFFARCH
HAROLHEAROLEITAIN
ERETOOUNEATURENES