Route Transposition and Phenomenon

[doranchak]

Jarlve, I've done symbol merges in the past but did not consider their effect on pivots. Your recent symbol merge test provoked my curiosity: If we merge certain symbols together, will more pivots appear? The answer is yes, but they may just be easy to produce this way.

Examples:

Merging



Merging



Merging



As you increase the number of merged symbols, you increase the chance of more pivots showing up. But I'm wondering if I can limit the search to the ones whose axes fall on positions divisible by 39 (or 13), and/or involve symbols that aren't so high frequency in the ciphertext. Or maybe new pivots that correlate to increases in periodic bigrams. I'll poke around with it some more.

[doranchak]

I will do a little more experimenting soon. I am just wondering if false cycles and true cycles have slightly different characteristics with regards to the spacing between ciphertext. Is there a way to tell the difference?

That is the sort of question that troubles me. It may be that the difference is so small, it would be hard to detect quantitatively. These sorts of unknowns usually lead me down the path of "hey, let's throw a machine learning classifier at it and see what it can learn, since it's too hard for me to discover the exact relationships." For example, you could produce a big pile of homophonic test ciphers, and train the machine learning algorithm to discover the clusters of symbols that are homophones. Then see how well it would behave with test ciphers it has never seen before.

The fun part of that approach is looking into the trained algorithm to see what it might have discovered about the symbol relationships we're looking for. You can see that in some kinds of algorithms (decision trees, genetic algorithms, etc). But they are much harder to see in neural networks, even if the predictions are accurate. If only we could devote a full time group of researchers to these kinds of things!

[Jarlve]

Does that mean the plaintext is still readable or have too many swaps occurred to preserve readability?

Not readable.

Are you shuffling the plaintext to simulate transposition?

No, just wanted to shift the focus to the encoding entirely.

[Jarlve]

I wanted to reproduce this observation. Here are my numbers, do they look the same as yours?

Yes, looks good.

Will try to reply to the other stuff later on.

[doranchak]

Still working through my symbol merge tests, and noticed this potential 5-gram that forms at period 19 if you merge the half filled square and circle. Cipher is shown at width 19 for improved visualization of the repeats. Perhaps you've encountered this already.

[Jarlve]

I will do a little more experimenting soon. I am just wondering if false cycles and true cycles have slightly different characteristics with regards to the spacing between ciphertext. Is there a way to tell the difference?

One thing you can do is look at minimal hypothetical scenarios and figure it out from there.

I made a cipher with only 2 symbols, the placement of these symbols is totally random. Then encoded to 4 symbols, the encoding is totally random. Can you infer which number pairs are true cycles by looking at distances? I haven't tried myself and will leave it up to you.

Code: Select all

1 3 2 4 3 1 4 1 2 1 2 2 1 1 3 3 4 2 1 4
1 3 2 3 4 2 3 4 4 3 2 3 2 2 3 1 2 2 1 4
4 3 4 3 2 4 1 2 2 4 4 4 2 1 1 4 4 2 1 2
2 3 1 3 1 1 3 2 4 1 3 2 4 4 4 4 1 1 4 1
4 3 2 4 4 2 3 3 1 4 2 2 4 3 2 2 3 1 1 3
1 2 2 3 4 1 1 3 4 2 2 2 4 2 3 2 4 2 1 3
3 3 1 2 3 1 3 4 2 4 4 2 3 3 1 2 1 4 2 4
3 3 1 1 1 1 2 1 4 1 4 3 2 4 3 3 2 3 1 4
2 4 2 4 2 1 1 1 1 1 1 2 4 1 4 1 1 3 3 3
1 4 3 2 4 3 2 3 4 2 3 3 3 4 4 3 3 1 4 3


[Jarlve]

Without any further cycle start and stop work, I have been thinking about variations on the idea. Jarlve, you might like this one. I modified the spreadsheet to calculate the total count of positions covered for each cycle, then average the count of positions covered by each count of consecutive alternations. Below, x axis is CA, y axis is average positions covered, Zodiac 340.

One alternation is "AB" right. So the expected or average number of positions covered per should be about 340/63*CA. And your graph shows a drop at 10 cycle alternations. I don't know what to think, what is your interpretation?

[Jarlve]

Jarlve, I've done symbol merges in the past but did not consider their effect on pivots. Your recent symbol merge test provoked my curiosity: If we merge certain symbols together, will more pivots appear? The answer is yes, but they may just be easy to produce this way.

That's interesting. You could make a 17 by 20 heatmap (while running symbol merges) where each point is the pivot axis to find out if there are regions in which they appear more often. I recommend then to allow pivots to wrap around the cipher.

After that is done you could do the same for a 340 character part of the 408 or one of your emulation ciphers to see if there's any major difference in counts.

[Jarlve]

I made a small bit of progress,

You will need some time to digest this, it will make your head spin.

Remember that adding an 18th column to the 340 increases untranposed period 5 bigrams to 35 from 30? It seems related to the transposed bigram bump from period 68 to 78 (especially with assumed cycle "2z"):

Code: Select all

340:

Period 5: 19, 30
----------------------------------
Period 68: 30, 19
Period 69: 31, 21
Period 70: 31, 23
Period 71: 31, 15
Period 72: 30, 17
Period 73: 30, 21
Period 74: 29, 30
Period 75: 29, 22
Period 76: 30, 20
Period 77: 31, 18
Period 78: 31, 19


To understand it better, note that a period 4 transposition equals a period 85 untransposition, and a period 5 transposition equals a period 68 untransposition. After applying period 4 transposition to a 340 character capped version of the 408 transposed bigrams spread out from period 79 to 92:

Code: Select all

408 (340 chars):

Period 79: 31, 19
Period 80: 31, 16
Period 81: 36, 16
Period 82: 39, 19
Period 83: 43, 18
Period 84: 43, 19
Period 85: 46, 21
Period 86: 47, 20 <---
Period 87: 44, 20
Period 88: 41, 23
Period 89: 37, 17
Period 90: 32, 20
Period 91: 30, 24
Period 92: 30, 18


And after applying a period 5 transposition to the 408 transposed bigrams spread out from period 64 to 74:

Code: Select all

408 (340 chars):

Period 64: 30, 28
Period 65: 32, 19
Period 66: 39, 16
Period 67: 45, 15
Period 68: 46, 16 <---
Period 69: 43, 13
Period 70: 44, 27
Period 71: 40, 24
Period 72: 37, 18
Period 73: 34, 21
Period 74: 31, 19


It can also be done the other way around. Applying a period 68 untransposition will cause an untransposed period 5 bigram peak. And applying a period 85 untransposition causes an untransposed period 4 bigram peak. What we could do is apply a period (68+85)/2 untransposition to cause a weaker untransposed period 4 to 5 bigram peak.

It is interesting that after assuming cycle "2z" in the 340 transposed bigrams go up so sharply in the 68 to 85 range peaking at 78 which is neither period 68 or 85 but instead an intermediate. I wonder if it could be another possible indication of transposition misalignment or tranposition mismatch.

I also wonder if a period 78 untransposition would then be primary to a period 4 or 5 transposition since a period 4 or 5 transposition would not cause the peak to be directly at or around 78. 78 is very much in between of 68 and 85. 4, 5, 68 and 85 are factors of 340, it all seems vaguely connected. If period 78 untransposition is primary to period 4 or 5 tranposition then the assumed cycle "2z" observation is primary to the add 18th column observation.

Period 78 untransposition, what could it mean?

[smokie treats]

I have been under the weather and unable to keep up, so I will have to look at your posts a bit later. Sorry. But I think I will have a solution to message 1 soon.