Homophonic substitution

[smokie treats]

I'm going to read your approach after you get the results, and we can go from there.

A preliminary analysis of M3 shows that it is much more cyclic than the 340. There are a few missing symbols in some of the higher scoring cycles, but only a few. Not sure why. Top half is comparable to bottom half.

M3.cyclestats.png

The scattergraph of count versus total score shows basically the same thing. Most of the dots are bunched up really close together in the respective count columns, showing the strong cycling. Symbols 5 and 13 are on the far right because of their high count and they cycle together. But there are a few missing symbols in that cycle too. Symbols 1, 3 and 7 have count 13, but they do not cycle together well, even though I thought that they did at first. I had to check manually to see what was going on. Their scores are high and similar, I am guessing, because they have such high count. With 13 count, there are going to be some short to medium false cycles with some of the other 50+ symbols just because of probabilities.

M3.scatter.png

I am a little bit burned out tonight, so I lazily marked the latest cycle hillclimber with colored boxes showing symbol groups that need further investigation as possible cycles. Because of the strong cycling overall, I am guessing that maybe at least half of them are grouped together correctly, if not more. There are 91 bigram repeats on my list. I haven't tried to solve it yet.

M3.cyclehillclimber.png

[daikon]

Nothing to report regarding wildcard solves in Z340. I'll let it run overnight, just to be sure, and then move on to the next set of possible wildcards.

Didn't notice that there is a new Mystery Cipher — #3. I'll check it out tomorrow too.

[smokie treats]

Thanks for introducing me to the world of 340 bigram repeats. We have way fewer of them than we should, and look at all of those that have suspected wildcards in them. Why? Maybe it is because the suspected wildcards are high in count. I think that the relationship between symbol count, cycling, randomness, bigram repeats, and whatever else is fascinating. The fact that a large proportion of the bigram repeats include symbols that don't cycle with other symbols. Is that significant?

340.biram.repeats.png

I marked the symbols that I strongly suspect as being wildcards in green, the weaker candidates in grey. So nineteen out of 46 repeats include symbols 19 or 20. Let's say, hypothetically, that Zodiac created these 19 repeats during the masking process. So then take these out for a minute, and we have 46 - 19 = 27 bigram repeats that were created by the cipher's cycle step and not masked. Total count of symbols 19 and 20 = 36. With 36 symbols, and masking bigram repeats with only one symbol per repeat, we can mask 72 repeats. 72 + 27 = 99 bigrams to start with. Isn't that about what we should have without the mystery step?

So those are interesting numbers. And I am also thinking about randomization of cycles and how that can increase or decrease the number of bigram repeats. I did just a little bit of experimentation, and it seemed like randomization will lower the number of repeats. But not every time. When I make the Tolkien message, I actually randomized several times until I had a high number of repeats to work with. So that is something that can be studied further also.

Daikon, I read your post. I have privately tried to expand many of the suspected wildcard symbols and used combinations of them. I have a huge spreadsheet of messages with that happening. It didn't work for me, but I wasn't as patient as you, and am not as good at solving these messages. Someday I would like to do that and then merge suspected cycle symbols to decrease multiplicity. If you make a cyclic message and randomly plop... I think about 40 but can't remember exactly... wildcards so that ZKD cannot solve. And then you expand those wildcards and merge the known cycle symbols, the program will solve.

EDIT: I am going to have to find ways to study the cycle behavior or each individual symbol in a different way so that I can compare them in different ways. I think that it may be interesting the compare the cycle activities of the symbols in the 340 bigram repeat list, and make the same comparison with practice messages. Later.

[Mr lowe]

Not sure if this has been performed yet.. Can you take out the top two, three or four top symbol counts and replace them with a space. Making sure the space is not a part of the code! Maybe show the space as an underscore. I know it's reducing the size of the code but I'm wondering if it may give a partial break that could lead you to what he used as a format for a substitution system. I'm in NY and along way from a real computer so I can't run that myself for a few weeks. The Reasoning behind this is that the human brain only needs partial words to read. It's ea-y to re-d a s-nte-ce ev-n if -t is mis--ng bi-s and piec-s. Wlel not so Esay but you get the piutcre.

[smokie treats]

I tried to take out the alleged wildcards, and it didn't solve. I think that replacing with a space is similar to what daikon was talking about. Your are welcome to try. I just have a little laptop with a spreadsheet and ZKD. And intermittent OCD.

[daikon]

Tried to post earlier, but my internet is super flaky today, so I couldn't until now. I've now tested the following sets of wildcards:
{'+','B','p'}
{'+','B'}
{'+','p'}
{'B','p'}
{'+','B','W'}
{'+','W'}
{'+','B','F'}
{'+','F'}
No obvious solutions. The top scores were also nowhere near the expected level for a good solve.
I believe this exhausts all possible combinations of the top 3 symbols: '+', 'B' and 'p'. The next 4: '|', 'c', 'F' and 'O', all repeat 10 times, but the number of combinations of 2-3 symbols out of a set of 7 (42+210) is kind of large, considering it takes several hours to get a reliable indication that it doesn't solve to anything. I guess I'll keep going through all of them, since I don't have any other better ideas to test at the moment. I think Smokie Treats likes 'F' as another possible candidate for a wildcard, so I'll prioritize it.

[Soze]

OCD was obvious yesterday when you said you were going to bed but struck at it again. Sucks doesn't it. To get something on your mind and can't shake it cause you think if I just tried this or maybe if I did that. Keep up the work though. You will figure it out.

Soze

[daikon]

Not sure if this has been performed yet.. Can you take out the top two, three or four top symbol counts and replace them with a space. Making sure the space is not a part of the code! Maybe show the space as an underscore. I know it's reducing the size of the code but I'm wondering if it may give a partial break that could lead you to what he used as a format for a substitution system. I'm in NY and along way from a real computer so I can't run that myself for a few weeks. The Reasoning behind this is that the human brain only needs partial words to read. It's ea-y to re-d a s-nte-ce ev-n if -t is mis--ng bi-s and piec-s. Wlel not so Esay but you get the piutcre.

Yep, that's exactly the idea of "wildcards" we are working on in this message thread. Or was it another thread? I lost track. :)

[smokie treats]

OCD was obvious yesterday when you said you were going to bed but struck at it again. Sucks doesn't it. To get something on your mind and can't shake it cause you think if I just tried this or maybe if I did that. Keep up the work though. You will figure it out.

Funny, I started joining Zodiac sites about 3 years ago so that I could have something else Zodiac-like to do without working on the 340, which doesn't take as much time and mental energy. That lasts only for a while. When the Ross Sullivan thread on this site started showing no immediate prospects for new information, I drifted over to this thread. And here I am. Somebody find out some new info about Ross Sullivan, please.

So, with the suspected wildcards being so prominent on the bigram repeat list. They could be 1:1 substitutes, but if they are, then it would seem to me that they would also have to map to high frequency letters so that they would appear on the bigram repeat list. But the high count could have something to do with that also. So I will look at the other messages in the suite and draw some comparisons to see of the high count 1:1's appear prominently on the respective bigram repeat lists.

Thanks for doing all of this work daikon. I understand that you are expanding the suspected wildcards to multiple symbols, correct? After you are done we will have to check to see if Haze or Tolkien have lower multiplicities; I don't know off of the top of my head. There is a certain point where the solvers won't solve with expanded wildcards. I played around with this in the previous thread. See: viewtopic.php?f=81&t=267&start=130.

[daikon]

Thanks for doing all of this work daikon. I understand that you are expanding the suspected wildcards to multiple symbols, correct?

Yep.

After you are done we will have to check to see if Haze or Tolkien have lower multiplicities; I don't know off of the top of my head.

For Tolkien one, I expanded to 115 symbols (top 3 symbols are wildcards) and got a pretty good solve. It's actually solvable if you only expand to 102 symbols (top 2). For Purple Haze, which is a much harder cipher even without wildcards, I expanded to 97 symbols. It ended up being fewer than for Tolkien because 2 out of 3 wildcards are not the most frequent symbols in the cipher. Expanding 2 wildcards wouldn't solve, it only worked with 3, unlike with Tolkien cipher. So my solver can work with multiplicities up to about 3 characters per unique symbol (i.e. length of cipher divided by number of unique symbols, which is not the classic definition of multiplicity, but I find it easier to work with and remember "5" or "4" instead of "0.185" or "0.238"). But it takes a few hours and many-many restarts to get the correct solution, instead of seconds for lower multiplicities. Expanding the top 3 symbols in Z340 gives you 107 total symbols, and even less with other symbols, so if it's anything like Tolkien/Purple Haze, it should be solvable. We just need to hit the 3 wildcards exactly. Assuming Z used wildcards of course. :)