Page 3 of 8

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Thu Jul 09, 2015 5:30 pm
by daikon
I'm glad to be of some small help with my suggestions. :) Using 6-grams should improve the solves even further, and 7-grams are not that impractical. If you keep each 7-gram score to 1 byte which should be plenty for a log score, you "only" need 8,031,810,176 bytes (26 to the power of 7) to keep the 7-gram stats in memory in an optimized for speed array, or less than 8Gb of RAM total. Not entirely out of the realm of possible for modern computers. How much improvement you get from 7-grams vs 6-grams is still remains to be seen, as it will greatly depend on the size of your corpus at that point.

Jarlve wrote:I don't want to go to much into program details but it's very similar to simulated annealing (performs about the same also) and it was something I came up with myself before I even learned of SA. In general my program has not much intelligence and relies on it's speed.


You don't actually need "much intelligence" to solve many problems. If you think about it, hill-climb algorithm is very dumb to begin with. I always thought that in the simplest terms it sounds just like how a 5-year old would approach solving a problem. :) You just nudge the solution a bit in a random direction and if it gets better, you keep it, otherwise you go back to the old solution and nudge some more. What could be simpler or dumber than that? And yet, it can solve a wide range of very complex optimization problems. It just takes a lot of nudging and a few clever improvements to speed up the whole process. So not much intelligence and a lot of speed is a *very* good thing.

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Thu Jul 09, 2015 6:00 pm
by daikon
Ok, here's my third, and hopefully last, attempt at unsolvable ciphers. Even though the previous cipher already doesn't get solved by any of the currently available auto-solvers, the next version of AZD that Jarlve is working on was able to crack it. Let's see if this cipher proves to be unsolvable:

Code: Select all
vKeO<]r4]GdgRq<aa@
tHZ_LWAo1MiXBI[2r3
+gCYNSDnE^T56P+JUF
_K+7Xf+bV8cGm9+ZoA
kde@a1q2+gBl3+CnQ5
r6+Rm7[Dp8+O]4<EYL
WFo9Mi^GIhNq@gO+hH
SZnJ+1P+KTA_d+2Xf+
bU3cBk5+CoDlHe64iY
EgF^LVGn7Mi_ZQm8RA
a<s9p@WSo1[P<BXNTC
n2OiYDIE^4UFoG_V35
ghJWZn6LiXAQ+KSBYd
+7^f+bT8cCk9+DoElH
eF_MUG@1+ghNhZnhJf
VRm2[AP<+hW<+]OQ]K
BX4SCoR3r5+IhdTDn[
6q7+g+L+8E9@+P+HUF
YJ+1^f+bV2cGk3+ZoA
lKeBm5+CnQ6r7+gDk8
+EohMq9Rl@[Z0dI4<]


I even managed to mimic Z340 somewhat, with an abundance of pluses and a lovely signature at the end, which is *not* a random filler. In theory it should be much easier to solve, as it is quite a bit longer than Z340, and nearly the length of Z408, if you trim the random filler at the end of Z408. Just as before, this is a straight homophonic substitution cipher. Every ciphertext symbol translates to a single unique plaintext letter. The message is in plain English, with the number of rare words kept to an absolute minimum - 5, each of which is a proper noun, and I allowed myself only one spelling mistake, to be consistent with Zodiac's writing style. The content of the message is entirely my own, but it would not be unexpected for Zodiac to write something similar. Just as before, I've also confirmed that the plaintext does get solved by ZKD and AZD if I lower the number of unique symbols in the ciphertext (i.e. if the multiplicity is improved).

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Fri Jul 10, 2015 1:22 pm
by daikon
So it seems I was able to stump all current auto-solvers after all? And only on the third try. I'm crackproof! :) I'll wait a couple of days before revealing the plaintext, just in case someone is still trying.

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 5:22 am
by Jarlve
Yes, I can't come up with a solve. Though your 3rd cipher scores much higher than the 340. I don't know what you did but well done!

Edit: I think you tried to induce high multiplicity by assigning some high counts to a few symbols and low counts to the bulk. It's something which I have thought of aswell in relation of the 340 and that we may need to come up with a better calculation of the cipher difficulty then multiplicity.

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 6:01 am
by glurk
I've worked on this a bit as well, and have no solve. But it's a weird cipher. Daikon, in your first post here, you said:

And I'm not talking about carefully crafted plaintext that doesn't have certain common letters (like passages from the novel "Gadsby"), or that has an abundance of rare letters, or a very high, or very low IoC.


So, in this one, the + symbol must be either E, T, A, O, I, or N. Assuming you didn't change the frequencies too much. And based on the patterns that ZKD finds, there are a fair number of repeats. Very odd.

You got me, though. Good job. You might as well post the plaintext, I think me and Jarlve are the only people here that work on these things. :)

-glurk

EDIT: Just to add, this is honestly the strangest cipher I've ever seen. If it turns out to be fairly normal English text, I bow down to you.

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 10:46 am
by Jarlve
Maybe the repeats/patterns can be explained by the plaintext being very repetitive. The same thing repeating over and over again, or use of a very limited vocabulary.

Edit: that would also artificially raise the multiplicity of the cipher, I believe. Yes, I think this is what daikon's guilty of! :)

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 1:20 pm
by doranchak
Here's a row with a spike in columnar repetitions:

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 1:55 pm
by doranchak
I did some experiments and I believe Jarlve is correct. There is a lot of repetition in the plain text.

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 2:09 pm
by doranchak
Daikon, don't post the solution yet, I think we all need the challenge to be extended a little. :)

Re: Not all homophonic substitutions can be auto-solved

PostPosted: Sat Jul 11, 2015 2:58 pm
by doranchak
OK, I cracked it (daikon, please confirm the solution I pm'd you), but it required many manual steps. Hint: Reduce the multiplicity before trying to crack this.

It will be interesting to determine how to get an auto-solver to uncover this kind of plaintext.