Rijndael 128/256 decryption

I tried many answers from SO and all over the web but still without success.

I use following tool to encrypt.

text to encrypt: tom
key: exo123exo1exo123
input (textfield or selected file above) is: text/binary
Convert output to: [i leave this unselected]
Mode: CTR
Ciphers: Rijndael-128 and Rijndael-256

After getting result I move here:

and encode it with base64.

Then I copy string and send it as a parameter to my function:

public String authenticate(String base64EncodedData){

    byte[] input = Base64.decodeBase64(base64EncodedData);
    byte[] ivBytes = "1234567812345678".getBytes();


    Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");

    cipher.init(
        Cipher.DECRYPT_MODE,
        new SecretKeySpec("exo123exo1exo123".getBytes(), "AES"),
        new IvParameterSpec(ivBytes)
    );

    byte[] plainText = new byte[cipher.getOutputSize(input.length)];
    int plainTextLength = cipher.update(input, 0, input.length, plainText, 0);
    plainTextLength += cipher.doFinal(plainText, plainTextLength);

    return new String(plainText);  
  }

Result I got is always something similar to this (no matter if I use Rijndael-128 or 256 encrypted string):

.�v�Y�

When I try to return input value - I get the encrypted string. So base64 works just fine. What I do wrong? Am slowly getting mad here. Thank you.

Solution

There are some problem with your assumptions and your code:

The output of the first tool is already Base 64 encoded. RIJNDAEL-128: r0GR and RIJNDAEL-256: yAVy. It doesn't need to be encoded a second time. It automatically selects this option, because binary data cannot be printed.
There is no native Rijndael-256 in Java, you will have to use BouncyCastle for this. Rijndael-128 is supposed to be AES which means that both have a block size of 128-bit.
The IV almost certainly needs to consist of zero bytes. For example:
```
byte[] ivBytes = new byte[16];
Arrays.fill(ivBytes, (byte)0); // might not be necessary
```
Note that CTR mode doesn't use an IV, but rather a nonce.
Always specify an encoding when retrieving bytes: "exo123exo1exo123".getBytes("UTF-8"). It is probably best to use UTF-8 everywhere. If you send data across systems that use different system encodings, it will lead to hard to find problems.

On second look at the online tool is not usable for anything, because it is unclear how it works. What I found:

Any key produces a ciphertext regardless of size, which suggests that the "key" that you enter is actually hashed and there exist a million ways that could be done. I could decrypt it with the above method, so the key is actually uses as-is without hashing. This suggests that the key is filled up with 0x00 bytes until a valid key size. When the key size is too big, it is probably truncated.
Plaintext of 16 characters encrypts to a ciphertext of 16 bytes (encoded 24) and a plaintext of 17 characters encrypts to a ciphertext of 32 bytes (encoded 44). This means that no discernible padding is used which might actually be zero padding.

And here is the deal breaker:

Every time you encrypt something, you get a different ciphertext. In CBC mode it means a random IV is generated before encrypting. The problem is that this IV is not shown. So there is no way to decrypt this fully. So if you encrypt more than 16 characters you can recover all but the first 16 characters when you decrypt it in the way I describe it in the first part of this answer.

You shouldn't make your system dependent on closed source online tools. Write your own encryption tool. Some general advice:

IV should be generated randomly every time you encrypt something. It is not secrect, but should be unique. Simply prepend it to the ciphertext after encryption and slice it off before decryption.
Use an authenticated mode like CCM or GCM where possible. It is harder to do authenticated encryption yourself, but the other way to go would be to use encrypt-then-mac paradigm.