Truncated output from CFB mode when calling from C#

Question

I have pretty annoying issue which I'm unable to solve for 2 days. I have an encrypt() method which makes use of Crypto++ library written in C++. The method is implemented as follows:

string CRijndaelHelper::Encrypt(string text)
{
    CFB_Mode< AES >::Encryption e;
    e.SetKeyWithIV(_key, sizeof(_key), _iv);

    string cipher, encoded;

    // CFB mode must not use padding. Specifying
    //  a scheme will result in an exception
    StringSource ss(text, true,
        new StreamTransformationFilter(e,
            new StringSink(cipher)
        ));     

    return cipher;
};

When I call this method within the native environment it works perfectly, encrypting the whole 26Kb xml string without any problem.

Eventually I've had to implement the encryption in C# code, for that purpose I wrote the following wrapper code in native dll for later use with PInvoke:

extern "C"
API_EXPORT char* _cdecl encrypt(char* contents)
{   
    string cont(contents);
    CRijndaelHelper rij;
    string transformedText = rij.Encrypt(cont);     
    return marshalReturn(transformedText);  
}

While the PInvoke part looks as follows:

[DllImport("EncryptDecrypt.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.Cdecl)]
[return: MarshalAs(UnmanagedType.LPStr)]
public static extern string encrypt([MarshalAs(UnmanagedType.LPStr)] string contents);

And everything looks working perfect except that I get in transformedText variable only first 540 bytes encrypted and that's all.

Please advise.

Answer 1

... returns a string which is correctly encrypted when invoked from native code

The problem is not you C++ encrypt, which uses a std::string. The problem is with marshalling it back to managed code as a char*.

Change CRijndaelHelper::Encrypt to the following to remove the embedded NULL that will be sprinkled liberally with a probability of 1/255:

#include <cryptopp/base64.h>
using CryptoPP::Base64Encoder;
...

string CRijndaelHelper::Encrypt(string text)
{
    CFB_Mode< AES >::Encryption e;
    e.SetKeyWithIV(_key, sizeof(_key), _iv);

    string cipher, encoded;

    // CFB mode must not use padding. Specifying
    //  a scheme will result in an exception
    StringSource ss(text, true,
        new StreamTransformationFilter(e,
            new Base64Encoder(
                new StringSink(cipher)
        )));     

    return cipher;
};

With the Base64 encoding, the marshaling as a char* will not truncate the encryption results on the first NULL byte encountered.

Then, for decryption:

StringSource ss(cipher, true,
    new Base64Decoder(
        new StreamTransformationFilter(d,
            new StringSink(text)
    )));