Access uint8_t vector in Java via SWIG

Question

I'm working on a Java <-> C++ bridge using SWIG; my c++ method is as follows:

std::vector<uint8_t> ArduGrabClass::grab() {
    IMAGE_FORMAT fmt = { IMAGE_ENCODING_I420, 100 };
    BUFFER *buffer = arducam_capture(camera_instance, &fmt, 352000);
    return vector<uint8_t>(buffer->data, buffer->data + buffer->length);
}

That is cut down, the real method holds onto the buffer object in the c++ class so that the java side can release it. However I need to access this data on the Java side, preferably in a ByteBuffer otherwise in an array. However I don't know how to do it; the JNI method NewDirectByteBufer seems suitable but SWIG is handling all the JNI stuff so I don't know how to tell SWIG to use the NewDirectByteBufer method.

Answer 1

It's totally feasible to write a typemap to call NewDirectByteBuffer, but as written that's definitely not the right way to wrap your code. The problem as written is two fold:

1. You've returned a vector by value, but that means its memory is going to expire as soon as it goes out of scope, leaving the ByteBuffer referring to memory that's no longer valid
2. If you did extend the lifespan of the vector to avoid that issue there's currently nothing that would free that memory when the ByteBuffer get garbage collected so you'd leak memory
3. The call to arducam_capture itself needs to be paired with a call to arducam_release_buffer somehow - you could add that into your ArduGrabClass::grab() function, but see below
4. Even if you did all this you'd still be doing quite a few copies of the data, which somewhat defeats the point of using a ByteBuffer in Java. (There's at least going to be one copy in the constructor for the vector, maybe more depending on how you did solve it.

So if you want to use the std::vector copy, I'd recommend just using the existing SWIG support for std::vector, or maybe copying it into a new array in some JNI code. I can demonstrate how to do that if you are interested, but for now I've focused on the question of how to use ByteBuffer correctly with arducam_capture.

To have a Java method that returns a ByteBuffer and gets cleaned up afterwards we want to arrange to use a Cleaner that spots when the ByteBuffer goes out of scope.

If we change our wrapped function to return the BUFFER * and not a copy of it in a std::vector we can use some SWIG support in the internals of the generated conversion to call an extra function we add into our wrapper that calls NewDirectByteBuffer for us. And then, inside that code we can also register the ByteBuffer with a Runnable that will call arducam_release_buffer for us.

An example of a SWIG interface that does this for us is:

%module test

%pragma(java) modulecode=%{
   // Create a single cleaner thread for all our buffers to register with
   // note package level access is deliberate
   static final java.lang.ref.Cleaner cc = java.lang.ref.Cleaner.create();
   static {
     // actually load our shared object!
     System.loadLibrary("test");
   }
%}

// later on when we implement toBuffer() we need the environment pointer.
// This adds it into our method call automatically
%typemap(in,numinputs=0) JNIEnv * %{
  $1 = jenv;
%}

// Our native implementation of toBuffer on BUFFER is going to return
// a ByteBuffer straight up for us, with no need for conversions
%typemap(jtype) jobject toBuffer "java.nio.ByteBuffer"
%typemap(jstype) jobject toBuffer "java.nio.ByteBuffer"

// When we hand the ByteBuffer off to a caller we need to register
// something to do the clean up for us. This is where we register it.
%typemap(javaout) jobject toBuffer {
  java.nio.ByteBuffer buf = $jnicall; // actually call the native code
  System.out.println("In toBuffer"); // To prove it worked!

  // Our cleaner instance lives in the module itself
  $module.cc.register(buf, new Runnable(){
    public void run() {
      System.out.println("in buffer cleanup java side");
      // We add a (private) cleanup function we can just call here
      BUFFER.this.cleanup();
    }
  });

  // Now it's registered actually let them have it
  return buf;
}

// Every time we return a BUFFER * call toBuffer() on it instead
%typemap(javaout) BUFFER * {
  return new $javaclassname($jnicall, $owner).toBuffer();
}

// Since we're going to call toBuffer() the return type is different
%typemap(jstype) BUFFER * "java.nio.ByteBuffer"

// Hide lots of thing about the BUFFER class outside our package
%typemap(javaclassmodifiers) BUFFER "class"
%javamethodmodifiers BUFFER::cleanup() "private";
%javamethodmodifiers BUFFER::toBuffer "";



// BEGIN Faked definitions, just for testing...
%{
typedef struct {
  void *data;
  int len;
} BUFFER;

typedef struct {
  enum { IMAGE_ENCODING_I420 } a;
  int n;
} IMAGE_FORMAT;

BUFFER * arducam_capture(void * instance, IMAGE_FORMAT *fmt, int timeout) {
  (void)instance; (void)fmt; (void)timeout;
  BUFFER *buf = malloc(sizeof *buf);
  *buf = (BUFFER){
    .data = malloc(100),
    .len = 100,
  };
  return buf;
}

void arducam_release_buffer(BUFFER *instance) {
  free(instance->data);
  free(instance);
}
%}
// END TESTING

// All the details of BUFFER are not public for Java users
%nodefaultctor BUFFER;
%nodefaultdtor BUFFER;
typedef struct {} BUFFER;

// In addition to what's really in the buffer object we want to add another
// two methods. 
%extend BUFFER {
        // toBuffer() is used by our internals when returning a BUFFER
        jobject toBuffer(JNIEnv *jenv) const {
                // Swig provides JCALLx macros for us, but they are not usable inside %extend :(
%#ifdef __cplusplus__
                return jenv->NewDirectByteBuffer($self->data, $self->len);
%#else
                return (*jenv)->NewDirectByteBuffer(jenv, $self->data, $self->len);
%#endif
        }

        // Cleanup is used when the buffer is dead
        void cleanup() {
                arducam_release_buffer($self);
        }
}


%inline %{
// Now, wrap a modified version of your code
BUFFER *do_capture() {
    void *camera_instance = NULL;
    IMAGE_FORMAT fmt = { IMAGE_ENCODING_I420, 100 };
    BUFFER *buffer = arducam_capture(camera_instance, &fmt, 352000);
    return buffer;
}

%}

And then to try it out:

public class run {
        public static void main(String[] argv) {
                for (int i = 0; i < 100; ++i) {
                        java.nio.ByteBuffer buf = test.do_capture();
                        System.gc();
                }
        }
}

Which we can then build and run:

swig3.0 -java -Wall test.i
gcc -Wall -Wextra -o libtest.so -I/usr/lib/jvm/default-java/include/ -I/usr/lib/jvm/default-java/include/linux test_wrap.c -shared 
javac *.java  -Xlint:deprecation 
LD_LIBRARY_PATH=. java run
In toBuffer
in buffer cleanup java side
......