Search code examples
javaandroidimagemobilearcore

ARCore: How to display saved images from device on a surface


I am a newbie in AR. I was wondering if there is a way where you can display picture that you saved on a phone to a surface using ARCore for android. Like I was able to use the Sceneform example with a precoded image but now I would like to show a photo from my picture folder to a surface. I read about texture but I am not sure how that works either. Thank you for your tips.

EDIT: so I create a class where the user can select an image from there I am saving through Uri

public void onActivityResult(int requestCode, int resultCode, Intent data) {

    super.onActivityResult(requestCode, resultCode, data);
    if (resultCode == this.RESULT_CANCELED) {
        return;
    }
    if (requestCode == GALLERY) {
        if (data != null) {
            contentURI = data.getData();
            try {
                Bitmap bitmap = MediaStore.Images.Media.getBitmap(this.getContentResolver(), contentURI);
                String path = saveImage(bitmap);
                Toast.makeText(imageSelector.this, "Image Saved!", Toast.LENGTH_SHORT).show();

                //set the image selected to bitmap so that imageView can display it
                imageview.setImageBitmap(bitmap);

                Intent intent = new Intent(this, HelloSceneformActivity.class);
                intent.putExtra("imageUri", contentURI.toString());
                startActivity(intent);

And from the ARclass side I called the intent and parse it as follow

//start the intent for the image chosen from the library
    Bundle extras = getIntent().getExtras();
    Uri myUri = Uri.parse(extras.getString("imageUri"));

    // When you build a Renderable, Sceneform loads its resources in the background while returning
    // a CompletableFuture. Call thenAccept(), handle(), or check isDone() before calling get().
    ModelRenderable.builder()
            .setSource(this, myUri)
            .build()
            .thenAccept(renderable -> andyRenderable = renderable)
            .exceptionally(
                    throwable -> {
                        Toast toast =
                                Toast.makeText(this, "Unable to load andy renderable", Toast.LENGTH_LONG);
                        toast.setGravity(Gravity.CENTER, 0, 0);
                        toast.show();
                        return null;
                    });

I am able to select the picture but program crashes and I get this error

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.google.ar.sceneform.samples.hellosceneform/com.google.ar.sceneform.samples.hellosceneform.HelloSceneformActivity}: java.lang.IllegalArgumentException: Unable to parse url: 'content://com.google.android.apps.photos.contentprovider/-1/1/content%3A%2F%2Fmedia%2Fexternal%2Fimages%2Fmedia%2F460/ORIGINAL/NONE/1926270799'

Caused by: java.lang.IllegalArgumentException: Unable to parse url: 'content://com.google.android.apps.photos.contentprovider/-1/1/content%3A%2F%2Fmedia%2Fexternal%2Fimages%2Fmedia%2F460/ORIGINAL/NONE/1926270799'
        at com.google.ar.sceneform.utilities.LoadHelper.remoteUriToInputStreamCreator(Unknown Source:56)
        at com.google.ar.sceneform.utilities.LoadHelper.fromUri(Unknown Source:40)
        at com.google.ar.sceneform.rendering.Renderable$Builder.build(Renderable.java:343)
        at com.google.ar.sceneform.rendering.ModelRenderable$Builder.build(ModelRenderable.java:44)
        at com.google.ar.sceneform.samples.hellosceneform.HelloSceneformActivity.onCreate(HelloSceneformActivity.java:92)
        at android.app.Activity.performCreate(Activity.java:7174)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1220)

Solution

  • According to https://github.com/google-ar/sceneform-android-sdk/issues/79 you can't load from custom URIs at the moment:

    in 1.0 we interpret all file:// uris as being relative to the assets folder which is why we fail to load from external storage. Consequently, in order to load a Texture from a Uri, you'll need to either put in in the assets folder (i.e. bundled into your apk), or host it online where it's available via an http:// scheme.

    This implies only http:// and file:// URIs are supported at the moment and the latter can only be relative to the assets folder. That is confirmed in malik-at-work's answer on https://github.com/google-ar/sceneform-android-sdk/issues/64:

    The URI based file loading isn't looking at external storage. Right now it's only looking for application assets using file:// and remote uris starting with http://

    EDIT: A workaround I'd suggest is using the ViewRenderable in combination with a layout with a single ImageView inside. The ViewRenderable can create 2D-Layouts in the Scene, you just have to set your image into the ImageView like you would in a usual Android-App and position / rotate the resulting object to your liking. You can find an example-usage of the ViewRenderable in the solarsystem-Sceneform-example, the sliders for the speeds are realized with it.

    EDIT 2: It seems that this is fixed in Sceneform 1.4 since the first mentioned issue has now been closed.