ARCore Visualizing video, image and 3D object

Question

So far I have my ARCore project display videos on top of images pretty consistently, but for more in depth understanding I wanted to be able to blend in 3D objects and still images as well. So say for a poster with three scanned image points, I would like to have one playing a video, other display a cube and third display an image. Kind of multiple visualizers displaying different objects. My problem is I can't seem to integrate the simple cube and I feel that's either because:

-My visualizer object is a Quad and that dabbles with the placement of the object

-Not calling my cube prefab correctly, considering it is added to the visualizer prefab

-Also I can't seem to call an image instead of a video, and that's my lack of knowledge over here.

My AugmentedImageVisualizer seems pretty catered to video playback:

public class AugmentedImageVisualizer : MonoBehaviour

[SerializeField] private VideoClip[] _videoClips;

public AugmentedImage Image;
public GameObject[] Models;
private VideoPlayer _videoPlayer;

void Start()
{
    _videoPlayer = GetComponent<VideoPlayer>();
    _videoPlayer.loopPointReached += OnStop;
}

private void OnStop(VideoPlayer source)
{
  gameObject.SetActive(false);
}

void Update()
{
    if (Image == null || Image.TrackingState != TrackingState.Tracking)
    {
      Models[Image.DatabaseIndex].SetActive(false);

      return;
    }

    if (!_videoPlayer.isPlaying)
    {
      _videoPlayer.clip = _videoClips[Image.DatabaseIndex];
      _videoPlayer.Play();

      Models[Image.DatabaseIndex].SetActive(true);
    }

    transform.localScale = new Vector3(Image.ExtentX, Image.ExtentZ, 1);
}

The AugmentedImageController is pretty straight forward, yet I have not edited anything to accomodate the addition of a 3D object:

public class AugmentedImageController : MonoBehaviour

[SerializeField] private AugmentedImageVisualizer _augmentedImageVisualizer;

private readonly Dictionary <int, AugmentedImageVisualizer> _visualizers =
          new Dictionary <int, AugmentedImageVisualizer>();

private readonly List<AugmentedImage> _images =
          new List<AugmentedImage>();

private void Update()
{
  if (Session.Status != SessionStatus.Tracking)
  {
    return;
  }

  Session.GetTrackables(_images, TrackableQueryFilter.Updated);
  VisualizeTrackables();
}

private void VisualizeTrackables()
{
  foreach (var image in _images)
  {
    var visualizer = GetVisualizer(image);

    if (image.TrackingState == TrackingState.Tracking && visualizer == null)
    {
      AddVisualizer(image);
    }
    else if (image.TrackingState == TrackingState.Stopped && visualizer != null)
    {
      RemoveVisualizer(image, visualizer);
    }
  }
}

private void AddVisualizer(AugmentedImage image)
{
  var anchor = image.CreateAnchor(image.CenterPose);
  var visualizer = Instantiate(_augmentedImageVisualizer, anchor.transform);
  visualizer.Image = image;
  _visualizers.Add(image.DatabaseIndex, visualizer);
}

private void RemoveVisualizer(AugmentedImage image, AugmentedImageVisualizer visualizer)
{
  _visualizers.Remove(image.DatabaseIndex);
  Destroy(visualizer.gameObject);
}

private AugmentedImageVisualizer GetVisualizer(AugmentedImage image)
{
  AugmentedImageVisualizer visualizer;
  _visualizers.TryGetValue(image.DatabaseIndex, out visualizer);
  return visualizer;
}

This is my visualizer object, which might cause the restrictions for adding varied media, but I can't seem to make sense of how to modify it.

Wondering if I could add multiple visualizers that I can add new objects, videos and images to. (like the way videos already are in the project) Goal is to allow adding extra files to display on visualizer

I was thinking of a way to make the Visualizer script on an empty and call a quad for videos, an empty with added 3D objects and another quad for images. But it feels like that is straying away from what I have so far.

Not sure what path would work, keep to the one AugmentedImageVisualizer or is there a way to create multiple Visualizers for displaying different objects. How to make it more modular, in order to display other media but still keep the functionality of adding and removing videos, objects and images and retain the order of anchoring

Answer 1

I assume that you are setting prefabs from your assets into the AugmentedImageVisualizer class, have you tried to Instantiate them?

The default ARCore augmented doesn't assign prefabs from assets, the script is in the prefab with the 3D object.

Also try to move the object at the center of the image, because you may be casting the objects far away from your view.

Sorry for my english.

Example:

Put this in the start void for each item in models.

GameObject.Instantiate(Models[Image.DatabaseIndex], Image.CenterPose.position, new Quaternion());