Search code examples
c#unity-game-enginehololens

How to capture HoloLens picture to programm variable


I'm currently working on a ML project with the HoloLens 1 and aiming to process a video taken by the front camera. Therefore I need to access the camera data from my c# Unity project. I assume there must be a way to store this in some data type(Tensor/Array/...).

I found several ways to store videos or pictures directly to the disc but none for directly using the data in my program. There might be a possibility to firstly save the video to the disc and afterwards access the saved data from my program, but that seems quite computing-intensive.

Do you have any suggestions how to make the data of the images available in the program?


Solution

  • You can definitely save the picture frame to memory, you can start with PhotoCapture class to capture the picture and stored as a variable. Please refer to this link to learn more about how to use it: UnityEngine.Windows.WebCam.PhotoCapture.

    And the key point is when you call the async method TakePhotoAsync(onCapturedPhotoToMemoryCallback), you need to pass in a function as a parameter, it will be invoked once the photo has been stored to memory. And in this function, you will get an PhotoCaputrerFrame instance as a parameter that contains the image captured from the camera.