Search code examples
androidmediapipe

How to use mediapipe graphs like the mediapipe API-s


I'm struggling to understand how am I meant to interact with the "graph" and "calculator" stuff from the mediapipe library. More specifically, I'd like to write some Android code that uses landmarks from the holistic (pose + hands in my case) solution, with the final goal of writing a flutter application that compiles both for Android and iOS.

I've managed to build a few of the sample apps (thanks to docker), and I think I roughly understand what the graphs do. However, I don't understand how to interact with them from within the code. The Hello World! for Android tutorial doesn't really explain this. There are examples that include this type of behaviour (e.g. here), but I don't really know where is all the required information coming from (e.g. how would I find out the right functions and string constants to get holistic landmarks?)

For example, in Python I could get data via something like holistic.process(image).pose_landmarks, and then compute e.g. position or angle of hips. And as far as I can see there are some similar Android API-s available, although not for all solutions - including holistic. So what if I don't want to wait for the API-s development, and want to use the graphs instead? That part is not so clear.

As a bonus, please do feel free to drop any links which further explain/document the "graph" and "calculator" stuff I've mentioned earlier, as I've not used anything like this before and find it a little difficult to google.

Thanks.


Solution

  • I've realised there are docs available: https://google.github.io/mediapipe/framework_concepts/framework_concepts.html

    Not sure how I've missed this...