Search code examples
mongodbhadoopsqoop

Moving HDFS data into MongoDB

I am trying to move HDFS data into MongoDB. I know how to export data into mysql by using sqoop. I dont think I can use sqoop for MongoDb. I need help understanding how to do that.

Solution

  • The basic problem is that mongo stores its data in BSON format (binary JSON), while you hdfs data may have different formats (txt, sequence, avro). The easiest thing to do would be to use pig to load your results using this driver:

    https://github.com/mongodb/mongo-hadoop/tree/master/pig

    into mongo db. You'll have to map your values to your collection - there's a good example on the git hub page.