Search code examples
c#macosmonothriftaccumulo

Connecting from C# to Accumulo


I am new to working with Accumulo. I need to read/write data from a remote Accumulo through C#. The only code sample/documentation for C#, I have found is - Accumulo createBatchScanner range not working as expected

I attempted to compile the code in Xamarin Studio, on a Mac.
The issue I am encountering is with this line:

AccumuloProxy.Client client = new AccumuloProxy.Client(protocol);

Error CS0246: The type or namespace name AccumuloProxy' could not be found. Are you missingorg.apache.accumulo.proxy.thrift' using directive? (CS0246) (AccumuloIntegratorPrototype)

Where can I find the DLLs to add to my CSharp project related to AccumuloProxy client? Is there a way I can generate the same?

Here is a code fragment:

namespace AccumuloIntegratorPrototype
{
    class MainClass
    {
        static byte[] GetBytes(string str)
        {
            return Encoding.ASCII.GetBytes(str);
        }

        static string GetString(byte[] bytes)
        {
            return Encoding.ASCII.GetString(bytes);
        }

        public static void Main (string[] args)
        {
            try
            {
                /** connect **/
                TTransport transport = new TSocket("xxx.xx.x.xx", 42424);
                transport = new TFramedTransport(transport);
                TCompactProtocol protocol = new TCompactProtocol(transport);
                transport.Open();

                AccumuloProxy.Client client = new AccumuloProxy.Client(protocol);

Solution

  • Thanks to all for the pointers.
    Was able to complete my project.
    These are my notes.

    A. Versions:
    Accumulo 1.5
    Thrift 0.90
    Mono 3.2.5

    B. Strategy/option used to connect to Accumulo from C#:
    Accumulo Proxy API

    C. Accumulo Proxy with C# bindings:
    Performed the following actions on a node running Accumulo
    1. Installed Mono 3.2.5
    2. Installed Thrift 0.90
    3. Configured Accumulo proxy service
    Modified the file $ACCUMULO_HOME/proxy/proxy.properties;
    Specifically updated the instance name, and zookeeper
    4. Started the proxy daemon-

    ${ACCUMULO_HOME}/bin/accumulo proxy -p ${ACCUMULO_HOME}/proxy/proxy.properties
    

    5.Generated the c# bindings, using the proxy.thrift IDL file

    thrift --gen csharp $ACCUMULO_HOME/proxy/thrift/proxy.thrift
    

    This resulted in the creation of a directory called gen-csharp under ${ACCUMULO_HOME}/proxy/thrift/
    6. The files under gen-csharp are needed in the C# project, in section D, below.
    7. Thrift.dll, is also needed.

    D. C# project - Accumulo Client:
    1. Created a project of type library.
    2. Added the files under gen-csharp in step C5, above to the library
    3. Added reference to thrift.dll
    4. Built the library

    E. Connecting to Accumulo from C#
    In the C# project that reads/writes to Accumulo,
    1. Added reference - thrift.dll
    2. Added reference to the library built in section D, above
    3. On the Accumulo server, started the proxy (refer step C4, above)

    Here is some sample code, to read data, to try this functionality out..

    using System;
    using System.Text;
    using System.Collections.Generic;
    
    using Thrift.Protocol;
    using Thrift.Transport;
    
    
    namespace AccumuloIntegratorPrototype
    {
    class MainClass
    {
        static byte[] GetBytes(string str)
        {
            return Encoding.ASCII.GetBytes(str);
        }
    
    
        static string GetString(byte[] bytes)
        {
            return Encoding.ASCII.GetString(bytes);
        }
    
        public static void Main (string[] args)
        {
    
            try
            {
                String accumuloProxyServerIP = "xxx.xxx.x.xx";//IP
                int accumuloProxyServerPort = 42424;//Port Number
    
                TTransport transport = new TSocket(accumuloProxyServerIP, accumuloProxyServerPort);
                transport = new TFramedTransport(transport);
                TCompactProtocol protocol = new TCompactProtocol(transport);
                transport.Open();
    
                String principal = "root";//Application ID
                Dictionary<string, string> passwd = new Dictionary<string,string>();
                passwd.Add("password", "xxxxx");//Password
    
                AccumuloProxy.Client client = new AccumuloProxy.Client(protocol);
                byte[] loginToken = client.login(principal, passwd);//Login token
    
    
                //{{
                //Read a range of rows from Accumulo
                var bScanner = new BatchScanOptions();
                Range range = new Range();
    
                range.Start = new Key();
                range.Start.Row = GetBytes("d001");
    
                //Need the \0 only if you need to get a single row back
                //Otherwise, its not needed
                range.Stop = new Key();
                range.Stop.Row = GetBytes("d001\0");
    
                bScanner.Ranges = new List<Range>();
                bScanner.Ranges.Add(range);
    
                String scanId = client.createBatchScanner(loginToken, "departments", bScanner);
    
    
                var more = true;
                while (more)
                {
                    var scan = client.nextK(scanId, 10);
                    more = scan.More;
    
                    foreach (var entry in scan.Results)
                    {
                        Console.WriteLine("Row = " + GetString(entry.Key.Row));
                        Console.WriteLine("{0} {1}:{2} [{3}]  {4}    {5}", GetString(entry.Key.Row), GetString(entry.Key.ColFamily), GetString(entry.Key.ColQualifier), GetString(entry.Key.ColVisibility), GetString(entry.Value),(long)entry.Key.Timestamp);
                    }
                }
    
                client.closeScanner(scanId);
    
                client.Dispose();
                transport.Close();
    
                }catch (Exception e)
                {
                    Console.WriteLine(e);
                }
            //}}
    
    
    
        }
    }
    }