Search code examples
cassandraignitecassandra-3.0sqlline

Integrating Apache Cassandra with Apache Ignite

I'm trying to integrate Apache Ignite with Apache Cassandra(3.11.2) as I want to use Ignite to cache the data present in my already existing Cassandra database.

After going through the online resources, I've done the following till now:

  1. Downloaded Apache Ignite.
  2. Copied all the folders present in "libs/optional/" to "libs/"(I don't know which ones will be required for Cassandra).
  3. Created 3 xmls in the config folder i.e. "cassandra-config.xml", "connection-settings.xml" and "persistance-settings.xml". Currently I'm using the same node(172.16.129.68) for both Cassandra and Ignite.

cassandra-config.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- Cassandra connection settings -->
<import resource="connection-settings.xml" />

<!-- Persistence settings for 'cache1' -->
<bean id="cache1_persistence_settings" class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
        <constructor-arg type="org.springframework.core.io.Resource" value="file:/home/cass/apache_ignite/apache-ignite-fabric-2.4.0-bin/config/persistance-settings.xml" />
</bean>
<!-- Ignite configuration -->
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="cacheConfiguration">
        <list>
            <!-- Configuring persistence for "cache1" cache -->
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="cache1"/>
                <property name="readThrough" value="true"/>
                <property name="writeThrough" value="true"/>
                <property name="writeBehindEnabled" value="true"/>
                <property name="cacheStoreFactory">
                    <bean class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
                        <property name="dataSourceBean" value="cassandraAdminDataSource"/>
                        <property name="persistenceSettingsBean" value="cache1_persistence_settings"/>
                    </bean>
                </property>
            </bean>
        </list>
    </property>
    <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <!--
                    Ignite provides several options for automatic discovery that can be used
                    instead os static IP based discovery. For information on all options refer
                    to our documentation: http://apacheignite.readme.io/docs/cluster-config
                -->
                <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
                <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                    <property name="addresses">
                        <list>
                            <!-- In distributed environment, replace with actual host IP address. -->
                            <value>172.16.129.68:47500..47509</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>

connection-settings.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

<bean id="loadBalancingPolicy" class="com.datastax.driver.core.policies.TokenAwarePolicy">
    <constructor-arg type="com.datastax.driver.core.policies.LoadBalancingPolicy">
        <bean class="com.datastax.driver.core.policies.RoundRobinPolicy"/>
    </constructor-arg>
</bean>

<bean id="cassandraAdminDataSource" class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
    <property name="port" value="9042"/>
    <property name="contactPoints" value="172.16.129.68"/>
    <property name="readConsistency" value="ONE"/>
    <property name="writeConsistency" value="ONE"/>
    <property name="loadBalancingPolicy" ref="loadBalancingPolicy"/>
</bean>

persistance-settings.xml

<persistence keyspace="test" table="epc_table">
    <keyPersistence class="java.lang.String" strategy="PRIMITIVE" column="imsi"/>
    <valuePersistence strategy="BLOB"/>
</persistence>

  1. I run the following command to start Ignite from bin folder.

    ignite.sh ../config/cassandra-config.xml

Now, I want to take a look at the cassandra table via sqlline. I've tried the following:

./sqlline.sh -u jdbc:cassandra://172.16.129.68:9042/test //(test is the name of the keyspace)

I get the following output:

No known driver to handle "jdbc:cassandra://172.16.129.68:9042/test". Searching for known drivers...
java.lang.NullPointerException
sqlline version 1.3.0
0: jdbc:cassandra://172.16.129.68:9042/test>

I've also tried:

./sqlline.sh -u jdbc:ignite:thin://172.16.129.68

but when I use "!tables", I'm not able to see any table.

What exactly has been missing? How to access/modify the tables present in Cassandra using sqlline?

Operating System: RHEL 6.5

Solution

  • Apache Ignite is a key-value database and there are no tables created by default that you are able to view with JDBC connector. CacheStore is a way to integrate Ignite with external DB or any other storage, and it loads data as a key-value pair.

    In your config you said Ignite that you want to store and load entries in/from Cassandra, but still Ignite doesn't know entries structure (BTW Ignite really doesn't care what objects were putted into it).

    To be able to list tables and do queries on it, you need to create tables. For that you need to have ignite-indexing in /lib directory and set QueryEntity or indexed types if you have annotated POJOs. Here is example of such configuration:

    <bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="mycache"/>
<!-- Configure query entities -->
<property name="queryEntities">
    <list>
        <bean class="org.apache.ignite.cache.QueryEntity">
            <property name="keyType" value="java.lang.Long"/>
            <property name="valueType" value="org.apache.ignite.examples.Person"/>

            <property name="fields">
                <map>
                    <entry key="id" value="java.lang.Long"/>
                    <entry key="orgId" value="java.lang.Long"/>
                    <entry key="firstName" value="java.lang.String"/>
                    <entry key="lastName" value="java.lang.String"/>
                    <entry key="resume" value="java.lang.String"/>
                    <entry key="salary" value="java.lang.Double"/>
                </map>
            </property>

            <property name="indexes">
                <list>
                    <bean class="org.apache.ignite.cache.QueryIndex">
                        <constructor-arg value="id"/>
                    </bean>
                    <bean class="org.apache.ignite.cache.QueryIndex">
                        <constructor-arg value="orgId"/>
                    </bean>
                    <bean class="org.apache.ignite.cache.QueryIndex">
                        <constructor-arg value="salary"/>
                    </bean>
                </list>
            </property>
        </bean>
    </list>
</property>

    If you configure that, you'll get an ability to enlist and query that tables over SQLine. (Please note, that you cannot query data that are not loaded into Ignite. To load them, you may use IgniteCache.get() with enabled readThrough option or IgniteCache.loadCache() to load everything from Cassandra table).

    To query Cassandra with JDBC, you need a JDBC driver for it, try, for example DBSchema.