How to add a database source to Kylo

Question

I’d like to create a new Data Ingest type feed in Kylo, to import data from an external database into Hive. On the second step of the feed wizard there’s a drop-down for Source Database Connection but my database isn’t listed. How do I add my database to this list?

Answer 1

Kylo retrieves the list of database connections from Apache NiFi by looking for controller services of type DBCPConnectionPool. It uses the Database Connection URL, the Database Driver Class Name, and the Database User properties of the controller service to create its own connection to the database. The password comes from Kylo’s application.properties file.

Here’s how to add a new database to the Source Database Connection list in Kylo:

1. Configure a new controller service under the root process group in Apache NiFi to connect to your database.
2. Copy the JDBC driver jar file for your database to Kylo’s plugin folder. This is usually at /opt/kylo/kylo-services/plugin/.
3. Add the password for the database to Kylo’s application.properties file. This is usually in /opt/kylo/kylo-services/conf/.
   • The name of the property should follow the format: nifi.service.<controller-service-name>.password
   • Where <controller-service-name> is the name of the controller service in all lowercase and spaces are substituted with underscores.
   • For example, the property for the MySQL controller service from Kylo’s sandbox is: nifi.service.mysql.password
4. Restart the kylo-services service and the new Source Database Connection should be available: service kylo-services restart