Search code examples
bigdataknime

Knime too slow - performance


I just started to use KNIME and it suppose managed a huge mount of data, but isn't, it's slow and often not response. I'll manage more data than that I'm using now, What am I doing wrong?. I set in my configuration file "knime.ini":

-XX:MaxPermSize=1024m
-Xmx2048m

I also read data from a database node (millions of rows) but I can't limit it by SQL (I don't really mind, I need this data).

SELECT * FROM foo LIMIT 1000

error:

WARN     Database Reader     com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'LIMIT 0' at line 1

Solution

  • see http://tech.knime.org/forum/knime-users/knime-performance-reading-from-a-database for the rest of this discussion and solutions...