mysqldatabaseoracleapache-kafkapentaho
Improve pentaho table input step performance
I am using table input step to extract data from oracle database and load into Apache Kafka producer(output step) by doing some concatenation.It is taking nearly 6 min to extract 2 million rows(table containing 50 columns).By seeing its log i found table input step taking longer time.How can i improve the performance in table input step?
Solution
Also, the log may be misleading: if the ConcatField or Kafka fields are slow, then the Input table will be slow not by itself, but just because it is not able to send data to the next steps.
First check the suspected Input table is guilty: remove the ConcatFields and Kafka steps and replace them by a Dummy step. If it still takes 6 min, then it is pure Oracle DBA optimization.
- Join newest price from price-table for all products
- Mysql ERROR: ASCII '\0' while importing sql file on linux server
- User registration with CodeIgniter
- Paging Results: Handling invalid passed data
- I'm stuck with foreign key constraint fails error. .NET 8, EF Core 8.0.2
- find number of rows in returned mysql result (nodejs)
- Get records that have total count of over X amount
- Excluding records between dates AND with an associating value
- MYSQL Update query, replace part string containing URL
- MySQL Not Equal To Certain Length
- How can I solve "Error: MySQL shutdown unexpectedly"?
- How do I turn off the mysql password validation?
- How to create table like another table with additional columns in MYSQL
- How import also column name in CSV file
- Export Column Names to CSV
- split characters and numbers in MySQL
- If only CI fails, what kind of trouble could there be for django?
- Perform an INNER JOIN with more than 2 tables in MySQL
- Imploding a list for use in a Python MySQL IN clause
- SQL check if timestamp is greater than 24 hours from now (MYSQL)
- SQL - Query to find if a string contains part of the value in Column
- MySQL stored function: CONTAINS SQL or NO SQL?
- MySQL "delayed commit ok done" and performance
- MySQL Query performance - best partial match
- MySQL: How to reset or change the MySQL root password?
- How to reset the root password in MySQL 8.0.11?
- Count 'IN' value in MySQL from multiple non-normalized columns
- coupon code statistics
- How to return NULL when result is empty?
- Extract dirname from full path stored in database