I try to convert a Procedure that join 8 tables with INNER AND OUTER JOIN, my understanding is that the Lookup task is the one to use and I should break these joins into smaller block, it takes a long time to load when I do this, since each of these tables had 10-40mill. rows and I have 8 tables to go thru, currently this Stored Procedure took 3-4min to run, by converting this to 8 Lookup tasks, it ran for 20min. has anyone run into this issue before and know a work around for this.
Thanks
Lookup task run very slow when hitting big tables
Vladimir Sapronov
Dear tm9t9,
I haven’t tried this (with 10 to 40 mil. rows) before, but according to my experience natural-join (just use “table1.row1 = table2.row1”) is easier and less complex;
Or you can just INDEX the tables; but this is also difficult for transaction/processing files/tables.
xRuntime