As we have seen in most of the Data Warehousing projects, we try to run the reference data loads in parallel and increase the number of threads so that we can maximize the resources allocated to the Database Server. With Kalido also we can run the parallel loads but we need to keep a small thing in mind that we cannot run the parallel loads on the same object in Kalido Warehouse. Meaning; if we have a GL Account CBE and this CBE is being seeded from two source Systems SRC1 and SRC 2, we cannot run the loads from the 2 source systems at the same time. For sure these can run back to back but not in parallel with each other. And the reason for this is Kalido locks the object and only one load can update that object(in our example GL Account CBE). So if we try running another load while one load is in progress,the second load will error out. Consider yourself lucky, as Kalido generates a log file which can be decrypted by a human being with an error message "Another Load is in Progress".
So if we have feeds from various source systems loading data into the CBE, they should be executed sequentially. A dependency has to be build saying that when one load is complete, trigger the next load on the same object. For sure we can execute the different feeds for different objects and in that way parallelism can be achieved but not on the same object. We could increase the number of threads while loading the data, but not greater than 4(loads start degrading after 4)
So the thought for the day is "Never Ever run multiple data feeds on the same object in Kalido at the same time"
As usual Enjoy working with this great tool....
Nilesh Makhija
No comments:
Post a Comment