Incremental Loads
I am getting data from seven or eight joined tables. Each table has a "LastChangDate" field but I'm not sure if it would be better to check each table for a change date - which would only tell me the table has changed, not the one or two columns I'm using - or process the entire ETL again after loading new rows and look for measures that have changed in the OLTP.
I suppose I could do a monster WHERE clause that looked at the LastChangeDate in every table. That would filter many records I guess. It's not like I'm expecting anyone to actually answer a question on this site. It's a pity. I'd have more respect for the authors/publishers and might even be inclined to purchase additional books if they ever bothered to check their forum.
|