|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.jhuapl.edu.sages.etl.strategy.ETLStrategyTemplate
public abstract class ETLStrategyTemplate
Design Pattern: Strategy
ETLStrategyTemplate
defines the various methods that the SAGES ETL process will need to fulfill. The
actual implementation can vary depending on the target database
(i.e. Postgresql supports Savepoints, MSAccess does not; i.e. Variations in SQL syntax)
Field Summary | |
---|---|
protected static Set<String> |
ignorableErrorCodes
|
protected SagesOpenCsvJar |
m_socj
|
protected SqlStateHandler |
m_sqlStateHandler
|
Constructor Summary | |
---|---|
ETLStrategyTemplate()
|
Method Summary | |
---|---|
String |
addFlagColumn(String tableToModify)
Adds ETL_FLAG column to a table * |
void |
alterCleanseTableAddFlagColumn(Connection c,
Savepoint save1,
Savepoint createCleanseSavepoint)
Adds ETL_FLAG column to cleanse table |
void |
alterStagingTableAddFlagColumn(Connection c,
Savepoint save1,
Savepoint createCleanseSavepoint)
Adds ETL_FLAG column to existing staging table |
Savepoint |
buildCleanseTable(Connection c,
SagesOpenCsvJar socj,
Savepoint save1)
build ETL_CLEANSE_TABLE SQL: "CREATE TABLE..." - all columns have text sql-datatype - column definitions built from csv file header |
Savepoint |
buildEtlStatusTable(Connection c,
SagesOpenCsvJar socj,
Savepoint save1)
build ETL_STATUS_TABLE ********************************** SQL: "CREATE TABLE..." |
String[] |
determineHeaderColumns(File file)
determining the headers from CSV file |
int |
errorCleanup(SagesOpenCsvJar socj,
Savepoint savepoint,
Connection connection,
File currentCsv,
String failedDirPath,
Exception e)
|
void |
extractHeaderColumns(SagesOpenCsvJar socj)
extract headers from CSV file |
void |
generateSourceDestMappings(SagesOpenCsvJar socj)
determining source-to-destination column mappings |
void |
truncateCleanseAndStagingTables(DumbTestOpenCsvJar socj_dumb,
Connection c,
File file,
Savepoint baseLine)
truncate cleansing & staging tables |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected SagesOpenCsvJar m_socj
protected SqlStateHandler m_sqlStateHandler
protected static Set<String> ignorableErrorCodes
Constructor Detail |
---|
public ETLStrategyTemplate()
Method Detail |
---|
public void extractHeaderColumns(SagesOpenCsvJar socj) throws FileNotFoundException, IOException
FileNotFoundException
IOException
public String[] determineHeaderColumns(File file) throws FileNotFoundException, IOException
file
-
FileNotFoundException
IOException
public String addFlagColumn(String tableToModify)
tableToModify
- name of table to modify
public void truncateCleanseAndStagingTables(DumbTestOpenCsvJar socj_dumb, Connection c, File file, Savepoint baseLine) throws SagesEtlException, SQLException
socj_dumb
- DumbTestOpenCsvJar
c
- Connection
file
- csv file being processedbaseLine
- Savepoint
savepoint to rollback to if error occurs
SagesEtlException
SQLException
public Savepoint buildEtlStatusTable(Connection c, SagesOpenCsvJar socj, Savepoint save1) throws SagesEtlException, SQLException
SagesEtlException
SQLException
public Savepoint buildCleanseTable(Connection c, SagesOpenCsvJar socj, Savepoint save1) throws SagesEtlException, SQLException
SagesEtlException
SQLException
public void alterCleanseTableAddFlagColumn(Connection c, Savepoint save1, Savepoint createCleanseSavepoint) throws SQLException, SagesEtlException
SQLException
SagesEtlException
public void alterStagingTableAddFlagColumn(Connection c, Savepoint save1, Savepoint createCleanseSavepoint) throws SQLException, SagesEtlException
c
- - Connection
save1
- - Savepoint
createCleanseSavepoint
- - Savepoint
SQLException
SagesEtlException
public void generateSourceDestMappings(SagesOpenCsvJar socj)
socj
- SagesOpenCsvJar
public int errorCleanup(SagesOpenCsvJar socj, Savepoint savepoint, Connection connection, File currentCsv, String failedDirPath, Exception e)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |