org.jhuapl.edu.sages.etl.strategy
Class ETLStrategyTemplate

java.lang.Object
  extended by org.jhuapl.edu.sages.etl.strategy.ETLStrategyTemplate
Direct Known Subclasses:
ETLPostgresqlStrategy

public abstract class ETLStrategyTemplate
extends Object

Design Pattern: Strategy ETLStrategyTemplate defines the various methods that the SAGES ETL process will need to fulfill. The actual implementation can vary depending on the target database (i.e. Postgresql supports Savepoints, MSAccess does not; i.e. Variations in SQL syntax)

Author:
POKUAM1

Field Summary
protected static Set<String> ignorableErrorCodes
           
protected  SagesOpenCsvJar m_socj
           
protected  SqlStateHandler m_sqlStateHandler
           
 
Constructor Summary
ETLStrategyTemplate()
           
 
Method Summary
 String addFlagColumn(String tableToModify)
          Adds ETL_FLAG column to a table *
 void alterCleanseTableAddFlagColumn(Connection c, Savepoint save1, Savepoint createCleanseSavepoint)
          Adds ETL_FLAG column to cleanse table
 void alterStagingTableAddFlagColumn(Connection c, Savepoint save1, Savepoint createCleanseSavepoint)
          Adds ETL_FLAG column to existing staging table
 Savepoint buildCleanseTable(Connection c, SagesOpenCsvJar socj, Savepoint save1)
          build ETL_CLEANSE_TABLE SQL: "CREATE TABLE..." - all columns have text sql-datatype - column definitions built from csv file header
 Savepoint buildEtlStatusTable(Connection c, SagesOpenCsvJar socj, Savepoint save1)
          build ETL_STATUS_TABLE ********************************** SQL: "CREATE TABLE..."
 String[] determineHeaderColumns(File file)
          determining the headers from CSV file
 int errorCleanup(SagesOpenCsvJar socj, Savepoint savepoint, Connection connection, File currentCsv, String failedDirPath, Exception e)
           
 void extractHeaderColumns(SagesOpenCsvJar socj)
          extract headers from CSV file
 void generateSourceDestMappings(SagesOpenCsvJar socj)
          determining source-to-destination column mappings
 void truncateCleanseAndStagingTables(DumbTestOpenCsvJar socj_dumb, Connection c, File file, Savepoint baseLine)
          truncate cleansing & staging tables
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

m_socj

protected SagesOpenCsvJar m_socj

m_sqlStateHandler

protected SqlStateHandler m_sqlStateHandler

ignorableErrorCodes

protected static Set<String> ignorableErrorCodes
Constructor Detail

ETLStrategyTemplate

public ETLStrategyTemplate()
Method Detail

extractHeaderColumns

public void extractHeaderColumns(SagesOpenCsvJar socj)
                          throws FileNotFoundException,
                                 IOException
extract headers from CSV file

Throws:
FileNotFoundException
IOException

determineHeaderColumns

public String[] determineHeaderColumns(File file)
                                throws FileNotFoundException,
                                       IOException
determining the headers from CSV file

Parameters:
file -
Returns:
headers
Throws:
FileNotFoundException
IOException

addFlagColumn

public String addFlagColumn(String tableToModify)
Adds ETL_FLAG column to a table *

Parameters:
tableToModify - name of table to modify
Returns:
alter table statement

truncateCleanseAndStagingTables

public void truncateCleanseAndStagingTables(DumbTestOpenCsvJar socj_dumb,
                                            Connection c,
                                            File file,
                                            Savepoint baseLine)
                                     throws SagesEtlException,
                                            SQLException
truncate cleansing & staging tables

Parameters:
socj_dumb - DumbTestOpenCsvJar
c - Connection
file - csv file being processed
baseLine - Savepoint savepoint to rollback to if error occurs
Throws:
SagesEtlException
SQLException

buildEtlStatusTable

public Savepoint buildEtlStatusTable(Connection c,
                                     SagesOpenCsvJar socj,
                                     Savepoint save1)
                              throws SagesEtlException,
                                     SQLException
build ETL_STATUS_TABLE ********************************** SQL: "CREATE TABLE..."

Throws:
SagesEtlException
SQLException

buildCleanseTable

public Savepoint buildCleanseTable(Connection c,
                                   SagesOpenCsvJar socj,
                                   Savepoint save1)
                            throws SagesEtlException,
                                   SQLException
build ETL_CLEANSE_TABLE SQL: "CREATE TABLE..." - all columns have text sql-datatype - column definitions built from csv file header

Throws:
SagesEtlException
SQLException

alterCleanseTableAddFlagColumn

public void alterCleanseTableAddFlagColumn(Connection c,
                                           Savepoint save1,
                                           Savepoint createCleanseSavepoint)
                                    throws SQLException,
                                           SagesEtlException
Adds ETL_FLAG column to cleanse table

Throws:
SQLException
SagesEtlException

alterStagingTableAddFlagColumn

public void alterStagingTableAddFlagColumn(Connection c,
                                           Savepoint save1,
                                           Savepoint createCleanseSavepoint)
                                    throws SQLException,
                                           SagesEtlException
Adds ETL_FLAG column to existing staging table

Parameters:
c - - Connection
save1 - - Savepoint
createCleanseSavepoint - - Savepoint
Throws:
SQLException
SagesEtlException

generateSourceDestMappings

public void generateSourceDestMappings(SagesOpenCsvJar socj)
determining source-to-destination column mappings

Parameters:
socj - SagesOpenCsvJar

errorCleanup

public int errorCleanup(SagesOpenCsvJar socj,
                        Savepoint savepoint,
                        Connection connection,
                        File currentCsv,
                        String failedDirPath,
                        Exception e)


Copyright © 2013 Johns Hopkins University Applied Physics Laboratory. All rights reserved.