Logging
PipelineWise is generating log files at ~/.pipelinewise/<TARGET_ID>/<TAP_ID>/log
from every run started by the run_tap
command. Please check Running Pipelines
section for further details how to run tap when you do a data sync.
Every log file will match the following pattern: <TARGET_ID>-<TAP_ID>-<DATE>_<TIME>.<SYNC_ENGINE>.log.<STATUS>
Variables:
TARGET_ID: Unique identifier of the target. This is the
id
property from a Targets (Destinations) YAML.TAP_ID: Unique identifier of the tap. This is the
id
property from a Taps (Data Sources) YAML.DATE: Date when the tap started in
YYYYMMDD
formatTIME: Time when the tap started in
HH24MMSS
formatSYNC_ENGINE: One of
singer
orfastsync
. Check Replication Methods and Fast Sync section for further details.STATUS: One of
running
,failed
orsuccess
Example:
$ ls -lah ~/.pipelinewise/snowflake/fx/log/
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 00:02 snowflake-fx-20190508_000038.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 00:31 snowflake-fx-20190508_003014.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 01:04 snowflake-fx-20190508_010031.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 01:33 snowflake-fx-20190508_013036.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 02:02 snowflake-fx-20190508_020037.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 02:32 snowflake-fx-20190508_023029.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 03:02 snowflake-fx-20190508_030032.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 03:31 snowflake-fx-20190508_033031.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 04:02 snowflake-fx-20190508_040033.fastsync.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 04:04 snowflake-fx-20190508_040222.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 04:30 snowflake-fx-20190508_043015.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 05:01 snowflake-fx-20190508_050035.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 05:31 snowflake-fx-20190508_053030.singer.log.failed
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 06:02 snowflake-fx-20190508_060037.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 13K May 8 06:33 snowflake-fx-20190508_063032.singer.log.success
-rw-rw-r-- 1 pipelinewise pipelinewise 8K May 8 07:03 snowflake-fx-20190508_070036.singer.log.running