AnsweredAssumed Answered

error with sqoop incremental import with merge-key

Question asked by ronaldm on Apr 28, 2016
Latest reply on Jul 25, 2017 by maprcommunity

Running a sqoop job:

sqoop job --create testjob -- import --connect jdbc:mysql://localhost/testdb --username username --password password --table member_info -m 2 --check-column updated_ts --incremental lastmodified --target-dir /tmp/test --fields-terminated-by ',' --merge-key id


During the merge phase (I believe), encountered the following problem:

16/04/14 18:33:40 INFO mapreduce.Job: Task Id : attempt_1460535083590_0043_m_000002_0, Status : FAILED

Error: File maprfs:///user/mapr/_sqoop/be8ef6242d5d40458eaf3a8afad4b660_member_info/part-m-00000 is not under new path maprfs:/user/mapr/_sqoop/be8ef6242d5d40458eaf3a8afad4b660_member_info or old path maprfs:///tmp/test

  at org.apache.sqoop.mapreduce.MergeMapperBase.setup(

  at org.apache.sqoop.mapreduce.MergeTextMapper.setup(


  at org.apache.hadoop.mapred.MapTask.runNewMapper(


  at org.apache.hadoop.mapred.YarnChild$

  at Method)



  at org.apache.hadoop.mapred.YarnChild.main(



Sqoop merge uses a couple parameters (--new-data, --onto) that seem related to the above error.

However, sqoop import does not provide any parameters related to this.


Any ideas where I am doing this wrong?