AnsweredAssumed Answered

Error while using Sqoop Import

Question asked by girisht on Jun 16, 2016
Latest reply on Mar 2, 2017 by Karthee

Objective : Load Hive data(orc fmt) from Maprcluster1 to M7(MapR-DB/Hbase) in another cluster Maprcluster2.

Plan:

1. Using Sqoop import - connect to Hive using JDBC in Maprcluster1

2. directly import in to Mapr-DB/Hbase in Maprcluster2.

 

While trying to connect Hive through JDBC using Sqoop, I got below exception.

 

Note: I tried using transport mode = http/https - but no luck.

 

can some one pls shed some light on it?

 

 

sqoop import \

> -Dmapred.job.queue.name=qname \

> --connect 'jdbc:hive2://hostname:10000/db_name;auth=noSasl' \

> --driver org.apache.hive.jdbc.HiveDriver \

> --table test \

> --username username -P

 

 

cat: /opt/mapr/zookeeper/zookeeperversion: No such file or directory

Warning: /opt/mapr/sqoop/sqoop-1.4.6/bin/../../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: /opt/mapr/zookeeper/zookeeper- does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

16/06/16 11:38:18 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-mapr-1601

16/06/16 11:38:19 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

16/06/16 11:38:19 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.

16/06/16 11:38:19 INFO manager.SqlManager: Using default fetchSize of 1000

16/06/16 11:38:19 INFO tool.CodeGenTool: Beginning code generation

16/06/16 11:38:19 INFO jdbc.Utils: Supplied authorities: hostname:10000

16/06/16 11:38:19 INFO jdbc.Utils: Resolved authority: hostname:10000

16/06/16 11:38:19 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://hostname:10000/db_name;auth=noSasl

16/06/16 11:38:19 INFO jdbc.HiveConnection: Could not open client transport with JDBC Uri: jdbc:hive2://hostname:10000/db_name;auth=noSasl

16/06/16 11:38:19 INFO jdbc.HiveConnection: Transport Used for JDBC connection: null

16/06/16 11:38:19 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://hostname:10000/db_name;auth=noSasl: java.net.ConnectException: Connection refused

java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://hostname:10000/db_name;auth=noSasl: java.net.ConnectException: Connection refused

        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:232)

        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:177)

        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)

        at java.sql.DriverManager.getConnection(DriverManager.java:664)

        at java.sql.DriverManager.getConnection(DriverManager.java:247)

        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885)

        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)

        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:744)

        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)

        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)

        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)

        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)

        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)

        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)

        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)

        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)

        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)

        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:606)

        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)

        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused

        at org.apache.thrift.transport.TSocket.open(TSocket.java:187)

        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:205)

        ... 23 more

Caused by: java.net.ConnectException: Connection refused

        at java.net.PlainSocketImpl.socketConnect(Native Method)

        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

        at java.net.Socket.connect(Socket.java:589)

        at org.apache.thrift.transport.TSocket.open(TSocket.java:182)

        ... 24 more

16/06/16 11:38:19 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter

        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)

        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)

        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)

        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:606)

        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)

        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Outcomes