Sqoop : Oracle: Connection Reset Errors

Document created by sumesh_kurup on Feb 8, 2016
Version 1Show Document
  • View in full screen mode

Author: Sumesh Kurup

 

Original Publication Date: November 10, 2014

 

Problem:

When using the default Sqoop connector for Oracle, some data does get transferred, but during the map-reduce job a lot of errors are reported as below:

 

java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLRecoverableException: IO Error: Connection reset

 

  at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)

 

  at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)

 

  at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)

 

  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:665)

 

  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:351)

 

  at org.apache.hadoop.mapred.Child$4.run(Child.java:282)

 

  at java.security.AccessController.doPrivileged(Native Method)

 

  at javax.security.auth.Subject.doAs(Subject.java:415)

 

  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1122)

 

  at org.apache.hadoop.mapred.Child.main(Child.java:271)

 

Caused by: java.lang.RuntimeException: java.sql.SQLRecoverableException:

 

  IO Error: Connection reset at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:193)

 

  at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162)

 

... 9 more

 

Caused by: java.sql.SQLRecoverableException: IO Error: Connection reset

 

  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:498)

 

  at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)

 

  at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)

 

  at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)

 

  at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)

 

  at java.sql.DriverManager.getConnection(DriverManager.java:571)

 

  at java.sql.DriverManager.getConnection(DriverManager.java:215)

 

  at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:294)

 

  at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:187)

 

... 10 more

 

Caused by: java.net.SocketException: Connection reset

 

  at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)

 

  at java.net.SocketOutputStream.write(SocketOutputStream.java:159)

 

  at oracle.net.ns.DataPacket.send(DataPacket.java:210)

 

  at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:230)

 

  at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:321)

 

  at oracle.net.ns.NetInputStream.read(NetInputStream.java:268)

 

  at oracle.net.ns.NetInputStream.read(NetInputStream.java:190)

 

  at oracle.net.ns.NetInputStream.read(NetInputStream.java:107)

 

  at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)

 

  at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)

 

  at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)

 

  at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:350)

 

  at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)

 

  at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:407)

 

  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:416)

 

... 18 more

Solution

This problem occurs primarily due to the lack of a fast random number generation device on the host where the map tasks execute.
On typical Linux systems this can be addressed by setting the following property in the java.security file:

 

securerandom.source=file:/dev/../dev/urandom


The java.security file can be found under $JAVA_HOME/jre/lib/security directory. Alternatively, this property can also be specified on the command line via:

-D mapred.child.java.opts="-Djava.security.egd=file:/dev/../dev/urandom"


Please note that it’s very important to specify the path as /dev/../dev/urandom due to a Java bug 6202721.  Otherwise /dev/urandom will be ignored and substituted by /dev/random.

Attachments

    Outcomes