[reporting utils]$ date --date='-1 month'
Sun Dec 31 07:08:07 PST 2017
[reporting utils]$ date --date='1 month'
Sat Mar 3 07:08:15 PST 2018
[reporting utils]$ date
Wed Jan 31 07:10:07 PST 2018
Wednesday, January 31, 2018
Tuesday, January 30, 2018
Cloudera Impala query fails on Java UDFs - PermGen Space
UDF WARNING: Hive UDF path=hdfs://nameservice1/libs/esri-gis.jar
class=com.esri.hadoop.hive.ST_GeomFromText failed due to:
ImpalaRuntimeException: UDF::evaluate() ran into a problem.
CAUSED BY:
ImpalaRuntimeException: UDF failed to evaluate
CAUSED BY:
InvocationTargetException: null
CAUSED BY:
OutOfMemoryError: PermGen space
UDF WARNING: Hive UDF path=hdfs://nameservice1/libs/esri-gis.jar
class=com.esri.hadoop.hive.ST_GeodesicLengthWGS84 failed due to:
ImpalaRuntimeException: UDF::evaluate() ran into a problem.
CAUSED BY:
ImpalaRuntimeException: UDF failed to evaluate
CAUSED BY:
InvocationTargetException: null
CAUSED BY:
NumberFormatException: For input string: "?"
Couldn't open transport for reshdp-xxx:22000
(connect() failed: Connection refused)
Couldn't open transport for
reshdp-xxx:22000 (connect() failed: Connection refused)
Couldn't open transport for
reshdp-xxx:22000 (connect() failed: Connection refused)
Couldn't open transport for
reshdp-xxx:22000 (connect() failed: Connection refused)
Couldn't open transport for
reshdp-xxx:22000 (connect() failed: Connection refused)
UDF WARNING: Hive UDF path=hdfs://nameservice1/libs/esri-gis.jar
class=com.esri.hadoop.hive.ST_GeomFromText failed due to:
ImpalaRuntimeException: UDF::evaluate() ran into a problem.
CAUSED BY: ImpalaRuntimeException: UDF failed to evaluate
CAUSED BY: InvocationTargetException: null
CAUSED BY: OutOfMemoryError: PermGen space
Started getting this error after kerberizing my cluster. I suspect that Java was loading kerberos libraries and running out of space for the UDFs when it came time to fix.
Solution: Add memory to the JAVA_TOOL_OPTION for impala:
Daemon Safety Valve Config
JAVA_TOOL_OPTIONS=-XX:MaxPermSize=512m
Restart impala. Should fix it.
Subscribe to:
Posts (Atom)