配置Apache Hop经Kerberos访问HDFS遇浏览失败问题求助
问题:Hop中无法通过文件输入组件访问Kerberos认证的远程HDFS
我需要创建文件转换管道,从Kerberos认证的远程HDFS下载输入文件并将输出文件存储回该HDFS。本地已完成以下配置:
- 安装Kerberos并获取密钥;
- 设置环境变量(JAVA_HOME、HADOOP_CONF_DIR和HADOOP_OPTS);
- 安装Hadoop客户端,通过CLI命令
hdfs dfs -ls hdfs://clusterhadoop/user测试远程HDFS读写成功; - 在
hop/config/hop-config.json中添加环境变量; - 将Hadoop依赖包
hadoop-common-3.1.1.jar和hadoop-hdfs-3.1.1.jar放入hop/plugins/tech/下新建的hdfs/lib目录; - 在插件资源管理器中添加HDFS,主类设为
org.apache.hadoop.fs.hdfs.HdfsFileSystem并指定上述jar路径。
但配置完成后,使用CSV File Input或TextFile Input组件查看HDFS内容时失败,报错如下:
Error browsing to location: 'hdfs://clusterhadoop/user/toto.csv'
FileNotFolderException: Could not list the contents of "file:///home/ubuntu/hop/hdfs:/clusterhadoop/user" because it is not a folder.
Root cause: FileNotFolderException: Could not list the contents of "file:///home/ubuntu/hop/hdfs:/clusterhadoop/user" because it is not a folder.
org.apache.commons.vfs2.FileNotFolderException: Could not list the contents of "file:///home/ubuntu/hop/hdfs:/clusterhadoop/user" because it is not a folder. at org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1107) at org.apache.hop.ui.core.vfs.HopVfsFileDialog.populateFolder(HopVfsFileDialog.java:941) at org.apache.hop.ui.core.vfs.HopVfsFileDialog.refreshBrowser(HopVfsFileDialog.java:898) at org.apache.hop.ui.core.vfs.HopVfsFileDialog.refreshAll(HopVfsFileDialog.java:1278) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.apache.hop.ui.core.gui.BaseGuiWidgets.lambda$getListener$1(BaseGuiWidgets.java:213) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91) at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5857) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1617) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:5067) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4519) at org.apache.hop.ui.core.vfs.HopVfsFileDialog.open(HopVfsFileDialog.java:584) at org.apache.hop.ui.core.dialog.BaseDialog.presentFileDialog(BaseDialog.java:278) at org.apache.hop.ui.core.dialog.BaseDialog.presentFileDialog(BaseDialog.java:172) at org.apache.hop.pipeline.transforms.fileinput.text.TextFileInputDialog.lambda$open$8(TextFileInputDialog.java:484) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91) at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5857) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1617) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:5067) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4519) at org.apache.hop.ui.core.dialog.BaseDialog.defaultShellHandling(BaseDialog.java:528) at org.apache.hop.pipeline.transforms.fileinput.text.TextFileInputDialog.open(TextFileInputDialog.java:501) at org.apache.hop.ui.hopgui.file.pipeline.delegates.HopGuiPipelineTransformDelegate.editTransform(HopGuiPipelineTransformDelegate.java:195) at org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.editTransform(HopGuiPipelineGraph.java:3368) at org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.editTransform(HopGuiPipelineGraph.java:2210) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:97) at org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:182) at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40) at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132) at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:5042) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4522) at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:478) at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:356)
恳请帮忙解决该问题,感谢!
内容的提问来源于stack exchange,提问作者user30133690
相关产品推荐
相关产品推荐

