hadoop運行wordcount卡住了,Windows7系統運行hadoop報Failed to locate the winutils binary in

 2023-10-15 阅读 22 评论 0

摘要:程序運行的過程中,報Failed to locate the winutils binary in the hadoop binary path ?Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries. 通過斷點調試、查看源碼發現程序需要根據HADOOP_HOME找到winutils.exe,由于win機

程序運行的過程中,報Failed to locate the winutils binary in the hadoop binary path ?Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries.

通過斷點調試、查看源碼發現程序需要根據HADOOP_HOME找到winutils.exe,由于win機器并沒有配置該環境變量,所以程序報 null\bin\winutils.exe。

hadoop運行wordcount卡住了??

private static String checkHadoopHome() {// first check the Dflag hadoop.home.dir with JVM scopeString home = System.getProperty("hadoop.home.dir");// fall back to the system/user-global env variableif (home == null) {home = System.getenv("HADOOP_HOME");}try {// couldn't find either setting for hadoop's home directoryif (home == null) {throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");}if (home.startsWith("\"") && home.endsWith("\"")) {home = home.substring(1, home.length()-1);}// check that the home setting is actually a directory that existsFile homedir = new File(home);if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {thrownew IOException("Hadoop home directory " + homedir+ " does not exist, is not a directory, or is not an absolute path.");}home = homedir.getCanonicalPath();} catch (IOException ioe) {if (LOG.isDebugEnabled()) {LOG.debug("Failed to detect a valid hadoop home directory", ioe);}home = null;}    return home;}private static String HADOOP_HOME_DIR = checkHadoopHome();public static final String getQualifiedBinPath(String executable) throws IOException {// construct hadoop bin path to the specified executableString fullExeName = HADOOP_HOME_DIR + File.separator + "bin" + File.separator + executable;File exeFile = new File(fullExeName);if (!exeFile.exists()) {throw new IOException("Could not locate executable " + fullExeName+ " in the Hadoop binaries.");}return exeFile.getCanonicalPath();}/** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath();public static final String getWinUtilsPath() {String winUtilsPath = null;try {if (WINDOWS) {winUtilsPath = getQualifiedBinPath("winutils.exe");}} catch (IOException ioe) {LOG.error("Failed to locate the winutils binary in the hadoop binary path",ioe);}return winUtilsPath;}

找到原因后就去網上問了度娘,找到了解決方案,很簡單,如下:

  1.下載winutils的windows版本

locate、  GitHub上,有人提供了winutils的windows的版本,項目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下載此項目的zip包,下載后是文件名是hadoop-common-2.2.0-bin-master.zip,隨便解壓到一個目錄

  2.配置環境變量

  增加用戶變量HADOOP_HOME,值是下載的zip包解壓的目錄,然后在系統變量path里增加$HADOOP_HOME\bin 即可。  

hadoop啟動不了start、  再次運行程序,正常執行。

轉載于:https://www.cnblogs.com/itboys/p/6675153.html

版权声明:本站所有资料均为网友推荐收集整理而来,仅供学习和研究交流使用。

原文链接:https://hbdhgg.com/3/138953.html

发表评论:

本站为非赢利网站,部分文章来源或改编自互联网及其他公众平台,主要目的在于分享信息,版权归原作者所有,内容仅供读者参考,如有侵权请联系我们删除!

Copyright © 2022 匯編語言學習筆記 Inc. 保留所有权利。

底部版权信息