How To Use a Patched Hadoop with HBase 0.89
The new HBase 0.89 dev release uses Maven for its build process. When patching hadoop, copying the jar to the lib directory used to be enough. For Maven you must modify the pom.xml to tell it to use the local hadoop core jar file.
The correct way to do it is to follow
The correct way to do it is to follow
with it pointing to my hadoop jar as directed by the above link was not working. Maven was spitting out
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.
Missing:
----------
1) org.apache.hadoop:hadoop-core:jar:0.20.2
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=org.apache.hadoop -DartifactId=hadoop-core -Dversion=0.20.2 -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=org.apache.hadoop -DartifactId=hadoop-core -Dversion=0.20.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) org.apache.hbase:hbase:jar:0.89.20100924
2) org.apache.hadoop:hadoop-core:jar:0.20.2
----------
1 required artifact is missing.
for artifact:
org.apache.hbase:hbase:jar:0.89.20100924
from the specified remote repositories:
central (http://repo1.maven.org/maven2),
codehaus (http://repository.codehaus.org/),
temp-hadoop (http://people.apache.org/~rawson/repo/),
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Thu Nov 18 10:45:00 UTC 2010
[INFO] Final Memory: 34M/205M
[INFO] ------------------------------------------------------------------------
I tried using the first recommended command, but that failed.
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).
Project ID: com.agilejava.docbkx:docbkx-maven-plugin
POM Location: Artifact [com.agilejava.docbkx:docbkx-maven-plugin:pom:2.0.10]
Validation Messages:
[0] 'dependencies.dependency.version' is missing for com.agilejava.docbkx:docbkx-maven-base:jar
Reason: Failed to validate POM for project com.agilejava.docbkx:docbkx-maven-plugin at Artifact [com.agilejava.docbkx:docbkx-maven-plugin:pom:2.0.10]
Instead, I used the original pom.xml file and replaced the jar file in the repository.
cp ${APPSCALE_HOME}/AppDB/hadoop/hadoop-${HADOOP_VER}/hadoop-${HADOOP_VER}-core.jar ~.m2/repository/org/apache/hadoop/hadoop-core/0.20.3-append-r964955-1240/hadoop-core-0.20.3-append-r964955-1240.jar
That's a hack, but it works. Now HBase picks up my version of the hadoop jar. I'll try doing it the right way some other time.
Furthermore, there was an incompatibility with the newer version of HBase because column families were returning ":" appended to each one. Stripping off that last character was the last step to upgrading to the newest HBase version for AppScale.
