A more-or-less customised version of Hadoop/HBase is used in where I work. One day, I really wanted to use HBase filters and began to code like everyone else:
(:import [org.apache.hadoop.hbase.util Bytes]
[org.apache.hadoop.hbase.client HTable]
[org.apache.hadoop.hbase.filter
RowFilter
CompareFilter$CompareOp
RegexStringComparator])
; ...
(defn daily-row-filter
[]
(RowFilter. (CompareFilter$CompareOp/EQUAL
(RegexStringComparator. "^[0-9a-fA-F-]{36}:[0-9]{8}$"))))
; ...
(hb/with-scanner [results (hb/scan ht :filter (daily-row-filter)
; ...
)])
The programme happily compiled on my laptop, but I always got a class not found in
CompareFilter$CompareOp/EQUAL
Frustrated, I checked my colleague's code in Java and found what he wrote was:
import org.apache.hadoop.hbase.filter.CompareOp;
// ...
new RowFilter(CompareOp.EQUAL, new RegexStringComparator(...));
WHAT? It turned out that our HBase deviated from the standard. Here's the solution.
In project.clj, add this not-so-standard hbase.jar:
:resource-paths [ "/usr/lib/hbase/hbase.jar" ]
This way, you can successfully
lein jar
. The downside is that you cannot compile it in a standard environment any more.
In core.clj, do this:
(:import
[org.apache.hadoop.hbase.filter
RowFilter
CompareOp
RegexStringComparator])
(defn daily-row-filter
[]
(RowFilter. CompareOp/EQUAL
(RegexStringComparator. "^[0-9a-fA-F-]{36}:[0-9]{8}$")))
(hb/with-scanner [results (hb/scan ht
:filter (daily-row-filter) ] ; ...
)
Hope I can help someone, or at least I'm helping myself of the future.
沒有留言:
張貼留言