mt7620n 玩玩看之一

從刀哥那搞來了一塊 mt7620n (他的blog) ，玩玩看。這塊板子原本的用途是 Wifi-HDD 。
和他一樣，在 Ubuntu 14.04.1 上採用 debootstrap 先裝個 chroot 環境，再開發，順便偷用他的 chroot.sh
用 debootstrap 或 VirtualBox 裝 32-bit VM 都可以。事實上 32-bit VM 只要 1GB 記憶體 2*CPU 就編得很順了。

S3:~/mtk7620$ sudo debootstrap  --arch=i386  wheezy mt7620_wheezy http://ftp.tw.debian.org/debian
S3:~/mtk7620$ wget https://github.com/hbtsai/devutils/raw/master/chroot.sh
S3:~/mtk7620$ chmod +x chroot.sh
S3:~/mtk7620$ ./chroot.sh mt7620_wheezy

進去後先把 /root/.bashrc改一下，加進 PS1='\u@mtk7620:\W\$ ' 再安裝必要的套件:

root@mtk7620:~# apt-get install vim build-essential git screen libncurses5-dev locales bison flex gawk zlib1g-dev liblzma-dev
root@mtk7620:~# cd /bin
root@mtk7620:~# ln -sf bash sh

把 locale 做好

修改 /etc/locale.gen 打開 en_US.UTF-8
root@mtk7620:source# apt-get install locales
root@mtk7620:source# locale-gen
export LC_ALL=en_US.UTF-8

嗯，對，我比較喜歡 dash ... 可是不改過不了 XD。
接著就用 SDK 裡面附的 QuickStart Guide 來處理。因為文件是 confidential 的關係，這邊只能講個密訣 XDDD

• 裝 32 位元的就好， 64bit gcc 不用裝
• 跳過 uboot 那段
• 直接進 source 做 make menuconfig 照手冊做
• 不能編譯 pcitools-3.0.0 的問題，在比較新的 git repo 裡的應該修好了

make dep
make
mkdir romfs

Delete all versions of a column with Clojure-HBase

To delete all versions of a given column, we call Delete.deleteColumns(family, quantifier) in Java. However, it is not yet implemented in Clojure-HBase. I have submitted an issue and a pull-request. Before the pull-request is accepted, you can delete a row in the right way like this:

(hb/with-table [ht (hb/table TABLE)]
  (io!
    (.delete ht (.deleteColumns (new Delete (tobyte ROWKEY))
                                (tobyte COLUMNFAMILY) (tobyte COLUMNQUANTIFIER)))))

DON'T DO

I did something stupid like calling:

(hb/with-table [ht (hb/table TABLE)]
  (hb/delete ROWKEY :with-timestamp-before
             3000000000   ; DON'T TO THIS
             [[:column CF CQ]]))

In fact, HBase inserts a DELETE mark with the timestamp, meaning that you can no longer put a new value before it is expired.

node.js + Express 3.x + session + cookie

It wasted me more than 6 hours to Google through the topic without reaching an answer. As I tried to write some examples in book "Node.js: Up and Running" in LiveScript, I bumped into this weird behaviour. Chrome v26 / Linux did not store my sessionID in the cookie, or more exactly, it stored but I couldn't retrieve it as I reload the page, resulting in that session is always regenerated. Interestingly, Firefox works properly, and so do Chrome v22/Linux and Chrome v29/Windows. I tried to write a small piece in both PHP and LiveScript. Cookie works. So the problem must be Express 3.x.

# For finding the root cause of 7-24 on Chrome 26

require! express

app = express!
app.use express.cookieParser!

app.get '/visit', (req, res) !->
        visit = parseInt req.cookies.visit or 0
        res.cookie 'visit', visit + 1
        res.send 'Visit #' + visit

app.get '/clear', (req, res) !->
        res.clearCookie 'visit'
        res.send 'Cookie cleared.'

app.listen 8001

I noticed that from Express 2.x to 3.x, the connect / anonymous function has been changed from (req, res) to (req, res, next). Nice bet, the answer is 42. Here's the code that works:

app.configure !->
        app.use express.cookieParser 'sign'
        app.use express.session {
                secret: 'secretKey'
                store: store
        }
        app.use (req, res, next) !->
                sess = req.session
                # console.log "store = "
                # console.log store
                # console.log "sess.email = " + sess.email
                res.render 'socket.jade', {email: sess.email || 'noSessMail'}
                console.log "SessionID in Express = " + req.sessionID
                next and next!   # NOTICE THIS LINE

I don't know how to use express.cookieSession(), though. Not many examples are there. You can find the Git repository to my practice here.

Not-so-standard HBase with Filters in Clojure

A more-or-less customised version of Hadoop/HBase is used in where I work. One day, I really wanted to use HBase filters and began to code like everyone else:

(:import [org.apache.hadoop.hbase.util Bytes]
         [org.apache.hadoop.hbase.client HTable]
         [org.apache.hadoop.hbase.filter
          RowFilter
          CompareFilter$CompareOp
          RegexStringComparator])
; ...
(defn daily-row-filter
  []
  (RowFilter. (CompareFilter$CompareOp/EQUAL
                (RegexStringComparator. "^[0-9a-fA-F-]{36}:[0-9]{8}$"))))
; ...
(hb/with-scanner [results (hb/scan ht :filter (daily-row-filter)
                            ; ...
)])

The programme happily compiled on my laptop, but I always got a class not found in CompareFilter$CompareOp/EQUAL Frustrated, I checked my colleague's code in Java and found what he wrote was:

import org.apache.hadoop.hbase.filter.CompareOp;
// ...
new RowFilter(CompareOp.EQUAL, new RegexStringComparator(...));

WHAT? It turned out that our HBase deviated from the standard. Here's the solution. In project.clj, add this not-so-standard hbase.jar:

            :resource-paths [ "/usr/lib/hbase/hbase.jar" ]

This way, you can successfully lein jar. The downside is that you cannot compile it in a standard environment any more. In core.clj, do this:

(:import
            [org.apache.hadoop.hbase.filter
            RowFilter
            CompareOp
            RegexStringComparator])

(defn daily-row-filter
  []
  (RowFilter. CompareOp/EQUAL
              (RegexStringComparator. "^[0-9a-fA-F-]{36}:[0-9]{8}$")))

  (hb/with-scanner [results (hb/scan ht
                                     :filter (daily-row-filter) ] ; ...
  )

Hope I can help someone, or at least I'm helping myself of the future.

MSP430 Lauchpad + Forth 初開始

Prolog: There are tons of articles about Forth on MSP430 in English. I would thus like to write some introductory in Chinese.

小弟長久以來在FigTaiwan論壇上潛水，一直沒有產出，深感抱歉。今晚來寫點介紹性的文章，給軟體業的朋友們參考。

Forth (符式) 是一個 postfix (正式名稱叫逆波蘭表示法) 的雙堆疊語言，詳細的介紹網路上很多了，本篇的重點在如何在 Linux 下，把一個 Forth 環境裝到 MSP430 Lauchpad 上面。

1. 按照這張圖，把 TX/RD 設成硬體 UART:

用 USB 線把 MSP430 Lauchpad 和電腦接起來，dmesg 可以看到以下字樣:

[55571.993498] usb 3-1: Product: Texas Instruments MSP-FET430UIF
[55571.993503] usb 3-1: Manufacturer: Texas Instruments
...
[55571.999657] cdc_acm 3-1:1.0: ttyACM0: USB ACM device

2. 跟著這個網頁的說明，在 Linux 下燒錄第一個 MSP430 的程式 (這邊使用 Ubuntu, 11.x 版之後應該都可以):

$ git clone http://github.com/wendlers/msp430-harduart.git
$ sudo apt-get install mspdebug gcc-msp430
$ make

3. 編譯完成後，就可以燒錄了。

$ sudo make flash-target

mspdebug rf2500 "prog ./bin/firmware.elf"
MSPDebug version 0.19 - debugging tool for MSP430 MCUs
Copyright (C) 2009-2012 Daniel Beer 
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Trying to open interface 1 on 005
Initializing FET...
FET protocol version is 30394216
Configured for Spy-Bi-Wire
Set Vcc: 3000 mV
Device ID: 0x2553
Device: MSP430G2553
Code memory starts at 0xc000
Number of breakpoints: 2
Erasing...
Programming...
Writing  364 bytes to c000 [section: .text]...
Writing  106 bytes to c16c [section: .rodata]...
Writing   32 bytes to ffe0 [section: .vectors]...
Done, 502 bytes written

4. 到這邊一切順利的話，可以先玩玩看測試用的小程式。

$ sudo screen /dev/ttyACM0 9600

如果你和我一樣，看到恆亮的紅綠燈，screen 沒有顯示任何字，可以改用 minicom:

$ sudo apt-get install minicom
$ minicom -b 9600 -D /dev/ttyACM0

然後按下 reset 鍵，就可以看到 Logo 了:

***************
MSP430 harduart
***************

PRESS any key to start echo example ... 

5. 按任意鍵之後，就可以開始打字。每打一個字母，紅燈就會 toggle 一次。

6. 玩得開心之後，按 Ctrl-A x 離開 Minicom 。如果離不開，直接把 USB 線拔掉即可。 如果你想用 gcc-msp430 來寫程式，那到這個步驟就可以了。下面要開始進入 Forth

7. 去 4E4th 下載 Forth 主程式。 4E4th 主程式大概 6K, 也就是說還有 8K 的空間，可以讓你寫自己的程式。

8. 燒錄:

$ sudo mspdebug rf2500 "prog ./4e4th.a43"
MSPDebug version 0.19 - debugging tool for MSP430 MCUs
Copyright (C) 2009-2012 Daniel Beer 
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Trying to open interface 1 on 007
Initializing FET...
FET protocol version is 30394216
Configured for Spy-Bi-Wire
Set Vcc: 3000 mV
Device ID: 0x2553
Device: MSP430G2553
Code memory starts at 0xc000
Number of breakpoints: 2
Erasing...
Programming...
Writing   34 bytes to 1080...
Writing 4096 bytes to e000...
Writing 3482 bytes to f000...
Writing   32 bytes to ffe0...
Done, 7644 bytes written

9. 4E4th 的 Wiki 上有很詳細的說明，不過他們的 PDF 檔不是標準格式，我用 Chrome 打不開，另存新檔後用 evince 就可以看。

10. 奇怪的是，Forth 的範例我沒辦法使用 minicom, 但是用 screen 跑卻沒問題。可以試試這個:

$ sudo screen /dev/ttyACM0

打指令:
red cclr        -- 紅燈滅
red cset        -- 紅燈亮
green cclr      -- 綠燈滅
green cset      -- 綠燈亮

已經可以使用 Forth 指令了:
11 22 33 ok 
. 33 ok 
. 22 ok 
. 11 ok 

奇怪的是，它的堆疊沒有檢查下限?
. 12771 ok
. -3823 ok
. -7228 ok

11. 定義自己的 word 也沒問題:

: TEST ." Hello Forth!" ; ok 
TEST Hello Forth!ok 

12. 為了 MSP430 特製指令在這裡，需要參考一下: Glossary 完整的符式字的定義在這裡。

13. 再多玩一下...

red . . [Enter] 33 1 ok
即 adr = 0x21 (Port1 輸出), m = bit0
green . . [Enter] 33 64 ok 
即 adr = 0x21 (Port1 輸出), m = bit6

14. 這塊板子可以幹什麼呢? 這邊 有些介紹。

15. 如果想研究原始碼，就 clone 一份下來吧!

$ svn co http://www.forth-ev.de/repos/4e4th/

Access HBase in a map-reduce job (Clojure)

Sometimes a Hadoop cluster is secured and a map-reduce job needs to obtain an authentication token to access HBase. Here's how we do it in Java:

if (User.isSecurityEnabled()) {
    try {
        User.getCurrent().obtainAuthTokenForJob(job.getConfiguration(), job);
    } catch (IOException ioe) {
        LOG.error(job.getJobName()+ ": Failed to obtain current user.");
    } catch (InterruptedException ie) {
        LOG.info(job.getJobName()+ ": Interrupted obtaining user authentication token");
        Thread.interrupted();
    }
}

So, how to do it in Clojure? First, use clojure-hadoop to create a map-reduce job. You may not use (defjob/defjob), because we need to add something before job submission. Second, we have to make Job job = new Job(HBaseConfiguration.create()). An example follows.

(defn- hbase-set-kerberos [#^Job job]
  (if (User/isSecurityEnabled)
    (try
      (.obtainAuthTokenForJob (User/getCurrent) (.getConfiguration job) job)
      (catch IOException e
        (throw (IOException. "Failed to obtain current user.")))
      (catch InterruptedException e
        (throw (InterruptedException. "Interrupted obtaining user authentication token"))))))
 
(defn tool-run [^Tool this args]
  (doto (Job. (HBaseConfiguration/create))
    (.setJarByClass (.getClass this))
    (.setJobName (str "hbfeeder.mrjob: " (second args)))
    (.setMapperClass (Class/forName "hbfeeder.mrjob_mapper"))
    (.setReducerClass (Class/forName "hbfeeder.mrjob_reducer"))
    (.setNumReduceTasks 0)
    (.setInputFormatClass TextInputFormat)
    (.setOutputFormatClass NullOutputFormat)
    (FileInputFormat/setInputPaths ^String (second args))
    (hbase-set-kerberos)       ; THIS LINE PLAYS THE MAGIC
    (.waitForCompletion true))
  0)

A runnable example can be found at clj-hbase-mapper-example on Github.

Dokuwiki + Slideshow + Syntaxhighlighter 3

Wonder why I can't Google anything like this. I was using Dokuwiki to make netty slides in Linux (as Libreoffice Impress is slow and easy to crash). Everything was fine, just that deck.js doesn't support Syntaxhighlighter 3 to the extent of Dokuwiki. And there is a final strike: DIV size varies according to the content, the fewer text in it, the larger font-size is. Ugly. So I downloaded S5 Reloaded plugin for Dokuwiki and tweaked quite a bit to make it work with Syntaxhighlighter. First, choose a theme for "presentation" only (hence the theme in the wiki is different from that on slides). I chose shThemeDark.css. Vim your syntaxhighlighter3/sxh3/styles/shThemeRDark.css, patch one line:

.syntaxhighlighter {
  background-color: #1b2426 !important;
  font-size: 14px !important;  /* ADD THIS */
}

Or it inherits font-size: 19px from S5. Then, patch s5reloaded/renderer.php.

<link rel="stylesheet" type="text/css" href="/dokuwiki/lib/plugins/syntaxhighlighter3/sxh3/styles/shCore.css"/>
<link rel="stylesheet" type="text/css" href="/dokuwiki/lib/plugins/syntaxhighlighter3/sxh3/styles/shThemeRDark.css"/>
<script type="text/javascript" charset="utf-8" src="/dokuwiki/lib/plugins/syntaxhighlighter3/sxh3/scripts/shCore.js"></script>
<script type="text/javascript" charset="utf-8" src="/dokuwiki/lib/plugins/syntaxhighlighter3/sxh3/scripts/shAutoloader.js"></script>

And put SyntaxHighlighter.autoloader in document_end(). That's all. Enjoy!