2012/04/21

sudo: no valid sudoers sources found, quitting

犯了一個天大的錯誤,就是編輯 sudoers 的時候,忘記把權限改回來,於是就失去了 sudo 的權限了...
sudo: /etc/sudoers is mode 0640, should be 0440
sudo: no valid sudoers sources found, quitting

如果你的機器就在旁邊的話,可以進入 recovery mode 或用 liveCD 把權限改回來。

如果是在 AWS呢?找到一篇教學:
Fixing Files on the Root EBS Volume of an EC2 Instance - Alestic.com

步驟
1. 把 Server A 的 EBS Volume detach,然後 attach 到另外一台 Sever B
2. 進入 Server B 把 device node mount,開始修復檔案
3. 最後再 attach 回原本的 Server A


後來才知道,要用 visudo 這個指令去修改才對...

2012/04/18

Linux下如何偵測網路介面是否有插上線

如果一台主機上有8個 network interface (eth0 - eth7),要怎麼知道我現在插上的網路線是對應到ethN?


$ sudo ethtool eth0
Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    Supports Wake-on: umbg
    Wake-on: g
    Current message level: 0x00000007 (7)
    Link detected: yes

Link detected: yes 表示是有插上線的,但前提是一定要先把 interface 叫起來 (ifconfig eth0 up),這個偵測才有用。


參考資料中還有提到其他的方法
$ grep "" /sys/class/net/eth0/*
...
/sys/class/net/eth0/carrier:1
...
/sys/class/net/eth0/operstate:up
...


參考資料
linux - How to detect the physical connected state of a network cable/connector? - Stack Overflow

Nginx 加速 web 存取速度

對於靜態資料不外乎就是 cache、CDN (Content delivery network),減少後端 server 的 loading。

讓 static content 去別台抓
location ^~ /static {
    proxy_pass http://cdn.server/static;
}

讓 static conetent 由 nginx 處理,不透過後端 http server
location ^~ /static {
    root /var/www
}

承上,並加入 cache 機制,第一次會去檔案讀取,之後就從 cache 讀取 (要注意 cache refresh的問題)
location ^~ /static {
    root /var/www;
    proxy_buffering on;
    proxy_cache_valid 200 120m;
    expires 30d;
}

如果 static file 在多個目錄,用 OR 的方式去match
location ~* (/images|/css|/js) {
   ...
}

如果 static file 分佈在各個目錄,可以利用下面的 rule 去 match 附檔名
location ~* \.(jpg|png|gif|jpeg|css|js|mp3|wav|swf|mov|doc|pdf|xls|ppt|docx|pptx|xlsx)$ {
   ...
}

2012/03/29

SimpleDB - "Too many value tests per predicate in the query expression"

當用Simple下Querey的時候,condition超過20個就會出現下面的錯誤
Too many value tests per predicate in the query expression
或
Client error : Too many value tests per predicate in the query expression.

像這類的語法都不行:
SELECT * FROM domain WHERE id='1' OR id='2' OR ... id='20' OR id='21'

SELECT * FROM domain WHERE id in ('1', '2', ... '20', '21')

SELECT * FROM domain WHERE id in ('1', '2', ... '20') OR id in ('21', ...)

只能考慮分批一次抓20個,然後再合併處理。



2012/05/07 發現一種可以破解20個條件限制的辦法:
SELECT * FROM `domain` WHERE 
  account in ('key1', 'key2', 'key3', 'key4', 'key5', 'key6', 'key7', 'key8', 'key9', 'key10', 
    'key11', 'key12', 'key13', 'key14', 'key15', 'key16', 'key17', 'key18', 'key19', 'key20') OR 
  dummy IS NOT NULL OR 
  account in ('key21', 'key22', 'key23', 'key24', 'key25', 'key26', 'key27', 'key28', 'key29', 'key30', 
    'key31', 'key32', 'key33', 'key34', 'key35', 'key36', 'key37', 'key38', 'key39', 'key40')
沒錯,就是加了"OR dummy IS NOT NULL"這個多餘的條件在中間就可以下達多個條件了!

不過當你有這樣的需求時,就應該想一下是不是設計面有問題,或是該用 relational database了。



參考資料
Amazon SimpleDB » Developer Guide » Amazon SimpleDB Concepts » Limits
Query 101: Building Amazon SimpleDB Queries

2012/03/28

架設 Subversion system + apache

安裝SVN
yum install subversion apache mod_dav_svn

在此我們不用 daemon mode (svnserve),而是使用 apache 來管理

設定帳密
htpasswd -c /etc/svn/passwd user

設定權限
[groups]
admin = user1, user2

[Project1:/]
@admin = rw

[Project2:/]
@admin = rw

[Project2:/Doc]
user3 = rw

設定apache configuration
/etc/httpd/conf.d/subversion.conf
LoadModule dav_svn_module     modules/mod_dav_svn.so
LoadModule authz_svn_module   modules/mod_authz_svn.so

<Location /svn>
  DAV svn
  SVNParentPath /var/svn

  SSLRequireSSL
  AuthType Basic
  AuthName "Authorization Realm"
  AuthUserFile /etc/svn/passwd
  AuthzSVNAccessFile /etc/svn/authz
  Require valid-user
</location>

建立新的 project
svnadmin create /var/svn/Project1

讓 httpd 有存取 svn 目錄的權限
chown -R apache:apache /var/svn/Project1

最後開啓網址 https://localhost/svn/Project1 試試看


2012/03/27

編譯 binary 的時候自動加入 release version 與 date

Makefile
SVN_VERSION = "\"`svn info | grep Revision | sed 's/Revision: //g'`\""
BUILD_TIME = "\"`date '+%Y/%m/%d %H:%M:%S'`\""

CFLAGS += -DSVN_VERSION=$(SVN_VERSION) -DBUILD_TIME=$(BUILD_TIME)

.c 檔
void show_version (void)
{
    printf("SVN Version: %s, Bulid time: %s\n", SVN_VERSION, BUILD_TIME);
}
Preprocessing 的時候 macro 會自動被帶換掉啦!

2012/03/25

用 shell script 做一個假的 DNS

很有趣的問題,突然跟我同事在想要怎麼快速做一個假的DNS (Domain Name Server)? 不就follow 鳥哥的教學,用 bind 架一個就好了?
有想過寫在 /etc/hosts,但發現 nslookup 並不會去查詢 hosts file

Anyway, geek 就喜歡搞怪,喜歡繞道而行,喜歡自己造輪子...

=> 想到用 netcat 當 dns proxy,把 dns query redirect 給 8.8.8.8,但遇到 special 的 domain name 時,就自己 return。



Step1: 測試 dns proxy
$ sudo nc -v -u -l -s 127.0.0.1 -p 53 -e 'nc -u 8.8.8.8 53'
$ nslookup - 127.0.0.1
> yahoo.com
Server:         10.32.100.199
Address:        10.32.100.199#53
Non-authoritative answer:
Name:   yahoo.com
Address: 209.191.122.70
Name:   yahoo.com
Address: 72.30.38.140
Name:   yahoo.com
Address: 98.139.183.24
> 
It's work!

想看整個過程,開三個session
$ mkfifo fifo1 fifo2
$ sudo nc -u -l -s 127.0.0.1 -p 53 -c -x > fifo1 < fifo2
Received 27 bytes from the socket
00000000  E2 20 01 00  00 01 00 00  00 00 00 00  05 79 61 68  . ...........yah
00000010  6F 6F 03 63  6F 6D 00 00  01 00 01                  oo.com.....
Sent 75 bytes to the socket
00000000  E2 20 81 80  00 01 00 03  00 00 00 00  05 79 61 68  . ...........yah
00000010  6F 6F 03 63  6F 6D 00 00  01 00 01 C0  0C 00 01 00  oo.com..........
00000020  01 00 00 09  80 00 04 62  8B B7 18 C0  0C 00 01 00  .......b........
00000030  01 00 00 09  80 00 04 D1  BF 7A 46 C0  0C 00 01 00  .........zF.....
00000040  01 00 00 09  80 00 04 48  1E 26 8C                  .......H.&.

$ nc -u 8.8.8.8 53 -x < fifo1 > fifo2
Sent 27 bytes to the socket
00000000  E2 20 01 00  00 01 00 00  00 00 00 00  05 79 61 68  . ...........yah
00000010  6F 6F 03 63  6F 6D 00 00  01 00 01                  oo.com.....
Received 75 bytes from the socket
00000000  E2 20 81 80  00 01 00 03  00 00 00 00  05 79 61 68  . ...........yah
00000010  6F 6F 03 63  6F 6D 00 00  01 00 01 C0  0C 00 01 00  oo.com..........
00000020  01 00 00 09  80 00 04 62  8B B7 18 C0  0C 00 01 00  .......b........
00000030  01 00 00 09  80 00 04 D1  BF 7A 46 C0  0C 00 01 00  .........zF.....
00000040  01 00 00 09  80 00 04 48  1E 26 8C                  .......H.&.

$ nslookup - 127.0.0.1
> yahoo.com
Server:         10.32.100.199
Address:        10.32.100.199#53

Non-authoritative answer:
Name:   yahoo.com
Address: 98.139.183.24
Name:   yahoo.com
Address: 209.191.122.70
Name:   yahoo.com
Address: 72.30.38.140
>


Step 2: 截取 dns query
$ sudo nc -u -l -s 127.0.0.1 -p 53 > fake_query
$ nslookup - 127.0.0.1
> yahoo.com


Step 3: 用 fake_query 去 get response
$ cat fake_query | nc -u 8.8.8.8 53 > fake_response


Step 4: 把 fake_response 內的 ip 換成特定的 ip
(這一步還沒做)


Step 5: Start fake dns
$ mkfifo fifo1 fifo2
$ sudo nc -u -l -s 127.0.0.1 -p 53 -c > fifo1 < fifo2 &
$ sh filter.sh < fifo1 > fifo2

filter.sh 裡面會截取 current query id,然後 replace fake response 裡面的
#!/bin/sh
QID=
COUNT=0
cat - | while read -n 1 byte
do
    if [ $COUNT -lt 2 ]; then
        QID+=$byte
    fi

    if [ $COUNT -eq 2 ]; then
        echo -n $QID$(cut -b 3- fake_response)
    fi

    COUNT=`expr $COUNT + 1`
done


結果
$ nslookup - 127.0.0.1
> yahoo.com
;; Got bad packet: bad compression pointer
51 bytes
65 9c 81 80 01 03 05 79 61 68 6f 6f 03 63 6f 6d
01 01 c0 0c 01 01 01 ad 04 d1 bf 7a 46 c0 0c 01
01 01 ad 04 48 1e 26 8c c0 0c 01 01 01 ad 04 62
8b b7 18
>

Oh no~ 還是有問題,初估應該是 dns packet header 哪邊需要動態更改...
Alright! 是該停止這浪費時間的行為了...



參考資料
想利用 shell script 做到這些事情代價真大,以下是必需了解的:
  • 以 hex mode 顯示 binary file
hexdump -C file
while read -n 1 byte; do
    ord=$(printf "%b" "${byte:-\000}" |
          od -t x1 |
          { read offset hex; echo $hex; })
    echo "$ord"
done
#!/bin/sh
cat - | while read LINE
do
  echo ${LINE}
done

try ls -al | script.sh