Introduction to NodeJS

1. What is node.js and what does it do?
Evented I/O for V8 JavaScript.
To provide a purely evented, non-blocking infrastructure to script highly concurrent programs.

2. Basic concept : blocking and non-blocking

[codesyntax lang=”javascript”]

// blocking
result = query('select * from t');
some_function(result);

// non-blocking
query('select * from t', function(result) {
    // callback
});

[/codesyntax]

3. Amazing Video by Ryan Dahl – Introduction to NodeJS



4. Download and install
[codesyntax lang=”bash”]

$ wget http://nodejs.org/dist/node-v0.2.6.tar.gz
$ tar zxf node-v0.2.6.tar.gz
$ cd node-v0.2.6
$ ./configure --prefix=/usr/local/node
$ make
$ make install

[/codesyntax]

5. Make the doc and read (optional)
[codesyntax lang=”bash”]

$ make doc
$ man doc/node.1

[/codesyntax]

6. Server side test script

[codesyntax lang=”javascript”]

// hello_world.js :
var sys = require('sys');
setTimeout(function() {
    sys.puts('world');
}, 2000);
sys.puts('hello');

[/codesyntax]

7. “hello world”
[codesyntax lang=”bash”]

$ /usr/local/node/bin/node hello_world.js

[/codesyntax]

8. Resources
Online chat room example written in node.js : https://github.com/ry/node_chat
NodeJS Quick Tour : Node-JS_Quick_Tour_V2.pdf
Slides from JSConf 2010 : jsconf2010.pdf

Posted in JavaScript | Tagged , | Leave a comment

Master-Slave MySQL-5.1 Replication

Environment:
RHEL: 5.3
MySQL: 5.1.54



Master :

1. MySQL user and group

[codesyntax lang=”bash”]

$ groupadd mysql
$ useradd -g mysql mysql

[/codesyntax]

2. Download and install MySQL

[codesyntax lang=”bash”]

$ wget http://mysql.ntu.edu.tw/Downloads/MySQL-5.1/mysql-5.1.54.tar.gz
$ tar xzf mysql-5.1.54.tar.gz
$ cd mysql-5.1.54
$ ./configure --prefix=/usr/local/webserver/mysql-5.1.54-master --enable-assembler --with-client-ldflags=-all-static --with-mysqld-ldflags=-all-static --with-extra-charsets=all --enable-thread-safe-client --with-big-tables --with-readline --with-ssl
$ make
$ make install

[/codesyntax]

3. Create database
[codesyntax lang=”bash”]

$ cd /usr/local/webserver/mysql-5.1.54-master/
# database data located at
$ ./bin/mysql_install_db --datadir=/data/mysql-5.1.54/master

[/codesyntax]

4. Master config
[codesyntax lang=”bash”]

$ cp /data/soft_misc/mysql-5.1.54/support-files/my-huge.cnf /data/mysql-5.1.54/master/my.cnf
$ vi /data/mysql-5.1.54/master/my.cnf
port            = 3406
socket          = /tmp/mysql.sock
datadir         = /data/mysql-5.1.54/master
# databases that need to backup
binlog-do-db     = appdb1
binlog-do-db     = appdb2
binlog-ignore-db = mysql
binlog-ignore-db = test

[/codesyntax]

5. Password for root
[codesyntax lang=”bash”]

$ /usr/local/webserver/mysql-5.1.54-master/bin/mysqladmin -uroot -hlocalhost -P3406 -p password root

[/codesyntax]

6. Launch master
[codesyntax lang=”bash”]

$ chown -R mysql.mysql /data/mysql-5.1.54/master/
$ /bin/sh /usr/local/webserver/mysql-5.1.54-master/bin/mysqld_safe --defaults-file=/data/mysql-5.1.54/master/my.cnf &

[/codesyntax]

7. Create account for replication
[codesyntax lang=”bash”]

$ mysql -hlocalhost -uroot -p
mysql> GRANT REPLICATION SLAVE ON *.* TO 'mysqlrepl'@'localhost' IDENTIFIED BY '123456';
mysql> FLUSH PRIVILEGES;

[/codesyntax]



Slave :

1. Copy master application and data as slave

[codesyntax lang=”bash”]

$ cd /usr/local/webserver/
$ cp -Rp mysql-5.1.54-master/ mysql-5.1.54-slave1/
$ cd /data/mysql-5.1.54/
$ cp -Rp master/ slave1/

[/codesyntax]

2. Slave Config
[codesyntax lang=”bash”]

$ vi /data/mysql-5.1.54/slave1/my.cnf
port            = 3416
socket          = /tmp/mysql-slave1.sock
datadir         = /data/mysql-5.1.54/slave1

server-id            = 11
master-host          = localhost
master-port          = 3406
master-user          = mysqlrepl
master-password      = 123456
# connection retry time
# master-connect-retry = 60
# read only for slave if needed
# read-only           = 1

# databases which need to backup
replicate-do-db     = appdb1
replicate-do-db     = appdb2

# databases ignored
replicate-ignore-db = mysql
replicate-ignore-db = test

[/codesyntax]

3. Launch Slave
[codesyntax lang=”bash”]

$ /bin/sh /usr/local/webserver/mysql-5.1.54-slave1/bin/mysqld_safe --defaults-file=/data/mysql-5.1.54/slave1/my.cnf &

[/codesyntax]



Test :

1. Check both master and slave were launched
[codesyntax lang=”bash”]

$ ps aux | grep mysql

[/codesyntax]

2. Try creating something in master
[codesyntax lang=”bash”]

# go to master
$ mysql -hlocalhost -uroot -p
mysql> SHOW MASTER STATUS;
mysql> CREATE DATABASE IF NOT EXISTS appdb1;
mysql> USE appdb1;
mysql> CREATE TABLE abc (id int(4));

[/codesyntax]

3. Check slave to see table ‘abc’
[codesyntax lang=”bash”]

$ mysql -hlocalhost -uroot -P3416 -S /tmp/mysql-slave1.sock -p
mysql> SHOW DATABASES;
mysql> USE appdb1;
mysql> SHOW TABLES;

[/codesyntax]

Posted in MySQL, RHEL | Tagged , | 2 Comments

Example of nginx.conf

Download example of nginx config file : nginx.conf

[codesyntax lang=”apache”]

# nobody nobody but you ...
user  nobody;

# max concurrent requests = worker_processes * worker_connections
worker_processes  8;

# 4 cpus (double core)
worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;

# max opened files of each process
worker_rlimit_nofile 51200;

# only open for dev environment
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

# pid number for nginx
pid        logs/nginx.pid;

events {
    # for large I/O
    use epoll;
    # max connections for each process
    worker_connections  51200;
}

# handle http requests
http {
    # map of files and mime-types
    include       mime.types;

    # default file type of http (can be text/html)
    default_type  application/octet-stream;

    # format of log file
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    # access log
    #access_log  logs/access.log  main;

    # timeout of keepalive connection
    keepalive_timeout  650;

    # efficient file transfer mode, using both sendfile and TCP_CORK, only for unix
    sendfile        on;
    tcp_nopush      on;

    # start gzip
    gzip  on;
    # min size of file
    gzip_min_length  1000;
    # buffer size
    gzip_buffers     4 8k;
    # file types which need gzip
    gzip_types       text/* text/css application/javascript application/x-javascript;
    # compression ratio (1 for least, 9 for most)
    gzip_comp_level  9;
    # whether allow compression for proxy request
    gzip_proxied     any;
    # header of "Vary: Accept-Encoding"
    gzip_vary        on;
    # gzip version (use 1.0 for squid)
    gzip_http_version 1.1

    # output buffer size
    output_buffers   4 32k;

    # size of output package
    postpone_output  1460;

    # for big http header (especially large cookies)
    client_header_buffer_size 128k;
    large_client_header_buffers 4 256k;

    # default charset
    charset utf-8;

    ########################################################################

    # deny access of default
    server {
        listen 80;
        server_name null;
        location / {
            root /dev/null;
            log_not_found off;
        }
    }

    # for magento-1.4.2-stable ################################################
    server {
        listen       80;
        server_name  www.mglocal.com;

        root /data/vhosts/mglocal/public_html;

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        location / {
            index  index.php index.html index.htm;
            if (-f $request_filename) {
                expires 30d;
                break;
            }
            if (-d $request_filename) {
                break;
            }
            if (!-e $request_filename) {
                # redirect all requests to magento
                rewrite ^(.+)$ /index.php last;
            }
        }

        location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;

            # type of project you are running
            fastcgi_param  MAGE_RUN_TYPE website;
            # the code you set in backend
            fastcgi_param  MAGE_RUN_CODE ft;

            include        fastcgi.conf;
        }

        # any access to /app/etc is forbidden
        location /app/etc {
            deny all;
        }

        access_log  /data/weblog/www.mglocal.com.access.log;
        error_log  /data/weblog/www.mglocal.com.error.log;
    }
    ###########################################################################

    # upsteam polling for www.kimbs.cn ########################################
    upstream www.kimbs.cn  {
	    server	127.0.0.1:8000;
	    server	127.0.0.1:8001;
    }

    server {
        listen	80;
        server_name  www.kimbs.cn;
        location / {
            # proxy
            proxy_pass	http://www.kimbs.cn;
            proxy_set_header	Host	$host;
            proxy_set_header	X-Real-IP	$remote_addr;
            proxy_set_header	X-Forwarded-For	$proxy_add_x_forwarded_for;
        }
        access_log  /data/weblog/www.kimbs.cn.access.log  access;
        error_log  /data/weblog/www.kimbs.cn.error.log  crit;
    }

    # 1st server
    server {
        listen       8000;
        server_name  127.0.0.1;
        root /data/vhosts/kbs/public_html;
        error_page   500 502 503 504  /50x.html;

        location = /50x.html {
            root   html;
        }

        location / {
            index  index.php index.html index.htm;
            if (-f $request_filename) {
                break;
            }
            if (-d $request_filename) {
                break;
            }
            rewrite ^(.+)$ /index.php last;
        }

        location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
            include        fastcgi.conf;
        }

        access_log  /data/weblog/www.kimbs.cn.access.log  access;
        error_log  /data/weblog/www.kimbs.cn.error.log  crit;
    }

    # 2nd server
    server {
        listen       8001;
        server_name  127.0.0.1;
        root /data/vhosts/kbs2/public_html;
        error_page   500 502 503 504  /50x.html;

        location = /50x.html {
            root   html;
        }

        location / {
            index  index.php index.html index.htm;
            if (-f $request_filename) {
                break;
            }
            if (-d $request_filename) {
                break;
            }
            rewrite ^(.+)$ /index.php last;
        }

        location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
            include        fastcgi.conf;
        }

        access_log  /data/weblog/www.kimbs.cn.access.log  access;
        error_log  /data/weblog/www.kimbs.cn.error.log  crit;
    }
    ###########################################################################

    # for drupal ##############################################################
    server {
        listen 80;
        server_name myproj.local;

        access_log /data/weblog/myproj.local.access.log access;
        error_log /data/weblog/myproj.local.error.log crit;

        root /data/vhosts/myproj/public_html;

        # prevent '413 Request Entity Too Large'
        client_max_body_size 32m;

        # rewrite
        location / {
            index index.php index.html index.htm;
            if (!-e $request_filename) {
                rewrite ^/(.*)$ /index.php?q=$1 last;
            }
        }

        # hide protected files
        location ~* .(engine|inc|info|install|module|profile|po|sh|.*sql|theme|tpl(.php)?|xtmpl)$|^(code-style.pl|Entries.*|Repository|Root|Tag|Template)$ {
            deny all;
        }

        # hide backup_migrate files
        location ~* ^/files/backup_migrate {
            deny all;
        }

        # serve static files directly
        location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|ico)$ {
            access_log  off;
            expires     30d;
        }

        # js and css files
        location ~ .*\.(js|css|xml)?$ {
            expires     1h;
        }

        # serve php
        location ~ \.php$ {
            fastcgi_pass                   127.0.0.1:9000;
            fastcgi_index                  index.php;
            fastcgi_param                  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            fastcgi_split_path_info        ^(.+\.php)(.*)$;
            fastcgi_param PATH_INFO        $fastcgi_path_info;

            # prevent timeout
            fastcgi_send_timeout           1800;
            fastcgi_read_timeout           1800;
            fastcgi_connect_timeout        1800;
            fastcgi_buffers                8 128k;

            include                        fastcgi.conf;
            include                        fastcgi_params;
        }
    }
    ###########################################################################

}

[/codesyntax]

Posted in Nginx | Tagged , , , | Leave a comment

Customizing Magento Module with Event-Observer

1. About the Module :
This module is for media sync when deploying magento as distributed application
Module located at : app/code/local/Kim/Syncmedia/
There are only 4 files list below to make that all :

2. app/code/local/Kim/Syncmedia/controllers/indexController.php

[codesyntax lang=”php”]

class Kim_Syncmedia_IndexController extends Mage_Core_Controller_Front_Action
{
    public function indexAction()
    {
        $params = $this->getRequest()->getParams();
        $model = Mage::getModel('syncmedia/observer');
        var_dump($params);
        var_dump($model);
        exit();
    }
}

[/codesyntax]

3. app/code/local/Kim/Syncmedia/etc/config.xml
[codesyntax lang=”xml”]

<?xml version="1.0"?>
<config>

    <!-- base configurations -->
    <modules>
        <Kim_Syncmedia>
            <!-- module version -->
            <version>0.1.0</version>
        </Kim_Syncmedia>
    </modules>

    <!-- global configurations -->
    <global>
        <!-- enable models and resources -->
        <models>
            <syncmedia>
                <class>Kim_Syncmedia_Model</class>
                <!-- <resourceModel>weblog_mysql4</resourceModel> -->
            </syncmedia>
        </models>

        <!-- events hooks -->
        <events>
            <cms_page_save_after>
                <observers>
                    <kim_catalog_observer>
                        <type>singleton</type>
                        <class>syncmedia/observer</class>
                        <method>sync_cms_image</method>
                    </kim_catalog_observer>
                </observers>
            </cms_page_save_after>

            <catalog_category_save_after>
                <observers>
                    <kim_catalog_observer>
                        <type>singleton</type>
                        <class>syncmedia/observer</class>
                        <method>sync_category_image</method>
                    </kim_catalog_observer>
                </observers>
            </catalog_category_save_after>

            <catalog_product_save_after>
                <observers>
                    <kim_catalog_observer>
                        <type>singleton</type>
                        <class>syncmedia/observer</class>
                        <method>sync_product_image</method>
                    </kim_catalog_observer>
                </observers>
            </catalog_product_save_after>
        </events>
    </global>

    <!-- frontend configurations -->
    <frontend>
        <routers>
            <syncmedia>
                <!-- module route -->
                <use>standard</use>
                <args>
                    <module>Kim_Syncmedia</module>
                    <frontName>syncmedia</frontName>
                </args>
            </syncmedia>
        </routers>
    </frontend>

[/codesyntax]

4. app/code/local/Kim/Syncmedia/Model/Observer.php

[codesyntax lang=”php”]

class Kim_Syncmedia_Model_Observer
{
    protected function _construct()
    {
        $this->_init('syncmedia/observer');
    }

    public function sync_cms_image($observer)
    {
        system('cp -Ru /data/vhosts/magento/public_html/media/*[.a-z] /data/vhosts/magento2/public_html/media/ &');
        system('chmod -R g+w /data/vhosts/magento2/public_html/media/ &');
        return $this;
    }

    public function sync_product_image($observer)
    {
        system('cp -Ru /ddata/vhosts/magento/public_html/media/catalog/product/* /ddata/vhosts/magento2/public_html/media/catalog/product/ &');
        system('chmod -R g+w /data/vhosts/magento2/public_html/media/catalog/product/ &');
        return $this;
    }

    public function sync_category_image($observer)
    {
        system('cp -Ru /data/vhosts/magento/public_html/media/catalog/category/* /data/vhosts/magento2/public_html/media/catalog/category/ &');
        system('chmod -R g+w /data/vhosts/magento2/public_html/media/catalog/category/ &');
        return $this;
    }

}

[/codesyntax]

5. Finally you need to enable the module with : app/etc/modules/Kim_Syncmedia.xml

[codesyntax lang=”xml”]

<?xml version="1.0"?>
<config>
    <modules>
        <Kim_Syncmedia>
            <active>true</active>
            <codePool>local</codePool>
        </Kim_Syncmedia>
    </modules>
</config>

[/codesyntax]

Posted in Magento | Tagged | 2 Comments

PHP-FPM for RHEL

1. Patch PHP-FPM to PHP
[codesyntax lang=”bash”]

$ wget http://php-fpm.org/downloads/php-5.2.17-fpm-0.5.14.diff.gz
$ wget http://cn.php.net/distributions/php-5.2.17.tar.bz2
$ tar -jxf php-5.2.17.tar.bz2
$ gzip -d php-5.2.17-fpm-0.5.14.diff.gz
$ patch -d php-5.2.17 -p1 < php-5.2.17-fpm-0.5.14.diff

[/codesyntax]

2. PHP Configure Options for CGI/FastCGI (for help)

[codesyntax lang=”bash”]

--disable-cgi                   Disable building CGI version of PHP
--enable-fastcgi                CGI: Enable FastCGI support in the CGI binary
--enable-force-cgi-redirect     CGI: Enable security check for internal server redirects.
                                Use this if you run the PHP CGI with Apache
--enable-discard-path           CGI: When this is enabled the PHP CGI binary can safely be placed
                                outside of the web tree and people will not be able to circumvent .htaccess security
--disable-path-info-check CGI:  If this is disabled, paths such as /info.php/test?a=b will fail to work
--enable-fpm                    FastCGI: If this is enabled, the fastcgi support
                                will include experimental process manager code
--with-fpm-conf=PATH            Set the path for php-fpm configuration file [PREFIX/etc/php-fpm.conf]
--with-fpm-log=PATH             Set the path for php-fpm log file [PREFIX/logs/php-fpm.log]
--with-fpm-pid=PATH             Set the path for php-fpm pid file [PREFIX/logs/php-fpm.pid]
--with-xml-config=PATH          FPM: use xml-config in PATH to find libxml

--with-ncurses[=DIR] Include ncurses support (CLI/CGI only) --enable-pcntl Enable pcntl support (CLI/CGI only) --with-libedit[=DIR] Include libedit readline replacement (CLI/CGI only) --with-readline[=DIR] Include readline support (CLI/CGI only)

[/codesyntax]

3. Install PHP
[codesyntax lang=”bash”]

$ ./configure --prefix=/usr/local/webserver/php-5.2.17 --with-config-file-path=/usr/local/webserver/php-5.2.17/etc --enable-fastcgi --enable-fpm --with-jpeg-dir= --with-png-dir --with-gd=/usr/local/gd --with-zlib-dir --with-curl --enable-mbstring=cn --with-mysql=/usr/local/webserver/mysql --with-mysqli=/usr/local/webserver/mysql/bin/mysql_config --enable-sysvshm --enable-sysvsem --enable-pcntl --enable-soap --with-freetype-dir --enable-gd-native-ttf --with-iconv-dir --with-imap=/usr/local/imap-c-client --with-imap-ssl --with-pdo-mysql=/usr/local/webserver/mysql
$ make all
$ make install

[/codesyntax]

4. Modify php.ini and php-fpm.conf
[codesyntax lang=”bash”]

$ cp php.ini-dist /usr/local/webserver/php-5.2.17/etc/php.ini
# open display_errors in php.ini

$ vi /usr/local/webserver/php-5.2.17/etc/php-fpm.conf
# uncomment these 2 lines
# <value name="user">nobody</value>
# <value name="group">nobody</value>

[/codesyntax]

5. Test and Start
[codesyntax lang=”bash”]

$ /usr/local/webserver/php-5.2.17/bin/php -v
$ /usr/local/webserver/php-5.2.17/sbin/php-fpm start

[/codesyntax]

Posted in PHP, RHEL | Tagged , | Leave a comment