Archive for the ‘Network service’ Category.

configure for syslog-ng , syslog-ng.conf , log server , remote log , syslog

 

// ----------------

Server: /etc/syslog-ng/syslog-ng.conf :

source remote_host {
        udp();
};

destination my_host { file("/var/log/$YEAR$MONTH$DAY.log"); };

log {
        source(remote_host);
        destination(my_host);
};

// —————-

 

Client: /etc/syslog-ng/syslog-ng.conf :

source src { unix-stream("/dev/log"); internal(); };

destination remote_server { udp("172.25.8.109" port(514)); };

log {
    source(src);
    destination(remote_server);
};

// —————- 改良版的 log server , 日期/時間比較容易看, remote server 的 IP address

 

其中 flush_lines 可以改大一點增加 log 的 performance ….

options {
  chain_hostnames(no);
  use_fqdn(no);
  use_dns(no);
  keep_hostname(no);
  flush_lines(0);
};

source remote_host {
  udp();
};

template t_rewritetime {
  template("${YEAR}-${MONTH}-${DAY} , ${HOUR}:${MIN}:${SEC} , ${HOST} , \"${MSG}\"\n");
  template_escape(no);
};

destination my_host {
  file("/var/log/$YEAR$MONTH$DAY-$HOUR.log" template(t_rewritetime) );
};

log {
  source(remote_host);
  destination(my_host);
};

// —— // —————- 改良版的 client , 保留一份 log 在本機備查

source src { unix-stream("/dev/log"); internal(); };

destination remote_server {
  udp( "172.25.8.109" port(514) );
  file("/var/log/local_log_file.log");
};

log {
  source(src);
  destination(remote_server);
};

// —–

log sample :

root@log_server:/etc/syslog-ng# tail -f /var/log/20111024-17.log
2011-10-24 , 17:21:06 , 172.25.x.107 , "crontab[12974]: (root) LIST (root)"
2011-10-24 , 17:22:01 , 172.25.x.108 , "CRON[4150]: pam_unix(cron:session): session opened for user root by (uid=0)"
2011-10-24 , 17:22:01 , 172.25.x.108 , "/USR/SBIN/CRON[4151]: (root) CMD (/usr/sbin/ntpdate 172.25.x.45)"
2011-10-24 , 17:22:01 , 172.25.x.108 , "CRON[4150]: pam_unix(cron:session): session closed for user root"
2011-10-24 , 17:22:49 , 172.30.x.43 , "sshd[5533]: Accepted keyboard-interactive/pam for monster from 172.30.x.89 port 56790 ssh2"
2011-10-24 , 17:22:49 , 172.30.x.43 , "sshd[5533]: pam_unix(sshd:session): session opened for user monster by (uid=0)"
2011-10-24 , 17:22:51 , 172.30.x.43 , "sshd[5533]: pam_unix(sshd:session): session closed for user monster"

// —- 測試 syslog 的 C – sample code :

 

*PS: ubuntu 要先 apt-get update 再裝 apt-get install gcc build-essential

…. // —- 測試 syslog 的 PHP – sample code : , 跟 C 的版本長得一樣 ….

openlog("myScriptLog", LOG_PID | LOG_PERROR, LOG_LOCAL0);
syslog(LOG_WARNING, "Unauthorized client...." );
closelog();

….

URL: syslog-ng performance tuning 1.

 

 

email checker

也許根據以下的 smtp command 可以用來 check email 名單的正確性

URL: http://www.ip-address.org/verify/email-checker.php

Resolving host name "gmail-smtp-in.l.google.com"...
Connecting to host address "209.85.225.27"...
Connected.
S 220 mx.google.com ESMTP m8si9351085icw.142
C HELO ip-address.org
S 250 mx.google.com at your service
C MAIL FROM: <info@ip-address.org>
S 250 2.1.0 OK m8si9351085icw.142
C RCPT TO: <xxx@gmail.com>
S 250 2.1.5 OK m8si9351085icw.142
C DATA
S 354 Go ahead m8si9351085icw.142
This host states that the address is valid.
Disconnected.

amazon aws ami LAMP / NGINX / PHP / memcache steps

// —–


Using NGINX + PHP5-FPM
sudo apt-get install nginx php5-fpm php5-mysql
cd /etc/init.d/
sudo service apache2 stop
sudo update-rc.d -f apache2 remove

http://yoodey.com/how-install-configure-nginx-php-fpm-memcached-work-drupal-7-ubuntu-1010-maverick

Install Memcached
sudo apt-get install memcached libmemcached-dev php5-memcached php5-memcache

// ====================

Getting a new virtual machine up and running from the default AMI is easy.  Here’s my cookbook:
Install basic services

    yum install mysql mysql-server mysql-devel
    yum install httpd httpd-devel
    yum install php php-devel php-mysql php-gd php-dom php-pear php-json
    yum install svn

Configure those services to start at boot

    chkconfig –level 345 httpd on
    chkconfig –level 345 mysqld on
    chkconfig –list

Install APC

I couldn’t find a package, and this bug report explains why we have to use the beta.

    yum install gcc pcre-devel
    pecl install apc-beta
    echo extension=apc.so > /etc/php.d/apc.ini

Install memcached

    yum install libevent libevent-devel
    curl -O http://memcached.googlecode.com/files/memcached-1.4.5.tar.gz
    cd memcached-1.4.5
    ./configure
    make ; make install
    /usr/local/bin/memcached -u nobody -d -m 30 -l 127.0.0.1 -p 11211

// ----
<pre>
[root@ip-10-130-9-23 log]# cat /etc/fstab
#
LABEL=/     /           ext4    defaults,noatime  1   1
tmpfs       /dev/shm    tmpfs   defaults        0   0
devpts      /dev/pts    devpts  gid=5,mode=620  0   0
sysfs       /sys        sysfs   defaults        0   0
proc        /proc       proc    defaults        0   0
[root@ip-10-130-9-23 log]# cat /etc/mtab
/dev/xvda1 / ext4 rw,noatime 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
devpts /dev/pts devpts rw,gid=5,mode=620 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0

</pre>

Apache 是 Internet 世界最偉大的 project

以前 打 www.apache.org 可能就是看到 apache web server 的 download , document ….等等, 但隨著越來越多開發者把他們的 project 標記成 apache license , apache 於是變成一個巨大的 project 集中地 – The Apache Software Foundation ….

apache web server 是排名第一的 web server 這不用說, 今天認識了 Lucene , Solr 這個應該算是 apache 第二大

Apache Lucene Java 分類是 database
http://projects.apache.org/projects/lucene_java.html

Apache Solr 是被分類在 web-framework, network-server
http://projects.apache.org/projects/solr.html

 

FreeBSD + apache with large connection settings

solved 如下:

一、  網站的 Search功能因為使用 Ajax的方式,在大量User使用下,造成 Mysql 連線問題而影響 Apahce.

二、  因為新網站為全新製作,所有舊有的網址(包含 Search Agent、各大討論區、子公司…)會全部導回首頁 (Rewrite)…

在大量存取下,短時間產生Mysql、Apache連線,影響mysql與apache 的效能…

 

 

 

problem: 大量出現…

Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max

===> MaxClients  設太大

這兩個有設

FreeBSD: append to /etc/sysctl (or you can use sysctl -w to add these)

kern.maxfiles=65536
kern.maxfilesperproc=32768

如下:

一、 網站的 Search功能因為使用 Ajax的方式,在大量User使用下,造成 Mysql 連線問題而影響 Apahce.

二、 因為新網站為全新製作,所有舊有的網址(包含 Search Agent、各大討論區、子公司…)會全部導回首頁 (Rewrite)

在大量存取下,短時間產生MysqlApache連線,影響mysqlapache 的效能…

Subversion Server Notes

Amazon AWS EC2 service / cloud / 雲端

Dear EC2 Customer,
You recently reached a limit on the volume of email you were able to send out of SMTP port 25 on your instance:

Instance ID: i-9b8924ce
* IP Address: 1.2.3.4
* Start date: 2011-06-01 22:47 +0000

In order to maintain the quality of EC2 addresses for sending email, we enforce default limits on the amount of email that can be sent from EC2 accounts. If you wish to send larger amounts of email from EC2, you can apply to have these limits removed from your account by filling out our online request form.

If you are unaware of your instance having sent emails, we advise checking your instance application(s) to confirm that this activity was intended. It is your responsibility to ensure that your instances and all applications are secured against unauthorized use. For suggestions on securing your instances, visit aws.amazon.com/security.

Regards,
Your Amazon Web Services EC2 team

—–

5/20
amzn-ami-2011.02.1.x86_64-ebs (ami-6ef08e3c)
Description: Amazon Linux AMI x86_64 EBS
Status: available
Platform: Amazon Linux
Image size: 8 GB
Visibility: Public
Owner: amazon (137112412989)

processor    : 0
vendor_id    : GenuineIntel
cpu family    : 6
model        : 23
model name    : Intel(R) Xeon(R) CPU           E5430  @ 2.66GHz
stepping    : 10
cpu MHz        : 2666.760
cache size    : 6144 KB
fpu        : yes
fpu_exception    : yes
cpuid level    : 13
wp        : yes
flags        : fpu tsc msr pae cx8 cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht pbe syscall nx lm constant_tsc up arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority
bogomips    : 5333.52
clflush size    : 64
cache_alignment    : 64
address sizes    : 38 bits physical, 48 bits virtual
power management:

5/14
Install MySQL

# yum install mysql mysql-server mysql-libs

Start the service and configure it to start automatically at boot time

# /sbin/service mysqld start

# chkconfig –levels 235 mysqld on

# yum install httpd php php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc
# /etc/init.d/httpd restart
# chkconfig –levels 235 httpd on

5/13

/etc/ssh/sshd_config and set “PasswordAuthentication” to YES

5/12

Amazon Linux AMI : http://aws.amazon.com/amazon-linux-ami/

 

2011.0512:

Amazon Machine Images = AMI

http://aws.amazon.com/articles/4261?_encoding=UTF8&jiveRedirect=1

Linux

Depending on your distribution, you can likely install everything you need via yum or apt-get; Apache Friends XAMPP, BitNami LAMPStack, BitNami LAPPStack

 

Amazon EC2 AMIs

BitNami LAMPStack

 

 

參考設定: ADSL / DDWRT 動態 IP 用 DynDNS 設定 DNS 及 Google Apps

—–

/home/ec2-user/inadyn/bin/linux/inadyn –username userid –password userpassword –alias demo.monster.tw &

 

amazon linux AMI : install LAMP command :

yum install httpd php php-mysql php-gd  php-pear php-xml php-xmlrpc

 

 

 

 

2010. Nov 26, : 原來這就是 Amazon Elastic Compute Cloud (Amazon EC2) 呀…..

 

URL : Amazon 送的網站 http://amazon.monster.tw/ , 申請很簡單, 一下子就有 free 的 VM 可用囉.

Continue reading ‘Amazon AWS EC2 service / cloud / 雲端’ »

ebay token usage

這邊有個圖, 跟 ebay API 串接時的 token / session id 等動作說明得很清楚

Continue reading ‘ebay token usage’ »

Online 畫草圖工具 / Online draft editing tools

tracking system 新點子

上週聽了 ora 公司的產品說明, 覺得不錯.

它是 tracking system 產品, 可是更 friendly / 非侵入式的作法, 它利用 switch 或 SLB設備的 mirror port 把流經的 traffic 導進這個 program , 由這個 program 來進行 traffic 分析.

架構簡圖:

420dbc62a3d3bc44984d3a927888b7fe

 

若要自行作的話可能會需要看懂這個

Programming with pcap
http://www.tcpdump.org/pcap.htm

The Sniffer’s Guide to Raw Traffic (a libpcap tutorial)
http://yuba.stanford.edu/~casado/pcap/section1.html

不錯的配色網站 – design your own palette / pattern / color

CodeIgniter MVC CRUD + memcache = Secret Message http://msg.monster.com.tw

這個 idea 是來自這個網站 : https://privnote.com/ , 簡單的說它是一個經由 https 保護傳送內容的網路服務 , user 用這個服務把臨時要給朋友的機密/私人資料譬如 password / URL 之類的 data 存放在這網站上, 然後此系統會給 user 一個唯一網址, 再把這網址給朋友, 開啟這個唯一網址後, 就可以看到這個訊息, 系統同時會把這訊息從系統中刪除… 哈哈! 看懂了嗎?

我的改進是用 memcache 的 expire 機制, 設定 10分鐘後, 若 user 沒讀過此訊息, 訊息會 ‘自動’ 銷毀… 我的站若再去申請 ssl 加密的話就跟那個站功能是類似的了.

codeigniter 的 MVC 架構, 讓我很快的把這個 idea implement 出來了 , 基本上就是一個簡單版的 CRUD

我的 Secret Message 服務長像非常的陽春 :P , 僅用簡單的 HTML , 若有空再加上 style 美化一下版面, 不過基本功能是有的了, 大家用看看, 有 idea 或意見請再告訴我.

目前 message 不提供 HTML / VBB , 但是有簡單的 skype 版表情符號 ( icon 正在慢慢搜集中 ) ….

Secret Message 網址是: http://msg.monster.com.tw/

 

dc34889242b1d25e1f4c4d477e85c96c

4185a388c19629704edc284514c6cfe5

CI 表情符號改寫例:

0fe1a40bfd47887c51ed60fc2af4a3a0

3a21610697fc7f7b5983a1358a9f7713d5617ad16b86596c9c44642a08193875

[monster] monster.tw 記錄 / memo / ubuntu / 裝 圖形介面 / GUI / gnome desktop / ip address / setting

昨天晚上去一位朋友公司幫忙用他們的 server , 回來就想要做幾件事:

1. 記錄一下自己的 server 某些重要檔案, 也可供其他人參考

2. 備份重要 file

3. 隨時準備好最新版的救援CD/Live CD/Live USB , 最新版的 Monster LAMP pack 也要燒好收著.

4. 準備一個 “萬用” 含各類 driver 的 linux kernel , 以供救援使用.

5. 搞懂 booting steps / GRUB setting


a93c35eaf5a0019c7f7a42f461dae3de

root@park:~# cat /etc/fstab

# /etc/fstab: static file system information.
#
# Use 'vol_id --uuid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
#              

proc            /proc           proc    defaults        0       0
# / was on /dev/sda1 during installation
UUID=46312252-8925-4e60-ab5f-af240b4b440f /               ext3    relatime,errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=bf9a64f1-3eb8-433c-8311-8d971be82dc2 none            swap    sw              0       0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0

service / daemon 管理工具

sysv-rc-conf text based utility :

sudo apt-get install sysv-rc-conf

另一套 GUI

sudo apt-get install gnome-system-tools

monster.tw 是裝這個 apt-get install rcconf

列出 runlevels 的 program : chkconfig

裝 圖形介面 / GUI / gnome desktop

sudo apt-get install ubuntu-desktop --no-install-recommends

啟動 desktop manager :

/etc/init.d/gdm start

stop x server / 關掉 x window: /etc/init.d/gdm stop

改 IP address

cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 172.30.0.42
        netmask 255.255.255.0
        network 172.30.0.0
        broadcast 172.30.0.255
        gateway 172.30.0.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 8.8.8.8
        dns-search monster.tw

重新啟動 networking
/etc/init.d/networking restart

若要使用 DHCP 的話

auto eth0
iface eth0 inet dhcp

加上 default route

gateway 1.2.3.4

或另一個 static route

up route add -net 1.2.3.0 netmask 255.255.255.0 gw 1.2.3.4

change host name:

改 /etc/hostname
改 /etc/hosts

若是用 vmware 之類的軟體導致 clone 出來的 ubuntu 網路 interface 跑掉, 可以砍掉
/etc/udev/rules.d/70-persistent-net.rules
後 reboot , 就可以解決了.

Google 提供 Public 的 DNS , IP 是 8.8.8.8 帥氣又好記!

URL: http://code.google.com/speed/public-dns/

有 unix server 的人 /etc/resolv.conf 可以設它
nameserver 8.8.8.8
windows 使用者也可看這篇說明來設定
http://code.google.com/speed/public-dns/docs/using.html
好處是
Why should you try Google Public DNS?

By using Google Public DNS you can:

    * Speed up your browsing experience.
    * Improve your security.
    * Get the results you expect with absolutely no redirection.

ADSL / DDWRT 動態 IP 用 DynDNS 設定 DNS 及 Google Apps

DYNDNS URL :

DynDNS Update Clients : https://www.dyndns.com/support/clients/

command liek this :

/bin/inadyn -u userid -p userpassword -a xxx.blogdns.net --update_period_sec 30 --log_file /tmp/dyn.log --background &

先到 https://www.dyndns.com/ 申請 “Custom DNS Service” (每年USD$29元)

9a0f558b84a2fc2fc37bc48cac6bc8cc

69a4e8e3d3a4b2e669d7a1cebe8c3099

  • The domain is registered elsewhere: You will need to log into your account with the domain’s current registrar, and change the nameserver list to the following:

Custom DNS Nameservers

Server IP Address
ns1.mydyndns.org   (Required) 204.13.248.76
ns2.mydyndns.org   (Required) 204.13.249.76
ns3.mydyndns.org 208.78.69.76
ns4.mydyndns.org 91.198.22.76
ns5.mydyndns.org 203.62.195.76

328bb1a335f9c817287b5d8c2a4790e7

在 router (我的是ddwrt) 設定 DDNS – ADSL 每次換 IP 時 ddwrt 就會通知 dyndns 改 DNS 的 IP address , 其他的 DN 也可以用 CNAME 設成跟這個 host name 一樣.

f277b6d220032cfb73de9035297d66ad

去 google apps 申請一個

361a485a50b729989390eb79680c6764

除了 mail 要設 MX record 比較複雜, 其他的只要設 CNAME record 就可以了, 很簡單!

34dc58a397d612b8f7c90915cc305bb9

mail.monster.tw 是 web mail 界面的網址 跟 收信的 domain 是不同的

ee0ae7e24f93714f64fa2338133b211e

[Great idea!] 10 Minute Mail / 十分鐘電郵

URL : http://www.10minutemail.com/10MinuteMail/

很多網站要申請加入會員都需要一個 email address , 這個十分鐘電郵就是可以用來作這件事, 收到會員的啟用連結後, 就可以跟這個 email 說再見了!

60f1ce7cabdd8ce41f573a9b25405a1b

programmer … 超好用的 online tools – URL Encode/URL Decode/base64 encode/base64 decode/UUencode/UUdecode

有趣的 ASCII ART

有趣的 ASCII ART : http://www.network-science.de/ascii/
Font: larry3d   Reflection: no   Adjustment: left   Stretch: no      Width: 80     Text: monster

                                   __
                                  /\ \__
  ___ ___     ___     ___     ____\ \ ,_\    __   _ __
/' __` __`\  / __`\ /' _ `\  /',__\\ \ \/  /'__`\/\`'__\
/\ \/\ \/\ \/\ \_\ \/\ \/\ \/\__, `\\ \ \_/\  __/\ \ \/
\ \_\ \_\ \_\ \____/\ \_\ \_\/\____/ \ \__\ \____\\ \_\
 \/_/\/_/\/_/\/___/  \/_/\/_/\/___/   \/__/\/____/ \/_/ 

squid proxy server setting

改這個檔 /etc/squid/squid.conf

allow 內網可以透過 proxy

就放在那個 INSERT YOUR ….

acl officenet src 172.30.0.0/24
http_access allow officenet

Protected: 用 ajax load 廣告圖的 delay

This post is password protected. To view it please enter your password below: