遇到一个奇怪的问题执行certbot会报错，moudle conflict和 No module，但是yum install certbot的时候没有报错。 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 certbot Traceback (most recent call last): File "/bin/certbot", line 7, in <module> from certbot.main import main File "/usr/lib/python2.7/site-packages/certbot/main.py", line 17, in <module> from certbot import account File "/usr/lib/python2.7/site-packages/certbot/account.py", line 17, in <module> from acme import messages File "/usr/lib/python2.7/site-packages/acme/messages.py", line 7, in <module> from acme import challenges File "/usr/lib/python2.7/site-packages/acme/challenges.py", line 11, in <module> import requests File "/usr/lib/python2.7/site-packages/requests/__init__.py", line 58, in <module> from . import utils File "/usr/lib/python2.7/site-packages/requests/utils.py", line 32, in <module> from .exceptions import InvalidURL File "/usr/lib/python2.7/site-packages/requests/exceptions.py", line 10, in <module> from .packages.urllib3.exceptions import HTTPError as BaseHTTPError File "/usr/lib/python2.7/site-packages/requests/packages/__init__.py", line 95, in load_module raise ImportError("No module named '%s'" % (name,)) ImportError: No module named 'requests.packages.urllib3' pip install requests urllib3 pyOpenSSL --force --upgrade certbot An unexpected error occurred: VersionConflict: (setuptools 0.9.8 (/usr/lib/python2.7/site-packages), Requirement.parse('setuptools>=1.0')) pip install --upgrade pip setuptools certbot ls meat 弄了很久决定抛弃yum直接使用 pip安装certbot，安装完成后，发现不再报错： ...

客户需要部署一套 moodle 教学系统。 去moodle官网大致看了一圈，发现moodle 是一个典型的PHP web应用。 其实这种LAMP (Linux, Apache, MySQL, PHP/Perl/Python)的应用， 我一般会用docker componse快速部署的，比如这个docker componse看上去就很不错。 但是客户不想用docker，要求直接在vm上部署。 初步确认部署环境为： nginx(let's encrypt) + php 7.2 + pg 10 + Centos 7.4 。 安装软件 初始化主机 1 2 3 4 5 6 7 8 9 10 hostnamectl set-hostname deoops.com # disable passwd login; use ssh-key only vi /etc/ssh/sshd_config yum update -y yum upgrade -y init 6 # add remi repo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm 安装nginx 1 2 3 4 5 6 7 8 9 10 11 yum install nginx yum -y install yum-utils yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional ## 安装let's encrypt certbot yum install certbot-nginx systemctl enable nginx systemctl start nginx ## 签发证书 certbot --nginx certonly ls -alh /etc/nginx/ 安装php dependency 1 yum --enablerepo=remi,remi-php72 install php-fpm php-common php-opcache php-pecl-apcu php-cli php-pear php-pdo php-mysqlnd php-pgsql php-pecl-mongodb php-pecl-redis php-pecl-memcache php-pecl-memcached php-gd php-mbstring php-mcrypt php-xml 配置nginx + php-fpm 详细的配置内容看这里 ...

工作需要定时备份postgresql slave数据库数据数据，服务器上运行了两个slave实例，隶属于两个不同的master。 备份 两个slave server实例分别监听在 5432和 4432端口 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 #!/bin/bash # # Daily PostgreSQL maintenance: vacuuming and backuping. # ## set -e for port in 5432 4432; do BACKDIR="/data/pg_back/$port" [ -d $BACKDIR ] || mkdir -p $BACKDIR echo "[`date`] begin Maintaining pg on port $port" # no need to use -U option for DB in $(psql -l -t -p $port |awk '{ print $1}' |grep -vE '^-|:|^List|^Name|template[0|1]|postgres|\|'); do ### swith form 'awk and grep' hacks to psql options and 'select sql' ### which is more dbaer professioner :) for DB in $(psql -AqXtc 'SELECT datname FROM pg_database WHERE datistemplate = false;'); do echo " [`date`] Maintaining $DB" PREFIX="$BACKDIR/$DB" # NO need to do `vacuum` on slaves # do `vacunm` on master instead # echo 'VACUUM' | psql -U postgres -hlocalhost -d $DB DUMP="$PREFIX.`date '+%Y%m%d'`.sql.gz" # no need for -U postgres option pg_dump -p $port $DB | gzip -c > $DUMP PREV="$PREFIX.`date -d'1 day ago' '+%Y%m%d'`.sql.gz" # md5sum -b $DUMP > $DUMP.md5 md5=($(md5sum -b $DUMP)) echo $md5 > $DUMP.md5 if [ -f $PREV.md5 ] && diff $PREV.md5 $DUMP.md5; then rm -f $PREV $PREV.md5 fi ## delete too old backup TOOOLD="$PREFIX.`date -d'15 day ago' '+%Y%m%d'`.sql.gz" [ -f $TOOOLD ] || rm -f $TOOOLD done echo "[`date`] Maintain pg on port $port finished" done 参考：Automatic Offsite PostgreSQL Backups Without a Password ...

一般postgres高可用集群是一个master配一个slave，但是开发这边需要做db的读写分离，所以运维这边 又添加了一台slave专门暴露出来做读操作。原来的slave还是只做备份。 HA 一主一从高可用的配置可以参考下面这篇文章 postgres streaming replication，有时间的话我可能会搬运一下 :) 安装配置 因为pg数据库集群已经配置好了一主一从，所以在master主机上不需要配置pg_hba.conf, 或者CREATE ROLE等等。 添加第二个slave需要注意以下两点： 等待pg_basebackupreplicas stream数据同步完成后，再启动 postgresql-9.6 service; 修改PG_DATA_DIR目录的权限; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 rm /etc/yum.repos.d/pgdg-96-redhat.repo yum install https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm yum install -y postgresql96 yum install -y postgresql96-server yum install -y postgresql96-contrib vi /usr/lib/systemd/system/postgresql-9.6.service mkdir /data/pg9.6 chown postgres:postgres /data/pg9.6/ ls -alh /data/pg9.6/ pg_basebackup --help ## you can add option --checkpoint=fast for an instance backup ## qhich is not recommend pg_basebackup -X stream -D /data/pg9.6/ -P -R -h 10.3.3.3 -U replicator ls /data/pg9.6/ cat /data/pg9.6/recovery.conf vi /data/pg9.6/postgresql.conf pwd systemctl start postgresql-9.6.service ls -alh /data/pg9.6/ chown -R postgres:postgres /data/pg9.6 chmod 700 /data/pg9.6 systemctl start postgresql-9.6.service netstat -nlp | grep 5432 su - postgres systemctl enable postgresql-9.6.service check 在master主机上查看pg_stat_replication表数据，验证第二个slave是否正常工作： ...

几天之前试用过了kong效果不理想 ，今天来使用下小米出品（存疑？）的 orange网关。 一个明显的区别是 kong的后端存储使用了postgresql，orange使用的是mysql。 好了，废话不多说，贴出安装部署的过程如下： 安装 mysql 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 #!/bin/bash wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm sudo yum update sudo yum install mysql-server sudo systemctl start mysqld sudo systemctl enable mysqld sudo mysql_secure_installation vi save_your_root_pwd git clone https://github.com/sumory/orange.git cd orange/ ls cd install/ ls head orange-v0.6.4.sql head -n 100 orange-v0.6.4.sql head -n 50 orange-v0.6.4.sql mysql -V mysql -u root -p cd orange/ cd install/ ls mysql -u o_usr -p o_database < orange-v0.6.4.sql mysql -u o_usr -p o_database orange 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 #!/bin/bash yum remove kong-community-edition ## get rid of annoying lua 5.1 version conflict cd /data/nginx/conf/ cp api.conf api.conf.bak.18.03.09 ## always backup conf files : ) nginx -s stop netstat -nlp | grep 443 mv /usr/sbin/nginx /usr/sbin/nginx_old yum install yum-utils yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo yum install openresty openresty-resty -y git clone https://github.com/sumory/orange.git git clone https://github.com/sumory/lor.git pwd mv lor ~ mv orange ~ cd ls cd lor/ make install cd ../orange/ make install cd ln -s /usr/local/bin/orange /bin/orange ln -s /usr/local/openresty/nginx/sbin/nginx /bin/nginx mv /home/deoops/orange.conf /home/deoops/nginx.conf /usr/local/orange/conf/ vi /usr/local/orange/conf/orange.conf ## make sure orange.conf has the right mysql server info vi /data/nginx/conf/api.conf chown root:root /usr/local/orange/conf/*.conf orange start netstat -nlp | grep 443 小结 用了几天，感觉UI比起kong来说简单些，开箱即用的功能比kong也多一些。 稳定性还有待进一步的观察。 ...

updated: kong不满足要求，后面调研了另外一个API产品 orange 2015年接触openresty的时候接触过kong，了解到kong是基于openresty二次开发的商业产品， 正好目前新公司要调研稳定好用的API Gateway产品，所以本文简单记录下我对kong的安装配置感受。 安装 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 yum install https://bintray.com/kong/kong-community-edition-rpm/download_file?file_path=centos/7/kong-community-edition-0.12.1.el7.noarch.rpm kong systemctl stop nginx netstat -nlp netstat -nlp|grep 80 ls date vi /etc/kong/kong.conf.default vipw su - postgres create user kong; create database kong ownner kong; create database kong owner kong; cd /etc/kong/ cp kong.conf.default kong.conf vi kong.conf kong migrations up psql -U kong tail /media/data/pgdata/log/postgresql-Wed.log vi /media/data/pgdata/pg_hba.conf systemctl restart postgresql-10.service kong migrations up kong kong check kong start netstat -nlp | grep 80 which nginx curl localhost: netstat -nlp vi /usr/local/kong/nginx-kong.conf which -a nginx kong kong restart --nginx-conf /etc/nginx/nginx.conf cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.tmpl vi /etc/nginx/nginx.conf.tmpl kong restart --nginx-conf /etc/nginx/nginx.conf.tmpl which -a npm 安装控制台 使用 ...

假设一个golang项目的三个源文件a.go,b.go, c.go，都定义了function inint(){}函数， 其中c.go文件初始化了一个全局变量globalVar，同时a.go 或者b.go的init func 引用了这个全局变量globalVar。 那么这个时候就会出现一个问题，在a.go和 b.go的init func中 globalVar的引用是空值。 示例 文件结构 1 2 3 4 5 6 7 8 9 ❯ tree . ├── a.go ├── b.go ├── c.go ├── go.mod └── main.go 0 directories, 5 files 源代码 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 // file `a.go` package main import "fmt" func init() { // globalVar is empty fmt.Println("globalVar in a.go:", globalVar) } // file `b.go` package main import "fmt" func init() { // globalVar is empty fmt.Println("globalVar in b.go:", globalVar) } // file `c.go` package main import ( "fmt" "time" ) func init() { globalVar = initVar() fmt.Println("globalVar in c.go:", globalVar) } func initVar() string { time.Sleep(20 * time.Millisecond) return "late is better than never" } // file `main.go` package main import "fmt" var globalVar = "" func main() { fmt.Println("vim-go") } result 1 2 3 4 5 6 ❯ go build -o demo ❯ ./demo globalVar in a.go: globalVar in b.go: globalVar in c.go: late is better than never vim-go 解决办法 简单的解决办法可以是重命名c.go为0a.go保证0a.go中的init最早执行完成， ...

换了份工作，新公司是做加密币交易所的，服务器都在国外。所以有机会接触到了微软的azure云服务， 服务器基本都在亚太区新加坡。 ps：这是我第一次实操单台配置32c64g的虚拟机，纪念一下。以前工作中最多就8c16g，数量的话两三百台机器。 history 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 uname -a # kernel info cat /etc/redhat-release df -alh # query disk info ifconfig ping 10.0.0.6 ssh 10.0.0.6 ls .ssh/ mv azagent .ssh/id_rsa ls -alh .ssh/id_rsa ssh 10.0.0.6 ping jd.com yum install yum-utils sudo yum install yum-utils yum -y upgrade sudo yum -y upgrade sudo yum -y update sudo yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo sudo yum install openresty # install web server sudo vi /etc/ssh/sshd_config systemctl status sshd systemctl restart sshd sudo systemctl restart sshd yum install ansible sudo yum install ansible sudo systemctl status openresty sudo systemctl enable openresty sudo systemctl start openresty curl -I localhost uptime date sudo install git sudo yum install git locate openresty rpm -qc openresty # query configuration file wget https://copr.fedorainfracloud.org/coprs/dheche/prometheus/repo/epel-7/dheche-prometheus-epel-7.repo ls head dheche-prometheus-epel-7.repo suod mv dheche-prometheus-epel-7.repo /etc/yum.repos.d/prometheus.repo sudo mv dheche-prometheus-epel-7.repo /etc/yum.repos.d/prometheus.repo sudo yum install prometheus sudo yum install prometheus-node sudo vi /etc/yum.repos.d/prometheus.repo sudo yum install prometheus sudo yum install prometheus2 sudo yum install node_exporter sudo yum install alertmanager vi /etc/hosts sudo vi /etc/hosts ssh ten sudo vi /etc/yum.conf ls /var/cache/yum/x86_64/7/prometheus/ ls /var/cache/yum/x86_64/7/prometheus/packages/ ls -alh /var/cache/yum/x86_64/7/prometheus/packages/ ls sudo yum install yum-utils history sudo yumdownloader prometheus ls ls -alh sudo yumdownloader prometheus2 # download installed rmp packages sudo yumdownloader node_exporter.x86_64 sudo yumdownloader alertmanager.x86_64 ls -alh scp node_exporter-0.15.2-1.el7.centos.x86_64.rpm ten: ssh ten ssh ten 'sudo systemctl enable node_exporter' ssh ten 'sudo systemctl start node_exporter' sudo systemctl start node_exporter sudo systemctl enable node_exporter sudo systemctl enable alertmanger sudo systemctl enable alertmanager sudo systemctl start alertmanager netstat -nlp sudo -i # su to root pwd ls rm prometheus-1.8.2-1.el7.centos.x86_64.rpm ls sudo systemctl enable prometheus sudo systemctl start prometheus cat/etc/passwd rpm -qc prometheus2 ls -alh /etc/default/prometheus id prometheus cat /etc/default/prometheus ls -alh /etc/prometheus/ sudo -i 总的说来敲了100来条指令，比较重要的是下面三条指令： ...

TLDR; find: 有很多过滤规则查找文件/目录/设备，而且可以递归查询某一个目录下的目录或文件，最后除了打印查询结果以外，还可以做其它操作（比如删除该文件）； locate: 则简单很多，根据关键字 在缓存index中 检索出含有该关键字的文件或者目录； fzf: 实时模糊查询，可以集成到常用的IDE中（比如fzf.vim)。 demo find The syntax of the Find command is: find [-H] [-L] [-P] [-D debugopts] [-Olevel] [starting-point...] [expression] 1 2 3 4 5 6 7 8 #!/bin/bash find . -name "*tar*gz" -delete find . -name "*tar*xz" -delete find . -name "*tar.xz" du -sh . find . -name "*zip*" -delete find . -type f | perl -lne 'print if -B' | xargs rm -f # delete all binary files under . recursivly locate The syntax of the Locate command is: ...

代理 (正向)代理 代理一般是指正向代理，比如翻墙软件shadowsocks就是一种正向代理。 shadowsocks通过socks 5协议 在代理服务器上， 代理我们（client）去访问被墙的资源（google/twitter/Facebook等服务器）。 A proxy server, sometimes referred to as a forward proxy, is a server that routes traffic between client(s) and another system, usually external to the network. By doing so, it can regulate traffic according to preset policies, convert and mask client IP addresses, enforce security protocols, and block unknown traffic. 反向代理 反向代理是说我们（client）被代理了， 我们自己还不知道。 我们以为和我们打交道的（处理我们的请求）的是nginx 服务器，其实nginx真正处理我们请求的是ngix后面的upstream在 处理我们的请求逻辑。 ...