List all azure containers in Python

Standard
from azure.storage.blob import BlobService
 
blob_service=BlobService(account_name='x', account_key='x', )
 
marker=None
 
while True:
  containers = blob_service.list_containers(marker=marker)
  for c in containers: 
    print c.name
 
  if containers.next_marker:
    marker = containers.next_marker
  else:
    break

Sublime正则替换修改Hosts文件为MikroTik配置

Standard

感谢老D博客整理更新这些科学上网Hosts

我需把它们配到MikroTik路由器上时格式要改一下
说真的,每次用正则都要查一下才行,这东西真难记

MikroTik:
[admin@MikroTik] ip dns static> add address=10.0.0.1 name=www.example.com

Sublime:
Find What: (\d+.\d+.\d+.\d+)\s+([A-Za-z0-9-]+.[A-Za-z0-9-]+)
Replace With: add address=$1 name=$2

A ton of data to import with a shell

Standard

schema_import.sh

#!/bin/bash
for f in *schema*.gz; do
  DBName=$(echo $f | cut -d - -f 5)
  echo "Create "${DBName}
  echo "create database if not exists ${DBName}" | /usr/bin/mysql -u root -pXXXXXX
  zcat ${f} | /usr/bin/mysql -u root -pXXXXXX ${DBName}
  echo "Created"
done

data_import.sh

#!/bin/bash
for f in *data*.gz; do
  DBName=$(echo $f | cut -d - -f 5)
  echo "Import "${DBName}
  zcat ${f} | /usr/bin/mysql -u root -pXXXXXX ${DBName}
  echo "Imported"
done

一大坨文件

/nfs/dbs/31/mysql-slave-31.lololololol.com-schema-user_cluster_1-11-27-16.sql.gz
/nfs/dbs/31/mysql-slave-31.lololololol.com-schema-user_cluster_2-11-27-16.sql.gz
...
 
/nfs/dbs/31/mysql-slave-31.lololololol.com-data-user_cluster_1-11-27-16.sql.gz
/nfs/dbs/31/mysql-slave-31.lololololol.com-data-user_cluster_2-11-27-16.sql.gz
...

ps:
公司收了一家美国的论坛服务商,那边留了个人交接.
放了个NFS,上去一看,好家伙.每台机器目录下有几千个小的备份文件.
问其why,答long story…
好吧…好在文件命名还算规律。

dump.sh

MYSQL_PORT=$1
FILE_NAME=$2
MYSQL_IP=x.x.x.x
MYSQL_USER=uid
MYSQL_PASS=pwd
MYSQL_CONN="-u${MYSQL_USER} -p${MYSQL_PASS} -h${MYSQL_IP} -P${MYSQL_PORT}"
#
# Collect all database names except for
# mysql, information_schema, and performance_schema
#
SQL="SELECT schema_name FROM information_schema.schemata WHERE schema_name NOT IN"
SQL="${SQL} ('mysql','information_schema','performance_schema')"
 
DBLISTFILE=/tmp/DatabasesToDump-${FILE_NAME}.txt
mysql ${MYSQL_CONN} -ANe"${SQL}" > ${DBLISTFILE}
 
DBLIST=""
for DB in `cat ${DBLISTFILE}` ; do DBLIST="${DBLIST} ${DB}" ; done
 
MYSQLDUMP_OPTIONS="--routines --triggers --single-transaction"
mysqldump ${MYSQL_CONN} ${MYSQLDUMP_OPTIONS} --databases ${DBLIST} | pv | gzip > db-${FILE_NAME}.sql.gz
wget https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb && dpkg -i percona-release_0.1-4.$(lsb_release -sc)_all.deb && apt-get update && apt-get -y upgrade && apt-get -y install lvm2 xfsprogs percona-server-server-5.7
 
# add 2 x 128 ssd and 1 x 256 hdd
# ssd n enter enter enter enter t enter 8e enter w enter
fdisk /dev/sdc
fdisk /dev/sdd
 
# hdd
fdisk /dev/sde
 
pvcreate /dev/sdc1 /dev/sdd1 /dev/sde1 && vgcreate mysql /dev/sdc1 /dev/sdd1 && vgcreate data /dev/sde1 && lvcreate --name data --size 240G mysql && lvcreate --name backups --size 240G data
 
mkfs.xfs /dev/mapper/mysql-data && mkfs.xfs /dev/mapper/data-backups
 
edit /etc/fstab
/dev/mapper/mysql-data /var/lib/mysql auto  defaults,nobarrier 0 2
/dev/mapper/data-backups /data auto  defaults,nobarrier 0 2
 
service mysql stop && cd /var/lib && mv mysql mysql.old && mkdir mysql && mkdir /data && mount -a && mv mysql.old/* mysql/  && chown mysql:mysql mysql
 
vi /etc/security/limits.d/91-mysql.conf
mysql   soft    nofile 400000
mysql   hard    nofile 400000 
 
/etc/sysctl.conf
fs.file-max = 20000000
net.ipv4.tcp_fin_timeout = 10
kernel.pid_max = 65535
kernel.randomize_va_space = 1
net.core.netdev_max_backlog=32768
net.core.rmem_max = 8388608
net.core.somaxconn = 16384
net.core.wmem_max = 8388608
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.log_martians = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.ip_forward=0
net.ipv4.ip_local_port_range = 2000 65000
net.ipv4.tcp_max_syn_backlog=8192
net.ipv4.tcp_rmem = 4096 87380 8388608
net.ipv4.tcp_syncookies=0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_wmem = 4096 87380 8388608
vm.overcommit_memory = 1
vm.swappiness = 0
fs.aio-max-nr = 500000
 
sysctl -p
 
echo session required pam_limits.so >> /etc/pam.d/common-session
 
vi /etc/mysql/percona-server.conf.d/mysqld.cnf 
[mysqld]
user   = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket   = /var/run/mysqld/mysqld.sock
port   = 3306
basedir    = /usr
datadir    = /var/lib/mysql
tmpdir   = /tmp
lc-messages-dir  = /usr/share/mysql
explicit_defaults_for_timestamp
innodb_use_native_aio=1
max_connections=4096
wait_timeout=5
interactive_timeout=120
myisam_sort_buffer_size=1024M
sort_buffer_size=1024M
innodb_file_per_table=ON
skip-name-resolve
default-storage-engine=InnoDB
max_allowed_packet=64M
# 48 hours
expire_logs_days = 3
server-id              = 108
innodb_data_file_path = ibdata1:10M:autoextend
#innodb_buffer_pool_size = 28000M
innodb_flush_method = O_DIRECT
innodb_file_per_table
bind-address = 0.0.0.0
log-error    = /var/log/mysql/error.log
#log-erorr = /dev/null
log_error_verbosity=3
# Recommended in standard MySQL setup
#sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_ALL_TABLES
sql_mode=""
symbolic-links=0
 
# Forum requires this
lower_case_table_names  = 1
master-info-repository=TABLE
relay-log-info-repository=TABLE
replicate-do-table              = account.identity_data
replicate-do-table              = account.signature_data
replicate-do-table              = account.account_usertypes
replicate-do-table              = account.banned_words
replicate-wild-do-table         = forum\_skeleton.%
replicate-wild-do-table = domain\_%.%
replicate-wild-do-table = user\_cluster\_%.%
slave-skip-errors               = 1062
open-files-limit=400000
myisam-recover-options=FORCE,BACKUP

Installing SoftEther VPN Server

Standard
apt-get update;
apt-get install build-essential;
 
cd;
wget http://www.softether-download.com/files/softether/v4.21-9613-beta-2016.04.24-tree/Linux/SoftEther_VPN_Server/64bit_-_Intel_x64_or_AMD64/softether-vpnserver-v4.21-9613-beta-2016.04.24-linux-x64-64bit.tar.gz;
tar zxf softether-vpnserver-v4.21-9613-beta-2016.04.24-linux-x64-64bit.tar.gz;
 
cd vpnserver/;
make;
 
cd;
mv vpnserver /usr/local/;
 
vi /etc/init.d/vpnserver;
#!/bin/sh
DAEMON=/usr/local/vpnserver/vpnserver
LOCK=/var/lock/subsys/vpnserver
test -x $DAEMON || exit 0
case "$1" in
start)
$DAEMON start
touch $LOCK
;;
stop)
$DAEMON stop
rm $LOCK
;;
restart)
$DAEMON stop
sleep 3
$DAEMON start
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit 0
 
chmod +x /etc/init.d/vpnserver;
/etc/init.d/vpnserver start;

Jenkins安装配置

Standard

详细参考:http://jenkins-php.org

apt-get install default-jdk
wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -
sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
apt-get update
apt-get install jenkins
 
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin git phing checkstyle cloverphp crap4j dry htmlpublisher jdepend plot pmd violations warnings xunit
java -jar jenkins-cli.jar -s http://localhost:8080 safe-restart
 
apt-get install phpunit phpcodesniffer phploc pdepen phpmd phpcpd phpdox php5-xdebug libxml2-utils
1.
curl -L https://raw.githubusercontent.com/sebastianbergmann/php-jenkins-template/master/config.xml | java -jar jenkins-cli.jar -s http://localhost:8080 create-job php-template
 
Or add the template manually:
 
cd $JENKINS_HOME/jobs
mkdir php-template
cd php-template
wget https://raw.github.com/sebastianbergmann/php-jenkins-template/master/config.xml
cd ..
chown -R jenkins:jenkins php-template/
Reload Jenkins configuration, for instance using the Jenkins CLI:
java -jar jenkins-cli.jar -s http://localhost:8080 reload-configuration
 
2.Click on "New Job".
3.Enter a "Job name".
4.Select "Copy existing job" and enter "php-template" into the "Copy from" field.
5.Click "OK".
6.Uncheck the "Disable Build" option.
7.Fill in your "Source Code Management" information.
8.Configure a "Build Trigger", for instance "Poll SCM".
9.Click "Save".

Dynamic DynamoDB

Standard

http://dynamic-dynamodb.readthedocs.org/en/latest/cloudformation_template.html
https://aws.amazon.com/cn/blogs/aws/auto-scale-dynamodb-with-dynamic-dynamodb/

This example will configure Dynamic DynamoDB to:
- Scale up your DynamoDB table when the consumed reads 90% of the total provisioned reads
- Scale up your DynamoDB table when the consumed writes 90% of the total provisioned writes
- Scale up your reads with 50%
- Scale up your writes with 40%
- Scale down your DynamoDB table when the consumed reads 30% of the total provisioned reads
- Scale down your DynamoDB table when the consumed writes 40% of the total provisioned writes
- Scale down your reads with 40%
- Scale down your writes with 70%
- Check for changes every 5 minutes
 
Command:
dynamic-dynamodb --table-name my-table \
--reads-upper-threshold 90 \
--reads-lower-threshold 30 \
--increase-reads-with 50 \
--decrease-reads-with 40 \
--writes-upper-threshold 90 \
--writes-lower-threshold 40 \
--increase-writes-with 40 \
--decrease-writes-with 70 \
--check-interval 300

搞不懂AWS为何不在界面上自带着就把这东西做进去-_-