MySQL 5.7.6でGTIDのローリング有効化ができるようになったので、システム全体を一度にシャットダウンしなくてもOK

March 10, 2015, 1:02 am

≫ Next: MySQL 5.7.6のPerformance SchemaでInnoDBのALTER TABLE進捗どうですか

≪ Previous: MySQL 5.7.6でCREATE USERせずにGRANTステートメントを叩くとワーニング

MySQL 5.7.6メモそのいくつか。
今までgtid-mode= ONとOFFのマスター, スレーブは混在できなかったので、ONにするときは一度レプリケーション群を全部止めて起動しなおさなければいけなかった。それが、出来るようになったという話。

MySQL Bugs: #71543: A new GTID_MODE is needed to evaluate/migrate to GTID: ANONYMOUS_IN-GTID_OUT.
MySQL :: WL#7083: GTIDS: set gtid_mode=ON online

Percona Server 5.6.22では一足先にリリースされてましたね。やってることは同じだけど実装が違うっぽい予感。
Online GTID rollout now available in Percona Server 5.6

[root@f51faa7d23c3 ~]# mysqld --verbose --help | less
..
  --gtid-mode=name    Controls whether Global Transaction Identifiers (GTIDs)
                      are enabled. Can be OFF, OFF_PERMISSIVE, ON_PERMISSIVE,
                      or ON. OFF means that no transaction has a GTID.
                      OFF_PERMISSIVE means that new transactions (committed in
                      a client session using GTID_NEXT='AUTOMATIC') are not
                      assigned any GTID, and replicated transactions are
                      allowed to have or not have a GTID. ON_PERMISSIVE means
                      that new transactions are assigned a GTID, and replicated
                      transactions are allowed to have or not have a GTID. ON
                      means that all transactions have a GTID. ON is required
                      on a master before any slave can use
                      MASTER_AUTO_POSITION=1. To safely switch from OFF to ON,
                      first set all servers to OFF_PERMISSIVE, then set all
                      servers to ON_PERMISSIVE, then wait for all transactions
                      without a GTID to be replicated and executed on all
                      servers, and finally set all servers to GTID_MODE = ON.
..

( ´-｀).oO（なんかmysqld --verbose --helpとドキュメントで設定できる値が違うぞ。。正解はOFF, OFF_PERMISSIVE, ON_PERMISSIVE, ONの4つでした。mysqldが正解。

まずはフツーにgtid_mode= OFFでレプリケーションを組む。
この時点から既に、マスターでもスレーブでもバイナリーログに@@gtid_next= 'ANNONYMOUS'が出力されている。

[root@2d38323d5eb5 ~]# mysqlbinlog /usr/local/mysql/data/bin.000003
..
# at 360
#150217 19:17:16 server id 161  end_log_pos 425 CRC32 0xc1a16911        Anonymous_GTID  last_committed=1        sequence_numbe
r=2
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 425
#150217 19:17:16 server id 161  end_log_pos 513 CRC32 0x3893ffcc        Query   thread_id=2     exec_time=0     error_code=0
SET TIMESTAMP=1424168236/*!*/;
create database d1
/*!*/;
..

[root@f51faa7d23c3 ~]# mysqlbinlog /usr/local/mysql/data/bin.000002
..
# at 360
#150217 19:17:16 server id 161  end_log_pos 425 CRC32 0x0f90114c        Anonymous_GTID  last_committed=1        sequence_numbe
r=2
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 425
#150217 19:17:16 server id 161  end_log_pos 513 CRC32 0x3893ffcc        Query   thread_id=2     exec_time=0     error_code=0
SET TIMESTAMP=1424168236/*!*/;
create database d1
/*!*/;
..

コイツが緩衝材の役割を果たしてくれるっぽい。スレーブ側でOFF_PERMISSIVEに変更。

mysql> SELECT @@gtid_mode;
+-------------+
| @@gtid_mode |
+-------------+
| OFF         |
+-------------+
1 row in set (0.00 sec)

mysql> SET GLOBAL gtid_mode= 'OFF_PERMISSIVE';
Query OK, 0 rows affected (0.05 sec)

mysql> SELECT @@gtid_mode;
+----------------+
| @@gtid_mode    |
+----------------+
| OFF_PERMISSIVE |
+----------------+
1 row in set (0.00 sec)

この時点では目だった変化はない(CHANGE MASTER TO master_auto_position= 1もできない)し、スレーブにクエリーを発行しても

[root@f51faa7d23c3 ~]# mysqlbinlog /usr/local/mysql/data/bin.000003
..
# at 426
#150217 19:24:49 server id 162  end_log_pos 491 CRC32 0xedeaa731        Anonymous_GTID  last_committed=1        sequence_number=2
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 491
#150217 19:24:49 server id 162  end_log_pos 603 CRC32 0xe284a3f5        Query   thread_id=6     exec_time=0     error_code=0
SET TIMESTAMP=1424168689/*!*/;
create database slave_only
..
/*!*/;

まだGTIDらしきものはバイナリーログに入っていない。

スレーブをON_PERMISSIVEに。

mysql> SET GLOBAL gtid_mode= 'ON_PERMISSIVE';
Query OK, 0 rows affected (0.03 sec)

mysql> SELECT @@gtid_mode;
+---------------+
| @@gtid_mode   |
+---------------+
| ON_PERMISSIVE |
+---------------+
1 row in set (0.00 sec)

mysql> DROP DATABASE slave_only;
Query OK, 0 rows affected (0.02 sec)

[root@f51faa7d23c3 ~]# mysqlbinlog /usr/local/mysql/data/bin.000004
..
# at 153
#150217 19:26:46 server id 162  end_log_pos 218 CRC32 0x9a581cd8        GTID    last_committed=0        sequence_number=1
SET @@SESSION.GTID_NEXT= '1e2c9249-b68b-11e4-ae86-0242ac1100a2:1'/*!*/;
# at 218
#150217 19:26:46 server id 162  end_log_pos 315 CRC32 0xb8794fef        Query   thread_id=11    exec_time=0     error_code=0
SET TIMESTAMP=1424168806/*!*/;
drop database slave_only
/*!*/;

スレーブで実行したクエリーにはGTIDが振られるようになった。マスターから流れてきたクエリーに対しては

mysql> INSERT INTO t1 VALUES (3, 'three');
Query OK, 1 row affected (0.01 sec)

[root@f51faa7d23c3 ~]# mysqlbinlog /usr/local/mysql/data/bin.000004
..
# at 1239
#150217 19:28:16 server id 161  end_log_pos 1304 CRC32 0x1fb41374       Anonymous_GTID  last_committed=5        sequence_number=6
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 1304
#150217 19:28:16 server id 161  end_log_pos 1379 CRC32 0x7dd37323       Query   thread_id=8     exec_time=0     error_code=0
SET TIMESTAMP=1424168896/*!*/;
BEGIN
/*!*/;
# at 1379
#150217 19:28:16 server id 161  end_log_pos 1483 CRC32 0x2e8e2a4a       Query   thread_id=8     exec_time=0     error_code=0
SET TIMESTAMP=1424168896/*!*/;
INSERT INTO t1 VALUES (3, 'three')
/*!*/;
..

[root@f51faa7d23c3 ~]# mysqlbinlog /usr/local/mysql/data/bin.000004
..
# at 315
#150217 19:28:16 server id 161  end_log_pos 380 CRC32 0xbed7303c        Anonymous_GTID  last_committed=1        sequence_number=2
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 380
#150217 19:28:16 server id 161  end_log_pos 455 CRC32 0x0dedb2e0        Query   thread_id=8     exec_time=0     error_code=0
SET TIMESTAMP=1424168896/*!*/;
BEGIN
/*!*/;
# at 455
#150217 19:28:16 server id 161  end_log_pos 559 CRC32 0x64c2139f        Query   thread_id=8     exec_time=0     error_code=0
use `d1`/*!*/;
SET TIMESTAMP=1424168896/*!*/;
INSERT INTO t1 VALUES (3, 'three')
/*!*/;
..

GTIDは振られない。うむうむ、いいんじゃないのこれ。
マスターも順番にOFF => OFF_PERMISSIVE => ON_PERMISSIVE => ONと続けてやると、

mysql> SET GLOBAL gtid_mode= OFF_PERMISSIVE;

# at 153
#150217 19:31:08 server id 161  end_log_pos 218 CRC32 0x0aec73f2        Anonymous_GTID  last_committed=0        sequence_numbe
r=1
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 218
#150217 19:31:08 server id 161  end_log_pos 342 CRC32 0xb46d3cb9        Query   thread_id=9     exec_time=0     error_code=0
SET TIMESTAMP=1424169068/*!*/;
create database off_permissive
/*!*/;

mysql> SET GLOBAL gtid_mode= ON_PERMISSIVE;

# at 153
#150217 19:31:15 server id 161  end_log_pos 218 CRC32 0x261d23fa        GTID    last_committed=0        sequence_number=1
SET @@SESSION.GTID_NEXT= '7fea486e-b687-11e4-ae6f-0242ac1100a1:1'/*!*/;
# at 218
#150217 19:31:15 server id 161  end_log_pos 339 CRC32 0x3ab38b3d        Query   thread_id=9     exec_time=0     error_code=0
SET TIMESTAMP=1424169075/*!*/;
create database on_permissive
/*!*/;

mysql> SET GLOBAL gtid_mode= ON;

# at 193
#150217 19:31:32 server id 161  end_log_pos 258 CRC32 0x175fe321        GTID    last_committed=0        sequence_number=1
SET @@SESSION.GTID_NEXT= '7fea486e-b687-11e4-ae6f-0242ac1100a1:2'/*!*/;
# at 258
#150217 19:31:32 server id 161  end_log_pos 394 CRC32 0xd9ff015c        Query   thread_id=9     exec_time=0     error_code=0
SET TIMESTAMP=1424169092/*!*/;
create database gtid_is_on_at_last
/*!*/;

master_auto_position= 1にするためにはマスター側のgtid_modeがONであることと、スレーブ側のgtid_modeがOFF_PERMISSIVE以上であることが必要。
あと、OFF <=> OFF_PERMISSIVE <=> ON_PERMISSIVE <=> ON以外のパスでgtid_modeを設定しようとすると、

ERROR 1788 (HY000): The value of @@GLOBAL.GTID_MODE can only be changed one step at a time: OFF <-> OFF_PERMISSIVE <-> ON_PERMISSIVE <-> ON. Also note that this value must be stepped up or down simultaneously on all servers. See the Manual for instructions.

といって怒られる。

割と柔軟性があるつくりなので、きっちり順々に上げなくてもなんとかなった。
(スレーブOFF_PERMISSIVE => ON_PERMISSIVE, マスター OFF_PERMISSIVE => ON_PERMISSIVE => ON, スレーブ ONとかやっても動き続ける)

これ是非MySQL 5.6にもバックポートしてほしいですね。
（5.6.23現在、@@global.gtid_modeはread_only variableなのでSET GLOBALで変更できない)

↧

MySQL 5.7.6のPerformance SchemaでInnoDBのALTER TABLE進捗どうですか

March 10, 2015, 1:02 am

≫ Next: MySQL 5.7.6のInnoDB日本語全文検索 MeCab Plugin

≪ Previous: MySQL 5.7.6でGTIDのローリング有効化ができるようになったので、システム全体を一度にシャットダウンしなくてもOK

MySQL 5.7.6で追加された新しいp_sのステージ情報から、 ~~合法的に~~ InnoDBに進捗どうですか？を聞けるようになったらしい。

MySQL :: MySQL 5.7 Reference Manual :: 14.13.11.1 Monitoring ALTER TABLE Progress for InnoDB Tables Using Performance Schema

setup_instrumentsでalter table関連のやつ(デフォルトOFF)と

mysql> SELECT * FROM setup_instruments WHERE name LIKE 'stage/innodb/alter%';
+------------------------------------------------------+---------+-------+
| NAME                                                 | ENABLED | TIMED |
+------------------------------------------------------+---------+-------+
| stage/innodb/alter table (end)                       | NO      | NO    |
| stage/innodb/alter table (flush)                     | NO      | NO    |
| stage/innodb/alter table (insert)                    | NO      | NO    |
| stage/innodb/alter table (log apply index)           | NO      | NO    |
| stage/innodb/alter table (log apply table)           | NO      | NO    |
| stage/innodb/alter table (merge sort)                | NO      | NO    |
| stage/innodb/alter table (read PK and internal sort) | NO      | NO    |
+------------------------------------------------------+---------+-------+
7 rows in set (0.11 sec)

setup_consumersのevents_stages_*もデフォルトはOFF

mysql> SELECT * FROM setup_consumers WHERE name LIKE '%stages%';
+----------------------------+---------+
| NAME                       | ENABLED |
+----------------------------+---------+
| events_stages_current      | NO      |
| events_stages_history      | NO      |
| events_stages_history_long | NO      |
+----------------------------+---------+
3 rows in set (0.08 sec)

これを

mysql> UPDATE setup_instruments SET enabled= 'YES' WHERE name LIKE 'stage/innodb/alter%';
Query OK, 7 rows affected (0.03 sec)
Rows matched: 7  Changed: 7  Warnings: 0

mysql> UPDATE setup_consumers SET enabled= 'YES' WHERE name LIKE '%stages%';
Query OK, 3 rows affected (0.03 sec)
Rows matched: 3  Changed: 3  Warnings: 0

こうじゃ！

mysql> ALTER TABLE order_line ADD KEY (ol_dist_info);
..

mysql> SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED FROM events_stages_current;
+------------------------------------------------------+----------------+----------------+
| EVENT_NAME                                           | WORK_COMPLETED | WORK_ESTIMATED |
+------------------------------------------------------+----------------+----------------+
| stage/innodb/alter table (read PK and internal sort) |          12084 |         268542 |
+------------------------------------------------------+----------------+----------------+
1 row in set (0.01 sec)

mysql> SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED FROM events_stages_current;
+------------------------------------------------------+----------------+----------------+
| EVENT_NAME                                           | WORK_COMPLETED | WORK_ESTIMATED |
+------------------------------------------------------+----------------+----------------+
| stage/innodb/alter table (read PK and internal sort) |          28060 |         268542 |
+------------------------------------------------------+----------------+----------------+
1 row in set (0.00 sec)

mysql> SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED FROM events_stages_current;
+------------------------------------------------------+----------------+----------------+
| EVENT_NAME                                           | WORK_COMPLETED | WORK_ESTIMATED |
+------------------------------------------------------+----------------+----------------+
| stage/innodb/alter table (read PK and internal sort) |          47972 |         268542 |
+------------------------------------------------------+----------------+----------------+
1 row in set (0.00 sec)

mysql> SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED FROM events_stages_current;
+---------------------------------------+----------------+----------------+
| EVENT_NAME                            | WORK_COMPLETED | WORK_ESTIMATED |
+---------------------------------------+----------------+----------------+
| stage/innodb/alter table (merge sort) |         138643 |         287346 |
+---------------------------------------+----------------+----------------+
1 row in set (0.01 sec)

mysql> SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED FROM events_stages_current;
+-----------------------------------+----------------+----------------+
| EVENT_NAME                        | WORK_COMPLETED | WORK_ESTIMATED |
+-----------------------------------+----------------+----------------+
| stage/innodb/alter table (insert) |         243821 |         287346 |
+-----------------------------------+----------------+----------------+
1 row in set (0.00 sec)

mysql> SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED FROM events_stages_current;
Empty set (0.00 sec)

終わるとevents_stages_currentからは消える。

The WORK_COMPLETED column shows the number of pages processed. The WORK_ESTIMATED column provides an estimate of the remaining work, in numbers of pages.

MySQL :: MySQL 5.7 Reference Manual :: 14.13.11.1 Monitoring ALTER TABLE Progress for InnoDB Tables Using Performance Schema

ということなので、work_completedとwork_estimatedはページ数らしい。

ちなみに、↑だとあまりにコピペでしかないのでちょっと気を利かせて終了予想時刻を計算するクエリー↓も書いたんですが

mysql> SELECT thread_id, event_name, sql_text, @progress:= (work_completed / work_estimated) * 100 AS progress, @elapsed:= (timer_current - timer_start) / power(10, 12) AS elapsed, @elapsed * (100 / @progress) - @elapsed AS estimated FROM (SELECT stage.thread_id, stage.event_name, work_completed, work_estimated, (SELECT timer_start FROM events_statements_current WHERE sql_text LIKE 'SELECT thread_id, event_name,%') AS timer_current, statement.timer_start, sql_text FROM events_stages_current AS stage JOIN events_statements_current AS statement USING(thread_id)) AS dummy;
+-----------+------------------------------------------------------+-------------------------------------+-------------+-------------+--------------------+
| thread_id | event_name                                           | sql_text                            | progress    | elapsed     | estimated          |
+-----------+------------------------------------------------------+-------------------------------------+-------------+-------------+--------------------+
|        28 | stage/innodb/alter table (read PK and internal sort) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 7.330386000 | 1.877416142 | 23.734006530694177 |
+-----------+------------------------------------------------------+-------------------------------------+-------------+-------------+--------------------+
1 row in set (0.15 sec)

mysql> SELECT thread_id, event_name, sql_text, @progress:= (work_completed / work_estimated) * 100 AS progress, @elapsed:= (timer_current - timer_start) / power(10, 12) AS elapsed, @elapsed * (100 / @progress) - @elapsed AS estimated FROM (SELECT stage.thread_id, stage.event_name, work_completed, work_estimated, (SELECT timer_start FROM events_statements_current WHERE sql_text LIKE 'SELECT thread_id, event_name,%') AS timer_current, statement.timer_start, sql_text FROM events_stages_current AS stage JOIN events_statements_current AS statement USING(thread_id)) AS dummy;
+-----------+------------------------------------------------------+-------------------------------------+--------------+--------------+------------------+
| thread_id | event_name                                           | sql_text                            | progress     | elapsed      | estimated        |
+-----------+------------------------------------------------------+-------------------------------------+--------------+--------------+------------------+
|        28 | stage/innodb/alter table (read PK and internal sort) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 46.969643300 | 33.385053874 | 37.6928839778295 |
+-----------+------------------------------------------------------+-------------------------------------+--------------+--------------+------------------+
1 row in set (0.01 sec)

mysql> SELECT thread_id, event_name, sql_text, @progress:= (work_completed / work_estimated) * 100 AS progress, @elapsed:= (timer_current - timer_start) / power(10, 12) AS elapsed, @elapsed * (100 / @progress) - @elapsed AS estimated FROM (SELECT stage.thread_id, stage.event_name, work_completed, work_estimated, (SELECT timer_start FROM events_statements_current WHERE sql_text LIKE 'SELECT thread_id, event_name,%') AS timer_current, statement.timer_start, sql_text FROM events_stages_current AS stage JOIN events_statements_current AS statement USING(thread_id)) AS dummy;
+-----------+---------------------------------------+-------------------------------------+--------------+--------------+--------------------+
| thread_id | event_name                            | sql_text                            | progress     | elapsed      | estimated          |
+-----------+---------------------------------------+-------------------------------------+--------------+--------------+--------------------+
|        28 | stage/innodb/alter table (merge sort) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 50.831565800 | 40.169081343 | 38.854810033960106 |
+-----------+---------------------------------------+-------------------------------------+--------------+--------------+--------------------+
1 row in set (0.00 sec)

mysql> SELECT thread_id, event_name, sql_text, @progress:= (work_completed / work_estimated) * 100 AS progress, @elapsed:= (timer_current - timer_start) / power(10, 12) AS elapsed, @elapsed * (100 / @progress) - @elapsed AS estimated FROM (SELECT stage.thread_id, stage.event_name, work_completed, work_estimated, (SELECT timer_start FROM events_statements_current WHERE sql_text LIKE 'SELECT thread_id, event_name,%') AS timer_current, statement.timer_start, sql_text FROM events_stages_current AS stage JOIN events_statements_current AS statement USING(thread_id)) AS dummy;
+-----------+-----------------------------------+-------------------------------------+--------------+--------------+--------------------+
| thread_id | event_name                        | sql_text                            | progress     | elapsed      | estimated          |
+-----------+-----------------------------------+-------------------------------------+--------------+--------------+--------------------+
|        28 | stage/innodb/alter table (insert) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 83.429283200 | 61.092267798 | 12.134140789914134 |
+-----------+-----------------------------------+-------------------------------------+--------------+--------------+--------------------+
1 row in set (0.00 sec)

だがしかしこれ、work_completedとwork_estimatedの値はALTER TABLE全体を通して全てのページ数を表現している(ので、途中でステージが変わってもprogressとして算出している値は常に進む)んだけど、各ステージごとの処理のスピードは違うので、そこまでアテにはならないかも知れない（が、それを言ったらSHOW ENGINE INNODB STATUSで見るのも似たようなもので。。）

体感ではステージがread PK and internal sort > merge sort >> insert >> その他くらいの順で時間を占めているので、そのあたりを加味すればありかな。。

↧

MySQL 5.7.6のInnoDB日本語全文検索 MeCab Plugin

March 10, 2015, 1:02 am

≫ Next: MySQL 5.7では暗黙のテンポラリーテーブルにもInnoDBが使われる

≪ Previous: MySQL 5.7.6のPerformance SchemaでInnoDBのALTER TABLE進捗どうですか

MySQL :: MySQL 5.7 Reference Manual :: 12.9.9 InnoDB MeCab Full-Text Parser Plugin の内容のおさらい。

まず、基本的なライブラリーと辞書は(この記事を書いている時点では).tar.gzバイナリーに同梱されているっぽいのでそちらを使う。Oracle公式のyumリポジトリーからインストールできるrpmには含まれていないように見えるので、その場合は別途突っ込まないといけないはずだけど、libpluginmecab.soが何かにダイナミックリンクしているわけではないので、辞書だけ取ってきてmecabrcに設定すればいけるような気がする。詳しく調べてない。

この環境はバイナリーの.tar.gzを取ってきて、/usr/local/mysqlに展開したとして、

$ ll /usr/local/mysql/lib/plugin/*mecab*
-rwxr-xr-x 1 root root 3988451 Feb 10 20:28 /usr/local/mysql/lib/plugin/libpluginmecab.so

$ ll -R /usr/local/mysql/lib/mecab
/usr/local/mysql/lib/mecab/:
total 8
drwxr-xr-x 5 root root 4096 Feb 17 10:54 dic
drwxr-xr-x 2 root root 4096 Feb 23 21:59 etc
..

plugin_dirにあたるlib/pluginにlibpluginmecab.soが、その他InnoDB MeCab Pluginに必要な辞書(dic)とか設定ファイル(etc)をおさめたディレクトリがlib/mecabにある。

続いてmy.cnfをゴニョる。

$ vim /etc/my.cnf
..
[mysqld]
loose-mecab-rc-file= /usr/local/mysql/lib/mecab/etc/mecabrc
innodb_ft_min_token_size= 1
..

mecab-rc-fileはlib/mecab/etc/mecabrcのパスを絶対パスで[mysqld]セクションに記述する。loose-接頭辞をつけておかないとMySQLが起動しなくなるので注意(INSTALL PLUGIN前にこのオプションを渡そうとすると、"unknown option"って言われてmysqldが起動してくれない)

参考: MySQL の unknown option エラーはオプションに loose- プレフィックスをつけると回避できる - かみぽわーる

innodb_ft_min_token_sizeはこのサイズより小さい文字列はトークンにしないというオプションだが、暗黙のデフォルトは3。英語で"a"とか"to"とかそういう頻出単語をトークナイズしないようにするためと書いてある。CJKでは1にセットしてね、とも。

my.cnfの次はmecabrc(↑のmy.cnfに記述したパスにあるもの)をゴニョる。

$ vim /usr/local/mysql/lib/mecab/etc/mecabrc
..
dicdir =  /usr/local/mysql/lib/mecab/dic/ipadic_utf-8

dicdirに、使いたい辞書の入っているディレクトリを指定する。

# ll lib/mecab/dic/
total 12
drwxr-xr-x 2 root root 4096 Feb 17 10:54 ipadic_euc-jp
drwxr-xr-x 2 root root 4096 Feb 17 10:54 ipadic_sjis
drwxr-xr-x 2 root root 4096 Feb 17 10:54 ipadic_utf-8

5.7.6現在、euc-jp, sjis, utf-8の3つが入ってる。5.7.7のリリースノートを見ると eucjpms, cp932, utf8mb4に対応したよ！と書いてあって、マニュアルのページには同じ辞書を使うよ、と書いてある。

この状態で起動してやると、

$ bin/mysqld_safe &
$ less data/error.log
..
2015-03-04T02:30:12.628925Z 0 [Warning] unknown variable 'loose-mecab-rc-file=/usr/local/mysql/lib/mecab/etc/mecabrc'
..

まだINSTALL PLUGINしてないので、unknown variableとして扱われる。

mysql> INSTALL PLUGIN mecab SONAME 'libpluginmecab.so';
Query OK, 0 rows affected (0.20 sec)

$ tail data/error.log
2015-03-04T03:21:46.236551Z 2 [Note] Mecab: Trying createModel(--rcfile=/usr/local/mysql/lib/mecab/etc/mecabrc)
2015-03-04T03:21:46.436600Z 2 [Note] Mecab: Loaded dictionary charset is utf-8

認識したぽい。

mysql> SHOW CREATE TABLE articles;
+----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table    | Create Table                                                                                                                                                                                                                                                                      |
+----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| articles | CREATE TABLE `articles` (
  `seq` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `title` text,
  `content` longtext,
  `timestamp` datetime DEFAULT NULL,
  UNIQUE KEY `seq` (`seq`),
  KEY `timestamp` (`timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=1914065 DEFAULT CHARSET=utf8 |
+----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

こんな感じのWikipediaのデータを食わせたテーブルに

mysql> ALTER TABLE articles ADD FULLTEXT KEY (title, content);
Query OK, 0 rows affected (1 hour 2 min 12.62 sec)
Records: 0  Duplicates: 0  Warnings: 0

$ tail data/error.log
2015-03-04T03:39:02.484153Z 0 [ERROR] Mecab:
2015-03-04T03:40:06.299126Z 0 [ERROR] Mecab:
2015-03-04T03:45:13.146195Z 0 [ERROR] Mecab:
2015-03-04T03:56:10.793337Z 0 [ERROR] Mecab:
2015-03-04T03:56:21.149414Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.385256Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.421553Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.470875Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.600808Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.632828Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.941143Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.969369Z 0 [ERROR] Mecab:
2015-03-04T03:59:34.986076Z 0 [ERROR] Mecab:
2015-03-04T03:59:35.056543Z 0 [ERROR] Mecab:
2015-03-04T03:59:35.352131Z 0 [ERROR] Mecab:
2015-03-04T03:59:36.754206Z 0 [ERROR] Mecab:

なんかダイイングメッセージみたいに不明なエラー吐いてるけど(Mroongaと比較した感じでは"too long sentence"エラーのはず)取り敢えず無視して、

mysql> SELECT COUNT(*) FROM articles WHERE match(title, content) against('データベース');
+----------+
| COUNT(*) |
+----------+
|     3013 |
+----------+
1 row in set (0.01 sec)

mysql> explain SELECT COUNT(*) FROM articles WHERE match(title, content) against ('データベース');
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+------------------------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                        |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+------------------------------+
|  1 | SIMPLE      | NULL  | NULL       | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | Select tables optimized away |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+------------------------------+
1 row in set, 1 warning (0.01 sec)

mysql> SHOW WARNINGS;
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                                                               |
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select count(0) AS `COUNT(*)` from `wikipedia`.`articles` where (match `wikipedia`.`articles`.`title`,`wikipedia`.`articles`.`content` against ('データベース'))       |
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.02 sec)

引けてるっぽい。

mysql> explain SELECT * FROM articles WHERE match(title, content) against ('データベース') LIMIT 10;
+----+-------------+----------+------------+----------+---------------+-------+---------+-------+------+----------+-------------------------------------------+
| id | select_type | table    | partitions | type     | possible_keys | key   | key_len | ref   | rows | filtered | Extra                                     |
+----+-------------+----------+------------+----------+---------------+-------+---------+-------+------+----------+-------------------------------------------+
|  1 | SIMPLE      | articles | NULL       | fulltext | title         | title | 0       | const |    1 |   100.00 | Using where; Ft_hints: sorted, limit = 10 |
+----+-------------+----------+------------+----------+---------------+-------+---------+-------+------+----------+-------------------------------------------+
1 row in set, 1 warning (0.02 sec)

mysql> SELECT * FROM articles WHERE match(title, content) against ('データベース') LIMIT 10;
..
10 rows in set (0.01 sec)

バッファプールに載ってて単一条件ならまあまあ動くんだけど

mysql> SELECT * FROM articles WHERE match(title, content) against ('データベース') ORDER BY timestamp DESC LIMIT 10;
..
10 rows in set (0.62 sec)

スコア以外のところでソートするとやっぱり死ねるねぇ。。

【2015/03/10 18:57】
Ngramの方も書きました => 日々の覚書: MySQL 5.7.6のInnoDB日本語全文検索 ngram

↧

MySQL 5.7では暗黙のテンポラリーテーブルにもInnoDBが使われる

March 10, 2015, 1:29 am

≫ Next: MySQL 5.7.6のInnoDB日本語全文検索 ngram

≪ Previous: MySQL 5.7.6のInnoDB日本語全文検索 MeCab Plugin

取り敢えずダミーデータを突っ込んだテーブルを自己結合しつつぐりぐりソートしてテンポラリーテーブルを作らせる。

$ perl -M"Digest::MD5 'md5_hex'" -e 'for ($n = 1; $n <= 1000000; $n++) { printf("%d\t%s\n", $n, md5_hex($n)); }'> /tmp/md5

mysql> create table t1 (num serial, val varchar(32));
Query OK, 0 rows affected (0.01 sec)

mysql> LOAD DATA INFILE '/tmp/md5' INTO TABLE t1;
Query OK, 1000000 rows affected (8.88 sec)
Records: 1000000  Deleted: 0  Skipped: 0  Warnings: 0

mysql> explain SELECT * FROM t1 LEFT JOIN t1 AS t2 USING(num) LEFT JOIN t1 AS t3 USING(num) ORDER BY t1.val ASC, t2.val DESC, t3.val ASC;
+----+-------------+-------+------------+--------+---------------+------+---------+-----------+--------+----------+---------------------------------+
| id | select_type | table | partitions | type   | possible_keys | key  | key_len | ref       | rows   | filtered | Extra                      |
+----+-------------+-------+------------+--------+---------------+------+---------+-----------+--------+----------+---------------------------------+
|  1 | SIMPLE      | t1    | NULL       | ALL    | NULL          | NULL | NULL    | NULL      | 996250 |   100.00 | Using temporary; Using filesort |
|  1 | SIMPLE      | t2    | NULL       | eq_ref | num           | num  | 8       | d1.t1.num |      1 |   100.00 | NULL                            |
|  1 | SIMPLE      | t3    | NULL       | eq_ref | num           | num  | 8       | d1.t1.num |      1 |   100.00 | NULL                      |
+----+-------------+-------+------------+--------+---------------+------+---------+-----------+--------+----------+---------------------------------+
3 rows in set, 1 warning (0.00 sec)

mysql> SHOW WARNINGS;
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                                                                                                                                                                                                                          |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select `d1`.`t1`.`num` AS `num`,`d1`.`t1`.`val` AS `val`,`d1`.`t2`.`val` AS `val`,`d1`.`t3`.`val` AS `val` from `d1`.`t1` left join `d1`.`t1` `t2` on((`d1`.`t1`.`num` = `d1`.`t2`.`num`)) left join `d1`.`t1` `t3` on((`d1`.`t1`.`num` = `d1`.`t3`.`num`)) where 1 order by `d1`.`t1`.`val`,`d1`.`t2`.`val` desc,`d1`.`t3`.`val` |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

5.7.6。

mysql> SELECT * FROM t1 LEFT JOIN t1 AS t2 USING(num) LEFT JOIN t1 AS t3 USING(num) ORDER BY t1.val ASC, t2.val DESC, t3.val ASC;
1000000 rows in set (38.72 sec)

# pt-ioprofile
Tue Mar 10 17:21:50 JST 2015
Tracing process ID 15352
     total      pread     pwrite      lseek filename
  0.092978   0.000000   0.092895   0.000083 /usr/local/mysql/data/ibtmp1
  0.000578   0.000578   0.000000   0.000000 /usr/local/mysql/data/d1/t1.ibd

ibtmp1はテンポラリーテーブル専用のテーブルスペースファイルで、REDOログを書かない(テンポラリーテーブルはクラッシュリカバリーされないため)

5.6.23。

mysql [localhost] {msandbox} (d1) > SELECT * FROM t1 LEFT JOIN t1 AS t2 USING(num) LEFT JOIN t1 AS t3 USING(num) ORDER BY t1.val ASC, t2.val DESC, t3.val ASC;
1000000 rows in set (24.80 sec)

# pt-ioprofile
Tue Mar 10 17:26:44 JST 2015
Tracing process ID 15829
     total      pread       read      write       open      close      lseek filename
  9.197881   8.124282   0.000000   1.009717   0.000035   0.063840   0.000007 /home/mysql/sandboxes/msb_5_6_23/tmp/MY7xramM
  6.187275   5.180143   0.000000   0.933478   0.000064   0.073582   0.000008 /home/mysql/sandboxes/msb_5_6_23/tmp/MYK6H2em
  1.000256   0.000000   0.129345   0.863924   0.006977   0.000000   0.000010 /home/mysql/sandboxes/msb_5_6_23/tmp/MYWMZd7b
  0.304381   0.000000   0.216208   0.088095   0.000035   0.000012   0.000031 /home/mysql/sandboxes/msb_5_6_23/tmp/#sql_3dd5_0.MYD
  0.019654   0.000000   0.000014   0.000084   0.019523   0.000020   0.000013 /home/mysql/sandboxes/msb_5_6_23/tmp/#sql_3dd5_0.MYI
  0.000195   0.000000   0.000030   0.000080   0.000039   0.000040   0.000006 /home/mysql/sandboxes/msb_5_6_23/tmp/MY2rane3

いつもどおり、MyISAMなテンポラリーテーブルを作ってる。

↧

MySQL 5.7.6のInnoDB日本語全文検索 ngram

March 10, 2015, 2:56 am

≫ Next: MySQL 5.7.6でmysql_install_db(とmysqld --initialize)がSSL証明書のファイルを作っているぽいのであとで調べたい

≪ Previous: MySQL 5.7では暗黙のテンポラリーテーブルにもInnoDBが使われる

日々の覚書: プレビュー "MySQL 5.7.6のInnoDB日本語全文検索 MeCab Plugin" に引き続き、今度はngramなトークナイザー。ドキュメントはここ。

MeCab Pluginと違って特にINSTALL PLUGINとかせずに即使える。

mysql> ALTER TABLE articles ADD FULLTEXT KEY (title, content) WITH PARSER ngram;
Query OK, 0 rows affected (6 hours 25 min 20.67 sec)
Records: 0  Duplicates: 0  Warnings: 0

( д ) ﾟﾟ 6時間半！？しょうがないとはいえ、MeCabのときは1時間半でお釣りが来てたのにと思うとちょっとしょんぼり。

時間もさることながら、このやり方(データをインポートしてからALTER TABLEでインデックス追加)だと容量がかなり持っていかれる。

[root@v157-7-235-254 mysql]# while true ; do
> date ; df -h
> echo ""
> sleep 60
> done
Tue Mar 10 10:13:41 JST 2015
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       97G   16G   76G  18% /
tmpfs                 499M     0  499M   0% /dev/shm
/dev/vda1             485M   32M  428M   7% /boot
..
Tue Mar 10 16:12:45 JST 2015
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       97G   74G   18G  81% /
tmpfs                 499M     0  499M   0% /dev/shm
/dev/vda1             485M   32M  428M   7% /boot
..
Tue Mar 10 16:44:51 JST 2015
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       97G   26G   66G  28% /
tmpfs                 499M     0  499M   0% /dev/shm
/dev/vda1             485M   32M  428M   7% /boot


# lsof -p `pidof mysqld`
COMMAND  PID  USER   FD   TYPE             DEVICE    SIZE/OFF    NODE NAME
mysqld  3049 mysql  cwd    DIR              253,0        4096  919114 /usr/local/mysql/data
mysqld  3049 mysql  rtd    DIR              253,0        4096       2 /
mysqld  3049 mysql  txt    REG              253,0   193624979  919090 /usr/local/mysql/bin/mysqld
mysqld  3049 mysql  mem    REG              253,0        5624  914553 /lib64/libaio.so.1.0.1
mysqld  3049 mysql  DEL    REG              253,0              914859 /lib64/libfreebl3.so
mysqld  3049 mysql  mem    REG              253,0    49199027 2231548 /usr/local/mysql/lib/mecab/dic/ipadic_utf-8/sys.dic
mysqld  3049 mysql  mem    REG              253,0     3463716 2231541 /usr/local/mysql/lib/mecab/dic/ipadic_utf-8/matrix.bin
mysqld  3049 mysql  mem    REG              253,0      262496 2231546 /usr/local/mysql/lib/mecab/dic/ipadic_utf-8/char.bin
mysqld  3049 mysql  mem    REG              253,0     3988451 2231491 /usr/local/mysql/lib/plugin/libpluginmecab.so
mysqld  3049 mysql  mem    REG              253,0       65928  914006 /lib64/libnss_files-2.12.so
mysqld  3049 mysql  DEL    REG              253,0              913936 /lib64/libc-2.12.so
mysqld  3049 mysql  DEL    REG              253,0              915115 /lib64/libgcc_s-4.4.7-20120601.so.1.#prelink#.JjQM5q
mysqld  3049 mysql  DEL    REG              253,0              913990 /lib64/libm-2.12.so
mysqld  3049 mysql  DEL    REG              253,0              263139 /usr/lib64/libstdc++.so.6.0.13.#prelink#.htv5XJ
mysqld  3049 mysql  DEL    REG              253,0              914081 /lib64/librt-2.12.so
mysqld  3049 mysql  DEL    REG              253,0              913976 /lib64/libdl-2.12.so
mysqld  3049 mysql  DEL    REG              253,0              913940 /lib64/libcrypt-2.12.so.#prelink#.3QGWZ8
mysqld  3049 mysql  DEL    REG              253,0              913960 /lib64/libpthread-2.12.so.#prelink#.xdy3QH
mysqld  3049 mysql  DEL    REG              253,0              913930 /lib64/ld-2.12.so
mysqld  3049 mysql  mem    REG              253,0        5690 2231545 /usr/local/mysql/lib/mecab/dic/ipadic_utf-8/unk.dic
mysqld  3049 mysql    0r   CHR                1,3         0t0    3788 /dev/null
mysqld  3049 mysql    1w   REG              253,0       15868  919127 /usr/local/mysql/data/error.log
mysqld  3049 mysql    2w   REG              253,0       15868  919127 /usr/local/mysql/data/error.log
mysqld  3049 mysql    3uW  REG              253,0    79691776  919158 /usr/local/mysql/data/ibdata1
..
mysqld  3049 mysql   45uW  REG              253,0 11286872064 1305615 /usr/local/mysql/data/wikipedia/articles.ibd
mysqld  3049 mysql   46uW  REG              253,0       98304 1305604 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_000000000000013a_INDEX_1.ibd
mysqld  3049 mysql   47uW  REG              253,0       98304 1305607 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_000000000000013a_INDEX_2.ibd
mysqld  3049 mysql   48uW  REG              253,0       98304 1305611 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_000000000000013a_INDEX_3.ibd
mysqld  3049 mysql   49uW  REG              253,0       98304 1305613 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_000000000000013a_INDEX_4.ibd
mysqld  3049 mysql   50uW  REG              253,0       98304 1305614 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_000000000000013a_INDEX_5.ibd
mysqld  3049 mysql   51uW  REG              253,0       98304 1305616 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_000000000000013a_INDEX_6.ibd
mysqld  3049 mysql   52uW  REG              253,0       98304 1305603 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_DELETED.ibd
mysqld  3049 mysql   53uW  REG              253,0       98304 1305605 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_DELETED_CACHE.ibd
mysqld  3049 mysql   54uW  REG              253,0       98304 1305606 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_BEING_DELETED.ibd
mysqld  3049 mysql   55uW  REG              253,0       98304 1305610 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_BEING_DELETED_CACHE.ibd
mysqld  3049 mysql   56uW  REG              253,0       98304 1305612 /usr/local/mysql/data/wikipedia/FTS_00000000000000f7_CONFIG.ibd
mysqld  3049 mysql   58u   REG              253,0 10842275840 1046745 /data/tmp/ibDDeVbj (deleted)
mysqld  3049 mysql   59u   REG              253,0   696254464 1046746 /data/tmp/ibaNLTUS (deleted)
mysqld  3049 mysql   60u   REG              253,0  1790967808 1046747 /data/tmp/ibNm7Vmi (deleted)
mysqld  3049 mysql   61u   REG              253,0  1623195648 1046748 /data/tmp/ibl17OJP (deleted)
mysqld  3049 mysql   62u   REG              253,0  8006926336 1046749 /data/tmp/ibBoSvqQ (deleted)
mysqld  3049 mysql   63u   REG              253,0  8006926336 1046750 /data/tmp/ibudnROb (deleted)
mysqld  3049 mysql   64u   REG              253,0 10904141824 1046751 /data/tmp/ibNbFRxL (deleted)
mysqld  3049 mysql   65u   REG              253,0   698351616 1046752 /data/tmp/ibYchSgl (deleted)
mysqld  3049 mysql   66u   REG              253,0  1802502144 1046753 /data/tmp/ibYYJGHW (deleted)
mysqld  3049 mysql   67u   REG              253,0  1631584256 1046754 /data/tmp/ibQOsAMA (deleted)
mysqld  3049 mysql   68u   REG              253,0  8050966528 1046755 /data/tmp/ibiSAO6i (deleted)
mysqld  3049 mysql   69u   REG              253,0  8050966528 1046756 /data/tmp/ibGrGYaE (deleted)
mysqld  3049 mysql   70u   REG              253,0   361758720 1046757 /data/tmp/ibQvfudp (deleted)
mysqld  3049 mysql   71u   REG              253,0   359661568 1046758 /data/tmp/ibN0BTAJ (deleted)

最終的に元に戻るとはいえ、tmpdirに数GB単位のテンポラリーファイルをいくつもつくられてしかもそれが途中で解放されないのは結構つらい。。

まあそこはそれとして

mysql> CREATE TABLE t1 (val varchar(32));
Query OK, 0 rows affected (0.26 sec)

mysql> INSERT INTO t1 VALUES ('にっこにっこにー');
Query OK, 1 row affected (0.08 sec)

mysql> SELECT * FROM t1;
+--------------------------+
| val                      |
+--------------------------+
| にっこにっこにー         |
+--------------------------+
1 row in set (0.00 sec)

こんなサンプルがあるじゃろ？

mysql> SELECT @@ngram_token_size;
+--------------------+
| @@ngram_token_size |
+--------------------+
|                  2 |
+--------------------+
1 row in set (0.00 sec)

mysql> ALTER TABLE t1 ADD FULLTEXT KEY idx_2gram(val) WITH PARSER ngram;
Query OK, 0 rows affected (0.44 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> SET GLOBAL innodb_ft_aux_table = 'd1/t1';
Query OK, 0 rows affected (0.00 sec)

mysql> OPTIMIZE TABLE t1;
+-------+----------+----------+-------------------------------------------------------------------+
| Table | Op       | Msg_type | Msg_text                                                          |
+-------+----------+----------+-------------------------------------------------------------------+
| d1.t1 | optimize | note     | Table does not support optimize, doing recreate + analyze instead |
| d1.t1 | optimize | status   | OK                                                                |
+-------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.36 sec)

mysql> SELECT * FROM information_schema.innodb_ft_index_cache ORDER BY position;
+--------+--------------+-------------+-----------+--------+----------+
| WORD   | FIRST_DOC_ID | LAST_DOC_ID | DOC_COUNT | DOC_ID | POSITION |
+--------+--------------+-------------+-----------+--------+----------+
| にっ   |            2 |           2 |         1 |      2 |        0 |
| っこ   |            2 |           2 |         1 |      2 |        3 |
| こに   |            2 |           2 |         1 |      2 |        6 |
| っこ   |            2 |           2 |         1 |      2 |        9 |
| にっ   |            2 |           2 |         1 |      2 |        9 |
| こに   |            2 |           2 |         1 |      2 |        9 |
| にー   |            2 |           2 |         1 |      2 |       18 |
+--------+--------------+-------------+-----------+--------+----------+
7 rows in set (0.02 sec)

mysql> SELECT * FROM information_schema.innodb_ft_config;
+---------------------------+-------+
| KEY                       | VALUE |
+---------------------------+-------+
| optimize_checkpoint_limit | 180   |
| synced_doc_id             | 0     |
| stopword_table_name       |       |
| use_stopword              | 1     |
+---------------------------+-------+
4 rows in set (0.03 sec)

ngram_token_size変数でトークンのサイズを制御できる。この変数はオンライン変更できないので、my.cnfに設定してからmysqld再起動。

mysql> SET GLOBAL innodb_ft_aux_table = 'd1/t1';
Query OK, 0 rows affected (0.00 sec)

mysql> OPTIMIZE TABLE t1;
+-------+----------+----------+-------------------------------------------------------------------+
| Table | Op       | Msg_type | Msg_text                                                          |
+-------+----------+----------+-------------------------------------------------------------------+
| d1.t1 | optimize | note     | Table does not support optimize, doing recreate + analyze instead |
| d1.t1 | optimize | status   | OK                                                                |
+-------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.31 sec)

mysql> SELECT * FROM information_schema.innodb_ft_index_cache ORDER BY position;
+-----------+--------------+-------------+-----------+--------+----------+
| WORD      | FIRST_DOC_ID | LAST_DOC_ID | DOC_COUNT | DOC_ID | POSITION |
+-----------+--------------+-------------+-----------+--------+----------+
| にっこ    |            2 |           2 |         1 |      2 |        0 |
| っこに    |            2 |           2 |         1 |      2 |        3 |
| こにっ    |            2 |           2 |         1 |      2 |        6 |
| にっこ    |            2 |           2 |         1 |      2 |        9 |
| っこに    |            2 |           2 |         1 |      2 |        9 |
| こにー    |            2 |           2 |         1 |      2 |       15 |
+-----------+--------------+-------------+-----------+--------+----------+
6 rows in set (0.00 sec)

mysql> SELECT * FROM information_schema.innodb_ft_config;
+---------------------------+-------+
| KEY                       | VALUE |
+---------------------------+-------+
| optimize_checkpoint_limit | 180   |
| synced_doc_id             | 0     |
| stopword_table_name       |       |
| use_stopword              | 1     |
+---------------------------+-------+
4 rows in set (0.01 sec)

あ、あれ？再作成しなくても切りなおされるの？；

↧

MySQL 5.7.6でmysql_install_db(とmysqld --initialize)がSSL証明書のファイルを作っているぽいのであとで調べたい

March 11, 2015, 7:31 pm

≫ Next: MySQL 5.7.6からSSL周りのセットアップが扱いやすくなった

≪ Previous: MySQL 5.7.6のInnoDB日本語全文検索 ngram

MySQL 5.7.5のmysql_install_dbだと

$ bin/mysql_install_db --no-defaults --basedir=./ --datadir=./test_575 -v
2015-03-12 11:23:45 [NOTE]    Creating data directory ./test_575
2015-03-12 11:23:45 [NOTE]    Generating random password to /root/.mysql_secret...done.
2015-03-12 11:23:45 [NOTE]    Executing /opt/mysql/5.7.5/bin/mysqld --no-defaults --bootstrap --datadir=./test_575 --lc-messages-dir=./share --lc-messages=en_US --basedir=.
2015-03-12 11:23:46 [NOTE]    Creating system tables...done.
2015-03-12 11:23:46 [NOTE]    Filling system tables with data...done.
2015-03-12 11:23:47 [NOTE]    Filling help table with data...done.
2015-03-12 11:23:47 [NOTE]    Creating default user root@localhost
2015-03-12 11:23:47 [NOTE]    Creating default proxy root@localhost
2015-03-12 11:23:48 [NOTE]    Success!

.mysql_secretを作ってシステムテーブルを作ってroot@localhostを作るだけだったのが、5.7.6のmysql_install_dbでは

$ bin/mysql_install_db --no-defaults --basedir=./ --datadir=./test_mysql_install_db -v
2015-03-12 11:25:04 [WARNING] mysql_install_db is deprecated. Please consider switching to mysqld --initialize
2015-03-12 11:25:04 [NOTE]    Creating data directory ./test_mysql_install_db
2015-03-12 11:25:04 [NOTE]    Generating random password to /home/ttanaka/.mysql_secret...done.
2015-03-12 11:25:04 [NOTE]    Executing /usr/mysql/5.7.6/bin/mysqld --no-defaults --bootstrap --datadir=./test_mysql_install_db --lc-messages-dir=./share --lc-messages=en_US --basedir=.
2015-03-12 11:25:06 [NOTE]    Creating system tables...done.
2015-03-12 11:25:06 [NOTE]    Filling system tables with data...done.
2015-03-12 11:25:07 [NOTE]    Filling help table with data...done.
2015-03-12 11:25:07 [NOTE]    Creating default user root@localhost
2015-03-12 11:25:07 [NOTE]    Creating default proxy root@localhost
2015-03-12 11:25:07 [WARNING] The bootstrap log isn't empty:
2015-03-12 11:25:07 [WARNING] mysqld:
2015-03-12 11:25:07 [NOTE]    Generating SSL Certificates
OpenSSL 1.0.1e-fips 11 Feb 2013
server-cert.pem: OK
client-cert.pem: OK

$ ll test_mysql_install_db
合計 110640
-rw------- 1 mysql mysql     1679  3月 12 11:25 2015 ca-key.pem
-rw-r--r-- 1 mysql mysql     1082  3月 12 11:25 2015 ca.pem
-rw-r--r-- 1 mysql mysql     1086  3月 12 11:25 2015 client-cert.pem
-rw------- 1 mysql mysql     1679  3月 12 11:25 2015 client-key.pem
-rw-r----- 1 mysql mysql 50331648  3月 12 11:25 2015 ib_logfile0
-rw-r----- 1 mysql mysql 50331648  3月 12 11:25 2015 ib_logfile1
-rw-r----- 1 mysql mysql 12582912  3月 12 11:25 2015 ibdata1
drwxr-x--- 2 mysql mysql     4096  3月 12 11:25 2015 mysql
drwxr-x--- 2 mysql mysql     4096  3月 12 11:25 2015 performance_schema
-rw------- 1 mysql mysql     1675  3月 12 11:25 2015 private_key.pem
-rw-r--r-- 1 mysql mysql      451  3月 12 11:25 2015 public_key.pem
-rw-r--r-- 1 mysql mysql     1086  3月 12 11:25 2015 server-cert.pem
-rw------- 1 mysql mysql     1679  3月 12 11:25 2015 server-key.pem

そのあとにサーバー用, クライアント用のSSL証明書を作っているぽい。
( ´-｀).oO（mysqld: で止まってるワーニングメッセージがあるんだけど、これ MeCabのやつと一緒でOpenSSLのライブラリーから戻ってきた何かをうまくハンドルできてないんじゃないか。。

そして5.7.6のmysqld --initialize。

$ bin/mysqld --no-defaults --initialize-insecure --datadir=./test_mysqld --log-error-verbosity=3
2015-03-12T02:25:54.901505Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-03-12T02:25:54.901602Z 0 [Note] Ignoring --secure-file-priv value as server is running with --initialize(-insecure) or --bootstrap.
2015-03-12T02:25:54.901846Z 0 [Note] bin/mysqld (mysqld 5.7.6-m16) starting as process 23958 ...
2015-03-12T02:25:54.938092Z 0 [Note] Creating the data directory /usr/mysql/5.7.6/test_mysqld/
..
2015-03-12T02:25:56.202015Z 0 [Note] Plugin 'FEDERATED' is disabled.
2015-03-12T02:25:56.202631Z 0 [Warning] Failed to setup SSL
2015-03-12T02:25:56.202643Z 0 [Warning] SSL error: SSL context is not usable without certificate and private key
2015-03-12T02:25:56.203497Z 1 [Note] Creating the system database
2015-03-12T02:25:56.203521Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2015-03-12T02:25:56.204099Z 1 [Note] Creating the system tables
2015-03-12T02:25:57.234370Z 1 [Note] Filling in the system tables, part 1
2015-03-12T02:25:57.235332Z 1 [Note] Filling in the system tables, part 2
2015-03-12T02:25:57.235347Z 1 [Note] Filling in the mysql.help table
2015-03-12T02:25:57.969635Z 1 [Note] Bootstrapping complete
2015-03-12T02:25:57.969924Z 0 [Note] Giving 0 client threads a chance to die gracefully
2015-03-12T02:25:57.969945Z 0 [Note] Shutting down slave threads
2015-03-12T02:25:57.969955Z 0 [Note] Forcefully disconnecting 0 remaining clients
2015-03-12T02:25:57.970165Z 0 [Note] Binlog end
2015-03-12T02:25:57.970630Z 0 [Note] InnoDB: FTS optimize thread exiting.
2015-03-12T02:25:57.970950Z 0 [Note] InnoDB: Starting shutdown...
2015-03-12T02:25:59.100867Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2321861
2015-03-12T02:25:59.104330Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"

$ ll test_mysqld/
合計 110604
-rw-r----- 1 mysql mysql 50331648  3月 12 11:25 2015 ib_logfile0
-rw-r----- 1 mysql mysql 50331648  3月 12 11:25 2015 ib_logfile1
-rw-r----- 1 mysql mysql 12582912  3月 12 11:25 2015 ibdata1
drwxr-x--- 2 mysql mysql     4096  3月 12 11:25 2015 mysql
drwxr-x--- 2 mysql mysql     4096  3月 12 11:25 2015 performance_schema

同じくなんかSSL証明書を作ろうとしてるけど失敗してる。全く同じ動作ってわけではないぽい。ちょっと調べようか。

【2015/03/13 13:36】
調べてみたところ、作ろうとしているのではなく、使おうとして(証明書がないので)失敗しているのであった。
日々の覚書: MySQL 5.7.6からSSL周りのセットアップが扱いやすくなった

↧

MySQL 5.7.6からSSL周りのセットアップが扱いやすくなった

March 12, 2015, 9:35 pm

≫ Next: MySQL 5.7でLOAD DATA INFILEに失敗する時に疑うこと(--secure-file-privの暗黙のデフォルトが少し変わった)

≪ Previous: MySQL 5.7.6でmysql_install_db(とmysqld --initialize)がSSL証明書のファイルを作っているぽいのであとで調べたい

MySQL :: MySQL 5.7 Release Notes :: Changes in MySQL 5.7.6 (2015-03-09, Milestone 16)から抜粋。

MySQL Server from Community Edition distributions now tries to deploy with SSL support enabled automatically if no SSL options are specified explicitly and it finds any of the ca.pem, server-cert.pem, and server-key.pem files in the data directory. In this case, clients can use a secure connection merely by specifying --ssl on the command line.

今までは--ssl指定するとついでに--ssl-ca, --ssl-cert, --ssl-keyを指定してやらないといけなかったのが、datadirからその辺のファイルが見つかれば指定しなくてもよしなにやってくれるらしい。

あと、サーバー側の--sslはデフォルトで有効状態に。

というわけで、前回 mysqld --initializeがSSL証明書を作ろうとして失敗したと思っていたものは、実はSSLを有効化しようとしていた(そして、証明書がdatadirにないので失敗していた)だけだった。

mysqld --initializeで作ったdatadirで起動すると、*.pemファイルはないので、

$ bin/mysqld --no-defaults --initialize-insecure
..

$ ll data/*.pem
ls: cannot access data/*.pem: No such file or directory

$ bin/mysqld_safe --no-defaults

mysql> SHOW GLOBAL VARIABLES LIKE '%ssl%';
+---------------+----------+
| Variable_name | Value    |
+---------------+----------+
| have_openssl  | DISABLED |
| have_ssl      | DISABLED |
| ssl_ca        |          |
| ssl_capath    |          |
| ssl_cert      |          |
| ssl_cipher    |          |
| ssl_crl       |          |
| ssl_crlpath   |          |
| ssl_key       |          |
+---------------+----------+
9 rows in set (0.00 sec)

$ bin/mysql --no-defaults -uroot --ssl
ERROR 2026 (HY000): SSL connection error: SSL is required but the server doesn't support it

使えない。

mysql_install_dbで作ったdatadirには*.pemがあるので(ただし、mysql_install_db --insecureだと作られない)

$ ./bin/mysql_install_db --no-defaults --basedir=./ --datadir=./data
..

$ ll data/*.pem
-rw------- 1 mysql mysql 1679 Mar 13 13:27 data/ca-key.pem
-rw-r--r-- 1 mysql mysql 1082 Mar 13 13:27 data/ca.pem
-rw-r--r-- 1 mysql mysql 1086 Mar 13 13:27 data/client-cert.pem
-rw------- 1 mysql mysql 1679 Mar 13 13:27 data/client-key.pem
-rw------- 1 mysql mysql 1675 Mar 13 13:27 data/private_key.pem
-rw-r--r-- 1 mysql mysql  451 Mar 13 13:27 data/public_key.pem
-rw-r--r-- 1 mysql mysql 1086 Mar 13 13:27 data/server-cert.pem
-rw------- 1 mysql mysql 1679 Mar 13 13:27 data/server-key.pem

$ bin/mysqld_safe --no-defaults

mysql> SHOW GLOBAL VARIABLES LIKE '%ssl%';
+---------------+-----------------+
| Variable_name | Value           |
+---------------+-----------------+
| have_openssl  | YES             |
| have_ssl      | YES             |
| ssl_ca        | ca.pem          |
| ssl_capath    |                 |
| ssl_cert      | server-cert.pem |
| ssl_cipher    |                 |
| ssl_crl       |                 |
| ssl_crlpath   |                 |
| ssl_key       | server-key.pem  |
+---------------+-----------------+
9 rows in set (0.00 sec)

$ bin/mysql --no-defaults -uroot --ssl
mysql> status
--------------
bin/mysql  Ver 14.14 Distrib 5.7.6-m16, for Linux (x86_64) using  EditLine wrapper

Connection id:          4
Current database:
Current user:           root@localhost
SSL:                    Cipher in use is DHE-RSA-AES256-SHA
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         5.7.6-m16 Source distribution
Protocol version:       10
Connection:             Localhost via UNIX socket
Server characterset:    latin1
Db     characterset:    latin1
Client characterset:    latin1
Conn.  characterset:    latin1
UNIX socket:            /tmp/mysql.sock
Uptime:                 53 sec

Threads: 1  Questions: 13  Slow queries: 0  Opens: 105  Flush tables: 1  Open tables: 98  Queries per second avg: 0.245
--------------

何の面倒なこともなしにSSLが有効化できる。これはいい。

mysql_install_dbからは mysql_ssl_rsa_setupというコマンドを呼んでいるだけなので、これを手で叩いてやれば他の方法でセットアップしてもOK。

$ bin/mysql_ssl_rsa_setup
OpenSSL 1.0.1e-fips 11 Feb 2013
Generating a 2048 bit RSA private key
...............+++
..........+++
writing new private key to 'ca-key.pem'
-----
writing RSA key
Signature ok
subject=/CN=MySQL_Server_5.7.6-m16_Auto_Generated_CA_Certificate
Getting Private key
Generating a 2048 bit RSA private key
...........................................................................+++
...........................................................................+++
writing new private key to 'server-key.pem'
-----
writing RSA key
Signature ok
subject=/CN=MySQL_Server_5.7.6-m16_Auto_Generated_Server_Certificate
Getting CA Private Key
Generating a 2048 bit RSA private key
.................+++
...................+++
writing new private key to 'client-key.pem'
-----
writing RSA key
Signature ok
subject=/CN=MySQL_Server_5.7.6-m16_Auto_Generated_Server_Certificate
Getting CA Private Key
server-cert.pem: OK
client-cert.pem: OK
Generating RSA private key, 2048 bit long modulus
............................................................................................................................+++
............+++
e is 65537 (0x10001)
writing RSA key

$ ll data/*.pem
-rw------- 1 mysql mysql 1679 Mar 13 13:33 data/ca-key.pem
-rw-r--r-- 1 mysql mysql 1082 Mar 13 13:33 data/ca.pem
-rw-r--r-- 1 mysql mysql 1086 Mar 13 13:33 data/client-cert.pem
-rw------- 1 mysql mysql 1679 Mar 13 13:33 data/client-key.pem
-rw------- 1 mysql mysql 1675 Mar 13 13:34 data/private_key.pem
-rw-r--r-- 1 mysql mysql  451 Mar 13 13:34 data/public_key.pem
-rw-r--r-- 1 mysql mysql 1086 Mar 13 13:33 data/server-cert.pem
-rw------- 1 mysql mysql 1679 Mar 13 13:33 data/server-key.pem

SSLのセットアップがラクになっていい :)

↧

MySQL 5.7でLOAD DATA INFILEに失敗する時に疑うこと(--secure-file-privの暗黙のデフォルトが少し変わった)

March 19, 2015, 8:22 pm

≫ Next: MySQL 5.7.6のgenerated columnは関数インデックスの夢を見るか

≪ Previous: MySQL 5.7.6からSSL周りのセットアップが扱いやすくなった

日々の覚書: MySQL 5.7.6は--secure-file-privを設定してないとWarningを吐くようになったのちょっと続き。

--secure-file-privそのものについては↑の記事を読んでいただけると幸い。LOAD DATA INFILEやSELECT .. INTO OUTFILEの動きを制限する。

ドキュメントのsysvarのところに書いてあるけれど、
* -DINSTALL_LAYOUT=STANDALONE, WIN でビルドされた場合は暗黙のデフォルトなし
* -DINSTALL_LAYOUT=DEB, RPM, SLES, SVR4 でビルドされた場合は"/var/lib/mysql-files"
* それ以外の設定の場合、"CMAKE_INSTALL_PREFIX(MySQLから見るとbasedir)の下のmysql-files"
が暗黙のデフォルトになる。

ソースはこのへん。
mysql-server/install_layout.cmake at 35e498b12e4e64e639cb8096d1a7362fd3272fa6 · mysql/mysql-server

INSTALL_LAYOUTの取りうる値はこのへんなので、FREEBSD, OSX, TARGZあたりがbasedir/mysql-filesになるのだろうか(ところで、Oracle公式のLinux Genericのバイナリー.tar.gzってSTANDALONEでビルドしてあるんだけど、他のプラットフォーム用のでTARGZ使ってるところもあるのかな)
mysql-server/install_layout.cmake at 35e498b12e4e64e639cb8096d1a7362fd3272fa6 · mysql/mysql-server

取り敢えず、バイナリーの.tar.gzは未設定なのと、Oracle Linux 6用のrpm bundleに入ってるやつはこの通りになるのを確認した。OSXとか誰か試してほしい。

あと、新しくsecure-file-privにディレクトリパスでなくて"NULL"を指定すると、いかなるディレクトリーに対してもFile_privがらみの操作を行えなくできるんだけど、

$ bin/mysqld_safe --no-defaults &

mysql> SELECT @@secure_file_priv;
+--------------------+
| @@secure_file_priv |
+--------------------+
| NULL               |
+--------------------+
1 row in set (0.00 sec)

mysql> SELECT LOAD_FILE('/etc/hosts');
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| LOAD_FILE('/etc/hosts')                                                                                                                                                        |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 172.17.0.3    0a8104f4882d
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
 |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)



$ bin/mysqld_safe --no-defaults --secure-file-priv=NULL &

mysql> SELECT @@secure_file_priv;
+--------------------+
| @@secure_file_priv |
+--------------------+
| NULL               |
+--------------------+
1 row in set (0.00 sec)

mysql> SELECT LOAD_FILE('/etc/hosts');
+-------------------------+
| LOAD_FILE('/etc/hosts') |
+-------------------------+
| NULL                    |
+-------------------------+
1 row in set (0.00 sec)

:(；ﾞﾟ'ωﾟ'): 区別がつかねぇ。。

【2015/03/20 12:44】
区別が付かない件はばぐれぽしました。

MySQL Bugs: #76401: Can't distinguish secure_file_priv = NULL and ""

↧

MySQL 5.7.6のgenerated columnは関数インデックスの夢を見るか

March 23, 2015, 3:31 am

≫ Next: 2015年版 InnoDB Monitorの仲間たち（InnoDBエンジン本体のアレ）

≪ Previous: MySQL 5.7でLOAD DATA INFILEに失敗する時に疑うこと(--secure-file-privの暗黙のデフォルトが少し変わった)

MySQL 5.7.6-m16で導入されたgenerated columnについてメモ。タイトルでしゃらっと関数インデックスとか言っているけれど、俺はその機能を知らない。

MySQL :: MySQL 5.7 Reference Manual :: 13.1.14 CREATE TABLE Syntax

generated columnはあるカラムの値に関数を噛ませた結果をカラムとして実体化できたり、実体化せずに毎回計算できたりするカラム定義のことらしくて、

mysql57> CREATE TABLE t1 (num int PRIMARY KEY, val varchar(32)) Engine = MyISAM;
Query OK, 0 rows affected (0.01 sec)

mysql57> INSERT INTO t1 SET num = 1, val = 'one';
Query OK, 1 row affected (0.00 sec)

mysql57> SELECT * FROM t1;
+-----+------+
| num | val  |
+-----+------+
|   1 | one  |
+-----+------+
1 row in set (0.00 sec)

$ od -c t1.MYD
0000000 003  \0  \n 006  \0 376 001  \0  \0  \0 003   o   n   e  \0  \0
0000020  \0  \0  \0  \0
0000024

こんなテーブルに対して

mysql57> ALTER TABLE t1 ADD new_val varchar(96) AS (REPEAT(val, 3)) STORED KEY;
Query OK, 1 row affected (0.02 sec)
Records: 1  Duplicates: 0  Warnings: 0

"STORED"キーワードは演算結果を実体化してデータとして格納するオプション。STOREDオプションで作成されたgenerated columnにはインデックスを作成することができる。

mysql57> SELECT * FROM t1;
+-----+------+-----------+
| num | val  | new_val   |
+-----+------+-----------+
|   1 | one  | oneoneone |
+-----+------+-----------+
1 row in set (0.00 sec)

$ od -c t1.MYD
0000000 003  \0 024  \0  \0 374 001  \0  \0  \0 003   o   n   e  \t   o
0000020   n   e   o   n   e   o   n   e
0000030

$ od -c t1.MYI
..
*
0004000  \0 024  \0 001  \t   o   n   e   o   n   e   o   n   e  \0  \0
0004020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
..

一応、インデックスとしても実体化されているぽい。

mysql57> INSERT INTO t1 SET num = 2, val = 'two';
Query OK, 1 row affected (0.00 sec)

mysql57> INSERT INTO t1 SET num = 3, val = 'three';
Query OK, 1 row affected (0.00 sec)

mysql57> explain SELECT * FROM t1 WHERE new_val LIKE 'one%';
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref  | rows | filtered | Extra                 |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
|  1 | SIMPLE      | t1    | NULL       | range | new_val       | new_val | 99      | NULL |    1 |   100.00 | Using index condition |
+----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

mysql57> show warnings;
+-------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                                         |
+-------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select `d1`.`t1`.`num` AS `num`,`d1`.`t1`.`val` AS `val`,`d1`.`t1`.`new_val` AS `new_val` from `d1`.`t1` where (`d1`.`t1`.`new_val` like 'one%') |
+-------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

極めてフツーにカラムを作って、極めてフツーに値を突っ込んで、極めてフツーに検索しているのと同じ感じ（当たり前）

ただし、generated columnからフツーの(?)カラムに対する逆関数は定義できないので、

mysql57> INSERT INTO t1 SET num = 4, new_val = 'fourfourfour';
ERROR 3105 (HY000): The value specified for generated column 'new_val' in table 't1' is not allowed.

突っ込めない。
おお、噂（？）の3000番台エラーだ。

で、このgenerated columnでやりたいのって

mysql57> create table json (id int auto_increment, col1 varchar(1000), primary key(id));
Query OK, 0 rows affected (0.05 sec)

mysql57> INSERT INTO json(id, col1) VALUES (1, '{"id":1,"Name":"Farmer grandmas","price":50000,"Conditions":["farms",15]}');
Query OK, 1 row affected (0.00 sec)

mysql57> INSERT INTO json(id, col1) VALUES (2, '{"id":2,"Name":"Worker grandmas","price":300000,"Conditions":["factories",15]}');
Query OK, 1 row affected (0.00 sec)

mysql57> INSERT INTO json(id, col1) VALUES (3, '{"id":3,"Name":"Miner grandmas","price":1000000,"Conditions":["mines",15]}');
Query OK, 1 row affected (0.02 sec)

mysql57> INSERT INTO json(id, col1) VALUES (4, '{"id":4,"Name":"Yoshiaki Yamasaki"}');
Query OK, 1 row affected (0.01 sec)

mysql57> SELECT * FROM json;
+----+--------------------------------------------------------------------------------+-----------+
| id | col1                                                                           | gen_price |
+----+--------------------------------------------------------------------------------+-----------+
|  1 | {"id":1,"Name":"Farmer grandmas","price":50000,"Conditions":["farms",15]}      |     50000 |
|  2 | {"id":2,"Name":"Worker grandmas","price":300000,"Conditions":["factories",15]} |    300000 |
|  3 | {"id":3,"Name":"Miner grandmas","price":1000000,"Conditions":["mines",15]}     |   1000000 |
|  4 | {"id":4,"Name":"Yoshiaki Yamasaki"}                                            |      NULL |
+----+--------------------------------------------------------------------------------+-----------+
4 rows in set (0.00 sec)

mysql57> ALTER TABLE json ADD price int unsigned AS (json_extract(col1, 'price')) STORED;
Query OK, 4 rows affected (0.07 sec)
Records: 4  Duplicates: 0  Warnings: 0

mysql57> ALTER TABLE json ADD KEY (price);
Query OK, 0 rows affected (0.05 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql57> SELECT * FROM json WHERE price > 500000;
+----+----------------------------------------------------------------------------+---------+
| id | col1                                                                       | price   |
+----+----------------------------------------------------------------------------+---------+
|  3 | {"id":3,"Name":"Miner grandmas","price":1000000,"Conditions":["mines",15]} | 1000000 |
+----+----------------------------------------------------------------------------+---------+
1 row in set (0.00 sec)

こーゆーことで合ってますかね？ :)

( ´-｀).oO（サンプルのJSONは日本語JSON UDFの一番詳しいスライドから拝借した

ところでADD price int unsigned AS (..) STORED KEYってやるとPRIMARYつけてなくてもPRIMARY KEYにしたがるんだけどたぶんバグだよな。。

【2015/03/23 19:44】
とりあえずばぐれぽしてみた。
MySQL Bugs: #76450: generated column with "KEY" makes PRIMARY KEY

↧

2015年版 InnoDB Monitorの仲間たち（InnoDBエンジン本体のアレ）

March 25, 2015, 3:50 am

≫ Next: MySQL 5.7でやっと(?)1テーブル複数トリガーが仕掛けられるようになった

≪ Previous: MySQL 5.7.6のgenerated columnは関数インデックスの夢を見るか

日々の覚書: InnoDB Monitorの仲間たち（InnoDBエンジン本体のアレ）で"CREATE TABLE innodb_monitor .."的なものを紹介しました。あれから2年。

というかよく調べてみたら1年前。

MySQL :: MySQL 5.7 Release Notes :: Changes in MySQL 5.7.4 (2014-03-31, Milestone 14)

InnoDB: New global configuration parameters, innodb_status_output and innodb_status_output_locks, allow you to dynamically enable and disable the standard InnoDB Monitor and InnoDB Lock Monitor for periodic output. Previously, enabling and disabling these monitors for periodic output required creating and dropping specially named tables (innodb_monitor and innodb_lock_monitor). For additional information, see InnoDB Monitors.

というわけで今後は"SET GLOBAL innodb_status_output{,_locks} = 1"で有効化ですね。
CREATE TABLE innodb_lock_monitor (a int); ってしたらフツーにテーブルが出来ただけで何も吐いてくれなくて困ったのでメモを残しておきます。

MySQL :: MySQL 5.7 Reference Manual :: 14.15.2 Enabling InnoDB Monitors

あ、デフォルトでエラーログに吐くのは相変わらずですが、エラーログでなくdatadir/innodb_status.pidファイルに吐くように設定できるオプションがあります。pid部分は文字列じゃなくてプロセスIDが入るので、自分で名前や場所を変えることは今のところできなさげ。

MySQL :: MySQL 5.7 Reference Manual :: 14.12 InnoDB Startup Options and System Variables

↧

MySQL 5.7でやっと(?)1テーブル複数トリガーが仕掛けられるようになった

March 26, 2015, 8:00 pm

≫ Next: MySQLのperformance_schemaでどれくらいの情報が見られるのか

≪ Previous: 2015年版 InnoDB Monitorの仲間たち（InnoDBエンジン本体のアレ）

As of MySQL 5.7.2, it is possible to define multiple triggers for a given table that have the same trigger event and action time. For example, you cannot have two BEFORE UPDATE triggers for a table. By default, triggers that have the same trigger event and action time activate in the order they were created. To affect trigger order, specify a trigger_order clause that indicates FOLLOWS or PRECEDES and the name of an existing trigger that also has the same trigger event and action time. With FOLLOWS, the new trigger activates after the existing trigger. With PRECEDES, the new trigger activates before the existing trigger.
MySQL :: MySQL 5.7 Reference Manual :: 13.1.16 CREATE TRIGGER Syntax

試してみよう。

5.6のフツーの動作は

mysql56> CREATE TRIGGER before_insert_1 BEFORE INSERT ON t1 FOR EACH ROW INSERT INTO t2 SET num = NEW.num, val = NEW.val;
Query OK, 0 rows affected (0.02 sec)

mysql56> INSERT INTO t1 VALUES (1, 'one');
Query OK, 1 row affected (0.03 sec)

mysql56> SELECT * FROM t1;
+-----+------+
| num | val  |
+-----+------+
|   1 | one  |
+-----+------+
1 row in set (0.00 sec)

mysql56> SELECT * FROM t2;
+-----+------+
| num | val  |
+-----+------+
|   1 | one  |
+-----+------+
1 row in set (0.00 sec)

mysql56> CREATE TRIGGER before_insert_2 BEFORE INSERT ON t1 FOR EACH ROW UPDATE t2 SET val = '残念だったな' WHERE num = NEW.num;
ERROR 1235 (42000): This version of MySQL doesn't yet support 'multiple triggers with the same action time and event for one table'

もちろんこう。

5.7.6だと

mysql57> CREATE TRIGGER before_insert_1 BEFORE INSERT ON t1 FOR EACH ROW INSERT INTO t2 SET num = NEW.num, val = NEW.val;
Query OK, 0 rows affected (0.01 sec)

mysql57> INSERT INTO t1 VALUES (1, 'one');
Query OK, 1 row affected (0.01 sec)

mysql57> SELECT * FROM t1;
+-----+------+
| num | val  |
+-----+------+
|   1 | one  |
+-----+------+
1 row in set (0.00 sec)

mysql57> SELECT * FROM t2;
+-----+------+
| num | val  |
+-----+------+
|   1 | one  |
+-----+------+
1 row in set (0.00 sec)

mysql57> CREATE TRIGGER before_insert_2 BEFORE INSERT ON t1 FOR EACH ROW UPDATE t2 SET val = '残念だったな' WHERE num = NEW.num;
Query OK, 0 rows affected (0.01 sec)

mysql57> INSERT INTO t1 VALUES (2, 'two');
Query OK, 1 row affected (0.04 sec)

mysql57> SELECT * FROM t1;
+-----+------+
| num | val  |
+-----+------+
|   1 | one  |
|   2 | two  |
+-----+------+
2 rows in set (0.00 sec)

mysql57> SELECT * FROM t2;
+-----+--------------------+
| num | val                |
+-----+--------------------+
|   1 | one                |
|   2 | 残念だったな       |
+-----+--------------------+
2 rows in set (0.00 sec)

mysql57> SHOW TRIGGERS\G
*************************** 1. row ***************************
             Trigger: before_insert_1
               Event: INSERT
               Table: t1
           Statement: INSERT INTO t2 SET num = NEW.num, val = NEW.val
              Timing: BEFORE
             Created: 2015-03-27 02:53:23.66
            sql_mode: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
*************************** 2. row ***************************
             Trigger: before_insert_2
               Event: INSERT
               Table: t1
           Statement: UPDATE t2 SET val = '残念だったな' WHERE num = NEW.num
              Timing: BEFORE
             Created: 2015-03-27 02:53:47.85
            sql_mode: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
2 rows in set (0.00 sec)

当たり前だけど張れた。順番の制御は、FOR EACH ROWのあとにPRECEDESまたはFOLLOWSで既存のトリガーを指定する(省略時は最後のトリガーの更に次に追加されるのかな)

mysql57> DROP TRIGGER before_insert_2;
Query OK, 0 rows affected (0.01 sec)

mysql57> CREATE TRIGGER before_insert_3 BEFORE INSERT ON t1 FOR EACH ROW PRECEDES before_insert_1 UPDATE t2 SET val = '残念だったな' WHERE num = NEW.num;
Query OK, 0 rows affected (0.02 sec)

mysql57> INSERT INTO t1 VALUES (3, 'three');
Query OK, 1 row affected (0.00 sec)

mysql57> SELECT * FROM t1;
+-----+-------+
| num | val   |
+-----+-------+
|   1 | one   |
|   2 | two   |
|   3 | three |
+-----+-------+
3 rows in set (0.00 sec)

mysql57> SELECT * FROM t2;
+-----+--------------------+
| num | val                |
+-----+--------------------+
|   1 | one                |
|   2 | 残念だったな       |
|   3 | three              |
+-----+--------------------+
3 rows in set (0.00 sec)

mysql57> SHOW TRIGGERS\G
*************************** 1. row ***************************
             Trigger: before_insert_3
               Event: INSERT
               Table: t1
           Statement: UPDATE t2 SET val = '残念だったな' WHERE num = NEW.num
              Timing: BEFORE
             Created: 2015-03-27 02:56:45.90
            sql_mode: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
*************************** 2. row ***************************
             Trigger: before_insert_1
               Event: INSERT
               Table: t1
           Statement: INSERT INTO t2 SET num = NEW.num, val = NEW.val
              Timing: BEFORE
             Created: 2015-03-27 02:53:23.66
            sql_mode: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: utf8_general_ci
2 rows in set (0.00 sec)

UPDATE t2が先に実行されるのでt2にレコードがなくて空ぶって、そのあとINSERT INTO t2でコピーされる。
ちょっと便利になるかも知れない。

↧

MySQLのperformance_schemaでどれくらいの情報が見られるのか

March 30, 2015, 11:56 pm

≫ Next: MySQLのperformance_schemaをフルパワーにするとどんな情報が見られるのか

≪ Previous: MySQL 5.7でやっと(?)1テーブル複数トリガーが仕掛けられるようになった

MySQL :: MySQL 5.6 Reference Manual :: 22 MySQL Performance Schema

5.6以降「よくなったよ！」「確かにね！でもメモリー」「」みたいな感じで扱われて結局まだONにしているインスタンスの少ないPerformance Schemaなのでちょっと練習中。

tpcc-mysqlとかでやると割と綺麗なクエリーをしているので面白い情報が出ない(ibdata1のfsyncに時間がかかってることなんて知ってるよ！って感じになる)ので、isucon2のアプリで試してみることにしますた。モリスさん++

livedoor Techブログ : 自家製 #isucon2 のつくりかた

MySQLは5.6がいいのでMySQL :: Download MySQL Yum Repositoryで入れる。

$ sudo yum localinstall -y http://dev.mysql.com/get/mysql-community-release-el6-5.noarch.rpm
$ sudo yum install -y mysql-community-*

らくちん。
吊るしでPerformance Schemaはいくらか有効になっているので(中の人いわく、「よく使いそうなものはデフォルトでONになっている」)まずはそのまま見てみる。めぼしそうなもの(とりあえずcount_starカラムが0じゃないレコードを含むテーブル)を探してみるとこんなのがあった。

↓にずらーと出力結果ばかり並べているので先に結論。

events_statements_summary_by_digest マジ優秀。
これを見るためだけに(メモリーが許すなら)performance_schemaをONにしておく価値はある。

* events_statements_summary_by_account_by_event_name
* events_statements_summary_by_host_by_event_name
* events_statements_summary_by_thread_by_event_name
* events_statements_summary_by_user_by_event_name
* events_statements_summary_global_by_event_name

mysql> SELECT * FROM events_statements_summary_by_account_by_event_name WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+------------+-----------+----------------------+------------+----------------+----------------+----------------+----------------+---------------+------------+--------------+-------------------+---------------+-------------------+-----------------------------+------------------------+----------------------+----------------------------+------------------+------------------------+-----------------+-----------------------+----------------+---------------+---------------+-------------------+------------------------+
| USER       | HOST      | EVENT_NAME           | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT | SUM_LOCK_TIME | SUM_ERRORS | SUM_WARNINGS | SUM_ROWS_AFFECTED | SUM_ROWS_SENT | SUM_ROWS_EXAMINED | SUM_CREATED_TMP_DISK_TABLES | SUM_CREATED_TMP_TABLES | SUM_SELECT_FULL_JOIN | SUM_SELECT_FULL_RANGE_JOIN | SUM_SELECT_RANGE | SUM_SELECT_RANGE_CHECK | SUM_SELECT_SCAN | SUM_SORT_MERGE_PASSES | SUM_SORT_RANGE | SUM_SORT_ROWS | SUM_SORT_SCAN | SUM_NO_INDEX_USED | SUM_NO_GOOD_INDEX_USED |
+------------+-----------+----------------------+------------+----------------+----------------+----------------+----------------+---------------+------------+--------------+-------------------+---------------+-------------------+-----------------------------+------------------------+----------------------+----------------------------+------------------+------------------------+-----------------+-----------------------+----------------+---------------+---------------+-------------------+------------------------+
| isucon2app | localhost | statement/sql/select |       2738 | 35991922775000 |       83997000 |    13145333000 |   160056655000 |  195344000000 |          0 |            0 |                 0 |       1653914 |          29363625 |                           0 |                    557 |                  556 |                          0 |                0 |                      0 |            1558 |                     0 |              0 |          4986 |           557 |              1558 |                      0 |
| isucon2app | localhost | statement/sql/insert |        148 |  3698267239000 |       81107000 |    24988292000 |  1354468459000 |  113248000000 |          0 |            0 |             82048 |             0 |                 0 |                           0 |                      0 |                    0 |                          0 |                0 |                      0 |               0 |                     0 |              0 |             0 |             0 |                 0 |                      0 |
| isucon2app | localhost | statement/sql/update |         94 |  1648364786000 |     5976157000 |    17535795000 |    42219182000 |    5534000000 |          0 |            0 |                94 |             0 |            385118 |                           0 |                      0 |                    0 |                          0 |                0 |                      0 |               0 |                     0 |             94 |            94 |             0 |                 0 |                      0 |
+------------+-----------+----------------------+------------+----------------+----------------+----------------+----------------+---------------+------------+--------------+-------------------+---------------+-------------------+-----------------------------+------------------------+----------------------+----------------------------+------------------+------------------------+-----------------+-----------------------+----------------+---------------+---------------+-------------------+------------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM events_statements_summary_by_account_by_event_name WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
                       USER: isucon2app
                       HOST: localhost
                 EVENT_NAME: statement/sql/select
                 COUNT_STAR: 2738
             SUM_TIMER_WAIT: 35991922775000
             MIN_TIMER_WAIT: 83997000
             AVG_TIMER_WAIT: 13145333000
             MAX_TIMER_WAIT: 160056655000
              SUM_LOCK_TIME: 195344000000
                 SUM_ERRORS: 0
               SUM_WARNINGS: 0
          SUM_ROWS_AFFECTED: 0
              SUM_ROWS_SENT: 1653914
          SUM_ROWS_EXAMINED: 29363625
SUM_CREATED_TMP_DISK_TABLES: 0
     SUM_CREATED_TMP_TABLES: 557
       SUM_SELECT_FULL_JOIN: 556
 SUM_SELECT_FULL_RANGE_JOIN: 0
           SUM_SELECT_RANGE: 0
     SUM_SELECT_RANGE_CHECK: 0
            SUM_SELECT_SCAN: 1558
      SUM_SORT_MERGE_PASSES: 0
             SUM_SORT_RANGE: 0
              SUM_SORT_ROWS: 4986
              SUM_SORT_SCAN: 557
          SUM_NO_INDEX_USED: 1558
     SUM_NO_GOOD_INDEX_USED: 0
1 row in set (0.00 sec)

うーん、ちょっと大まかすぎるかな。。sum_sort_scanとかsum_sort_merge_passesとかは出てるから、どちらかというと全体の傾向をグラフにした方がいいような気がする(や、それならinformation_schemaで今のところ足りてるけど、こっちはupdateかselectか分けて見られるってことだよね)

* events_statements_summary_by_digest

mysql> SELECT * FROM events_statements_summary_by_digest WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+-------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------+----------------+----------------+----------------+---------------+------------+--------------+-------------------+---------------+-------------------+-----------------------------+------------------------+----------------------+----------------------------+------------------+------------------------+-----------------+-----------------------+----------------+---------------+---------------+-------------------+------------------------+---------------------+---------------------+
| SCHEMA_NAME | DIGEST                           | DIGEST_TEXT                                                                                                                                                                                                                                                                                                                                                                                        | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT | SUM_LOCK_TIME | SUM_ERRORS | SUM_WARNINGS | SUM_ROWS_AFFECTED | SUM_ROWS_SENT | SUM_ROWS_EXAMINED | SUM_CREATED_TMP_DISK_TABLES | SUM_CREATED_TMP_TABLES | SUM_SELECT_FULL_JOIN | SUM_SELECT_FULL_RANGE_JOIN | SUM_SELECT_RANGE | SUM_SELECT_RANGE_CHECK | SUM_SELECT_SCAN | SUM_SORT_MERGE_PASSES | SUM_SORT_RANGE | SUM_SORT_ROWS | SUM_SORT_SCAN | SUM_NO_INDEX_USED | SUM_NO_GOOD_INDEX_USED | FIRST_SEEN          | LAST_SEEN           |
+-------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------+----------------+----------------+----------------+---------------+------------+--------------+-------------------+---------------+-------------------+-----------------------------+------------------------+----------------------+----------------------------+------------------+------------------------+-----------------+-----------------------+----------------+---------------+---------------+-------------------+------------------------+---------------------+---------------------+
| isucon2     | 68ff5779415a5d171d700f4a20fd47d0 | SELECT `stock` . `seat_id` , `variation` . `name` AS `v_name` , `ticket` . `name` AS `t_name` , `artist` . `name` AS `a_name` FROM `stock` JOIN `variation` ON `stock` . `variation_id` = `variation` . `id` JOIN `ticket` ON `variation` . `ticket_id` = `ticket` . `id` JOIN `artist` ON `ticket` . `artist_id` = `artist` . `id` WHERE `order_id` IS NOT NULL ORDER BY `order_id` DESC LIMIT ?  |        556 | 19618662892000 |    12253383000 |    35285364000 |   160056655000 |   62943000000 |          0 |            0 |                 0 |          4931 |          22848064 |                           0 |                    556 |                  556 |                          0 |                0 |                      0 |             556 |                     0 |              0 |          4931 |           556 |               556 |                      0 | 2015-03-31 14:50:03 | 2015-03-31 14:50:21 |
| isucon2     | 32f72c3a9a23b6bdf3847bbdddeb16c7 | SELECT COUNT ( * ) FROM `variation` INNER JOIN `stock` ON `stock` . `variation_id` = `variation` . `id` WHERE `variation` . `ticket_id` = ? AND `stock` . `order_id` IS NULL                                                                                                                                                                                                                       |        351 |  8184198181000 |     6442619000 |    23316803000 |    50954069000 |   27705000000 |          0 |            0 |                 0 |           351 |           2878902 |                           0 |                      0 |                    0 |                          0 |                0 |                      0 |             351 |                     0 |              0 |             0 |             0 |               351 |                      0 | 2015-03-31 14:50:03 | 2015-03-31 14:50:21 |
| isucon2     | 886e82ad6d641a9be797d0b5b1f2bf73 | SELECT `seat_id` , `order_id` FROM `stock` WHERE `variation_id` = ?                                                                                                                                                                                                                                                                                                                                |        402 |  5404182832000 |     3303525000 |    13443240000 |    38198522000 |   13646000000 |          0 |            0 |                 0 |       1646592 |           1646592 |                           0 |                      0 |                    0 |                          0 |                0 |                      0 |               0 |                     0 |              0 |             0 |             0 |                 0 |                      0 | 2015-03-31 14:50:06 | 2015-03-31 14:50:21 |
+-------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------+----------------+----------------+----------------+---------------+------------+--------------+-------------------+---------------+-------------------+-----------------------------+------------------------+----------------------+----------------------------+------------------+------------------------+-----------------+-----------------------+----------------+---------------+---------------+-------------------+------------------------+---------------------+---------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM events_statements_summary_by_digest WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
                SCHEMA_NAME: isucon2
                     DIGEST: 68ff5779415a5d171d700f4a20fd47d0
                DIGEST_TEXT: SELECT `stock` . `seat_id` , `variation` . `name` AS `v_name` , `ticket` . `name` AS `t_name` , `artist` . `name` AS `a_name` FROM `stock` JOIN `variation` ON `stock` . `variation_id` = `variation` . `id` JOIN `ticket` ON `variation` . `ticket_id` = `ticket` . `id` JOIN `artist` ON `ticket` . `artist_id` = `artist` . `id` WHERE `order_id` IS NOT NULL ORDER BY `order_id` DESC LIMIT ?
                 COUNT_STAR: 556
             SUM_TIMER_WAIT: 19618662892000
             MIN_TIMER_WAIT: 12253383000
             AVG_TIMER_WAIT: 35285364000
             MAX_TIMER_WAIT: 160056655000
              SUM_LOCK_TIME: 62943000000
                 SUM_ERRORS: 0
               SUM_WARNINGS: 0
          SUM_ROWS_AFFECTED: 0
              SUM_ROWS_SENT: 4931
          SUM_ROWS_EXAMINED: 22848064
SUM_CREATED_TMP_DISK_TABLES: 0
     SUM_CREATED_TMP_TABLES: 556
       SUM_SELECT_FULL_JOIN: 556
 SUM_SELECT_FULL_RANGE_JOIN: 0
           SUM_SELECT_RANGE: 0
     SUM_SELECT_RANGE_CHECK: 0
            SUM_SELECT_SCAN: 556
      SUM_SORT_MERGE_PASSES: 0
             SUM_SORT_RANGE: 0
              SUM_SORT_ROWS: 4931
              SUM_SORT_SCAN: 556
          SUM_NO_INDEX_USED: 556
     SUM_NO_GOOD_INDEX_USED: 0
                 FIRST_SEEN: 2015-03-31 14:50:03
                  LAST_SEEN: 2015-03-31 14:50:21
1 row in set (0.00 sec)

これは判りやすい。ステートメント(pt-query-digestみたいに可変部分は握りつぶしてくれる)単位でSelect_scanやSum_rows_examinedが見られる。ステキ。
Percona Serverのスローログ + pt-query-digest並みの情報量な気がする。パーセンタイルな情報はないけどまあいいや。

* events_waits_summary_by_account_by_event_name
* events_waits_summary_by_host_by_event_name
* events_waits_summary_by_instance
* events_waits_summary_by_thread_by_event_name
* events_waits_summary_by_user_by_event_name
* events_waits_summary_global_by_event_name

mysql> SELECT * FROM events_waits_summary_by_account_by_event_name WHERE count_star <> 0 AND event_name <> 'idle' ORDER BY sum_timer_wait DESC LIMIT 3;
+------------+-----------+--------------------------------------+------------+----------------+----------------+----------------+----------------+
| USER       | HOST      | EVENT_NAME                           | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT |
+------------+-----------+--------------------------------------+------------+----------------+----------------+----------------+----------------+
| isucon2app | localhost | wait/io/table/sql/handler            |   29763732 | 28437697326885 |          48125 |         955185 |  1355498746525 |
| isucon2app | localhost | wait/io/file/innodb/innodb_data_file |        417 |  2104885627075 |        2718100 |     5047687260 |  1191383874065 |
| isucon2app | localhost | wait/io/file/innodb/innodb_log_file  |        350 |   865270214655 |        7838215 |     2472200500 |    58576612710 |
+------------+-----------+--------------------------------------+------------+----------------+----------------+----------------+----------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM events_waits_summary_by_account_by_event_name WHERE count_star <> 0 AND event_name <> 'idle' ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
          USER: isucon2app
          HOST: localhost
    EVENT_NAME: wait/io/table/sql/handler
    COUNT_STAR: 29763732
SUM_TIMER_WAIT: 28437697326885
MIN_TIMER_WAIT: 48125
AVG_TIMER_WAIT: 955185
MAX_TIMER_WAIT: 1355498746525
1 row in set (0.00 sec)

どこのイベントで待ちが発生したかの情報なんだけど、なんかやっぱりちょっと粒度が荒い。
ibdata1で待ってるのは知ってるよ！でもdouble write切れないんだよ！って感じ。

* file_summary_by_event_name

mysql> SELECT * FROM file_summary_by_event_name WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+--------------------------------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+--------------------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------------------+------------+----------------+----------------+----------------+----------------+
| EVENT_NAME                           | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT | COUNT_READ | SUM_TIMER_READ | MIN_TIMER_READ | AVG_TIMER_READ | MAX_TIMER_READ | SUM_NUMBER_OF_BYTES_READ | COUNT_WRITE | SUM_TIMER_WRITE | MIN_TIMER_WRITE | AVG_TIMER_WRITE | MAX_TIMER_WRITE | SUM_NUMBER_OF_BYTES_WRITE | COUNT_MISC | SUM_TIMER_MISC | MIN_TIMER_MISC | AVG_TIMER_MISC | MAX_TIMER_MISC |
+--------------------------------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+--------------------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------------------+------------+----------------+----------------+----------------+----------------+
| wait/io/file/innodb/innodb_data_file |       1519 |  2481586224040 |              0 |     1633697065 |  1191383874065 |        163 |     3469540305 |              0 |       21285495 |      896338905 |                  4734976 |        1029 |    206577099575 |               0 |       200755170 |     20881438270 |                 119422976 |        327 |  2271539584160 |              0 |     6946603895 |  1191383874065 |
| wait/io/file/innodb/innodb_log_file  |        434 |  1022073887450 |              0 |     2355008810 |    58576612710 |          6 |       27545210 |              0 |        4590740 |       22502480 |                    69632 |         211 |    262934620795 |               0 |      1246135275 |     58576612710 |                   7907840 |        217 |   759111721445 |              0 |     3498210485 |    41712168190 |
| wait/io/file/sql/FRM                 |       1663 |   374523815050 |              0 |      225209600 |   205524644710 |        739 |   124995936065 |              0 |      169141665 |    29619563050 |                   167156 |         105 |       176487850 |               0 |         1680525 |        15076215 |                     26609 |        819 |   249351391135 |              0 |      304458000 |   205524644710 |
+--------------------------------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+--------------------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------------------+------------+----------------+----------------+----------------+----------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM file_summary_by_event_name WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
               EVENT_NAME: wait/io/file/innodb/innodb_data_file
               COUNT_STAR: 1519
           SUM_TIMER_WAIT: 2481586224040
           MIN_TIMER_WAIT: 0
           AVG_TIMER_WAIT: 1633697065
           MAX_TIMER_WAIT: 1191383874065
               COUNT_READ: 163
           SUM_TIMER_READ: 3469540305
           MIN_TIMER_READ: 0
           AVG_TIMER_READ: 21285495
           MAX_TIMER_READ: 896338905
 SUM_NUMBER_OF_BYTES_READ: 4734976
              COUNT_WRITE: 1029
          SUM_TIMER_WRITE: 206577099575
          MIN_TIMER_WRITE: 0
          AVG_TIMER_WRITE: 200755170
          MAX_TIMER_WRITE: 20881438270
SUM_NUMBER_OF_BYTES_WRITE: 119422976
               COUNT_MISC: 327
           SUM_TIMER_MISC: 2271539584160
           MIN_TIMER_MISC: 0
           AVG_TIMER_MISC: 6946603895
           MAX_TIMER_MISC: 1191383874065
1 row in set (0.00 sec)

WAIT, READ, WRITE, MISCに分かれているので多少見やすそうだけど、粒度が荒くてちょっとつらい。
対して、↓の

* file_summary_by_instance

mysql> SELECT * FROM file_summary_by_instance WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+----------------------------------+--------------------------------------+-----------------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+--------------------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------------------+------------+----------------+----------------+----------------+----------------+
| FILE_NAME                        | EVENT_NAME                           | OBJECT_INSTANCE_BEGIN | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT | COUNT_READ | SUM_TIMER_READ | MIN_TIMER_READ | AVG_TIMER_READ | MAX_TIMER_READ | SUM_NUMBER_OF_BYTES_READ | COUNT_WRITE | SUM_TIMER_WRITE | MIN_TIMER_WRITE | AVG_TIMER_WRITE | MAX_TIMER_WRITE | SUM_NUMBER_OF_BYTES_WRITE | COUNT_MISC | SUM_TIMER_MISC | MIN_TIMER_MISC | AVG_TIMER_MISC | MAX_TIMER_MISC |
+----------------------------------+--------------------------------------+-----------------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+--------------------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------------------+------------+----------------+----------------+----------------+----------------+
| /var/lib/mysql/ibdata1           | wait/io/file/innodb/innodb_data_file |       140132153275264 |        819 |  1604593676840 |        2331560 |     1959210715 |  1191383874065 |        157 |     3398451980 |        3504655 |       21645855 |      896338905 |                  4636672 |         626 |    181296946215 |         7489790 |       289611630 |     20881438270 |                  88735744 |         36 |  1419898278645 |        2331560 |    39441618755 |  1191383874065 |
| /var/lib/mysql/ib_logfile0       | wait/io/file/innodb/innodb_log_file  |       140132153275968 |        431 |  1022063454720 |         382690 |     2371376700 |    58576612710 |          6 |       27545210 |         382690 |        4590740 |       22502480 |                    69632 |         211 |    262934620795 |         7044730 |      1246135275 |     58576612710 |                   7907840 |        214 |   759101288715 |         934780 |     3547202120 |    41712168190 |
| /var/lib/mysql/isucon2/stock.ibd | wait/io/file/innodb/innodb_data_file |       140132153361152 |        310 |   273430299835 |        3560095 |      882033075 |     9891016520 |          0 |              0 |              0 |              0 |              0 |                        0 |         239 |     12590424770 |         8717170 |        52679550 |       827383095 |                  15532032 |         71 |   260839875065 |        3560095 |     3673800900 |     9891016520 |
+----------------------------------+--------------------------------------+-----------------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+--------------------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------------------+------------+----------------+----------------+----------------+----------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM file_summary_by_instance WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
                FILE_NAME: /var/lib/mysql/ibdata1
               EVENT_NAME: wait/io/file/innodb/innodb_data_file
    OBJECT_INSTANCE_BEGIN: 140132153275264
               COUNT_STAR: 819
           SUM_TIMER_WAIT: 1604593676840
           MIN_TIMER_WAIT: 2331560
           AVG_TIMER_WAIT: 1959210715
           MAX_TIMER_WAIT: 1191383874065
               COUNT_READ: 157
           SUM_TIMER_READ: 3398451980
           MIN_TIMER_READ: 3504655
           AVG_TIMER_READ: 21645855
           MAX_TIMER_READ: 896338905
 SUM_NUMBER_OF_BYTES_READ: 4636672
              COUNT_WRITE: 626
          SUM_TIMER_WRITE: 181296946215
          MIN_TIMER_WRITE: 7489790
          AVG_TIMER_WRITE: 289611630
          MAX_TIMER_WRITE: 20881438270
SUM_NUMBER_OF_BYTES_WRITE: 88735744
               COUNT_MISC: 36
           SUM_TIMER_MISC: 1419898278645
           MIN_TIMER_MISC: 2331560
           AVG_TIMER_MISC: 39441618755
           MAX_TIMER_MISC: 1191383874065
1 row in set (0.00 sec)

こっちはファイル名まで見られるのでいくらか良い感じ。更新が集中しているテーブルとか見られる。グラフ化したい。

* objects_summary_global_by_type

mysql> SELECT * FROM objects_summary_global_by_type WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+-------------+---------------+---------------+------------+----------------+----------------+----------------+----------------+
| OBJECT_TYPE | OBJECT_SCHEMA | OBJECT_NAME   | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT |
+-------------+---------------+---------------+------------+----------------+----------------+----------------+----------------+
| TABLE       | isucon2       | stock         |   29755167 | 25975000162265 |          38885 |         872795 |    60737987310 |
| TABLE       | isucon2       | order_request |        343 |  2256830278395 |         177870 |     6579680030 |  1355498746525 |
| TABLE       | isucon2       | artist        |       3653 |   139255591860 |          25795 |       38120775 |    14495216505 |
+-------------+---------------+---------------+------------+----------------+----------------+----------------+----------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM objects_summary_global_by_type WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
   OBJECT_TYPE: TABLE
 OBJECT_SCHEMA: isucon2
   OBJECT_NAME: stock
    COUNT_STAR: 29755167
SUM_TIMER_WAIT: 25975000162265
MIN_TIMER_WAIT: 38885
AVG_TIMER_WAIT: 872795
MAX_TIMER_WAIT: 60737987310
1 row in set (0.00 sec)

こっちはファイルに触れる触れないに関わらずテーブル単位で待ちが発生した回数と時間の累計。
バッファプールだけで完結するSELECTならundoのためにibdata1とib_logfileに触ることはあっても.ibdファイルには触らないだろうから、とかそんなところが違うんだと思う。

* table_io_waits_summary_by_index_usage
* table_io_waits_summary_by_table

mysql> SELECT * FROM table_io_waits_summary_by_index_usage WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+-------------+---------------+---------------+----------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+-------------+-----------------+-----------------+-----------------+-----------------+-------------+-----------------+-----------------+-----------------+-----------------+--------------+------------------+------------------+------------------+------------------+--------------+------------------+------------------+------------------+------------------+--------------+------------------+------------------+------------------+------------------+
| OBJECT_TYPE | OBJECT_SCHEMA | OBJECT_NAME   | INDEX_NAME     | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT | COUNT_READ | SUM_TIMER_READ | MIN_TIMER_READ | AVG_TIMER_READ | MAX_TIMER_READ | COUNT_WRITE | SUM_TIMER_WRITE | MIN_TIMER_WRITE | AVG_TIMER_WRITE | MAX_TIMER_WRITE | COUNT_FETCH | SUM_TIMER_FETCH | MIN_TIMER_FETCH | AVG_TIMER_FETCH | MAX_TIMER_FETCH | COUNT_INSERT | SUM_TIMER_INSERT | MIN_TIMER_INSERT | AVG_TIMER_INSERT | MAX_TIMER_INSERT | COUNT_UPDATE | SUM_TIMER_UPDATE | MIN_TIMER_UPDATE | AVG_TIMER_UPDATE | MAX_TIMER_UPDATE | COUNT_DELETE | SUM_TIMER_DELETE | MIN_TIMER_DELETE | AVG_TIMER_DELETE | MAX_TIMER_DELETE |
+-------------+---------------+---------------+----------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+-------------+-----------------+-----------------+-----------------+-----------------+-------------+-----------------+-----------------+-----------------+-----------------+--------------+------------------+------------------+------------------+------------------+--------------+------------------+------------------+------------------+------------------+--------------+------------------+------------------+------------------+------------------+
| TABLE       | isucon2       | stock         | NULL           |   23155059 | 15405691875035 |          51590 |         665280 |    60737987310 |   23073139 | 14505802679445 |          51590 |         628320 |    29693793745 |       81920 |    899889195590 |         3240930 |        10984820 |     60737987310 |    23073139 |  14505802679445 |           51590 |          628320 |     29693793745 |        81920 |     899889195590 |          3240930 |         10984820 |      60737987310 |            0 |                0 |                0 |                0 |                0 |            0 |                0 |                0 |                0 |                0 |
| TABLE       | isucon2       | stock         | variation_seat |    6596170 | 10556789714240 |          48125 |        1600060 |    35806416030 |    6596170 | 10556789714240 |          48125 |        1600060 |    35806416030 |           0 |               0 |               0 |               0 |               0 |     6596170 |  10556789714240 |           48125 |         1600060 |     35806416030 |            0 |                0 |                0 |                0 |                0 |            0 |                0 |                0 |                0 |                0 |            0 |                0 |                0 |                0 |                0 |
| TABLE       | isucon2       | order_request | NULL           |         94 |  2256572540145 |       24644620 |    24006090570 |  1355498746525 |          0 |              0 |              0 |              0 |              0 |          94 |   2256572540145 |        24644620 |     24006090570 |   1355498746525 |           0 |               0 |               0 |               0 |               0 |           94 |    2256572540145 |         24644620 |      24006090570 |    1355498746525 |            0 |                0 |                0 |                0 |                0 |            0 |                0 |                0 |                0 |                0 |
+-------------+---------------+---------------+----------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+-------------+-----------------+-----------------+-----------------+-----------------+-------------+-----------------+-----------------+-----------------+-----------------+--------------+------------------+------------------+------------------+------------------+--------------+------------------+------------------+------------------+------------------+--------------+------------------+------------------+------------------+------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM table_io_waits_summary_by_index_usage WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
     OBJECT_TYPE: TABLE
   OBJECT_SCHEMA: isucon2
     OBJECT_NAME: stock
      INDEX_NAME: NULL
      COUNT_STAR: 23155059
  SUM_TIMER_WAIT: 15405691875035
  MIN_TIMER_WAIT: 51590
  AVG_TIMER_WAIT: 665280
  MAX_TIMER_WAIT: 60737987310
      COUNT_READ: 23073139
  SUM_TIMER_READ: 14505802679445
  MIN_TIMER_READ: 51590
  AVG_TIMER_READ: 628320
  MAX_TIMER_READ: 29693793745
     COUNT_WRITE: 81920
 SUM_TIMER_WRITE: 899889195590
 MIN_TIMER_WRITE: 3240930
 AVG_TIMER_WRITE: 10984820
 MAX_TIMER_WRITE: 60737987310
     COUNT_FETCH: 23073139
 SUM_TIMER_FETCH: 14505802679445
 MIN_TIMER_FETCH: 51590
 AVG_TIMER_FETCH: 628320
 MAX_TIMER_FETCH: 29693793745
    COUNT_INSERT: 81920
SUM_TIMER_INSERT: 899889195590
MIN_TIMER_INSERT: 3240930
AVG_TIMER_INSERT: 10984820
MAX_TIMER_INSERT: 60737987310
    COUNT_UPDATE: 0
SUM_TIMER_UPDATE: 0
MIN_TIMER_UPDATE: 0
AVG_TIMER_UPDATE: 0
MAX_TIMER_UPDATE: 0
    COUNT_DELETE: 0
SUM_TIMER_DELETE: 0
MIN_TIMER_DELETE: 0
AVG_TIMER_DELETE: 0
MAX_TIMER_DELETE: 0
1 row in set (0.01 sec)

更に細かく、インデックス単位でのWAIT, READ, WRITE, FETCH, INSERT, UPDATE, DELETE(table_io_waits_summary_by_tableの方はテーブル単位)
INDEX_NAME = 'NULL'ってWITH ROLLUP的な感じで合算なのかと思ったけどどうもそうではなさげ。。なんだろう。
使われてないインデックス(writeばっかりでfetchが少ないとかになるのかな)とか算出するsysスキーマがあったような気がする。これ使ってるのかな。

* table_lock_waits_summary_by_table

mysql> SELECT * FROM table_lock_waits_summary_by_table WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+-------------+---------------+-------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+-------------+-----------------+-----------------+-----------------+-----------------+-------------------+-----------------------+-----------------------+-----------------------+-----------------------+------------------------------+----------------------------------+----------------------------------+----------------------------------+----------------------------------+--------------------------+------------------------------+------------------------------+------------------------------+------------------------------+----------------------+--------------------------+--------------------------+--------------------------+--------------------------+---------------------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+-----------------------------+-----------------------------+-----------------------------+-----------------------------+-------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+---------------------+-------------------------+-------------------------+-------------------------+-------------------------+--------------------------+------------------------------+------------------------------+------------------------------+------------------------------+--------------------+------------------------+------------------------+------------------------+------------------------+----------------------+--------------------------+--------------------------+--------------------------+--------------------------+
| OBJECT_TYPE | OBJECT_SCHEMA | OBJECT_NAME | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT | COUNT_READ | SUM_TIMER_READ | MIN_TIMER_READ | AVG_TIMER_READ | MAX_TIMER_READ | COUNT_WRITE | SUM_TIMER_WRITE | MIN_TIMER_WRITE | AVG_TIMER_WRITE | MAX_TIMER_WRITE | COUNT_READ_NORMAL | SUM_TIMER_READ_NORMAL | MIN_TIMER_READ_NORMAL | AVG_TIMER_READ_NORMAL | MAX_TIMER_READ_NORMAL | COUNT_READ_WITH_SHARED_LOCKS | SUM_TIMER_READ_WITH_SHARED_LOCKS | MIN_TIMER_READ_WITH_SHARED_LOCKS | AVG_TIMER_READ_WITH_SHARED_LOCKS | MAX_TIMER_READ_WITH_SHARED_LOCKS | COUNT_READ_HIGH_PRIORITY | SUM_TIMER_READ_HIGH_PRIORITY | MIN_TIMER_READ_HIGH_PRIORITY | AVG_TIMER_READ_HIGH_PRIORITY | MAX_TIMER_READ_HIGH_PRIORITY | COUNT_READ_NO_INSERT | SUM_TIMER_READ_NO_INSERT | MIN_TIMER_READ_NO_INSERT | AVG_TIMER_READ_NO_INSERT | MAX_TIMER_READ_NO_INSERT | COUNT_READ_EXTERNAL | SUM_TIMER_READ_EXTERNAL | MIN_TIMER_READ_EXTERNAL | AVG_TIMER_READ_EXTERNAL | MAX_TIMER_READ_EXTERNAL | COUNT_WRITE_ALLOW_WRITE | SUM_TIMER_WRITE_ALLOW_WRITE | MIN_TIMER_WRITE_ALLOW_WRITE | AVG_TIMER_WRITE_ALLOW_WRITE | MAX_TIMER_WRITE_ALLOW_WRITE | COUNT_WRITE_CONCURRENT_INSERT | SUM_TIMER_WRITE_CONCURRENT_INSERT | MIN_TIMER_WRITE_CONCURRENT_INSERT | AVG_TIMER_WRITE_CONCURRENT_INSERT | MAX_TIMER_WRITE_CONCURRENT_INSERT | COUNT_WRITE_DELAYED | SUM_TIMER_WRITE_DELAYED | MIN_TIMER_WRITE_DELAYED | AVG_TIMER_WRITE_DELAYED | MAX_TIMER_WRITE_DELAYED | COUNT_WRITE_LOW_PRIORITY | SUM_TIMER_WRITE_LOW_PRIORITY | MIN_TIMER_WRITE_LOW_PRIORITY | AVG_TIMER_WRITE_LOW_PRIORITY | MAX_TIMER_WRITE_LOW_PRIORITY | COUNT_WRITE_NORMAL | SUM_TIMER_WRITE_NORMAL | MIN_TIMER_WRITE_NORMAL | AVG_TIMER_WRITE_NORMAL | MAX_TIMER_WRITE_NORMAL | COUNT_WRITE_EXTERNAL | SUM_TIMER_WRITE_EXTERNAL | MIN_TIMER_WRITE_EXTERNAL | AVG_TIMER_WRITE_EXTERNAL | MAX_TIMER_WRITE_EXTERNAL |
+-------------+---------------+-------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+-------------+-----------------+-----------------+-----------------+-----------------+-------------------+-----------------------+-----------------------+-----------------------+-----------------------+------------------------------+----------------------------------+----------------------------------+----------------------------------+----------------------------------+--------------------------+------------------------------+------------------------------+------------------------------+------------------------------+----------------------+--------------------------+--------------------------+--------------------------+--------------------------+---------------------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+-----------------------------+-----------------------------+-----------------------------+-----------------------------+-------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+---------------------+-------------------------+-------------------------+-------------------------+-------------------------+--------------------------+------------------------------+------------------------------+------------------------------+------------------------------+--------------------+------------------------+------------------------+------------------------+------------------------+----------------------+--------------------------+--------------------------+--------------------------+--------------------------+
| TABLE       | isucon2       | stock       |       3844 |     1580755330 |          38885 |         411180 |       84922530 |       3612 |     1451608235 |          38885 |         401555 |       84922530 |         232 |       129147095 |          192885 |          556325 |         9873325 |              1806 |             503861820 |                 48125 |                278740 |              84922530 |                            0 |                                0 |                                0 |                                0 |                                0 |                        0 |                            0 |                            0 |                            0 |                            0 |                    0 |                        0 |                        0 |                        0 |                        0 |                1806 |               947746415 |                   38885 |                  524755 |                13140050 |                     114 |                    40675635 |                      192885 |                      356510 |                     1143450 |                             0 |                                 0 |                                 0 |                                 0 |                                 0 |                   0 |                       0 |                       0 |                       0 |                       0 |                        0 |                            0 |                            0 |                            0 |                            0 |                  2 |                1497265 |                 525525 |                 748440 |                 971740 |                  116 |                 86974195 |                   418110 |                   749595 |                  9873325 |
| TABLE       | isucon2       | variation   |       2260 |      722340080 |          58905 |         319550 |        9551850 |       2216 |      705739265 |          58905 |         318395 |        9551850 |          44 |        16600815 |          180180 |          376915 |          656040 |              1108 |             360249120 |                114345 |                324940 |               9551850 |                            0 |                                0 |                                0 |                                0 |                                0 |                        0 |                            0 |                            0 |                            0 |                            0 |                    0 |                        0 |                        0 |                        0 |                        0 |                1108 |               345490145 |                   58905 |                  311465 |                 1170785 |                      20 |                     5014625 |                      180180 |                      250635 |                      358050 |                             0 |                                 0 |                                 0 |                                 0 |                                 0 |                   0 |                       0 |                       0 |                       0 |                       0 |                        0 |                            0 |                            0 |                            0 |                            0 |                  2 |                 860860 |                 338030 |                 430430 |                 522830 |                   22 |                 10725330 |                   423885 |                   487410 |                   656040 |
| TABLE       | isucon2       | artist      |       2236 |      466900280 |          25795 |         208670 |        3642485 |       2224 |      455622090 |          25795 |         204820 |        3642485 |          12 |        11278190 |          210595 |          939785 |         2962960 |              1112 |             224735280 |                 44660 |                201740 |               1176560 |                            0 |                                0 |                                0 |                                0 |                                0 |                        0 |                            0 |                            0 |                            0 |                            0 |                    0 |                        0 |                        0 |                        0 |                        0 |                1112 |               230886810 |                   25795 |                  207515 |                 3642485 |                       4 |                     2527140 |                      210595 |                      631785 |                     1158850 |                             0 |                                 0 |                                 0 |                                 0 |                                 0 |                   0 |                       0 |                       0 |                       0 |                       0 |                        0 |                            0 |                            0 |                            0 |                            0 |                  2 |                1629705 |                 699545 |                 814660 |                 930160 |                    6 |                  7121345 |                   438130 |                  1186570 |                  2962960 |
+-------------+---------------+-------------+------------+----------------+----------------+----------------+----------------+------------+----------------+----------------+----------------+----------------+-------------+-----------------+-----------------+-----------------+-----------------+-------------------+-----------------------+-----------------------+-----------------------+-----------------------+------------------------------+----------------------------------+----------------------------------+----------------------------------+----------------------------------+--------------------------+------------------------------+------------------------------+------------------------------+------------------------------+----------------------+--------------------------+--------------------------+--------------------------+--------------------------+---------------------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+-----------------------------+-----------------------------+-----------------------------+-----------------------------+-------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+---------------------+-------------------------+-------------------------+-------------------------+-------------------------+--------------------------+------------------------------+------------------------------+------------------------------+------------------------------+--------------------+------------------------+------------------------+------------------------+------------------------+----------------------+--------------------------+--------------------------+--------------------------+--------------------------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM table_lock_waits_summary_by_table WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 1\G
*************************** 1. row ***************************
                      OBJECT_TYPE: TABLE
                    OBJECT_SCHEMA: isucon2
                      OBJECT_NAME: stock
                       COUNT_STAR: 3844
                   SUM_TIMER_WAIT: 1580755330
                   MIN_TIMER_WAIT: 38885
                   AVG_TIMER_WAIT: 411180
                   MAX_TIMER_WAIT: 84922530
                       COUNT_READ: 3612
                   SUM_TIMER_READ: 1451608235
                   MIN_TIMER_READ: 38885
                   AVG_TIMER_READ: 401555
                   MAX_TIMER_READ: 84922530
                      COUNT_WRITE: 232
                  SUM_TIMER_WRITE: 129147095
                  MIN_TIMER_WRITE: 192885
                  AVG_TIMER_WRITE: 556325
                  MAX_TIMER_WRITE: 9873325
                COUNT_READ_NORMAL: 1806
            SUM_TIMER_READ_NORMAL: 503861820
            MIN_TIMER_READ_NORMAL: 48125
            AVG_TIMER_READ_NORMAL: 278740
            MAX_TIMER_READ_NORMAL: 84922530
     COUNT_READ_WITH_SHARED_LOCKS: 0
 SUM_TIMER_READ_WITH_SHARED_LOCKS: 0
 MIN_TIMER_READ_WITH_SHARED_LOCKS: 0
 AVG_TIMER_READ_WITH_SHARED_LOCKS: 0
 MAX_TIMER_READ_WITH_SHARED_LOCKS: 0
         COUNT_READ_HIGH_PRIORITY: 0
     SUM_TIMER_READ_HIGH_PRIORITY: 0
     MIN_TIMER_READ_HIGH_PRIORITY: 0
     AVG_TIMER_READ_HIGH_PRIORITY: 0
     MAX_TIMER_READ_HIGH_PRIORITY: 0
             COUNT_READ_NO_INSERT: 0
         SUM_TIMER_READ_NO_INSERT: 0
         MIN_TIMER_READ_NO_INSERT: 0
         AVG_TIMER_READ_NO_INSERT: 0
         MAX_TIMER_READ_NO_INSERT: 0
              COUNT_READ_EXTERNAL: 1806
          SUM_TIMER_READ_EXTERNAL: 947746415
          MIN_TIMER_READ_EXTERNAL: 38885
          AVG_TIMER_READ_EXTERNAL: 524755
          MAX_TIMER_READ_EXTERNAL: 13140050
          COUNT_WRITE_ALLOW_WRITE: 114
      SUM_TIMER_WRITE_ALLOW_WRITE: 40675635
      MIN_TIMER_WRITE_ALLOW_WRITE: 192885
      AVG_TIMER_WRITE_ALLOW_WRITE: 356510
      MAX_TIMER_WRITE_ALLOW_WRITE: 1143450
    COUNT_WRITE_CONCURRENT_INSERT: 0
SUM_TIMER_WRITE_CONCURRENT_INSERT: 0
MIN_TIMER_WRITE_CONCURRENT_INSERT: 0
AVG_TIMER_WRITE_CONCURRENT_INSERT: 0
MAX_TIMER_WRITE_CONCURRENT_INSERT: 0
              COUNT_WRITE_DELAYED: 0
          SUM_TIMER_WRITE_DELAYED: 0
          MIN_TIMER_WRITE_DELAYED: 0
          AVG_TIMER_WRITE_DELAYED: 0
          MAX_TIMER_WRITE_DELAYED: 0
         COUNT_WRITE_LOW_PRIORITY: 0
     SUM_TIMER_WRITE_LOW_PRIORITY: 0
     MIN_TIMER_WRITE_LOW_PRIORITY: 0
     AVG_TIMER_WRITE_LOW_PRIORITY: 0
     MAX_TIMER_WRITE_LOW_PRIORITY: 0
               COUNT_WRITE_NORMAL: 2
           SUM_TIMER_WRITE_NORMAL: 1497265
           MIN_TIMER_WRITE_NORMAL: 525525
           AVG_TIMER_WRITE_NORMAL: 748440
           MAX_TIMER_WRITE_NORMAL: 971740
             COUNT_WRITE_EXTERNAL: 116
         SUM_TIMER_WRITE_EXTERNAL: 86974195
         MIN_TIMER_WRITE_EXTERNAL: 418110
         AVG_TIMER_WRITE_EXTERNAL: 749595
         MAX_TIMER_WRITE_EXTERNAL: 9873325
1 row in set (0.00 sec)

テーブルごとのロック待ちの統計…なんだけど、i_sで確認するステータスとは違う変数名がならんでてどこがどう対応してるんだろう。
ブロックしてるクエリーとかも見られるi_sの方が優秀かなここは。

もういちど。

events_statements_summary_by_digest マジ優秀。

【2015/03/31 17:28】
フルパワーにしたらこんな感じだった。
日々の覚書: MySQLのperformance_schemaをフルパワーにするとどんな情報が見られるのか

↧

MySQLのperformance_schemaをフルパワーにするとどんな情報が見られるのか

March 31, 2015, 12:34 am

≫ Next: pt-online-schema-change(と、pt-table-checksumとかもろもろ)の--recursion-methodについて

≪ Previous: MySQLのperformance_schemaでどれくらいの情報が見られるのか

日々の覚書: MySQLのperformance_schemaでどれくらいの情報が見られるのかの続き。

* events_stages_summary_by_account_by_event_name
* events_stages_summary_by_host_by_event_name
* events_stages_summary_by_thread_by_event_name
* events_stages_summary_by_user_by_event_name
* events_stages_summary_global_by_event_name

mysql> SELECT * FROM events_stages_summary_by_account_by_event_name WHERE count_star <> 0 ORDER BY sum_timer_wait DESC LIMIT 3;
+------------+-----------+-------------------------+------------+----------------+----------------+----------------+----------------+
| USER       | HOST      | EVENT_NAME              | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT |
+------------+-----------+-------------------------+------------+----------------+----------------+----------------+----------------+
| isucon2app | localhost | stage/sql/Sending data  |       2509 | 39308705666000 |              0 |    15667080000 |   178525262000 |
| isucon2app | localhost | stage/sql/freeing items |       3000 |  7854180396000 |              0 |     2618060000 |    25104133000 |
| isucon2app | localhost | stage/sql/System lock   |       2743 |  2691080812000 |              0 |      981072000 |   244085523000 |
+------------+-----------+-------------------------+------------+----------------+----------------+----------------+----------------+
3 rows in set (0.00 sec)

デフォルトではOFFになっていたステージの情報(5.5まででいうprofiling)の情報が見られるようになる。
はかどりそうだけど、event_stageの情報をdigest(つまりクエリー単位)でサマライズして見られたらもっと良かったのになぁ。。
profileと同じようにピンポイントで有効にすれば本来いいはず。

* events_waits_summary_by_account_by_event_name
* events_waits_summary_by_host_by_event_name
* events_waits_summary_by_instance
* events_waits_summary_by_thread_by_event_name
* events_waits_summary_by_user_by_event_name
* events_waits_summary_global_by_event_name

mysql> SELECT * FROM events_waits_summary_by_account_by_event_name WHERE count_star <> 0 and event_name <> 'idle' and user = 'isucon2app' ORDER BY sum_timer_wait DESC;
+------------+-----------+------------------------------------------+------------+----------------+----------------+----------------+----------------+
| USER       | HOST      | EVENT_NAME                               | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT |
+------------+-----------+------------------------------------------+------------+----------------+----------------+----------------+----------------+
| isucon2app | localhost | wait/io/table/sql/handler                |   32424969 | 41901091208900 |          49280 |        1292060 |   107226966770 |
| isucon2app | localhost | wait/io/socket/sql/client_connection     |       6455 |  8661436551845 |         510125 |     1341818170 |    25122662950 |
| isucon2app | localhost | wait/io/file/innodb/innodb_log_file      |        364 |  1616073950645 |        8077685 |     4439763405 |   418325359760 |
| isucon2app | localhost | wait/io/file/innodb/innodb_data_file     |        657 |  1146723865595 |        1064525 |     1745393650 |   209724324810 |
| isucon2app | localhost | wait/synch/mutex/innodb/trx_mutex        |     699706 |    41160329980 |          20790 |          58520 |     3020631845 |
| isucon2app | localhost | wait/synch/mutex/mysys/THR_LOCK::mutex   |       9634 |    11706377760 |          26950 |        1215060 |    10035856215 |
| isucon2app | localhost | wait/lock/table/sql/handler              |      11344 |     6277293330 |          26950 |         553245 |      126401660 |
| isucon2app | localhost | wait/synch/mutex/innodb/trx_undo_mutex   |      82384 |     4336347400 |          24640 |          52360 |       41428695 |
| isucon2app | localhost | wait/synch/mutex/sql/THD::LOCK_thd_data  |      18669 |     2553290740 |          25410 |         136675 |       45770340 |
| isucon2app | localhost | wait/io/file/sql/FRM                     |        225 |      605735515 |         199045 |        2691920 |       43222025 |
| isucon2app | localhost | wait/synch/rwlock/innodb/fil_space_latch |        675 |       66989230 |          24640 |          98945 |         846615 |
| isucon2app | localhost | wait/io/file/sql/dbopt                   |          4 |       31597335 |         232925 |        7899045 |       27579475 |
| isucon2app | localhost | wait/synch/rwlock/sql/MDL_lock::rwlock   |         20 |        2514050 |          81235 |         125510 |         405405 |
+------------+-----------+------------------------------------------+------------+----------------+----------------+----------------+----------------+
13 rows in set (0.00 sec)

さっきとそんなに塩梅は変わらないものの、mutexの情報とか増えてる。

↓吊るしで同じクエリーを叩くとこれだけしか情報は出ない。

mysql> SELECT * FROM events_waits_summary_by_account_by_event_name WHERE count_star <> 0 and event_name <> 'idle' and user = 'isucon2app' ORDER BY sum_timer_wait DESC;
+------------+-----------+--------------------------------------+------------+----------------+----------------+----------------+----------------+
| USER       | HOST      | EVENT_NAME                           | COUNT_STAR | SUM_TIMER_WAIT | MIN_TIMER_WAIT | AVG_TIMER_WAIT | MAX_TIMER_WAIT |
+------------+-----------+--------------------------------------+------------+----------------+----------------+----------------+----------------+
| isucon2app | localhost | wait/io/table/sql/handler            |   30563374 | 30001032268130 |          48125 |         981365 |    65554244140 |
| isucon2app | localhost | wait/io/file/innodb/innodb_log_file  |        362 |   941252670050 |        7715785 |     2600145240 |    61855393600 |
| isucon2app | localhost | wait/io/file/innodb/innodb_data_file |        386 |   769021622600 |         872795 |     1992283755 |   110223802535 |
| isucon2app | localhost | wait/lock/table/sql/handler          |      10640 |     3912227935 |          25410 |         367675 |      136991855 |
| isucon2app | localhost | wait/io/file/sql/FRM                 |        160 |      453758305 |         201355 |        2835910 |       42650685 |
| isucon2app | localhost | wait/io/file/sql/dbopt               |          4 |       24409000 |         222530 |        6102250 |       21301280 |
+------------+-----------+--------------------------------------+------------+----------------+----------------+----------------+----------------+
6 rows in set (0.00 sec)

* mutex_instances

mysql> SELECT * FROM mutex_instances;
+-------------------------------------------------+-----------------------+---------------------+
| NAME                                            | OBJECT_INSTANCE_BEGIN | LOCKED_BY_THREAD_ID |
+-------------------------------------------------+-----------------------+---------------------+
| wait/synch/mutex/mysys/my_thread_var::mutex     |       140714335799552 |                NULL |
| wait/synch/mutex/mysys/my_thread_var::mutex     |       140714402908416 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714805632408 |                NULL |
| wait/synch/mutex/mysys/THR_LOCK::mutex          |       140714805616856 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714805660792 |                NULL |
| wait/synch/mutex/mysys/THR_LOCK::mutex          |       140714805657816 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714805659576 |                NULL |
| wait/synch/mutex/mysys/THR_LOCK::mutex          |       140714805645720 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714805679048 |                NULL |
| wait/synch/mutex/mysys/THR_LOCK::mutex          |       140714805586648 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714805601848 |                NULL |
| wait/synch/mutex/mysys/THR_LOCK::mutex          |       140714805602632 |                NULL |
| wait/synch/mutex/mysys/my_thread_var::mutex     |       140714201581952 |                NULL |
| wait/synch/mutex/sql/MDL_wait::LOCK_wait_status |              36850088 |                NULL |
| wait/synch/mutex/sql/THD::LOCK_thd_data         |              36853576 |                NULL |
| wait/synch/mutex/sql/MDL_wait::LOCK_wait_status |              42562344 |                NULL |
| wait/synch/mutex/sql/THD::LOCK_thd_data         |              42565832 |                NULL |
| wait/synch/mutex/sql/MDL_wait::LOCK_wait_status |              42577160 |                NULL |
| wait/synch/mutex/sql/THD::LOCK_thd_data         |              42580648 |                NULL |
| wait/synch/mutex/sql/MDL_wait::LOCK_wait_status |              42740984 |                NULL |
| wait/synch/mutex/sql/THD::LOCK_thd_data         |              42744472 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714604308568 |                NULL |
| wait/synch/mutex/sql/TABLE_SHARE::LOCK_ha_data  |       140714604309576 |                NULL |
| wait/synch/mutex/sql/MDL_wait::LOCK_wait_status |              42529224 |                NULL |
| wait/synch/mutex/sql/THD::LOCK_thd_data         |              42532712 |                NULL |
| wait/synch/mutex/sql/MDL_wait::LOCK_wait_status |              42595464 |                NULL |
| wait/synch/mutex/sql/THD::LOCK_thd_data         |              42598952 |                NULL |
+-------------------------------------------------+-----------------------+---------------------+
27 rows in set (0.00 sec)

これはむしろクエリー流している最中に見て、locked_by_thread_idが埋まりがちだったらインスタンスを分けるとかしたほうがいいのかしら。

パッと見これくらいなので、普段は吊るしのまま(バックグラウンドスレッドはOFFにしてもいい)か、setup_instrumentsのWHERE name LIKE 'stage%'を有効にだけしておいて、実際に計測したいときにsetup_consumersのevents_stages_currentをYesにしてやればいいですかね。

前にそんなことちょっと書いたなぁと思いつつ。
performance_schema_instrumentの設定方法 | GMOメディアエンジニアブログ
performance_schema_instrumentの設定方法 | GMOメディアエンジニアブログ

↧

pt-online-schema-change(と、pt-table-checksumとかもろもろ)の--recursion-methodについて

April 1, 2015, 1:45 am

≫ Next: MySQL 5.7のEXPLAINでようやく変なUsing whereが消えた

≪ Previous: MySQLのperformance_schemaをフルパワーにするとどんな情報が見られるのか

pt-online-schema-change(以下、pt-osc)をはじめとするPercona Toolkitのツール群には、--recursion-methodというオプションがあります。

pt-online-schema-change — Percona Toolkit Documentation

暗黙のデフォルトは"processlist,hosts"で、なんとなくprocesslist(SHOW PROCESSLISTからBinlog Dump Threadのホスト名を引っ張り挙げる)とhosts(SHOW SLAVE HOSTSの結果からホスト名とポート番号を引っ張り挙げる)をよしなにミックスしてくれそうな雰囲気なれど。

これ残念ながら単にprocesslistだけ指定した時と同じ結果になります。

( ´-｀).oO（全くトリアージされる気はしないけど、一応ばぐれぽも上げた
Bug #1438641 “Default value of --recursion-method, “processlist,...” : Bugs : Percona Toolkit

まあそれはいいとして、各--recursion-method個別の動作。動作環境は127.0.0.1:64057に5.7なマスター, 127.10.0.1:64056 に5.6なスレーブがぶら下がっている。

* processlist

$ PTDEBUG=1 pt-online-schema-change --execute --alter "Engine = InnoDB" h=127.10.0.1,D=d1,t=t1,u=root,P=64057 --recursion-method=processlist
..
# MasterSlave:4031 30199 Recursion methods: processlist
# MasterSlave:4041 30199 Connected to D=d1,P=64057,h=127.0.0.1,t=t1,u=root
# MasterSlave:4050 30199 SELECT @@SERVER_ID
# MasterSlave:4052 30199 Working on server ID 1057
# MasterSlave:4085 30199 Looking for slaves on D=d1,P=64057,h=127.0.0.1,t=t1,u=root using methods processlist
# MasterSlave:4092 30199 Finding slaves with _find_slaves_by_processlist
# MasterSlave:4154 30199 DBI::db=HASH(0x3051a08) SHOW GRANTS FOR CURRENT_USER()
# MasterSlave:4184 30199 DBI::db=HASH(0x3051a08) SHOW PROCESSLIST
..
# MasterSlave:4097 30199 Found 1 slaves
..
# MasterSlave:4041 30199 Connected to D=d1,P=64057,h=127.10.0.1,u=root
# MasterSlave:4050 30199 SELECT @@SERVER_ID
# MasterSlave:4052 30199 Working on server ID 1057
# MasterSlave:4058 30199 Server ID seen, or not what master said
..

mysql57> show processlist;
+----+------+------------------+------+------------------+------+----------------+------------------+
| Id | User | Host             | db   | Command          | Time | State          | Info             |
+----+------+------------------+------+------------------+------+----------------+------------------+
| 39 | root | 127.10.0.1:45008 | NULL | Binlog Dump GTID |  112 | Writing to net | NULL             |
| 46 | root | localhost        | NULL | Query            |    0 | starting       | show processlist |
+----+------+------------------+------+------------------+------+----------------+------------------+
2 rows in set (0.02 sec)

SHOW PROCESSLISTの結果からCommand =~ /Binlog Dump/なスレッドのHost部からホスト名を引いている。それ以外のパラメーターはもともとマスターに接続するために指定したDSN(h=127.0.0.1,D=d1,t=t1,u=root,P=64057)を流用するらしい。
なので、IPアドレスは127.10.0.1向きになっているけどポート番号がそのまま。

というわけで、
* マスターに接続するDSNに明示的にポートを指定している場合で
* スレーブでポート番号が違う場合
* マスターに接続するDSNに明示的にポートを指定していない場合(S= でソケットを指定しているとか)で
* スレーブが標準ポート(3306)以外を使っている場合
はprocesslistではスレーブの自動検出に失敗する。

* hosts

$ PTDEBUG=1 pt-online-schema-change --execute --alter "Engine = InnoDB" h=127.0.0.1,D=d1,t=t1,u=root,P=64057 --recursion-method=hosts
..
# MasterSlave:4031 30218 Recursion methods: hosts
# MasterSlave:4041 30218 Connected to D=d1,P=64057,h=127.0.0.1,t=t1,u=root
# MasterSlave:4050 30218 SELECT @@SERVER_ID
# MasterSlave:4052 30218 Working on server ID 1057
# MasterSlave:4085 30218 Looking for slaves on D=d1,P=64057,h=127.0.0.1,t=t1,u=root using methods hosts
# MasterSlave:4092 30218 Finding slaves with _find_slaves_by_hosts
# MasterSlave:4126 30218 DBI::db=HASH(0x3260a08) SHOW SLAVE HOSTS
# MasterSlave:4130 30218 Found some SHOW SLAVE HOSTS info
..
# MasterSlave:4097 30218 Found 1 slaves
# MasterSlave:4074 30218 Recursing from D=d1,P=64057,h=127.0.0.1,t=t1,u=root to D=d1,P=64056,h=,u=root
..
Cannot connect to D=d1,P=64056,h=,u=root
# pt_online_schema_change:8027 30218 0 slaves found
..

mysql57> show slave hosts;
+-----------+------+-------+-----------+--------------------------------------+
| Server_id | Host | Port  | Master_id | Slave_UUID                           |
+-----------+------+-------+-----------+--------------------------------------+
|      1056 |      | 64056 |      1057 | 5ceebb66-c191-11e4-b66a-001a4aa000e2 |
+-----------+------+-------+-----------+--------------------------------------+
1 row in set (0.00 sec)

SHOW SLAVE HOSTSの結果からHostとPortをパースして接続に行く。一見万能そうだが、--report-hostを指定していない場合は空欄になってしまって上手くパースできていない("Cannot connect to D=d1,P=64056,h=,u=root")
あと、MySQL 5.5.23からは--report-portを指定しなかった場合は暗黙のデフォルトとしてmysqldのポートが使われるのだが、それ以前の場合は3306が暗黙のデフォルトとして使われてしまうし、5.5.3より前のバージョンはそもそも--report-hostが設定されていない場合はSHOW SLAVE HOSTSに全く表示されない。

というわけで--report-hostと--report-portが正しく設定されている場合(5.5.23以降はreport-portは未指定でもいい)のみ使えるrecursion-methodで、これらはオンラインで変更できないオプションなので、pt-oscのためだけに変更するのはなんかコレジャナイ感しかしない。

MySQL :: MySQL 5.5 Reference Manual :: 13.7.5.34 SHOW SLAVE HOSTS Syntax

* dsn

というわけで、↑の条件を満たさない(非標準ポートで使っていて、--report-hostや--report-portが全てのmysqldに行き渡っているわけではない)場合に使えるrecursion-methodってこれしか残らないと思うんだけど、ぐぐっても(英語ですら)ほとんど情報がなかった。

mysql57> create table mysql.slave (id int, dsn varchar(255));
Query OK, 0 rows affected (0.67 sec)

mysql57> INSERT INTO mysql.slave VALUES (1, 'h=127.0.0.1,P=64056,u=root');
Query OK, 1 row affected (0.23 sec)

mysql57> SELECT * FROM mysql.slave;
+------+----------------------------+
| id   | dsn                        |
+------+----------------------------+
|    1 | h=127.0.0.1,P=64056,u=root |
+------+----------------------------+
1 row in set (0.12 sec)

ここは下準備。別にmysqlスキーマに作らなくてもいいし、テーブル名も何でもいい。pt-oscのオプションで任意の値を指定できる。
カラムも(マニュアルには色々書いてあるけど)`id`カラムと`dsn`カラムさえあればそれでいい。発行されるクエリーはSELECT dsn FROM .. ORDER BY idだったので、それが結果を返しさえすればOKだ。今のところは。

$ PTDEBUG=1 pt-online-schema-change --execute --alter "Engine = InnoDB" h=127.0.0.1,D=d1,t=t1,u=root,P=64057 --recursion-method="dsn=D=mysql,t=slave"
..
# MasterSlave:4598 30328 DSN table DSN: D=mysql,t=slave
..
# MasterSlave:4619 30328 SELECT dsn FROM `mysql`.`slave` ORDER BY id
# MasterSlave:4624 30328 DSN from DSN table: h=127.0.0.1,P=64056,u=root
..
# pt_online_schema_change:8027 30328 1 slaves found
# pt_online_schema_change:8052 30328 Will check slave lag on all slaves
# pt_online_schema_change:8070 30328 Checking slave replication filters
# MasterSlave:4268 30328 DBI::db=HASH(0x2a07458) SHOW MASTER STATUS
# MasterSlave:4272 30328 binlog_do_db= file=bin.000006 position=234290939 executed_gtid_set=5ceebb66-c191-11e4-b66a-001a4aa000e2:1-13,
# ecb991db-cb86-11e4-b75b-001a4aa000e2:1-613048 binlog_ignore_db=
# MasterSlave:4244 30328 DBI::db=HASH(0x2a07458) SHOW SLAVE STATUS
# MasterSlave:4570 30328 DBI::db=HASH(0x2a07458) SHOW VARIABLES LIKE 'slave_skip_errors'
..

この通り、--recursion-method="dsn=.."で指定したスキーマ, テーブルにクエリーを発行してそこからDSNを拾う。processlistの時と同じく、テーブルのdsnカラムに指定されていない部分はマスターに接続するときに使ったDSNをそのまま流用するらしい。ホスト名とポート番号だけでなくユーザーとパスワードも指定しておいた方がいい(や、rootでどのホストからでもアクセスできるならいいけど)

また、これをやっておくと多少遅延してもいいけど性能の悪いバックアップ用のスレーブのラグに引きずられて、pt-osc自体の進行が遅くなる…とかいうのも防げる。ビバ。

ちなみに接続先のDSNを間違えて、レプリケーションのつながっていないホストに接続してしまうと、

Waiting forever for new table `d1`.`_t1_new` to replicate to myhost...

( ´-｀).oO（マスターで_t1_newをCREATE TABLEした後にスレーブで_t1_newが作られてるか確認してスリープしながら無限ループしてるだけ(なので、すさまじく遅延したレプリケーション環境では出うる)んだけど本当にforever待ってしまうという。。

↧

MySQL 5.7のEXPLAINでようやく変なUsing whereが消えた

April 2, 2015, 10:50 pm

≫ Next: MySQL 5.7.6以降では暗黙のテンポラリーテーブルがあふれると死ぬ

≪ Previous: pt-online-schema-change(と、pt-table-checksumとかもろもろ)の--recursion-methodについて

や、EXPLAINの説明用の資料作ってて気付いたんですが、こんなサンプルテーブルに対して

mysql56> show create table cards\G
*************************** 1. row ***************************
       Table: cards
Create Table: CREATE TABLE `cards` (
  `seq` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `suite` varchar(6) NOT NULL,
  `number` tinyint(3) unsigned NOT NULL,
  UNIQUE KEY `seq` (`seq`),
  KEY `suite` (`suite`)
) ENGINE=InnoDB AUTO_INCREMENT=101 DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

mysql56> SELECT * FROM cards ORDER BY _rowid LIMIT 3;
+-----+--------------+--------+
| seq | suite        | number |
+-----+--------------+--------+
|   1 | ダイヤ       |      2 |
|   2 | クラブ       |      9 |
|   3 | スペード     |      3 |
+-----+--------------+--------+
3 rows in set (0.01 sec)

WHERE句だけをインデックスで解決するクエリーをEXPLAIN取ると

mysql56> explain extended SELECT * FROM cards WHERE suite= 'ハート' ORDER BY number;
+----+-------------+-------+------+---------------+-------+---------+-------+------+----------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key   | key_len | ref   | rows | filtered | Extra                                              |
+----+-------------+-------+------+---------------+-------+---------+-------+------+----------+----------------------------------------------------+
|  1 | SIMPLE      | cards | ref  | suite         | suite | 20      | const |   31 |   100.00 | Using index condition; Using where; Using filesort |
+----+-------------+-------+------+---------------+-------+---------+-------+------+----------+----------------------------------------------------+
1 row in set, 1 warning (0.00 sec)

mysql56> SELECT * FROM cards WHERE suite= 'ハート' ORDER BY number;
..
31 rows in set (0.00 sec)

mysql56> SHOW SESSION STATUS LIKE 'handler%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| Handler_commit             | 1     |
| Handler_delete             | 0     |
| Handler_discover           | 0     |
| Handler_external_lock      | 2     |
| Handler_mrr_init           | 0     |
| Handler_prepare            | 0     |
| Handler_read_first         | 0     |
| Handler_read_key           | 1     |
| Handler_read_last          | 0     |
| Handler_read_next          | 31    |
| Handler_read_prev          | 0     |
| Handler_read_rnd           | 0     |
| Handler_read_rnd_next      | 0     |
| Handler_rollback           | 0     |
| Handler_savepoint          | 0     |
| Handler_savepoint_rollback | 0     |
| Handler_update             | 0     |
| Handler_write              | 0     |
+----------------------------+-------+
18 rows in set (0.00 sec)

こんな風になるじゃないですか。31行をインデックスからフェッチしてUsing filesort。

それはいいんですが、インデックスだけでWHERE句を解決しているにも関わらずUsing whereが出るし、filteredは100.00以外の値を見た記憶がない。これ5.5からずっとそうだったし、バグなのか仕様なのかよくわからない。

これが5.7では、

mysql57> explain extended SELECT * FROM cards WHERE suite= 'ハート' ORDER BY number;
+----+-------------+-------+------------+------+---------------+-----------+---------+-------+------+----------+---------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key       | key_len | ref   | rows | filtered | Extra                                 |
+----+-------------+-------+------------+------+---------------+-----------+---------+-------+------+----------+---------------------------------------+
|  1 | SIMPLE      | cards | NULL       | ref  | idx_suite     | idx_suite | 17      | const |  100 |    31.00 | Using index condition; Using filesort |
+----+-------------+-------+------------+------+---------------+-----------+---------+-------+------+----------+---------------------------------------+
1 row in set, 2 warnings (0.00 sec)

mysql57> SELECT * FROM cards WHERE suite= 'ハート' ORDER BY number;
..
31 rows in set (0.00 sec)

mysql57> SHOW SESSION STATUS LIKE 'handler%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| Handler_commit             | 1     |
| Handler_delete             | 0     |
| Handler_discover           | 0     |
| Handler_external_lock      | 2     |
| Handler_mrr_init           | 0     |
| Handler_prepare            | 0     |
| Handler_read_first         | 0     |
| Handler_read_key           | 1     |
| Handler_read_last          | 0     |
| Handler_read_next          | 31    |
| Handler_read_prev          | 0     |
| Handler_read_rnd           | 0     |
| Handler_read_rnd_next      | 0     |
| Handler_rollback           | 0     |
| Handler_savepoint          | 0     |
| Handler_savepoint_rollback | 0     |
| Handler_update             | 0     |
| Handler_write              | 0     |
+----------------------------+-------+
18 rows in set (0.00 sec)

こうなってました。
テーブル全体がrowsの100件で、31.00%がフィルターされてってことなんでしょうかね？変なUsing whereは姿を見せなくなっています。

とはいえなんかこのfilteredカラム、ちょっと謎で、インデックスだけで解決できないクエリーにすると

mysql57> explain extended SELECT * FROM cards WHERE suite = 'ハート' AND number = 1 ORDER BY number;
+----+-------------+-------+------------+------+---------------+-----------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key       | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+-----------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | cards | NULL       | ref  | idx_suite     | idx_suite | 17      | const |   31 |    10.00 | Using where |
+----+-------------+-------+------------+------+---------------+-----------+---------+-------+------+----------+-------------+
1 row in set, 2 warnings (0.00 sec)

mysql57> SELECT * FROM cards WHERE suite = 'ハート' AND number = 1 ORDER BY number;
+-----+-----------+--------+
| seq | suite     | number |
+-----+-----------+--------+
|  41 | ハート    |      1 |
|  47 | ハート    |      1 |
|  50 | ハート    |      1 |
|  59 | ハート    |      1 |
|  96 | ハート    |      1 |
+-----+-----------+--------+
5 rows in set (0.00 sec)

rows= インデックスで解決できた31件、filterd= 10.00%は31 * 10.00% < 5とちょっと誤差が大きすぎる気がします。どうやって計算してるんだろ。

↧

MySQL 5.7.6以降では暗黙のテンポラリーテーブルがあふれると死ぬ

April 29, 2015, 11:21 pm

≫ Next: MySQL 5.7.5から複数のユーザーレベルロック(get_lock関数のアレ)を取れるようになった

≪ Previous: MySQL 5.7のEXPLAINでようやく変なUsing whereが消えた

MySQL 5.7.6から、JOINした時とかに作る暗黙のテンポラリーテーブルでMemoryストレージエンジンで収まらなくなった時に固定化するテンポラリーテーブル(Created_tmp_disk_tablesがカウントアップされるアレ)のストレージエンジンがInnoDBになった。

MySQL :: MySQL 5.7 Reference Manual :: 8.4.4 How MySQL Uses Internal Temporary Tables
MySQL :: MySQL 5.7 Reference Manual :: 5.1.4 Server System Variables

テンポラリーテーブル用(こっちはCREATE TEMPORARY TABLEで作るやつも含んだ言い方だと思いねえ)のテーブルスペースはREDOロギングを無効化してあったりといいこともたくさんあるんだけれど、mysqldを再起動しない限りibtmp1ファイルは決して小さくならない。

こんなエラーを見たことはないだろうか。俺はある。設定が古いサーバーだと特によくある。

mysql57> SELECT @@internal_tmp_disk_storage_engine;
+------------------------------------+
| @@internal_tmp_disk_storage_engine |
+------------------------------------+
| MYISAM                             |
+------------------------------------+
1 row in set (0.00 sec)

mysql57> SELECT * FROM (SELECT * FROM t1 WHERE val LIKE 'abc%') AS t1, (SELECT * FROM t1 WHERE val LIKE 'def%') AS t2 ORDER BY t1.num;
ERROR 126 (HY000): Incorrect key file for table '/tmp/#sql_2974_0.MYI'; try to repair it


$ tail error.log
..
2015-04-30T14:59:45.558360+09:00 87 [ERROR] /usr/mysql/5.7.7/bin/mysqld: Incorrect key file for table '/tmp/#sql_2974_0.MYI'; try to repair it
2015-04-30T14:59:45.573411+09:00 87 [ERROR] Got an error from unknown thread, /home/yoku0825/mysql-5.7.7-rc/storage/myisam/mi_write.c:223

暗黙のテンポラリーテーブルがtmpdirに作られて、容量が足りなくなるとこうなる。

$ ll /tmp
合計 792816
-rw-r----- 1 yoku0825 yoku0825 402935808  4月 30 14:59 2015 #sql_2974_0.MYD
-rw-r----- 1 yoku0825 yoku0825      1024  4月 30 14:59 2015 #sql_2974_0.MYI
-rw-rw-r-- 1 yoku0825 yoku0825 408888897  4月 30 14:45 2015 md5
drwx------ 2 yoku0825 tmux         4096  4月 17 15:45 2015 tmux-3012

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_00-lv_root
                       42G   40G   33M 100% /

暗黙のテンポラリーテーブルがDiskを食いきってクエリーがアボートされるとストレージ容量は元に戻る。
internal_tmp_disk_storage_engine= MyISAMのとき(MySQL 5.7.5とそれ以前)はこんな動作だった。

ところがInnoDBになるとibtmp1にこのデータを書くし、ibtmp1はmysqldの再起動まで容量が空かないので、

mysql57> SELECT @@internal_tmp_disk_storage_engine;
+------------------------------------+
| @@internal_tmp_disk_storage_engine |
+------------------------------------+
| INNODB                             |
+------------------------------------+
1 row in set (0.03 sec)

mysql57> SELECT * FROM (SELECT * FROM t1 WHERE val LIKE 'abc%') AS t1, (SELECT * FROM t1 WHERE val LIKE 'def%') AS t2 ORDER BY t1.num;
ERROR 1114 (HY000): The table '/tmp/#sql_2974_0' is full

$ tail error.log
..
2015-04-30T15:04:22.549484+09:00 87 [ERROR] InnoDB: posix_fallocate(): Failed to preallocate data for file ./ibtmp1, desired size 67108864 bytes. Operating system error number 28. Check that the disk is not full or a disk quota exceeded. Make sure the file system supports this function. Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
2015-04-30T15:04:22.590070+09:00 87 [Warning] InnoDB: 1048576 bytes should have been written. Only 794624 bytes written. Retrying again to write the rem

クエリーがアボートしても容量は開放されていない。

$ ll /usr/mysql/5.7.7/data/ibtmp1
-rw-r----- 1 yoku0825 yoku0825 485236736  4月 30 15:04 2015 /usr/mysql/5.7.7/data/ibtmp1

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_00-lv_root
                       42G   40G   17M 100% /

ベンチマーク取りながらうっかり100%突っ込んだ人とかならわかると思うんだけれど、DiskがFullな状態で更新かけるとbinlogに書き込もうとしたときにそこで詰まって

mysql57> INSERT INTO t1 VALUES (0, 'test');

何も言わずにmysqldは応答を返さない。

(gdb) bt
#0  0x00000038862aca3d in nanosleep () at ../sysdeps/unix/syscall-template.S:82
#1  0x00000038862ac8b0 in __sleep (seconds=0) at ../sysdeps/unix/sysv/linux/sleep.c:138
#2  0x0000000000e9a111 in wait_for_free_space (filename=0x7f3d6f428690 "./bin.000002", errors=)
    at /home/yoku0825/mysql-5.7.7-rc/mysys/errors.c:85
#3  0x0000000000e9147c in my_write (Filedes=19, Buffer=0x7f3d5772e0c2 "", Count=108, MyFlags=52)
    at /home/yoku0825/mysql-5.7.7-rc/mysys/my_write.c:90
#4  0x0000000000e7f579 in inline_mysql_file_write (info=0x1cfc0c8, need_append_buffer_lock=)
    at /home/yoku0825/mysql-5.7.7-rc/include/mysql/psi/mysql_file.h:1136
#5  my_b_flush_io_cache (info=0x1cfc0c8, need_append_buffer_lock=)
    at /home/yoku0825/mysql-5.7.7-rc/mysys/mf_iocache.c:1537
#6  0x0000000000e2a317 in MYSQL_BIN_LOG::flush_cache_to_file (this=0x1cfbdc0, end_pos_var=0x7f3d6fda9520)
    at /home/yoku0825/mysql-5.7.7-rc/sql/binlog.cc:8187
#7  0x0000000000e39a14 in MYSQL_BIN_LOG::ordered_commit (this=0x1cfbdc0, thd=0x7f3d55ffa000, all=,
    skip_commit=) at /home/yoku0825/mysql-5.7.7-rc/sql/binlog.cc:8467
#8  0x0000000000e39c62 in MYSQL_BIN_LOG::write_binlog_and_commit_engine (this=0x1cfbdc0, thd=0x7f3d55ffa000,
    all=) at /home/yoku0825/mysql-5.7.7-rc/sql/binlog.cc:7858
#9  0x0000000000e3a1d4 in MYSQL_BIN_LOG::commit (this=, thd=0x7f3d55ffa000, all=)
    at /home/yoku0825/mysql-5.7.7-rc/sql/binlog.cc:7643
#10 0x000000000079dc60 in ha_commit_trans (thd=0x7f3d55ffa000, all=,
    ignore_global_read_lock=) at /home/yoku0825/mysql-5.7.7-rc/sql/handler.cc:1615
#11 0x0000000000d1c0fe in trans_commit_stmt (thd=)
    at /home/yoku0825/mysql-5.7.7-rc/sql/transaction.cc:395
#12 0x0000000000c687ec in mysql_execute_command (thd=0x7f3d55ffa000) at /home/yoku0825/mysql-5.7.7-rc/sql/sql_parse.cc:4591
#13 0x0000000000c6acb0 in mysql_parse (thd=0x7f3d55ffa000, parser_state=)
    at /home/yoku0825/mysql-5.7.7-rc/sql/sql_parse.cc:5159
#14 0x0000000000c6c1c6 in dispatch_command (command=COM_QUERY, thd=0x7f3d55ffa000,
    packet=0x7f3d55c1b011 "INSERT INTO t1 VALUES (0, 'test')", packet_length=33)
    at /home/yoku0825/mysql-5.7.7-rc/sql/sql_parse.cc:1249
#15 0x0000000000d3a1dc in handle_connection (arg=)
    at /home/yoku0825/mysql-5.7.7-rc/sql/conn_handler/connection_handler_per_thread.cc:298
#16 0x0000000000f066b7 in pfs_spawn_thread (arg=0x7f3d693a2210)
    at /home/yoku0825/mysql-5.7.7-rc/storage/perfschema/pfs.cc:2147
#17 0x00000038866079d1 in start_thread (arg=0x7f3d6fdac700) at pthread_create.c:301
#18 0x00000038862e88fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

$ tail error.log
2015-04-30T15:12:50.592458+09:00 87 [ERROR] Disk is full writing './bin.000002' (Errcode: 28 - No space left on device). Waiting for someone to free space...
2015-04-30T15:12:50.592539+09:00 87 [ERROR] Retry in 60 secs. Message reprinted in 600 secs
2015-04-30T15:22:50.705673+09:00 87 [ERROR] Disk is full writing './bin.000002' (Errcode: 28 - No space left on device). Waiting for someone to free space...
2015-04-30T15:22:50.705730+09:00 87 [ERROR] Retry in 60 secs. Message reprinted in 600 secs
2015-04-30T15:32:50.831917+09:00 87 [ERROR] Disk is full writing './bin.000002' (Errcode: 28 - No space left on device). Waiting for someone to free space...
2015-04-30T15:32:50.831991+09:00 87 [ERROR] Retry in 60 secs. Message reprinted in 600 secs
2015-04-30T15:42:50.963577+09:00 87 [ERROR] Disk is full writing './bin.000002' (Errcode: 28 - No space left on device). Waiting for someone to free space...
2015-04-30T15:42:50.963636+09:00 87 [ERROR] Retry in 60 secs. Message reprinted in 600 secs
2015-04-30T15:52:51.098086+09:00 87 [ERROR] Disk is full writing './bin.000002' (Errcode: 28 - No space left on device). Waiting for someone to free space...
2015-04-30T15:52:51.098142+09:00 87 [ERROR] Retry in 60 secs. Message reprinted in 600 secs

ああ、~~一応ちゃんとリトライアウトはあるのか、よかった。。~~ 主人をibtmp1に殺されてからはや1時間くらいになりますが、リトライ *アウト* はしてなかった。まだクエリーはエラーになってくれない。たぶん、これからもなってくれない。

デフォルトのまま使うと間違ってでっかい暗黙のテンポラリーテーブル作るクエリー投げちゃった、てへ☆ が笑い事でなくなるので、

innodb_tmp_data_file_pathをautoextendじゃない形でサイズを決め打っておくか(innodb_data_file_pathと同じノリで記述できる。暗黙のデフォルトはibtmp1:12M:autoextend。コールドパラメーターなのでMySQLの再起動が必要) または internal_tmp_disk_storage_engineをMyISAMにセットしておくか（こっちはオンラインでOK）

秘伝のmy.cnfにinnodb_tmp_data_file_pathを足しておくか。。

↧

MySQL 5.7.5から複数のユーザーレベルロック(get_lock関数のアレ)を取れるようになった

April 30, 2015, 12:37 am

≫ Next: MySQL 5.7.5からはmysql.sock.lockというのを作るようになった

≪ Previous: MySQL 5.7.6以降では暗黙のテンポラリーテーブルがあふれると死ぬ

今までは

mysql56 1> SELECT @@pseudo_thread_id;
+--------------------+
| @@pseudo_thread_id |
+--------------------+
|                  3 |
+--------------------+
1 row in set (0.00 sec)

mysql56 1>SELECT get_lock('yoku0825', 1);
+-------------------------+
| get_lock('yoku0825', 1) |
+-------------------------+
|                       1 |
+-------------------------+
1 row in set (0.02 sec)

mysql56 2> SELECT is_used_lock('yoku0825');
+--------------------------+
| is_used_lock('yoku0825') |
+--------------------------+
|                        3 |
+--------------------------+
1 row in set (0.00 sec)

こんなロックを取った状態で次のロックを取ると

mysql56 1> SELECT get_lock('yoku08253', 1);
+--------------------------+
| get_lock('yoku08253', 1) |
+--------------------------+
|                        1 |
+--------------------------+
1 row in set (0.00 sec)

mysql56 2> SELECT is_used_lock('yoku0825');
+--------------------------+
| is_used_lock('yoku0825') |
+--------------------------+
|                     NULL |
+--------------------------+
1 row in set (0.00 sec)

何も言われずに古い方のロックはリリースされていた。
これが、

mysql57 1> SELECT @@pseudo_thread_id;
+--------------------+
| @@pseudo_thread_id |
+--------------------+
|                 89 |
+--------------------+
1 row in set (0.01 sec)

mysql57 1> SELECT get_lock('yoku0825', 1);
+-------------------------+
| get_lock('yoku0825', 1) |
+-------------------------+
|                       1 |
+-------------------------+
1 row in set (0.02 sec)

mysql57 2> SELECT is_used_lock('yoku0825');
+--------------------------+
| is_used_lock('yoku0825') |
+--------------------------+
|                       89 |
+--------------------------+
1 row in set (0.01 sec)

mysql57 1> SELECT get_lock('yoku08253', 1);
+--------------------------+
| get_lock('yoku08253', 1) |
+--------------------------+
|                        1 |
+--------------------------+
1 row in set (0.00 sec)

mysql57 2> SELECT is_used_lock('yoku0825');
+--------------------------+
| is_used_lock('yoku0825') |
+--------------------------+
|                       89 |
+--------------------------+
1 row in set (0.00 sec)

複数取っても勝手にリリースされなくなった。
ユーザーロックの状態もperformance_schemaで見られるようになった的なことが書いてあるんだけど

mysql57 2> SELECT * FROM performance_schema.metadata_locks;
Empty set (0.00 sec)

mysql57 2> SELECT @@performance_schema_max_metadata_locks;
+-----------------------------------------+
| @@performance_schema_max_metadata_locks |
+-----------------------------------------+
|                                      -1 |
+-----------------------------------------+
1 row in set (0.00 sec)

あれれ…まあいいや。
-1はautosizedって書いてあるけど、実際autoでいくつに設定されたんだかよくわからん。。

ところでアレだ。ロックを順次取れるようになると必ず現れてくるアイツ。そう、アイツ。

mysql57 2> SELECT get_lock('yoku0826', 1);
+-------------------------+
| get_lock('yoku0826', 1) |
+-------------------------+
|                       1 |
+-------------------------+
1 row in set (0.00 sec)

mysql57 2> SELECT get_lock('yoku08253', 1000);

mysql57 1> SELECT get_lock('yoku0826', 1);
ERROR 3058 (HY000): Deadlock found when trying to get user-level lock; try rolling back transaction/releasing locks and restarting lock acquisition.

Error 3058番。ちゃんと検出してくれるのはありがたいんだけど、この間も2番目のスレッドはget_lock関数で待ちっぱなしの状態で勝手にリリースはしてくれない。
release_lockかrelease_all_locksでリリースしてやらないとダメ。

mysql57 1> SELECT release_all_locks();
+---------------------+
| release_all_locks() |
+---------------------+
|                   2 |
+---------------------+
1 row in set (0.00 sec)

mysql57 2> SELECT get_lock('yoku08253', 1000);
+-----------------------------+
| get_lock('yoku08253', 1000) |
+-----------------------------+
|                           1 |
+-----------------------------+
1 row in set (1 min 30.79 sec)

InnoDBのデッドロックと違って *勝手にリリースしてくれない*ので、エラーをハンドルしたらrelease_lockするか接続を閉じればロックは開放されるのでそうしなければならない。
そう、お前のことだ、 *つなぎっぱなしのConnector/J氏*

MySQL :: MySQL 5.7 Release Notes :: Changes in MySQL 5.7.5 (2014-09-25, Milestone 15)

ここで"get_lock"って探すと詳しく書いてありました。

↧

MySQL 5.7.5からはmysql.sock.lockというのを作るようになった

May 10, 2015, 10:15 am

≫ Next: MySQL 5.6以降でクライアントに "Warning: Using a password on the command line interface can be insecure."と言われるのを防ぐいくつかの方法

≪ Previous: MySQL 5.7.5から複数のユーザーレベルロック(get_lock関数のアレ)を取れるようになった

A server could have its socket file taken over by a second server listening on different TCP/IP port but the same socket file. The socket file also would be unlinked by normal shutdown of the second server. To avoid this, the server now uses a lock file with the same name as the socket file and a .lock suffix. (For example, /tmp/mysql.sock has a lock file of /tmp/mysql.sock.lock.) The lock file contains the process ID of the server process that created the socket file. (Bug #17286856)
MySQL :: MySQL 5.7 Release Notes :: Changes in MySQL 5.7.5 (2014-09-25, Milestone 15)

やっぱり誰しも、--socketの設定間違えてUNIXソケットつぶしたことありますよねわかりますん！ orz

$ bin/mysqld --no-defaults --initialize --datadir=/tmp/mysql1
2015-05-10T17:04:32.148319Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-05-10T17:04:32.526383Z 0 [Warning] InnoDB: New log files created, LSN=45790
2015-05-10T17:04:32.562985Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2015-05-10T17:04:32.622323Z 0 [Warning] Failed to setup SSL
2015-05-10T17:04:32.622341Z 0 [Warning] SSL error: SSL context is not usable without certificate and private key
2015-05-10T17:04:32.622849Z 1 [Warning] A temporary password is generated for root@localhost: gOxQq0_u2l)B

$ bin/mysqld --no-defaults --socket=/tmp/mysql.sock --port=3306 --datadir=/tmp/mysql1
2015-05-10T17:04:54.881779Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-05-10T17:04:54.881843Z 0 [Warning] Insecure configuration for --secure-file-priv: Current value does not restrict location of generated files. Consider setting it to a valid, non-empty path.
2015-05-10T17:04:54.881879Z 0 [Note] bin/mysqld (mysqld 5.7.7-rc) starting as process 22478 ...
..
Version: '5.7.7-rc'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution

$ ll /tmp/mysql.sock*
srwxrwxrwx 1 yoku0825 yoku0825 0 May 11 02:04 /tmp/mysql.sock
-rw------- 1 yoku0825 yoku0825 6 May 11 02:04 /tmp/mysql.sock.lock

$ cat /tmp/mysql.sock.lock
22478

こうして起動したmysqldと
(ちなみに--no-defaultsで--log-timestamps=SYSTEMが打ち消されているので、ログの時間(UTF)とファイルシステムの時間(JST)が盛大にズレて見えている)
See also, 日々の覚書: MySQL 5.7で絶対に必要になると思うlog_timestampsの設定

$ bin/mysqld --no-defaults --initialize --datadir=/tmp/mysql2
2015-05-10T17:06:46.667942Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-05-10T17:06:47.268897Z 0 [Warning] InnoDB: New log files created, LSN=45790
2015-05-10T17:06:47.307634Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2015-05-10T17:06:47.367505Z 0 [Warning] Failed to setup SSL
2015-05-10T17:06:47.367526Z 0 [Warning] SSL error: SSL context is not usable without certificate and private key
2015-05-10T17:06:47.368373Z 1 [Warning] A temporary password is generated for root@localhost: au*pq3)kf6oR

$ bin/mysqld --no-defaults --socket=/tmp/mysql.sock --port=13306 --datadir=/tmp/mysql2
2015-05-10T17:07:18.521509Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-05-10T17:07:18.521566Z 0 [Warning] Insecure configuration for --secure-file-priv: Current value does not restrict location of generated files. Consider setting it to a valid, non-empty path.
2015-05-10T17:07:18.521601Z 0 [Note] bin/mysqld (mysqld 5.7.7-rc) starting as process 22603 ...
..
2015-05-10T17:07:18.648282Z 0 [ERROR] Another process with pid 22478 is using unix socket file.
2015-05-10T17:07:18.648287Z 0 [ERROR] Unable to setup unix socket lock file.
2015-05-10T17:07:18.648301Z 0 [ERROR] Aborting

$ ll /tmp/mysql.sock*
srwxrwxrwx 1 yoku0825 yoku0825 0 May 11 02:04 /tmp/mysql.sock
-rw------- 1 yoku0825 yoku0825 6 May 11 02:04 /tmp/mysql.sock.lock

$ cat /tmp/mysql.sock.lock
22478

datadir, portをずらしてsocketだけ一致させたmysqldを起動しようとすると、後から起動しようとした方はスタートアップ処理でmysql.sock.lockの存在を検知してAbortしてくれる。
なんか導入直後の5.7.5で試した時に転けるはいいけどmysql.sockはやっぱり消しちゃうみたいなダメな動作だった気がしていたんだけど、少なくとも5.7.7では正しく動作している。

↧

MySQL 5.6以降でクライアントに "Warning: Using a password on the command line interface can be insecure."と言われるのを防ぐいくつかの方法

May 17, 2015, 8:59 pm

≫ Next: 1つのbasedirに複数のMroongaさんをぶら下げる

≪ Previous: MySQL 5.7.5からはmysql.sock.lockというのを作るようになった

MySQL 5.6以降では、同梱のクライアント(mysqlコマンドラインクライアント, mysqladmin, mysqldumpあたりがよくありそう)に-pオプションで直接パスワードを渡すと、

$ mysql56 -utest -ptest -e "SELECT current_user()"
Warning: Using a password on the command line interface can be insecure.
+----------------+
| current_user() |
+----------------+
| test@localhost |
+----------------+

こんなのが出るようになりました。
読んで字のごとく、パスワードをコマンドラインインターフェイス(つまり-pオプション)で渡すのはセキュアじゃねーぞ、というワーニングです。
これ、標準エラー出力に吐くんですよね。

標準エラー出力に吐くということは、

$ crontab -l
1 3 * * * /usr/mysql/5.6.24/bin/mysqladmin -uflush -pxxxx flush-tables

週末に仕込んだこんなcronのエントリーがぼんぼんぼんぼんメールを投げてくるわけです。午前3時に。インセキュアだぞって。 '`,、'`,、( ´∀｀)'`,、'`,、

さてこのメッセージ、mysys_ssl/my_getopt.cc にハードコードされています。エラーログドリブンソースコードリーディングですね。

653 /**
654  * This function should be called to print a warning message
655  * if password string is specified on the command line.
656  */
657
658 void print_cmdline_password_warning()
659 {
660   static my_bool password_warning_announced= FALSE;
661
662   if (!password_warning_announced)
663   {
664     fprintf(stderr, "Warning: Using a password on the command line "
665             "interface can be insecure.\n");
666     (void) fflush(stderr);
667     password_warning_announced= TRUE;
668   }
669 }

mysql-server/my_getopt.cc at cac6fc837a5f72203058e4acc6b8b4dba8a98294 · mysql/mysql-server

( ´-｀).oO（password_warning_announcedってここ以外で使ってないので、if文なしでいきなりfprintfでいいんじゃないかって気がするんだけども。

これを出力させないように、いくつかの方法を考えてみました。セキュアかどうかは別問題です。出力させないことが大事です。

1) コマンドラインオプションから渡さずに、my.cnf系(~/.my.cnfとか、~/.mylogin.cnfも含む)で渡す。

5.6からは mysql_config_editorがありますね。かつてのわたしのお気に入りでした。日々の覚書: MySQL5.6 .mylogin.cnfで遊んでみる

$ mysql_config_editor set --login-path=mysql --user=test --password
Enter password:

$ mysql56 -e "SELECT current_user()"
+----------------+
| current_user() |
+----------------+
| test@localhost |
+----------------+

使　っ　て　ま　せ　ん　'`,、'`,、( ´∀｀)'`,、'`,、

MySQL 5.6にできたサーバーと、まだ5.6未満で稼動しているサーバーでスクリプト作り分けるのが ~~めんどくさい~~管理が煩雑になってしまうのが嫌。if文書きたくない。~/.my.cnfなら5.6未満でも使えますが、シェルスクリプトが ~/.my.cnf に依存するようになるので管理対象が増えて嫌。

あと.mylogin.cnf別にセキュアじゃなかったし => セキュアそうでセキュアじゃない少しセキュアな気分になれるmysql_config_editor , ~/.my.cnfだって平文で書くからセキュアじゃないし。

2) MYSQL_PWD環境変数を使ってパスワードを渡す。

これ知らない人多そう。MYSQL_PWD環境変数が設定されていてかつ -pオプションが指定されていない場合、MYSQL_PWD環境変数の中身がパスワードとして扱われます。つまりがパスワード未設定時のデフォルト値をオーバーライドしてくれる。

というわけでこれを使うと

$ MYSQL_PWD="test" mysql56 -utest -e "SELECT current_user()"
+----------------+
| current_user() |
+----------------+
| test@localhost |
+----------------+

全然セキュアになってませんが、インセキュアだと怒られることはなくなります。管理対象が増えるわけでなくてもとのcronエントリーを

$ crontab -l
1 3 * * * MYSQL_PWD=xxxx /usr/mysql/5.6.24/bin/mysqladmin -uflush flush-tables

とだけ書き換えればいいですね。らくちん。

3) mysqlコマンドラインクライアントだけダウングレードする。

yumでrpmを突っ込んでいる環境だとつらい気がしますが、バイナリーやソースコードから突っ込んでいるならこれも手です。
5.5 => 5.6のmysqlコマンドラインクライアントの新機能ってhistignoreや.mylogin.cnf対応くらいじゃないかと思うので(5.6 => 5.7はCtrl + C対応とsyslogがある)、別に最新じゃなくてもいいやって割り切りは十分アリじゃないかと思います。

4) ほげる。

オチは想像がついていると思いますが、print_cmdline_password_warningをほげります。もうバッサリreturn (void) 0; だけでいいんじゃないかな。

$ diff -c mysys_ssl/my_getopt.cc{.orig,}
*** mysys_ssl/my_getopt.cc.orig 2015-03-26 01:34:52.000000000 +0900
--- mysys_ssl/my_getopt.cc      2015-05-18 12:35:56.314531829 +0900
***************
*** 657,671 ****

  void print_cmdline_password_warning()
  {
!   static my_bool password_warning_announced= FALSE;
!
!   if (!password_warning_announced)
!   {
!     fprintf(stderr, "Warning: Using a password on the command line "
!             "interface can be insecure.\n");
!     (void) fflush(stderr);
!     password_warning_announced= TRUE;
!   }
  }


--- 657,663 ----

  void print_cmdline_password_warning()
  {
!   return (void) 0;
  }


$ client/mysql -utest -ptest -S /usr/mysql/5.6.24/data/mysql.sock -e "SELECT current_user()"
+----------------+
| current_user() |
+----------------+
| test@localhost |
+----------------+

ちょっと別件でmysqlコマンドラインクライアントをほげってrpmにすることが決まってるので、まさかの4) を採用するかも知れません。

↧

1つのbasedirに複数のMroongaさんをぶら下げる

May 27, 2015, 11:42 pm

≫ Next: 何も考えずに真っ新なCentOS 6.6にMySQL 5.7をyumで叩き込むメモ

≪ Previous: MySQL 5.6以降でクライアントに "Warning: Using a password on the command line interface can be insecure."と言われるのを防ぐいくつかの方法

複数バージョンのGroonga / Mroongaで挙動の違いを調べる時に、いちいち/usr/local/mysqlを複数作るのが面倒なので手順をメモ。

コマンドの羅列はここ。 https://gist.github.com/yoku0825/a85643cd9b5a4dcd8e1c

mysqldにINSTALL PLUGINする場合、SONAMEで指定されたファイルをplugin_dirから読み出すので、plugin_dir(暗黙のデフォルトはbasedir/lib/plugin) だけを打ち分けてやればOKなはず。

$ wget http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.24-linux-glibc2.5-x86_64.tar.gz
$ tar xzf mysql-5.6.24-linux-glibc2.5-x86_64.tar.gz
$ sudo ln -s $PWD/mysql-5.6.24-linux-glibc2.5-x86_64 /usr/local/mysql
$ wget http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.24.tar.gz
$ tar xzf mysql-5.6.24.tar.gz

$ sudo useradd mysql

ここまではフツウ。

$ sudo mkdir /usr/local/mysql/{stable_lib,nightly_lib}

plugin_dir用のディレクトリを2つ作ってやる。

$ wget http://packages.groonga.org/source/groonga/groonga-5.0.3.tar.gz
$ tar xzf groonga-5.0.3.tar.gz
$ cd groonga-5.0.3
$ ./configure --prefix=/usr/local/groonga_stable
$ make
$ sudo make install
$ cd ../

$ wget http://packages.groonga.org/nightly/groonga-5.0.3.2015.05.28.tar.gz
$ tar xzf groonga-5.0.3.2015.05.28.tar.gz
$ cd groonga-5.0.3.2015.05.28
$ ./configure --prefix=/usr/local/groonga_nightly
$ make
$ sudo make install
$ cd ../

2つのGroongaを違うディレクトリーにインストールして、

$ wget http://packages.groonga.org/source/groonga-normalizer-mysql/groonga-normalizer-mysql-1.0.9.tar.gz
$ tar xzf groonga-normalizer-mysql-1.0.9.tar.gz
$ cd groonga-normalizer-mysql-1.0.9
$ ./configure --prefix=/usr/local/groonga_stable PKG_CONFIG_PATH=/usr/local/groonga_stable/lib/pkgconfig
$ make
$ sudo make install
$ make clean

$ ./configure --prefix=/usr/local/groonga_nightly PKG_CONFIG_PATH=/usr/local/groonga_nightly/lib/pkgconfig
$ make
$ sudo make install
$ cd ../

$ wget http://packages.groonga.org/source/mroonga/mroonga-5.02.tar.gz
$ tar xzf mroonga-5.02.tar.gz
$ cd mroonga-5.02
$ ./configure --prefix=/usr/local/groonga_stable --with-mysql-source=../mysql-5.6.24 --with-mysql-config=/usr/local/mysql/bin/mysql_config PKG_CONFIG_PATH=/usr/local/groonga_stable/lib/pkgconfig
$ sed -ib 's|/usr/local/mysql/lib/plugin|/usr/local/mysql/stable_lib|' Makefile
$ make
$ sudo make install
$ make clean

$ ./configure --prefix=/usr/local/groonga_nightly --with-mysql-source=../mysql-5.6.24 --with-mysql-config=/usr/local/mysql/bin/mysql_config PKG_CONFIG_PATH=/usr/local/groonga_nightly/lib/pkgconfig
$ sed -ib 's|/usr/local/mysql/lib/plugin|/usr/local/mysql/nightly_lib|' Makefile
$ make
$ sudo make install
$ cd ..

groonga-normalizer-mysqlとMroongaを、PKG_CONFIG_PATHの値を書き換えつつmakeしてインストールする。
Mroongaのconfigureスクリプトの中でha_mroonga.soをインストールする先がmysql_config --pluginから取られているので、これをMakefileの中でいじってからmakeしてやると、それぞれのplugin_dirにインストールされます。手でコピーしてもいいです。

$ cd /usr/local/mysql
$ scripts/mysql_install_db --user=mysql --datadir=./stable_data
$ bin/mysqld_safe --no-defaults --user=mysql --datadir=./stable_data --socket=/tmp/stable.sock --port=13306 --plugin-dir=/usr/local/mysql/stable_lib &

$ scripts/mysql_install_db --user=mysql --datadir=./nightly_data
$ bin/mysqld_safe --no-defaults --user=mysql --datadir=./nightly_data --socket=/tmp/nightly.sock --port=23306 --plugin-dir=/usr/local/mysql/nightly_lib &

$ alias stable="/usr/local/mysql/bin/mysql -uroot -S /tmp/stable.sock -P 13306"
$ alias nightly="/usr/local/mysql/bin/mysql -uroot -S /tmp/nightly.sock -P 23306"

$ stable < /usr/local/groonga_stable/share/mroonga/install.sql
$ stable -e "create database d1"
$ stable stable < data.sql

$ nightly < /usr/local/groonga_stable/share/mroonga/install.sql
$ nightly -e "create database d1"
$ nightly nightly < data.sql

$ stable -e "select @@plugin_dir"
+------------------------------+
| @@plugin_dir                 |
+------------------------------+
| /usr/local/mysql/stable_lib/ |
+------------------------------+

$ nightly -e "select @@plugin_dir"
+-------------------------------+
| @@plugin_dir                  |
+-------------------------------+
| /usr/local/mysql/nightly_lib/ |
+-------------------------------+

$ ldd /usr/local/mysql/*_lib/ha_mroonga.so
/usr/local/mysql/nightly_lib/ha_mroonga.so:
        linux-vdso.so.1 =>  (0x00007fffeaf76000)
        libgroonga.so.0 => /usr/local/groonga_nightly/lib/libgroonga.so.0 (0x00007f00adc09000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f00ad8f7000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f00ad673000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f00ad2df000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f00ad0c8000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f00aceb2000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f00accae000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f00acaa5000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003517400000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f00ac888000)
/usr/local/mysql/stable_lib/ha_mroonga.so:
        linux-vdso.so.1 =>  (0x00007fff6c7ff000)
        libgroonga.so.0 => /usr/local/groonga_stable/lib/libgroonga.so.0 (0x00007f13f4307000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f13f3ff5000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f13f3d71000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f13f39dd000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f13f37c6000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f13f35b0000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f13f33ac000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f13f31a3000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003517400000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f13f2f86000)

あとはdatadirをそれぞれ用意してやって、データを流し込めばおしまいと。
mysqld_multiを使って起動するならこんな感じのmy.cnfにしておく。

$ sudo vim /etc/my.cnf
[mysqld_multi]
no-log

[mysqld1]
basedir   = /usr/local/mysql
mysqld    = /usr/local/mysql/bin/mysqld_safe
datadir   = /usr/local/mysql/stable_data
socket    = /tmp/stable.sock
port      = 13006
plugin-dir= /usr/local/mysql/stable_lib

[mysqld2]
basedir   = /usr/local/mysql
mysqld    = /usr/local/mysql/bin/mysqld_safe
datadir   = /usr/local/mysql/nightly_data
socket    = /tmp/nightly.sock
port      = 23006
plugin-dir= /usr/local/mysql/nightly_lib

$ mysqld_multi start 1,2
mysqld_multi log file version 2.16; run: Thu May 28 15:39:09 2015

Starting MySQL servers
150528 15:39:09 mysqld_safe Adding '/usr/lib64/libjemalloc.so.1' to LD_PRELOAD for mysqld
150528 15:39:09 mysqld_safe Adding '/usr/lib64/libjemalloc.so.1' to LD_PRELOAD for mysqld
150528 15:39:09 mysqld_safe Logging to '/usr/local/mysql/stable_data/error.log'.
150528 15:39:09 mysqld_safe Logging to '/usr/local/mysql/nightly_data/error.log'.
150528 15:39:09 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/stable_data
150528 15:39:09 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/nightly_data

これであとはやりたい放題。

↧