DRBD双节点高可用集群维护重启后MySQL服务无法启动的故障排查请求

大家好，我这边有个双节点高可用集群，用DRBD做数据同步。最近外部公司做完维护重启系统后，MySQL资源直接停摆了，折腾半天没搞定，想请各位大佬帮忙分析下问题根源。

一、DRBD状态信息 #

执行drbd-overview命令后，两台节点的输出如下：

主节点（clsstd1）： #

0:home    Connected Primary/Secondary UpToDate/UpToDate  C r-----
1:storage Connected Secondary/Primary UpToDate/UpToDate  C r-----
2:mysql   StandAlone Secondary/Unknown UpToDate/Outdated r-----

从节点（clsstd2）： #

0:home    Connected Secondary/Primary UpToDate/UpToDate  C r-----
1:storage Connected Primary/Secondary UpToDate/UpToDate  C r-----
2:mysql   StandAlone Primary/Unknown UpToDate/Outdated   r-----

二、已尝试的操作与发现 #

• 按照其他论坛的建议，执行了/etc/init.d/mysql start命令，但完全没用，MySQL还是启动失败。
• 检查两台节点的/var/lib/mysql目录时，发现节点2（clsstd2）没有ibdata1文件，但节点1（clsstd1）存在这个文件，不确定这是不是关键问题。

三、MySQL错误日志（clsstd2.err） #

节点2的MySQL启动日志报错信息如下：

mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql  
/usr/sbin/mysqld: Table ‘mysql.plugin’ doesn’t exist

[ERROR] Can’t open the mysql.plugin table. Please run mysql_upgrade to create it.

InnoBD: operating system error number 13 in a file operation.
InnoBD: The error means mysqld does not have access rights to the directory.
InnoBD: File name ./ibdata1
InnoBD: File operation call: ‘create’
InnoBD: Cannot continue operation

mysqld_safe mysqld from pid file /var/lib/mysql/clsstd2.pid ended

四、Pacemaker集群状态（crm_mon -1） #

执行crm_mon -1查看集群资源，所有服务资源都处于停止状态：

=========
Stack: openais
Current DC: clsstd1 – partition with quorum
Version: 1.1.5-1.1.e15-01e86afaaa6da8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
4 Resources configured.
==========

Online: [ clsstd1 clsstd2 ]

Full list of resources:
Resources group all
virtual_ip_1        (ocf::heartbeat:IPaddr2):       stopped
virtual_ip_2        (ocf::heartbeat:IPaddr2):       stopped
virtual_ip_3        (ocf::heartbeat:IPaddr2):       stopped
fs_home (ocf::heartbeat:Filesystem):    stopped
fs_mysql    (ocf::heartbeat:Filesystem):    stopped
fs_storage  (ocf::heartbeat:Filesystem):    stopped
mysql       (ocf::heartbeat:mysql):     stopped
httpd       (ocf::heartbeat:apache):        stopped
swengined   (lsb:/user/lib/ocf/resource.d/streamwide/swengine):   stopped

Master/Slave Set: ms_drbd_home [drbd_home]
Masters: [ clsstd1 ]
Slaves: [ clsstd2 ]

Master/Slave Set: ms_drbd_mysql [drbd_mysql]
Masters: [ clsstd1 ]
Slaves: [ clsstd2 ]

Master/Slave Set: ms_drbd_mysql [drbd_storage]
Masters: [ clsstd1 ]
Slaves: [ clsstd2 ]

Migration summary:
*  Node clsstd1:
*  Node clsstd2:

现在完全摸不着头绪，DRBD的mysql资源状态异常（StandAlone、Outdated），加上MySQL启动时的权限和表不存在错误，想请教下各位该从哪里入手排查？

备注：内容来源于stack exchange，提问作者Iván Jf