Erlang/OTP分布式聊天应用数据库服务nodedown错误排查求助

问题场景 #

基于Erlang/OTP开发分布式聊天应用，包含聊天服务器、Mnesia数据库逻辑，引入数据库服务（gen_server实现）作为中间层。聊天服务器与数据库均采用分布式架构，可通过聊天服务器生成带节点名的新用户节点，Mnesia表会同步到所有集群节点。

当前问题：直接调用数据库逻辑代码可正常访问数据，但通过数据库服务执行存储、查询等操作时，触发nodedown错误，所有节点实际处于运行状态。

启动与错误日志 #

数据库服务启动输出 #

(john@DESKTOP-RD414DV)7> database_server:start_link([node()|nodes()]).
{local,database_server} (<0.129.0>) starting...

查询操作错误输出 #

(john@DESKTOP-RD414DV)9> database_server:getalldb("mary@DESKTOP-RD414DV").
** exception exit: {{nodedown,database_server},
                    {gen_server,call,
                                [{local,database_server},
                                 {get_all_db,"mary@DESKTOP-RD414DV"}]}}
     in function  gen_server:call/2 (gen_server.erl, line 367)

核心代码片段 #

数据库逻辑代码（database_logic.erl） #

-export([install/1, get_db/1, get_all_dbe/1, delete_db/1, store_db/4]).
-include_lib("stdlib/include/qlc.hrl").
-record(userDetails, {node,username, location, gender}).

%%initialize database
install(Nodes) ->
  ok = mnesia:create_schema(Nodes),
  rpc:multicall(Nodes, application, start, [mnesia]),
  try
      mnesia:table_info(type,userDetails)
  catch
      exit:_ ->
        mnesia:create_table(userDetails, [{attributes, record_info(fields, userDetails)},
          {type, bag},
          {disc_copies, Nodes}])
  end.

store_db(Node, Username, Location, Gender) ->
  F = fun() ->
    mnesia:write(#userDetails{node =Node, username = Username, location = Location, gender = Gender})
      end,
  mnesia:transaction(F).

get_db(Node) ->
  F = fun() ->
    Query = qlc:q([X || X <- mnesia:table(userDetails),
      X#userDetails.node =:= Node]),
    Results = qlc:e(Query),
    lists:map(fun(Item) -> Item#userDetails.username end, Results)
      end,
  mnesia:transaction(F).

get_all_dbe(Node) ->
  F = fun() ->
    Query = qlc:q([X || X <- mnesia:table(userDetails),
      X#userDetails.node =:= Node]),
    Results = qlc:e(Query),
    lists:map(fun(Item) -> {Item#userDetails.username, Item#userDetails.location, Item#userDetails.gender} end, Results)
      end,
  mnesia:transaction(F).

delete_db(Node) ->
  F = fun() ->
    Query = qlc:q([X || X <- mnesia:table(userDetails),
      X#userDetails.node =:= Node]),
    Results = qlc:e(Query),

    FF = fun() ->
      lists:foreach(fun(Result) -> mnesia:delete_object(Result) end, Results)
         end,
    mnesia:transaction(FF)
      end,
  mnesia:transaction(F).

数据库服务代码（database_server.erl） #

-behaviour(gen_server).

-export([start_link/1, store/4, getalldb/1, delete/1]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,
  code_change/3]).

-define(SERVER, ?MODULE).

-record(database_server_state, {}).

start_link(Nodes) ->
  gen_server:start_link({local, ?SERVER}, ?MODULE, Nodes, []).

init(Nodes) ->
  process_flag(trap_exit, true),
  io:format("~p (~p) starting...~n", [{local, ?MODULE}, self()]),
  database_logic:install(Nodes),
  {ok, #database_server_state{}}.

store(Node, Username, Location, Gender) ->
  gen_server:call({local, ?MODULE}, {store_db, Node, Username, Location, Gender}).

getalldb(Node) ->
  gen_server:call({local, ?MODULE}, {get_all_db, Node}).

delete(Node) ->
  gen_server:call({local, ?MODULE}, {delete, Node}).

handle_call({store_db, Node, Username, Location, Gender}, _From, State = #database_server_state{}) ->
  database_logic:store_db(Node, Username, Location, Gender),
  io:format("userdetails are saved!"),
  {reply, ok, State};

handle_call({get_all_db, Node}, _From, State = #database_server_state{}) ->
  Y = database_logic:get_all_dbe(Node),
  {_, [{Name, Location, Gender}]} = Y,
  io:format("SENDER NAME: ~p\t LOCATION: ~p\t GENDER: ~p~n", [Name,Location,Gender]),
  {reply, ok, State};

handle_call({delete, Node},  _From, State = #database_server_state{}) ->
  database_logic:delete_db(Node),
  io:format("~p node data deleted!", [Node]),
  {reply, ok, State}.

handle_cast(_Request, State = #database_server_state{}) ->
  {noreply, State}.

handle_info(_Info, State = #database_server_state{}) ->
  {noreply, State}.

terminate(_Reason, _State = #database_server_state{}) ->
  ok.

code_change(_OldVsn, State = #database_server_state{}, _Extra) ->
  {ok, State}.

问题根源分析 #

1. gen_server本地注册限制：数据库服务使用{local, ?MODULE}进行本地注册，该方式仅允许当前节点的进程访问服务。在分布式场景下，跨节点调用或集群内其他节点尝试访问时，会因找不到本地注册的进程而抛出类似nodedown的错误（实际是进程不存在）。
2. Mnesia与gen_server的差异：直接调用数据库逻辑正常，是因为Mnesia本身是分布式数据库，操作会自动在集群节点间同步，无需依赖进程注册；而gen_server的访问受限于注册范围。
3. 代码逻辑小错误：install/1中mnesia:table_info(type, userDetails)参数顺序错误，应为mnesia:table_info(userDetails, type)，导致初始化时总是触发catch分支创建表；get_all_db的handle_call中硬编码匹配单条结果，存在崩溃风险。

解决方案 #

1. 改用全局注册gen_server #

将数据库服务的注册方式改为全局注册，允许集群内所有节点访问：

• 修改start_link/1：

start_link(Nodes) ->
  gen_server:start_link({global, ?SERVER}, ?MODULE, Nodes, []).

• 修改所有调用函数的目标标识：

store(Node, Username, Location, Gender) ->
  gen_server:call({global, ?MODULE}, {store_db, Node, Username, Location, Gender}).

getalldb(Node) ->
  gen_server:call({global, ?MODULE}, {get_all_db, Node}).

delete(Node) ->
  gen_server:call({global, ?MODULE}, {delete, Node}).

2. 修复Mnesia初始化逻辑 #

修正install/1中的错误，并增加schema已存在的容错处理：

install(Nodes) ->
  % 处理schema已存在的情况
  case mnesia:create_schema(Nodes) of
    ok -> ok;
    {error, {_, {already_exists, _}}} -> ok
  end,
  rpc:multicall(Nodes, application, start, [mnesia]),
  try
      % 修正table_info参数顺序
      mnesia:table_info(userDetails, type)
  catch
      exit:_ ->
        mnesia:create_table(userDetails, [{attributes, record_info(fields, userDetails)},
          {type, bag},
          {disc_copies, Nodes}])
  end.

3. 增强gen_server handle_call的容错性 #

修改get_all_db的handle_call逻辑，避免硬编码匹配结果数量导致崩溃：

handle_call({get_all_db, Node}, _From, State) ->
  case database_logic:get_all_dbe(Node) of
    {atomic, []} ->
      io:format("No data found for node ~p~n", [Node]),
      {reply, {ok, []}, State};
    {atomic, Results} ->
      lists:foreach(fun({Name, Loc, Gen}) ->
        io:format("SENDER NAME: ~p\t LOCATION: ~p\t GENDER: ~p~n", [Name, Loc, Gen])
      end, Results),
      {reply, {ok, Results}, State};
    {aborted, Reason} ->
      io:format("Query failed: ~p~n", [Reason]),
      {reply, {error, Reason}, State}
  end;

4. 确认集群节点连通性 #

执行以下命令验证集群状态：

• 查看当前节点列表：nodes().
• 手动连接未加入的节点：net_adm:ping('mary@DESKTOP-RD414DV').（返回pong则连接成功）

内容的提问来源于stack exchange，提问作者Ke_Sandaru