如何从Java传递Kerberos凭证调用HTTP接口终止YARN作业

核心逻辑 #

Kerberos认证场景下的HTTP请求需通过SPNEGO协议传递凭证，Java中可通过原生HttpURLConnection结合JAAS配置，或使用Apache HttpClient的Kerberos扩展来实现认证请求。

具体实现方案 #

方案1：使用Java原生HttpURLConnection #

1. 配置Kerberos与JAAS文件
   • 编写krb5.conf（指定KDC地址和域信息）：

     [libdefaults]
         default_realm = YOUR_REALM.COM
         dns_lookup_kdc = false
         dns_lookup_realm = false
         ticket_lifetime = 24h
         renew_lifetime = 7d
         forwardable = true
     
     [realms]
         YOUR_REALM.COM = {
             kdc = kdc.your-realm.com:88
             admin_server = kdc.your-realm.com:749
         }
     

   • 编写jaas.conf（配置客户端登录模块）：

     com.sun.security.jgss.krb5.initiate {
         com.sun.security.auth.module.Krb5LoginModule required
         useKeyTab=true
         keyTab="/path/to/your/service.keytab"
         principal="your-service-principal@YOUR_REALM.COM"
         storeKey=true;
     };
     

2. 编写认证请求代码

   import java.io.OutputStream;
   import java.net.HttpURLConnection;
   import java.net.URL;
   import javax.security.auth.Subject;
   import javax.security.auth.login.LoginContext;
   import javax.security.auth.login.LoginException;
   
   public class YarnJobTerminator {
       public static void main(String[] args) throws Exception {
           // 加载配置文件
           System.setProperty("java.security.auth.login.config", "/path/to/jaas.conf");
           System.setProperty("java.security.krb5.conf", "/path/to/krb5.conf");
           System.setProperty("sun.security.krb5.debug", "true"); // 调试用，可关闭
   
           // 登录获取Kerberos主体
           LoginContext loginContext = new LoginContext("com.sun.security.jgss.krb5.initiate");
           try {
               loginContext.login();
               Subject subject = loginContext.getSubject();
   
               // 在认证上下文内执行HTTP请求
               Subject.doAs(subject, () -> {
                   try {
                       URL url = new URL("http://yarn-rm-host:8088/ws/v1/cluster/apps/{your-app-id}/state");
                       HttpURLConnection conn = (HttpURLConnection) url.openConnection();
                       conn.setRequestMethod("PUT");
                       conn.setRequestProperty("Content-Type", "application/json");
                       conn.setDoOutput(true);
   
                       // 构造终止请求体
                       String requestBody = "{\"state\":\"KILLED\"}";
                       try (OutputStream os = conn.getOutputStream()) {
                           byte[] input = requestBody.getBytes("utf-8");
                           os.write(input, 0, input.length);
                       }
   
                       System.out.println("Response Code: " + conn.getResponseCode());
                       conn.disconnect();
                   } catch (Exception e) {
                       e.printStackTrace();
                   }
                   return null;
               });
           } catch (LoginException e) {
               e.printStackTrace();
           } finally {
               loginContext.logout();
           }
       }
   }
   

方案2：使用Apache HttpClient（更简洁） #

1. 添加Maven依赖

   <dependency>
       <groupId>org.apache.httpcomponents.client5</groupId>
       <artifactId>httpclient5</artifactId>
       <version>5.2.1</version>
   </dependency>
   <dependency>
       <groupId>org.apache.httpcomponents.client5</groupId>
       <artifactId>httpclient5-kerberos</artifactId>
       <version>5.2.1</version>
   </dependency>
   

2. 编写请求代码

   import org.apache.hc.client5.http.classic.methods.HttpPut;
   import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
   import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
   import org.apache.hc.client5.http.impl.classic.HttpClients;
   import org.apache.hc.client5.http.impl.auth.SPNegoSchemeFactory;
   import org.apache.hc.core5.http.ContentType;
   import org.apache.hc.core5.http.io.entity.StringEntity;
   
   public class YarnJobTerminatorHttpClient {
       public static void main(String[] args) throws Exception {
           // 配置Kerberos参数
           System.setProperty("java.security.krb5.conf", "/path/to/krb5.conf");
           System.setProperty("sun.security.krb5.debug", "true");
   
           // 创建支持SPNEGO认证的HttpClient
           CloseableHttpClient httpClient = HttpClients.custom()
                   .setDefaultAuthSchemeRegistry(registry -> {
                       registry.register("Negotiate", new SPNegoSchemeFactory(true));
                       return registry;
                   })
                   .build();
   
           try {
               HttpPut putRequest = new HttpPut("http://yarn-rm-host:8088/ws/v1/cluster/apps/{your-app-id}/state");
               String requestBody = "{\"state\":\"KILLED\"}";
               putRequest.setEntity(new StringEntity(requestBody, ContentType.APPLICATION_JSON));
   
               try (CloseableHttpResponse response = httpClient.execute(putRequest)) {
                   System.out.println("Response Code: " + response.getCode());
               }
           } finally {
               httpClient.close();
           }
       }
   }
   

   注：使用HttpClient时需确保JAAS配置正确，可通过系统属性java.security.auth.login.config指定配置文件路径。

关键注意事项 #

• 确保keytab文件权限正确（仅允许运行程序的用户读取）
• YARN ResourceManager的SPN格式通常为yarn/<hostname>@YOUR_REALM.COM，需与主体配置匹配
• 开启sun.security.krb5.debug=true可查看Kerberos认证详细日志，便于定位403的具体原因（如凭证失效、SPN不匹配等）

内容的提问来源于stack exchange，提问作者rajani