Hi there, let's break down this tricky cross-platform TLS behavior issue you're hitting in the Arti project—your core suspicion is absolutely on the right track, and the difference between OpenSSL and Apple's Secure Transport when handling buffer flushing is almost certainly the root cause here.

Why OpenSSL "magically" works on Linux #

OpenSSL’s architecture relies heavily on its BIO (Basic Input/Output) abstraction layer, which adds a lot of hidden behavior that can indirectly flush your custom DataStream buffer:

• When you call TLS write operations via native-tls’s Rust bindings, OpenSSL’s BIO layer often triggers automatic flushing under the hood. For example, socket-based BIOs will frequently flush data when the TLS record layer requires it (e.g., to send a complete TLS record fragment), or when OpenSSL’s internal flow control logic determines it’s time to push data to the network.
• OpenSSL’s SSL objects also perform implicit BIO flushes during key TLS lifecycle events—like after handshake completion, or when transitioning between session states. These flushes propagate down to your DataStream, pushing out any buffered data even if you didn’t explicitly call a flush method.
• Many default BIO configurations have "flush-on-write" semantics built in, so even small writes can trigger a full buffer flush without you knowing it.

In short, OpenSSL’s BIO layer acts as an intermediary that compensates for missing explicit flushes in your custom stream.

Why Secure Transport fails on macOS #

Apple’s Secure Transport takes a much more direct approach to IO, with no equivalent to OpenSSL’s BIO layer:

• Its core write API (SSLWrite) reads directly from the data source you provide (your DataStream) without any intermediate abstraction to handle buffer flushing. It only accesses data that’s already available in your stream’s current readable state.
• If your DataStream stores data in an internal buffer and doesn’t expose it until explicitly flushed, Secure Transport will never see that buffered data. This leads to incomplete request data being sent to the server, which eventually times out and closes the connection—matching the "connection closed via error" you’re seeing.
• Secure Transport expects the caller to manage all application-level buffer flushing explicitly. It doesn’t add any hidden logic to push data out of your custom stream.

How to validate and fix this #

To confirm your theory and resolve the issue:

1. Test with explicit flushing: Modify your code to call a manual flush on your DataStream immediately after writing data to the native-tls session. If the macOS issue disappears, that’s definitive proof the missing flush was the problem.
2. Adjust your DataStream logic:
   • Add an explicit flush method (async, since you’re using tokio) that pushes all buffered data to the underlying stream.
   • Consider adding automatic flush triggers in your stream’s write method—for example, flushing when the buffer reaches a certain size, or when the stream detects it’s being used with a TLS layer that requires explicit flushing.
3. Check native-tls crate specifics: The tokio-native-tls bindings might have platform-specific configuration options or documentation notes about Secure Transport’s flushing requirements that you can leverage.

Final note #

Cross-platform TLS issues like this are common because different implementations prioritize different design goals: OpenSSL prioritizes flexibility and self-containment (hence the BIO layer), while Secure Transport prioritizes tight integration with Apple’s system IO model. Your initial analysis was spot-on—this is all about implicit vs. explicit buffer flushing behavior between the two libraries.

If you get stuck on validating the behavior or adjusting your stream implementation, feel free to share more details about how your internal buffering is structured, and we can dive deeper into tailored fixes!