PyRFC/SAP-RFC是否支持大数据集自动Chunking功能？

Hey there, let's get straight to it: SAP-RFC doesn't offer built-in automatic chunking for large result sets—you’ll need to implement this logic yourself, even though I know that’s not the answer you were hoping for.

From community discussions and library issue trackers, the general consensus is that fetching large datasets (like your 100,000 rows in 1,000-row batches) requires custom pagination work. Here’s a straightforward way to set this up without too much hassle:

• First, verify if the RFC module you’re using supports pagination parameters. Most SAP RFCs designed for large datasets include parameters like MAX_ROWS (to define your batch size) and SKIP (to specify how many rows to skip before fetching the next batch).
• Set up a loop where you increment the SKIP value by your batch size (1000, in your case) with each RFC call. Keep looping until the returned result set has fewer rows than your batch size—that’s your signal you’ve fetched the last chunk of data.
• Add basic error handling to account for edge cases like network interruptions, and consider adding a checkpoint system to track how many rows you’ve successfully retrieved so you can resume fetching if something goes wrong.

It’s not the out-of-the-box solution you wanted, but this pattern is standard when working with large SAP datasets via RFC, and it’s fairly simple to implement once you have the right pagination parameters sorted.

内容的提问来源于stack exchange，提问作者guettli