如何提取、记录并编辑DataCompy的DataFrame对比结果？

Hey there! Let's break down your DataCompy questions with practical examples that fit right into your existing workflow.

1. Extracting Comparison Results & Generating Log Files #

DataCompy gives you both human-readable reports and structured DataFrame outputs to work with. Here's how to leverage both:

Extract Structured Results #

The datacompy.Compare object has built-in properties that let you access specific subsets of the comparison directly as Pandas DataFrames. For your use case, these are the most useful:

• comp.intersect_rows: Rows present in both DataFrames (may have value mismatches)
• comp.mismatch_rows: Rows that exist in both DataFrames but have conflicting values
• comp.df1_unq_rows: Rows that only exist in your "Original" DataFrame (df_db1)
• comp.df2_unq_rows: Rows that only exist in your "New" DataFrame (df_db2)

Generate Log Files #

You can save the full text report (the same one you print with comp.report()) to a log file, or export the structured DataFrames to CSV/Excel for later analysis. Add this code right after you run comp.matches() in your loop:

# Save the full comparison report to a text log
with open(f"comparison_log_{index}.txt", "w") as log_file:
    log_file.write(comp.report())

# Export structured mismatch/unique rows to CSV files for deeper dives
comp.mismatch_rows.to_csv(f"mismatch_rows_{index}.csv", index=False)
comp.df1_unq_rows.to_csv(f"original_only_rows_{index}.csv", index=False)
comp.df2_unq_rows.to_csv(f"new_only_rows_{index}.csv", index=False)

This will create separate logs and data files for each SQL query comparison, making it easy to track results across your loop.

2. Editing Comparison Results #

Since all the structured results are Pandas DataFrames, you can edit them just like any other DataFrame—delete rows, modify values, filter subsets, etc. Here are some common use cases:

Delete Specific Rows #

If you want to exclude certain rows from your results (e.g., rows with a specific ID or value), filter the DataFrame:

# Remove rows where a specific column has a value you want to exclude
filtered_mismatches = comp.mismatch_rows[comp.mismatch_rows["your_column_name"] != "value_to_exclude"]

# Drop rows by index
filtered_mismatches = filtered_mismatches.drop(index=[10, 20, 30])

Modify Result Content #

You can update values in the result DataFrames if needed (e.g., correcting a known data discrepancy):

# Update a specific value in the mismatch rows
filtered_mismatches.loc[filtered_mismatches["row_index"] == 123, "Original_your_column"] = "corrected_value"

Create a Custom Edited Report #

After editing the DataFrames, you can build your own custom report to document the changes. For example:

custom_report = f"""
Custom Edited Comparison Report (Query {index})
=============================================
Total mismatches after filtering: {len(filtered_mismatches)}

Filtered Mismatch Rows:
{filtered_mismatches.to_string()}
"""

# Save the custom report to a file
with open(f"edited_comparison_log_{index}.txt", "w") as f:
    f.write(custom_report)

内容的提问来源于stack exchange，提问作者shubham kumar