Python数组中查找无子集的唯一集合及Numpy数组布尔标记问题

Got it, let's work through your two problems with concrete, Python/NumPy-focused solutions. I'll break each down clearly so you can adapt them to your exact use case.

First, let's clarify: you want to extract the unique elements from your main array that do NOT appear in a specified subset. Here's a clean, efficient way to do this, whether you're working with standard Python lists or NumPy arrays:

import numpy as np

# Example main array (each element is a sub-array; adjust to your data shape)
main_array = np.array([[1,2], [3,4], [1,2], [5,6], [3,4], [7,8]])
# The subset of elements you want to exclude
exclude_subset = np.array([[1,2], [3,4]])

# Convert NumPy rows to tuples (since arrays aren't hashable for set operations)
main_unique_set = set(tuple(row) for row in main_array)
exclude_set = set(tuple(row) for row in exclude_subset)

# Calculate the set difference to get unique elements not in the subset
unique_non_subset = np.array(list(main_unique_set - exclude_set))

print("Unique elements not in the excluded subset:")
print(unique_non_subset)
# Output:
# [[5 6]
#  [7 8]]

How this works: #

• We convert each row of the NumPy arrays to tuples so they can be stored in Python sets (sets require hashable elements).
• Set difference (main_unique_set - exclude_set) instantly gives us elements that exist in the main array but not in the excluded subset.
• Finally, we convert the result back to a NumPy array for consistency with your workflow.

You mentioned you have a 100000×20 array, need to search along the 20-feature axis per record, and mark matches in a same-sized all-zero array (1 for matches, 0 otherwise). I'll cover two common scenarios here—pick the one that fits your definition of "information subset":

Scenario 1: Mark individual features that belong to a value subset #

If your "subset" is a collection of specific values (e.g., {2, 5, 7}), and you want to mark every position in the array where the feature equals one of these values:

import numpy as np

# Generate dummy data (replace with your actual dataset)
data = np.random.randint(0, 10, size=(100000, 20))
# The value subset to match
target_values = {2, 5, 7}
# Initialize all-zero marker array
markers = np.zeros_like(data, dtype=int)

# Use NumPy's vectorized operation to mark matches
markers[np.isin(data, list(target_values))] = 1

# Check the first 5 rows of results
print("First 5 rows of marker array:")
print(markers[:5])

Why this is efficient: #

• np.isin() creates a boolean array where each element is True if it's in the target subset.
• We use this boolean array as a mask to set corresponding positions in the marker array to 1. No slow loops needed—NumPy handles the vectorization under the hood, which is critical for large 100k-row datasets.

Scenario 2: Mark positions where a continuous feature subsequence matches #

If your "subset" is a continuous sequence of features (e.g., [3, 1, 4]), and you want to mark every position in each row that falls within this matching subsequence:

import numpy as np

# Generate dummy data
data = np.random.randint(0, 10, size=(100000, 20))
# The target continuous subsequence
target_subseq = np.array([3, 1, 4])
markers = np.zeros_like(data, dtype=int)

# Optimized sliding window approach (way faster than looping row-by-row)
subseq_length = len(target_subseq)
# Create sliding window view of the data along the feature axis
windows = np.lib.stride_tricks.sliding_window_view(data, window_shape=subseq_length, axis=1)
# Check which windows match the target subsequence
match_windows = np.all(windows == target_subseq, axis=2)

# Expand window matches to mark all positions in the subsequence
for i in range(subseq_length):
    markers[:, i:i+match_windows.shape[1]] += match_windows.astype(int)
# Ensure no duplicate marks (set values >1 back to 1)
markers[markers > 1] = 1

# Check how many rows have matches
matched_row_indices = np.where(markers.sum(axis=1) > 0)[0]
print(f"Found {len(matched_row_indices)} records containing the target subsequence")
print("Example marker row for a matched record:")
print(markers[matched_row_indices[0]])

How this optimization works: #

• sliding_window_view creates a view of the data where each row is split into overlapping windows of the subsequence length (no extra memory used).
• We compare all windows to the target subsequence in one vectorized operation, then expand those matches to mark every position in the subsequence. This avoids slow Python loops and handles the 100k rows efficiently.

内容的提问来源于stack exchange，提问作者Will