咨询可实现字符串转唯一ID的Python库及ID反查方法

Great question! You absolutely can map strings to unique positive integer IDs (and reverse the lookup) in Python—you don’t even need fancy external libraries, though there are simple built-in tools and patterns to make this straightforward. Let’s walk through the best options for your use case, since you need bidirectional lookup and have data stored in a TXT file:

This approach uses two dictionaries to maintain a direct, conflict-free mapping between strings and IDs, and saves the mapping to a file so it persists between sessions. IDs start at 1 (guaranteeing positive integers) and increment for new strings.

Example Code: #

import json
from pathlib import Path

# Define paths for your data and mapping file
DATA_FILE = "your_data.txt"
MAP_FILE = "string_id_mapping.json"

def load_mappings():
    """Load existing string-ID mappings from file, or initialize empty ones."""
    if Path(MAP_FILE).exists():
        with open(MAP_FILE, "r") as f:
            data = json.load(f)
            # Convert ID keys back to integers (JSON stores them as strings)
            return data["str_to_id"], {int(k): v for k, v in data["id_to_str"].items()}
    return {}, {}

def save_mappings(str_to_id, id_to_str):
    """Save the current mappings to file."""
    with open(MAP_FILE, "w") as f:
        json.dump({
            "str_to_id": str_to_id,
            "id_to_str": {k: v for k, v in id_to_str.items()}
        }, f)

def get_id(string, str_to_id, id_to_str):
    """Get the ID for a string, creating a new one if it doesn't exist."""
    if string not in str_to_id:
        # Assign the next available positive integer ID
        new_id = max(id_to_str.keys(), default=0) + 1
        str_to_id[string] = new_id
        id_to_str[new_id] = string
        save_mappings(str_to_id, id_to_str)
    return str_to_id[string]

def get_string(id_num, id_to_str):
    """Reverse lookup: get the string for a given ID."""
    return id_to_str.get(id_num, None)

# ----------------------
# Usage Example
# ----------------------
# Load existing mappings
str_to_id, id_to_str = load_mappings()

# Read strings from your TXT file
with open(DATA_FILE, "r") as f:
    strings = [line.strip() for line in f if line.strip()]

# Process each string to get its ID
for s in strings:
    string_id = get_id(s, str_to_id, id_to_str)
    print(f"String: '{s}' → ID: {string_id}")

# Reverse lookup example
sample_id = 1
sample_string = get_string(sample_id, id_to_str)
print(f"ID {sample_id} → String: '{sample_string}'")

Pros: #

• 100% conflict-free (no chance of two strings getting the same ID)
• No external dependencies
• Fast lookup for small-to-medium datasets

Cons: #

• Not ideal for extremely large datasets (dictionaries can use significant memory)

If you’re working with a huge number of strings, using SQLite (a built-in Python module) is more efficient. It handles persistence, unique constraints, and lookups seamlessly, with minimal memory overhead.

Example Code: #

import sqlite3

DATA_FILE = "your_data.txt"
DB_FILE = "string_ids.db"

def init_database():
    """Create the mapping table if it doesn't exist."""
    conn = sqlite3.connect(DB_FILE)
    cursor = conn.cursor()
    # Create table with auto-incrementing positive ID and unique string constraint
    cursor.execute('''
        CREATE TABLE IF NOT EXISTS string_mapping (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            string TEXT UNIQUE NOT NULL
        )
    ''')
    conn.commit()
    conn.close()

def get_id(string):
    """Get or create an ID for a string."""
    conn = sqlite3.connect(DB_FILE)
    cursor = conn.cursor()
    # Check if the string already exists
    cursor.execute("SELECT id FROM string_mapping WHERE string = ?", (string,))
    result = cursor.fetchone()
    if result:
        string_id = result[0]
    else:
        # Insert new string (auto-generates next ID starting at 1)
        cursor.execute("INSERT INTO string_mapping (string) VALUES (?)", (string,))
        string_id = cursor.lastrowid
        conn.commit()
    conn.close()
    return string_id

def get_string(id_num):
    """Reverse lookup: get the string for an ID."""
    conn = sqlite3.connect(DB_FILE)
    cursor = conn.cursor()
    cursor.execute("SELECT string FROM string_mapping WHERE id = ?", (id_num,))
    result = cursor.fetchone()
    conn.close()
    return result[0] if result else None

# ----------------------
# Usage Example
# ----------------------
# Initialize the database
init_database()

# Read strings from TXT file
with open(DATA_FILE, "r") as f:
    strings = [line.strip() for line in f if line.strip()]

# Process each string
for s in strings:
    string_id = get_id(s)
    print(f"String: '{s}' → ID: {string_id}")

# Reverse lookup example
sample_id = 2
sample_string = get_string(sample_id)
print(f"ID {sample_id} → String: '{sample_string}'")

Pros: #

• Efficient for large datasets (uses disk storage instead of loading all mappings into memory)
• Built-in unique constraint prevents duplicate IDs/strings
• Supports complex queries if needed later

Cons: #

• Slightly more setup than the manual dictionary approach

• Don’t use one-way hash functions (like hashlib or uuid.uuid5): These are one-way—you can’t reverse the lookup to get the original string. They also carry a small risk of hash collisions (two different strings producing the same ID), which breaks your uniqueness requirement.
• Always persist your mapping: If you don’t save the mapping to a file/database, you’ll lose the ID-string correspondence when your script ends.

内容的提问来源于stack exchange，提问作者saad