批量改名 + 编码统一:工程化最后一公里

Bulk Rename & Encoding: Production-Ready Output

分析做完了,要把结果文件交给老板/同事/客户 — 文件命名规范、编码统一、目录结构清晰,是工程化的最后一公里。

1. UTF-8 编码统一

import os


from pathlib import Path





for txt_file in Path('raw_data').rglob('*.txt'):


    try:


        with open(txt_file, 'rb') as f:


            raw = f.read()


        text = raw.decode('gbk', errors='ignore')


        with open(txt_file, 'w', encoding='utf-8') as f:


            f.write(text)


    except Exception as e:


        print(f'跳过 {txt_file}: {e}')

2. 批量改名(按时间 / 来源 / 关键词)

import re


from datetime import datetime





def rename_comment_file(filepath):


    p = Path(filepath)


    text = p.read_text(encoding='utf-8')[:500]





    date_match = re.search(r'(\d{4}-\d{2}-\d{2})', text)


    date = date_match.group(1) if date_match else datetime.now().strftime('%Y-%m-%d')





    source = 'unknown'


    if '淘宝' in text: source = 'taobao'


    elif '京东' in text: source = 'jd'


    elif '抖音' in text: source = 'douyin'





    words = jieba.lcut(text)


    keyword = max(set(words), key=words.count) if words else 'mixed'





    new_name = f'{date}_{source}_{keyword}.txt'


    new_path = p.parent / new_name


    p.rename(new_path)





for f in Path('messy_data/').glob('*.txt'):


    rename_comment_file(f)

3. 完整流水线整合

前面 8 节教程(基础/分词/统计/去重/清洗/批量改名)就是文本分析的工程化完整链路:

raw_data/


  → 1_read.py       # 读 CSV/TXT


  → 2_clean.py      # 正则清洗


  → 3_dedup.py      # 去重


  → 4_filter.py     # 过滤水军


  → 5_rename.py     # 批量改名


clean_data/         # 输出,可直接喂给分析模块


  2026-06-01_taobao_手机.txt


  2026-06-01_jd_电视.txt