sal/src/text

Text Processing Utilities

A collection of Rust utilities for common text processing operations.

Overview

This module provides functions for text manipulation tasks such as:

• Removing indentation from multiline strings
• Adding prefixes to multiline strings
• Normalizing filenames and paths
• Text replacement (regex and literal) with file operations

Functions

Text Indentation

dedent(text: &str) -> String

Removes common leading whitespace from multiline strings.

let indented = "    line 1\n    line 2\n        line 3";
let dedented = dedent(indented);
assert_eq!(dedented, "line 1\nline 2\n    line 3");

Features:

• Analyzes all non-empty lines to determine minimum indentation
• Preserves empty lines but removes all leading whitespace from them
• Treats tabs as 4 spaces for indentation purposes

prefix(text: &str, prefix: &str) -> String

Adds a specified prefix to each line of a multiline string.

let text = "line 1\nline 2\nline 3";
let prefixed = prefix(text, "    ");
assert_eq!(prefixed, "    line 1\n    line 2\n    line 3");

Filename and Path Normalization

name_fix(text: &str) -> String

Normalizes filenames by:

• Converting to lowercase
• Replacing whitespace and special characters with underscores
• Removing non-ASCII characters
• Collapsing consecutive special characters into a single underscore

assert_eq!(name_fix("Hello World"), "hello_world");
assert_eq!(name_fix("File-Name.txt"), "file_name.txt");
assert_eq!(name_fix("Résumé"), "rsum");

path_fix(text: &str) -> String

Applies name_fix() to the filename portion of a path while preserving the directory structure.

assert_eq!(path_fix("/path/to/File Name.txt"), "/path/to/file_name.txt");
assert_eq!(path_fix("./relative/path/to/DOCUMENT-123.pdf"), "./relative/path/to/document_123.pdf");

Features:

• Preserves paths ending with / (directories)
• Only normalizes the filename portion, leaving the path structure intact
• Handles both absolute and relative paths

Text Replacement

TextReplacer

A flexible text replacement utility that supports both regex and literal replacements with a builder pattern.

// Regex replacement
let replacer = TextReplacer::builder()
    .pattern(r"\bfoo\b")
    .replacement("bar")
    .regex(true)
    .add_replacement()
    .unwrap()
    .build()
    .unwrap();

let result = replacer.replace("foo bar foo baz"); // "bar bar bar baz"

Features:

• Supports both regex and literal string replacements
• Builder pattern for fluent configuration
• Multiple replacements in a single pass
• Case-insensitive matching (for regex replacements)
• File reading and writing operations

Multiple Replacements

let replacer = TextReplacer::builder()
    .pattern("foo")
    .replacement("qux")
    .add_replacement()
    .unwrap()
    .pattern("bar")
    .replacement("baz")
    .add_replacement()
    .unwrap()
    .build()
    .unwrap();

let result = replacer.replace("foo bar foo"); // "qux baz qux"

File Operations

// Replace in a file and get the result as a string
let result = replacer.replace_file("input.txt")?;

// Replace in a file and write back to the same file
replacer.replace_file_in_place("input.txt")?;

// Replace in a file and write to a new file
replacer.replace_file_to("input.txt", "output.txt")?;

Case-Insensitive Matching

let replacer = TextReplacer::builder()
    .pattern("foo")
    .replacement("bar")
    .regex(true)
    .case_insensitive(true)
    .add_replacement()
    .unwrap()
    .build()
    .unwrap();

let result = replacer.replace("FOO foo Foo"); // "bar bar bar"

Usage

Import the functions from the module:

use your_crate::text::{dedent, prefix, name_fix, path_fix, TextReplacer};

Examples

Cleaning up indented text from a template

let template = "
    <div>
        <h1>Title</h1>
        <p>
            Some paragraph text
            with multiple lines
        </p>
    </div>
";

let clean = dedent(template);
// Result:
// <div>
//     <h1>Title</h1>
//     <p>
//         Some paragraph text
//         with multiple lines
//     </p>
// </div>

Normalizing user-provided filenames

let user_filename = "My Document (2023).pdf";
let safe_filename = name_fix(user_filename);
// Result: "my_document_2023_.pdf"

let user_path = "/uploads/User Files/Report #123.xlsx";
let safe_path = path_fix(user_path);
// Result: "/uploads/User Files/report_123.xlsx"