feat: Enhance CLI with improved list and smart remove commands

- **Better UX**: Current project shown first with clear separation
- **Visual improvements**: Icons (🏠/📂), better formatting, size info
- **Smart guidance**: Context-aware usage examples and getting started tips

- **Safety first**: Always shows ALL matching indexes across projects
- **Intelligent handling**:
  - Single match: Clear location display with cross-project warnings
  - Multiple matches: Interactive selection with final confirmation
- **Prevents accidents**: No more deleting wrong indexes due to name conflicts
- **User-friendly**: 'c' to cancel, clear visual hierarchy, detailed info

- **Clean logging**: Hide debug messages for better CLI experience
- **Comprehensive search**: Always scan all projects for transparency
- **Error handling**: Graceful handling of edge cases and user input

- **Safer**: Eliminates risk of accidental index deletion
- **Clearer**: Users always know what they're operating on
- **Smarter**: Automatic detection and handling of common scenarios

README.md

@@ -5,7 +5,7 @@
 <p align="center">
   <img src="https://img.shields.io/badge/Python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue.svg" alt="Python Versions">
   <img src="https://github.com/yichuan-w/LEANN/actions/workflows/build-and-publish.yml/badge.svg" alt="CI Status">
-  <img src="https://img.shields.io/badge/Platform-Ubuntu%20%7C%20macOS%20(ARM64%2FIntel)-lightgrey" alt="Platform">
+  <img src="https://img.shields.io/badge/Platform-Ubuntu%20%26%20Arch%20%26%20WSL%20%7C%20macOS%20(ARM64%2FIntel)-lightgrey" alt="Platform">
   <img src="https://img.shields.io/badge/License-MIT-green.svg" alt="MIT License">
   <img src="https://img.shields.io/badge/MCP-Native%20Integration-blue" alt="MCP Integration">
 </p>
@@ -94,7 +94,9 @@ CC=$(brew --prefix llvm)/bin/clang CXX=$(brew --prefix llvm)/bin/clang++ uv sync
 
 **Linux:**
 ```bash
-sudo apt-get install libomp-dev libboost-all-dev protobuf-compiler libabsl-dev libmkl-full-dev libaio-dev libzmq3-dev
+# Ubuntu/Debian (For Arch Linux: sudo pacman -S blas lapack openblas libaio boost protobuf abseil-cpp zeromq)
+sudo apt-get update && sudo apt-get install -y libomp-dev libboost-all-dev protobuf-compiler libabsl-dev libmkl-full-dev libaio-dev libzmq3-dev
+
 uv sync
 ```
 
@@ -482,6 +484,9 @@ leann ask my-docs --interactive
 
 # List all your indexes
 leann list
+
+# Remove an index
+leann remove my-docs
 ```
 
 **Key CLI features:**
@@ -494,7 +499,7 @@ leann list
 <details>
 <summary><strong>📋 Click to expand: Complete CLI Reference</strong></summary>
 
-You can use `leann --help`, or `leann build --help`, `leann search --help`, `leann ask --help` to get the complete CLI reference.
+You can use `leann --help`, or `leann build --help`, `leann search --help`, `leann ask --help`, `leann list --help`, `leann remove --help` to get the complete CLI reference.
 
 **Build Command:**
 ```bash
@@ -532,6 +537,28 @@ Options:
   --top-k N                  Retrieval count (default: 20)
 ```
 
+**List Command:**
+```bash
+leann list
+
+# Lists all indexes across all projects with status indicators:
+# ✓ - Index is complete and ready to use
+# ✗ - Index is incomplete or corrupted
+```
+
+**Remove Command:**
+```bash
+leann remove INDEX_NAME [OPTIONS]
+
+Options:
+  --force, -f    Force removal without confirmation
+
+# Smart removal: automatically finds and safely removes indexes
+# - Shows all matching indexes across projects
+# - Requires confirmation for cross-project removal
+# - Interactive selection when multiple matches found
+```
+
 </details>
 
 ## 🏗️ Architecture & How It Works

packages/leann-backend-diskann/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "scikit_build_core.build"
 
 [project]
 name = "leann-backend-diskann"
-version = "0.2.9"
-dependencies = ["leann-core==0.2.9", "numpy", "protobuf>=3.19.0"]
+version = "0.3.0"
+dependencies = ["leann-core==0.3.0", "numpy", "protobuf>=3.19.0"]
 
 [tool.scikit-build]
 # Key: simplified CMake path

packages/leann-backend-hnsw/pyproject.toml

@@ -6,10 +6,10 @@ build-backend = "scikit_build_core.build"
 
 [project]
 name = "leann-backend-hnsw"
-version = "0.2.9"
+version = "0.3.0"
 description = "Custom-built HNSW (Faiss) backend for the Leann toolkit."
 dependencies = [
-    "leann-core==0.2.9",
+    "leann-core==0.3.0",
     "numpy",
     "pyzmq>=23.0.0",
     "msgpack>=1.0.0",

packages/leann-core/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "leann-core"
-version = "0.2.9"
+version = "0.3.0"
 description = "Core API and plugin system for LEANN"
 readme = "README.md"
 requires-python = ">=3.9"

packages/leann-core/src/leann/api.py

@@ -307,6 +307,23 @@ class LeannBuilder:
     def build_index(self, index_path: str):
         if not self.chunks:
             raise ValueError("No chunks added.")
+
+        # Filter out invalid/empty text chunks early to keep passage and embedding counts aligned
+        valid_chunks: list[dict[str, Any]] = []
+        skipped = 0
+        for chunk in self.chunks:
+            text = chunk.get("text", "")
+            if isinstance(text, str) and text.strip():
+                valid_chunks.append(chunk)
+            else:
+                skipped += 1
+        if skipped > 0:
+            print(
+                f"Warning: Skipping {skipped} empty/invalid text chunk(s). Processing {len(valid_chunks)} valid chunks"
+            )
+            self.chunks = valid_chunks
+            if not self.chunks:
+                raise ValueError("All provided chunks are empty or invalid. Nothing to index.")
         if self.dimensions is None:
             self.dimensions = len(
                 compute_embeddings(

packages/leann-core/src/leann/cli.py

@@ -84,6 +84,7 @@ Examples:
   leann search my-docs "query"                                           # Search in my-docs index
   leann ask my-docs "question"                                           # Ask my-docs index
   leann list                                                             # List all stored indexes
+  leann remove my-docs                                                   # Remove an index (local first, then global)
             """,
         )
 
@@ -148,6 +149,12 @@ Examples:
             type=str,
             help="Comma-separated list of file extensions to include (e.g., '.txt,.pdf,.pptx'). If not specified, uses default supported types.",
         )
+        build_parser.add_argument(
+            "--include-hidden",
+            action=argparse.BooleanOptionalAction,
+            default=False,
+            help="Include hidden files and directories (paths starting with '.') during indexing (default: false)",
+        )
         build_parser.add_argument(
             "--doc-chunk-size",
             type=int,
@@ -245,6 +252,13 @@ Examples:
         # List command
         subparsers.add_parser("list", help="List all indexes")
 
+        # Remove command
+        remove_parser = subparsers.add_parser("remove", help="Remove an index")
+        remove_parser.add_argument("index_name", help="Index name to remove")
+        remove_parser.add_argument(
+            "--force", "-f", action="store_true", help="Force removal without confirmation"
+        )
+
         return parser
 
     def register_project_dir(self):
@@ -333,8 +347,6 @@ Examples:
             return False
 
     def list_indexes(self):
-        print("Stored LEANN indexes:")
-
         # Get all project directories with .leann
         global_registry = Path.home() / ".leann" / "projects.json"
         all_projects = []
@@ -360,58 +372,293 @@ Examples:
         if (current_path / ".leann" / "indexes").exists() and current_path not in valid_projects:
             valid_projects.append(current_path)
 
-        if not valid_projects:
-            print(
-                "No indexes found. Use 'leann build <name> --docs <dir> [<dir2> ...]' to create one."
-            )
-            return
-
-        total_indexes = 0
-        current_dir = Path.cwd()
+        # Separate current and other projects
+        current_project = None
+        other_projects = []
 
         for project_path in valid_projects:
-            indexes_dir = project_path / ".leann" / "indexes"
-            if not indexes_dir.exists():
-                continue
-
-            index_dirs = [d for d in indexes_dir.iterdir() if d.is_dir()]
-            if not index_dirs:
-                continue
-
-            # Show project header
-            if project_path == current_dir:
-                print(f"\n📁 Current project ({project_path}):")
+            if project_path == current_path:
+                current_project = project_path
             else:
-                print(f"\n📂 {project_path}:")
+                other_projects.append(project_path)
 
-            for index_dir in index_dirs:
-                total_indexes += 1
-                index_name = index_dir.name
-                meta_file = index_dir / "documents.leann.meta.json"
-                status = "✓" if meta_file.exists() else "✗"
+        print("📚 LEANN Indexes")
+        print("=" * 50)
 
-                print(f"  {total_indexes}. {index_name} [{status}]")
-                if status == "✓":
-                    size_mb = sum(f.stat().st_size for f in index_dir.iterdir() if f.is_file()) / (
-                        1024 * 1024
-                    )
-                    print(f"     Size: {size_mb:.1f} MB")
+        total_indexes = 0
+        current_indexes_count = 0
 
-        if total_indexes > 0:
-            print(f"\nTotal: {total_indexes} indexes across {len(valid_projects)} projects")
-            print("\nUsage (current project only):")
-
-            # Show example from current project
-            current_indexes_dir = current_dir / ".leann" / "indexes"
+        # Show current project first (most important)
+        if current_project:
+            current_indexes_dir = current_project / ".leann" / "indexes"
             if current_indexes_dir.exists():
                 current_index_dirs = [d for d in current_indexes_dir.iterdir() if d.is_dir()]
+
+                print("\n🏠 Current Project")
+                print(f"   {current_project}")
+                print("   " + "─" * 45)
+
                 if current_index_dirs:
-                    example_name = current_index_dirs[0].name
-                    print(f'  leann search {example_name} "your query"')
-                    print(f"  leann ask {example_name} --interactive")
+                    for index_dir in current_index_dirs:
+                        total_indexes += 1
+                        current_indexes_count += 1
+                        index_name = index_dir.name
+                        meta_file = index_dir / "documents.leann.meta.json"
+                        status = "✅" if meta_file.exists() else "❌"
+
+                        print(f"   {current_indexes_count}. {index_name} {status}")
+                        if meta_file.exists():
+                            size_mb = sum(
+                                f.stat().st_size for f in index_dir.iterdir() if f.is_file()
+                            ) / (1024 * 1024)
+                            print(f"      📦 Size: {size_mb:.1f} MB")
+                else:
+                    print("   📭 No indexes in current project")
+        else:
+            print("\n🏠 Current Project")
+            print(f"   {current_path}")
+            print("   " + "─" * 45)
+            print("   📭 No indexes in current project")
+
+        # Show other projects (reference information)
+        if other_projects:
+            print("\n\n🗂️  Other Projects")
+            print("   " + "─" * 45)
+
+            for project_path in other_projects:
+                indexes_dir = project_path / ".leann" / "indexes"
+                if not indexes_dir.exists():
+                    continue
+
+                index_dirs = [d for d in indexes_dir.iterdir() if d.is_dir()]
+                if not index_dirs:
+                    continue
+
+                print(f"\n   📂 {project_path.name}")
+                print(f"      {project_path}")
+
+                for index_dir in index_dirs:
+                    total_indexes += 1
+                    index_name = index_dir.name
+                    meta_file = index_dir / "documents.leann.meta.json"
+                    status = "✅" if meta_file.exists() else "❌"
+
+                    print(f"      • {index_name} {status}")
+                    if meta_file.exists():
+                        size_mb = sum(
+                            f.stat().st_size for f in index_dir.iterdir() if f.is_file()
+                        ) / (1024 * 1024)
+                        print(f"        📦 {size_mb:.1f} MB")
+
+        # Summary and usage info
+        print("\n" + "=" * 50)
+        if total_indexes == 0:
+            print("💡 Get started:")
+            print("   leann build my-docs --docs ./documents")
+        else:
+            projects_count = len(
+                [
+                    p
+                    for p in valid_projects
+                    if (p / ".leann" / "indexes").exists()
+                    and list((p / ".leann" / "indexes").iterdir())
+                ]
+            )
+            print(f"📊 Total: {total_indexes} indexes across {projects_count} projects")
+
+            if current_indexes_count > 0:
+                print("\n💫 Quick start (current project):")
+                # Get first index from current project for example
+                current_indexes_dir = current_path / ".leann" / "indexes"
+                if current_indexes_dir.exists():
+                    current_index_dirs = [d for d in current_indexes_dir.iterdir() if d.is_dir()]
+                    if current_index_dirs:
+                        example_name = current_index_dirs[0].name
+                        print(f'   leann search {example_name} "your query"')
+                        print(f"   leann ask {example_name} --interactive")
+            else:
+                print("\n💡 Create your first index:")
+                print("   leann build my-docs --docs ./documents")
+
+    def remove_index(self, index_name: str, force: bool = False):
+        """Safely remove an index - always show all matches for transparency"""
+
+        # Always do a comprehensive search for safety
+        print(f"🔍 Searching for all indexes named '{index_name}'...")
+        all_matches = self._find_all_matching_indexes(index_name)
+
+        if not all_matches:
+            print(f"❌ Index '{index_name}' not found in any project.")
+            return False
+
+        if len(all_matches) == 1:
+            return self._remove_single_match(all_matches[0], index_name, force)
+        else:
+            return self._remove_from_multiple_matches(all_matches, index_name, force)
+
+    def _find_all_matching_indexes(self, index_name: str):
+        """Find all indexes with the given name across all projects"""
+        matches = []
+
+        # Get all registered projects
+        global_registry = Path.home() / ".leann" / "projects.json"
+        all_projects = []
+
+        if global_registry.exists():
+            try:
+                import json
+
+                with open(global_registry) as f:
+                    all_projects = json.load(f)
+            except Exception:
+                pass
+
+        # Always include current project
+        current_path = Path.cwd()
+        if str(current_path) not in all_projects:
+            all_projects.append(str(current_path))
+
+        # Search across all projects
+        for project_dir in all_projects:
+            project_path = Path(project_dir)
+            if not project_path.exists():
+                continue
+
+            index_dir = project_path / ".leann" / "indexes" / index_name
+            if index_dir.exists():
+                is_current = project_path == current_path
+                matches.append(
+                    {"project_path": project_path, "index_dir": index_dir, "is_current": is_current}
+                )
+
+        # Sort: current project first, then by project name
+        matches.sort(key=lambda x: (not x["is_current"], x["project_path"].name))
+        return matches
+
+    def _remove_single_match(self, match, index_name: str, force: bool):
+        """Handle removal when only one match is found"""
+        project_path = match["project_path"]
+        index_dir = match["index_dir"]
+        is_current = match["is_current"]
+
+        if is_current:
+            location_info = "current project"
+            emoji = "🏠"
+        else:
+            location_info = f"other project '{project_path.name}'"
+            emoji = "📂"
+
+        print(f"✅ Found 1 index named '{index_name}':")
+        print(f"   {emoji} Location: {location_info}")
+        print(f"   📍 Path: {project_path}")
+
+        if not force:
+            if not is_current:
+                print("\n⚠️  CROSS-PROJECT REMOVAL!")
+                print("   This will delete the index from another project.")
+
+            response = input(f"   ❓ Confirm removal from {location_info}? (y/N): ").strip().lower()
+            if response not in ["y", "yes"]:
+                print("   ❌ Removal cancelled.")
+                return False
+
+        return self._delete_index_directory(
+            index_dir, index_name, project_path if not is_current else None
+        )
+
+    def _remove_from_multiple_matches(self, matches, index_name: str, force: bool):
+        """Handle removal when multiple matches are found"""
+
+        print(f"⚠️  Found {len(matches)} indexes named '{index_name}':")
+        print("   " + "─" * 50)
+
+        for i, match in enumerate(matches, 1):
+            project_path = match["project_path"]
+            is_current = match["is_current"]
+
+            if is_current:
+                print(f"   {i}. 🏠 Current project")
+                print(f"      📍 {project_path}")
+            else:
+                print(f"   {i}. 📂 {project_path.name}")
+                print(f"      📍 {project_path}")
+
+            # Show size info
+            try:
+                size_mb = sum(
+                    f.stat().st_size for f in match["index_dir"].iterdir() if f.is_file()
+                ) / (1024 * 1024)
+                print(f"      📦 Size: {size_mb:.1f} MB")
+            except (OSError, PermissionError):
+                pass
+
+        print("   " + "─" * 50)
+
+        if force:
+            print("   ❌ Multiple matches found, but --force specified.")
+            print("   Please run without --force to choose which one to remove.")
+            return False
+
+        try:
+            choice = input(
+                f"   ❓ Which one to remove? (1-{len(matches)}, or 'c' to cancel): "
+            ).strip()
+            if choice.lower() == "c":
+                print("   ❌ Removal cancelled.")
+                return False
+
+            choice_idx = int(choice) - 1
+            if 0 <= choice_idx < len(matches):
+                selected_match = matches[choice_idx]
+                project_path = selected_match["project_path"]
+                index_dir = selected_match["index_dir"]
+                is_current = selected_match["is_current"]
+
+                location = "current project" if is_current else f"'{project_path.name}' project"
+                print(f"   🎯 Selected: Remove from {location}")
+
+                # Final confirmation for safety
+                confirm = input(
+                    f"   ❓ FINAL CONFIRMATION - Type '{index_name}' to proceed: "
+                ).strip()
+                if confirm != index_name:
+                    print("   ❌ Confirmation failed. Removal cancelled.")
+                    return False
+
+                return self._delete_index_directory(
+                    index_dir, index_name, project_path if not is_current else None
+                )
+            else:
+                print("   ❌ Invalid choice. Removal cancelled.")
+                return False
+
+        except (ValueError, KeyboardInterrupt):
+            print("\n   ❌ Invalid input. Removal cancelled.")
+            return False
+
+    def _delete_index_directory(
+        self, index_dir: Path, index_name: str, project_path: Path | None = None
+    ):
+        """Actually delete the index directory"""
+        try:
+            import shutil
+
+            shutil.rmtree(index_dir)
+
+            if project_path:
+                print(f"✅ Index '{index_name}' removed from {project_path.name}")
+            else:
+                print(f"✅ Index '{index_name}' removed successfully")
+            return True
+        except Exception as e:
+            print(f"❌ Error removing index '{index_name}': {e}")
+            return False
 
     def load_documents(
-        self, docs_paths: Union[str, list], custom_file_types: Union[str, None] = None
+        self,
+        docs_paths: Union[str, list],
+        custom_file_types: Union[str, None] = None,
+        include_hidden: bool = False,
     ):
         # Handle both single path (string) and multiple paths (list) for backward compatibility
         if isinstance(docs_paths, str):
@@ -455,6 +702,10 @@ Examples:
 
         all_documents = []
 
+        # Helper to detect hidden path components
+        def _path_has_hidden_segment(p: Path) -> bool:
+            return any(part.startswith(".") and part not in [".", ".."] for part in p.parts)
+
         # First, process individual files if any
         if files:
             print(f"\n🔄 Processing {len(files)} individual file{'s' if len(files) > 1 else ''}...")
@@ -467,8 +718,12 @@ Examples:
 
                 files_by_dir = defaultdict(list)
                 for file_path in files:
-                    parent_dir = str(Path(file_path).parent)
-                    files_by_dir[parent_dir].append(file_path)
+                    file_path_obj = Path(file_path)
+                    if not include_hidden and _path_has_hidden_segment(file_path_obj):
+                        print(f"  ⚠️  Skipping hidden file: {file_path}")
+                        continue
+                    parent_dir = str(file_path_obj.parent)
+                    files_by_dir[parent_dir].append(str(file_path_obj))
 
                 # Load files from each parent directory
                 for parent_dir, file_list in files_by_dir.items():
@@ -479,6 +734,7 @@ Examples:
                         file_docs = SimpleDirectoryReader(
                             parent_dir,
                             input_files=file_list,
+                            # exclude_hidden only affects directory scans; input_files are explicit
                             filename_as_id=True,
                         ).load_data()
                         all_documents.extend(file_docs)
@@ -577,6 +833,8 @@ Examples:
                     # Check if file matches any exclude pattern
                     try:
                         relative_path = file_path.relative_to(docs_path)
+                        if not include_hidden and _path_has_hidden_segment(relative_path):
+                            continue
                         if self._should_exclude_file(relative_path, gitignore_matches):
                             continue
                     except ValueError:
@@ -604,6 +862,7 @@ Examples:
                         try:
                             default_docs = SimpleDirectoryReader(
                                 str(file_path.parent),
+                                exclude_hidden=not include_hidden,
                                 filename_as_id=True,
                                 required_exts=[file_path.suffix],
                             ).load_data()
@@ -632,6 +891,7 @@ Examples:
                     encoding="utf-8",
                     required_exts=code_extensions,
                     file_extractor={},  # Use default extractors
+                    exclude_hidden=not include_hidden,
                     filename_as_id=True,
                 ).load_data(show_progress=True)
 
@@ -781,7 +1041,9 @@ Examples:
             paragraph_separator="\n\n",
         )
 
-        all_texts = self.load_documents(docs_paths, args.file_types)
+        all_texts = self.load_documents(
+            docs_paths, args.file_types, include_hidden=args.include_hidden
+        )
         if not all_texts:
             print("No documents found")
             return
@@ -918,6 +1180,8 @@ Examples:
 
         if args.command == "list":
             self.list_indexes()
+        elif args.command == "remove":
+            self.remove_index(args.index_name, args.force)
         elif args.command == "build":
             await self.build_index(args)
         elif args.command == "search":
@@ -929,10 +1193,15 @@ Examples:
 
 
 def main():
+    import logging
+
     import dotenv
 
     dotenv.load_dotenv()
 
+    # Set clean logging for CLI usage
+    logging.getLogger().setLevel(logging.WARNING)  # Only show warnings and errors
+
     cli = LeannCLI()
     asyncio.run(cli.run())
 

packages/leann-core/src/leann/embedding_compute.py

@@ -246,6 +246,16 @@ def compute_embeddings_openai(texts: list[str], model_name: str) -> np.ndarray:
     except ImportError as e:
         raise ImportError(f"OpenAI package not installed: {e}")
 
+    # Validate input list
+    if not texts:
+        raise ValueError("Cannot compute embeddings for empty text list")
+    # Extra validation: abort early if any item is empty/whitespace
+    invalid_count = sum(1 for t in texts if not isinstance(t, str) or not t.strip())
+    if invalid_count > 0:
+        raise ValueError(
+            f"Found {invalid_count} empty/invalid text(s) in input. Upstream should filter before calling OpenAI."
+        )
+
     api_key = os.getenv("OPENAI_API_KEY")
     if not api_key:
         raise RuntimeError("OPENAI_API_KEY environment variable not set")

packages/leann-core/src/leann/registry.py

@@ -2,11 +2,15 @@
 
 import importlib
 import importlib.metadata
+import logging
 from typing import TYPE_CHECKING
 
 if TYPE_CHECKING:
     from leann.interface import LeannBackendFactoryInterface
 
+# Set up logger for this module
+logger = logging.getLogger(__name__)
+
 BACKEND_REGISTRY: dict[str, "LeannBackendFactoryInterface"] = {}
 
 
@@ -14,7 +18,7 @@ def register_backend(name: str):
     """A decorator to register a new backend class."""
 
     def decorator(cls):
-        print(f"INFO: Registering backend '{name}'")
+        logger.debug(f"Registering backend '{name}'")
         BACKEND_REGISTRY[name] = cls
         return cls
 

packages/leann/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "leann"
-version = "0.2.9"
+version = "0.3.0"
 description = "LEANN - The smallest vector index in the world. RAG Everything with LEANN!"
 readme = "README.md"
 requires-python = ">=3.9"