refactor: improve test structure and fix main_cli example

- Move pytest configuration from pytest.ini to pyproject.toml
- Remove unnecessary run_tests.py script (use test extras instead)
- Fix main_cli_example.py to properly use command line arguments for LLM config
- Add test_readme_examples.py to test code examples from README
- Refactor tests to use pytest fixtures and parametrization
- Update test documentation to reflect new structure
- Set proper environment variables in CI for test execution

tests/README.md

@@ -1,52 +1,89 @@
 # LEANN Tests
 
-This directory contains automated tests for the LEANN project, primarily used in CI/CD pipelines.
+This directory contains automated tests for the LEANN project using pytest.
 
 ## Test Files
 
-### `test_ci_basic.py`
+### `test_readme_examples.py`
+Tests the examples shown in README.md:
+- The basic example code that users see first
+- Import statements work correctly
+- Different backend options (HNSW, DiskANN)
+- Different LLM configuration options
+
+### `test_basic.py`
 Basic functionality tests that verify:
 - All packages can be imported correctly
 - C++ extensions (FAISS, DiskANN) load properly
 - Basic index building and searching works for both HNSW and DiskANN backends
+- Uses parametrized tests to test both backends
 
 ### `test_main_cli.py`
 Tests the main CLI example functionality:
 - Tests with facebook/contriever embeddings
 - Tests with OpenAI embeddings (if API key is available)
+- Tests error handling with invalid parameters
 - Verifies that normalized embeddings are detected and cosine distance is used
 
-## Running Tests Locally
+## Running Tests
 
-### Basic tests:
+### Install test dependencies:
 ```bash
-python tests/test_ci_basic.py
+# Using extras
+uv pip install -e ".[test]"
 ```
 
-### Main CLI tests:
+### Run all tests:
 ```bash
-# Without OpenAI API key
-python tests/test_main_cli.py
+pytest tests/
 
-# With OpenAI API key
-OPENAI_API_KEY=your-key-here python tests/test_main_cli.py
+# Or with coverage
+pytest tests/ --cov=leann --cov-report=html
+
+# Run in parallel (faster)
+pytest tests/ -n auto
+```
+
+### Run specific tests:
+```bash
+# Only basic tests
+pytest tests/test_basic.py
+
+# Only tests that don't require OpenAI
+pytest tests/ -m "not openai"
+
+# Skip slow tests
+pytest tests/ -m "not slow"
+```
+
+### Run with specific backend:
+```bash
+# Test only HNSW backend
+pytest tests/test_basic.py::test_backend_basic[hnsw]
+
+# Test only DiskANN backend
+pytest tests/test_basic.py::test_backend_basic[diskann]
 ```
 
 ## CI/CD Integration
 
-These tests are automatically run in the GitHub Actions workflow:
+Tests are automatically run in GitHub Actions:
 1. After building wheel packages
 2. On multiple Python versions (3.9 - 3.13)
 3. On both Ubuntu and macOS
+4. Using pytest with appropriate markers and flags
+
+### pytest.ini Configuration
+
+The `pytest.ini` file configures:
+- Test discovery paths
+- Default timeout (600 seconds)
+- Environment variables (HF_HUB_DISABLE_SYMLINKS, TOKENIZERS_PARALLELISM)
+- Custom markers for slow and OpenAI tests
+- Verbose output with short tracebacks
 
 ### Known Issues
 
-- On macOS, there might be C++ standard library compatibility issues that cause tests to fail
-- The CI is configured to continue on macOS failures to avoid blocking releases
-- OpenAI tests are skipped if no API key is provided in GitHub secrets
-
-## Test Data
-
-Tests use the example data in `examples/data/`:
-- `PrideandPrejudice.txt` - Text file for testing
-- PDF files for document processing tests 
+- On macOS, tests may fail due to C++ standard library compatibility issues
+- Tests marked with `@pytest.mark.xfail` are expected to fail on macOS
+- OpenAI tests are automatically skipped if no API key is provided