Commit 7a16908
WIP: Phase 0.5 - Streaming foundation for sync performance
Add database schema and streaming architecture foundation for SPEC-19.
This establishes mtime-based change detection and reduces memory usage.
Database Schema:
- Add mtime (Float) and size (Integer) columns to Entity model
- Create Alembic migration 9d9c1cb7d8f5 for new nullable columns
- Verified unique index on (file_path, project_id) exists
Streaming Architecture:
- Implement _scan_directory_streaming() using os.scandir()
- Provides cached stat info (50% fewer network I/O on network filesystems)
- Add get_all_file_paths() repository method for efficient deletion detection
- Returns only file_path strings without loading entities/relationships
Testing:
- Add 6 comprehensive tests for streaming scan functionality
- Add 4 tests for get_all_file_paths() repository method
- All tests passing with proper error handling
Documentation:
- Add SPEC-19: Sync Performance and Memory Optimization
- Document Phase 0.5 completion status and future phases
Related: #383, #382
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>1 parent 32236cd commit 7a16908
7 files changed
Lines changed: 1156 additions & 1 deletion
File tree
- specs
- src/basic_memory
- alembic/versions
- models
- repository
- sync
- tests
- repository
- sync
0 commit comments