Add ignore_all_dot_files config to allow setting it to false; remove .serena from ALWAYS_IGNORED_DIRS#1456
Open
EarthmanWeb wants to merge 5 commits into
Open
Add ignore_all_dot_files config to allow setting it to false; remove .serena from ALWAYS_IGNORED_DIRS#1456EarthmanWeb wants to merge 5 commits into
EarthmanWeb wants to merge 5 commits into
Conversation
…port Adds a new project.yml option `ignore_all_dot_files` (default: true) that controls whether directories starting with "." are automatically excluded from symbol indexing. Previously, ALL dot-prefixed directories were silently ignored (including .github, .husky, etc.). Setting `ignore_all_dot_files: false` now allows Serena to index symbols within those directories. Changes: - ls_config.py: add ignore_all_dot_files field to LanguageServerConfig - ls.py: is_ignored_dirname() respects ignore_all_dot_files; .git always ignored - serena_config.py: add ignore_all_dot_files to ProjectConfig - ls_manager.py: pass ignore_all_dot_files through LanguageServerFactory - project.py: propagate ignore_all_dot_files to LanguageServerFactory - project.template.yml: document the new option - pyright_server.py: make _get_initialize_params an instance method; build exclude/include lists based on ignore_all_dot_files; handle workspace/configuration requests to reinforce include paths with Pyright
Serena stores project memories as .md files inside .serena/. With markdown configured as a language, symbol retrieval (get_symbols_overview, find_symbol) should work on these files. Cache and state files within .serena/ are non-.md and won't match the markdown LSP's source filter. The ignore_all_dot_files flag (default: true) still protects dot-prefixed directories in general. Users who set ignore_all_dot_files: false to access .serena/ memories opt in explicitly.
is_ignored_dirname("..") was returning True because ".." starts with
"." and ignore_all_dot_files defaults to True. This caused all
cross-package references with ".." in their relative path to be
silently dropped.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Member
|
Hi @EarthmanWeb . I am not sure this option is needed in practical applications, as we take into account the user's gitignore. Is that not enough? I also don't see how the workspace/configuration handler in pyright specifically has something to do with this new setting - rather we should use the existing mechanisms to identify what has to be ignored and handle it there. Or am I missing something? |
Root cause: When ignore_all_dot_files is set to false, the directory walker enters dot-prefixed directories that were previously skipped. If those directories contain broken symlinks (e.g. .sync-reload pointing to a non-existent target via ../), the following crash sequence occurs: 1. os.listdir() returns broken symlinks as valid directory entries 2. Path().resolve().relative_to() follows the symlink, yielding a relative path to the (non-existent) target 3. is_ignored_path() calls os.path.exists() on the resolved path 4. os.path.exists() returns False for the broken target 5. FileNotFoundError is raised, crashing the entire find_symbol call This was observed in a WordPress project where sibling theme directories (atlas-iconic/, atlas-swm/) contained .sync-reload symlinks pointing to ../atlas/.sync-reload which did not exist. The error manifested as all find_symbol calls failing with: "FileNotFoundError - File .../content/themes/atlas/.sync-reload not found, the ignore check cannot be performed" Changes: - ls.py is_ignored_path(): Return True (treat as ignored) instead of raising FileNotFoundError when a path does not exist. This is the correct semantic — a non-existent path cannot contain symbols and should be skipped, not crash the tool. - project.py _is_ignored_relative_path(): Same fix. Updated docstring to reflect the new behavior (no longer raises FileNotFoundError). - ls.py request_full_symbol_tree(): Add early broken-symlink check in the directory walker (os.path.islink + not os.path.exists) before attempting Path.resolve(), preventing the resolved path from ever reaching is_ignored_path. The is_ignored_path fix is defensive (handles any caller), while the directory walker fix is preventive (stops broken symlinks at source). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
Author
|
@MischaPanch Unignoring files in gitignore does NOT allow access to dot folders, that is the issue this is fixing, by providing an OPTIONAL ability to include dotfolders. Skipping dotfolders and especially .serena is BAKED IN currently, so even when trying to unignore them, the hardcoded skips do not allow it. Often I am working on files in dotfolders, including .serena itself, and want to use serena to index them. This fix allows that, and does not affect operations otherwise, since it defaults to True, which is the current default now |
EarthmanWeb
changed the title
MischaPanch
force-pushed
the
main
branch
2 times, most recently
from
420a0ba to
016ccbe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
PR #1203 replaced the blanket
dirname.startswith(".")with an explicit_ALWAYS_IGNORED_DIRSset — a solid improvement. However, two gaps remain:No user control over dot-directory indexing. Some dot-prefixed directories (
.github/,.devcontainer/) contain legitimate code that users may want symbol retrieval on. The current allowlist requires a code change to adjust..serenais in_ALWAYS_IGNORED_DIRS, blocking symbol retrieval on Serena's own memory files. Serena stores project memories as.mdfiles inside.serena/. Withmarkdownconfigured as a language,get_symbols_overviewandfind_symbolshould work on these files — they're Serena's own content, authored and managed through its tools. Cache and state files within.serena/are non-.mdand won't match the markdown LSP's source filter, so there's no noise risk.Changes
Commit 1:
ignore_all_dot_filesconfig option (rebased from #1101)ignore_all_dot_filesfield toProjectConfig(default:true, preserving current behavior)_from_dict→LanguageServerFactory→LanguageServerConfig→SolidLanguageServeris_ignored_dirname()checks_ALWAYS_IGNORED_DIRSfirst, then applies the dot-prefix filter only whenignore_all_dot_files=true_get_initialize_paramsrespects the setting for exclude/include listsproject.template.ymlCommit 2: Remove
.serenafrom_ALWAYS_IGNORED_DIRS.serena/memory files are valid symbol retrieval targets when markdown is a configured languageignore_all_dot_filesflag (default:true) still protects dot-prefixed directories in general.serena/memories setignore_all_dot_files: falseexplicitlySupersedes
mainwith the_from_dictwiring fix added)🤖 Generated with Claude Code