If a git object is a submodule link, kallithea tries to resolve the submodule url via the repos' .gitmodules file. A missing file, missing section or a missing url for that submodule will raise either KeyError or ValueError in kallithea/lib/vcs/backends/git/changeset.py lines 402-403 and 441-442 and cause a "500 Server Error".
The Error also breaks search indexing and likely some other functions.
To reproduce just add a submodule in a git repo, rm .gitmodules, and commit it.
helpers: very partial support for pygments >= 2.12
setup.py makes it clear that Kallithea only supports "Pygments >= 2.2.0, < 2.8". Later versions are known to cause minor breakage to tests, and it hasn't been validated that they don't expose other problems.
Pawel (aka momat) reported a problem with pygments >= 2.12 when subclassing HtmlFormatter. The wrap outfile parameter has been dropped, and he suggested this workaround.
The workaround seems safe and backwards compatible and nice to have, but we still don't recommend or support using Pygments >= 2.8 .
tests: try harder to prevent Git from reading external configuration
Tests would fail if ~/.gitconfig contained for example:
[init] defaultBranch = main
Setting GIT_CONFIG_GLOBAL=/dev/null prevents Git from reading ~/.gitconfig ... but isn't support by Dulwich and doesn't solve the problem.
We will thus work around this issue by setting HOME to the temp test location. That will prevent Dulwich from reading configuration files by making os.path.expanduser point to a location with known content.
We also set XDG_CONFIG_HOME to avoid reading ~/.config/git/config .
Setting GIT_CONFIG_NOSYSTEM will prevent reading /etc/gitconfig - that seems like a good idea too.
Address warnings like: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. import pkg_resources
When cloning or pulling from a repository which uses the evolve extension Kallithea would require write permissions for multiple commands: evoext_obshashrange_v1, _exttopics_heads and tns_heads.
The freeze for the Mercurial 7.0 release was done on February 28th. The release of 7.0.0 was scheduled for March 1st, but was delayed. It is likely that the release will happen soon without changes that would break Kallithea.
The tests pass and some quick manual testing of the web UI found no issues.
controller: Handle UnicodeDecodeError from webob decoding invalid URLs
webob will try to utf-8 decode all %-encoded bytes in URL-parameters, but will not handle Unicode erors ... and neither did Kallithea. Visiting a URL like http://localhost:5000/?%AD would thus give an unhandled exception showing "Internal Server Error" to the user, and logging the full traceback and:
WebApp Error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 0: invalid start byte
tests: address PytestDeprecationWarning for @pytest.yield_fixture
Adhere to the warning:
kallithea/tests/conftest.py:207: PytestDeprecationWarning: @pytest.yield_fixture is deprecated. Use @pytest.fixture instead; they are the same. @pytest.yield_fixture(scope="session")
setup: bump max versions of pip package dependencies
It is a bit risky to update versions on stable, but it is also risky to stay on old versions when there are better(?) options.
There are various reasons these max versions have been chosen. I have not analyzed problems seen when using higher versions, but the versions specified here seems like a safe progression of the broad baseline.
Latest pip version pip fail: WARNING: Ignoring version 5.0.5 of celery since it has invalid metadata: Requested celery<5.1,>=5 from .../celery-5.0.5-py3-none-any.whl (from Kallithea==0.7.0) has invalid metadata: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier pytz (>dev) ~^ Please use pip<24.1 if you need to use this version.
We already use setuptools<67 for the same reason.
Pip will keep noting that a newer pip version is available. Resist the temptation to upgrade.
repo groups: make it possible to remove own explicit permissions, now when group owners always have admin permissions
Until recently, group owners very given explicit admin permissions on repo group, and special care was taken to make sure they didn't remove themselves.
Now we always give admin permissions to owners, and don't care about the explicit permissions. We no longer add them when creating groups or changing owner. There is no migration step to remove redundant permissions, but we should allow group admins to remove them. This change will thus remove the mechanism for preventing removal of own/owner permissions.
tests: skip reading Git system and global configuration in test_vcs_operations
The parent changeset reduced the dependency on global configuration and made it possible to run tests without any global git configuration. But it is still unfortunate to even look at the global configuration when running tests.
Global configuration is already disabled for Mercurial by setting HGRCPATH.
Now do something similar for Git. According to the git man page, GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM set to /dev/null will make Git skip reading the configuration files on all platforms.
Note that the GIT_CONFIG variables were introduced in Git 2.32.0, so this will not work with all the Git versions supported by Kallithea.
tests: stabilize Git committer in test_vcs_operations
Git tries to find out name and email in this order:
1. The author can be set e.g. via the `--author` option of `git commit`. 2. If set, the environment variables GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, GIT_COMMITTER_NAME and GIT_COMMITTER_EMAIL are taken. 3. If set, various (global) config files are considered. 4. Unless disabled by the user.useconfigonly config, the names and emails are inferred from various system sources such as various fields from /etc/passwd, /etc/mailname and the environment variable EMAIL.
The author can be provided on the command line (1), but that is not possible for the committer.
It is not an option to modify Git’s configuration files, so the result of (3) depends on the system the tests run on, which should be avoided. A follow-up patch will try to instruct Git to not read the system Git configuration files.
(4) is also system-dependent. On some systems, (4) is disabled in the Git configuration. If enabled, Git will try to infer the committer name from the gecko field in /etc/passwd, but will fail if it is empty. The previous code passed the environment variable EMAIL to provide the corresponding email address.
By passing the names and emails via (2), we can set the author and committer name and email uniformly and prevent Git from using the system-dependent ways (3) and (4). This will replace the use of of EMAIL. The environment variables were introduced in 2005, so there should be no backwards compatibility problems.
The tests will specify --author explicitly in the cases where the actual name matters. We just need default values that can be used for committing when we don't care.
We set it as static defaults to: Author: test_regular <test_regular@example.com> Commit: test_admin <test_admin@example.com>
Based on changes and research by Manuel Jacob <me@manueljacob.de>.
The `packaging` vendored in setuptools cannot handle the broken syntax `Requires-Dist: pytz (>dev)` in venv/lib/python3.11/site-packages/celery-5.0.5.dist-info/METADATA .
The old celery version currently used by Kallithea is wrong, and setuptools has moved on after a reasonable grace period. We thus have to work around and avoid latest setuptools.
i18n: make sure 'en' in Accept-Language is recognized as having 100% coverage
The workaround in 7c7d6b5c07c7 no longer works after upstream addressed the main issue and released the changes in TurboGears 2.4.3 .
Setting `i18n.native = en` in the .ini works as a workaround.
The native language for translations is an implementation detail that users shouldn't have to configure, so we define it as a default value without making it explicit in the generated .ini template files.
Note that even though TG will figure out that languages like `en_US` should fall back to using the `en` `kallithea.mo`, it doesn't consider `en_US` native if `en` is in the native list but `en_US` isn't. We thus include the most common aliases for `en` in the list.
api: fix get_changeset() when incomplete raw_id is passed with with_reviews
Previously, ChangesetStatusModel was queried with the raw_id passed as an argument to the API function. When the raw_id was incomplete (i.e. shortened), no reviews were found. Using the full raw_id from the changeset instance fixes that.
Someone might argue that the caller is supposed to pass a full raw_id to the API function. However, in any case, the return value should not be incomplete without notice.
pullrequests: introduce limit to stop displaying additional changes
We've noticed some scalability issues when many descendants exist for the changesets in a pull request.
To resolve this issue, we instead do not display any additional changes at all if the amount of additional changes is larger than a configured limit.
(This occurred because we were merging a lot of heads in the repository.)
Throwing away some of the changesets would also keep the total amount more manageable, but this would result in an (for the user) unpredictable subset of changesets being shown.
This could be extended further with "too long to be shown - click here to show", but that was quite a bit of additional work and did not cover our use case of not allowing the display at all in case of too many additional changes.
After renaming a group, it would iterate over all the contained groups and repos and update their full path while logging the update from the/old/path to the the/new/path. Doing that, it would also visit the already renamed top level group, but since the full path of that one already had been updated, it would log it as renaming from the/new/path to the/new/path.
Fixed by logging when renaming in the first place, and skipping the top level repo group while iterating.
To avoid redundant logging, only log (and rename) if the name or parent actually change.
setup: use old importlib_metadata version to fix kombu failing on python < 3.8
Many libraries use the importlib_metadata library as fallback when running on Python versions older than 3.8 . For example setuptools when easy_install is used for install the Kallithea console_scripts entrypoints in the bin folder. The dependencies on importlib_metadata were indirect and without constrains on version number.
The problem is that Celery uses Kombu, which (on Python < 3.8) uses importlib_metadata in a way that is incompatible with importlib_metadata > 5.
Most obvious, building docs failed as: Running Sphinx v5.1.1
Configuration error: There is a programmable error in your configuration file:
Traceback (most recent call last): File ".../kallithea/venv/lib64/python3.7/site-packages/sphinx/config.py", line 347, in eval_config_file exec(code, namespace) File ".../kallithea/docs/conf.py", line 17, in <module> import kallithea File ".../kallithea/kallithea/__init__.py", line 45, in <module> CELERY_APP = celery.Celery() # needed at import time but is lazy and can be configured later File ".../kallithea/venv/lib64/python3.7/site-packages/celery/local.py", line 492, in __getattr__ [name]) File ".../kallithea/venv/lib64/python3.7/site-packages/celery/app/__init__.py", line 2, in <module> from celery import _state File ".../kallithea/venv/lib64/python3.7/site-packages/celery/_state.py", line 15, in <module> from celery.utils.threads import LocalStack File ".../kallithea/venv/lib64/python3.7/site-packages/celery/utils/__init__.py", line 16, in <module> from .nodenames import nodename, nodesplit, worker_direct File ".../kallithea/venv/lib64/python3.7/site-packages/celery/utils/nodenames.py", line 6, in <module> from kombu.entity import Exchange, Queue File ".../kallithea/venv/lib64/python3.7/site-packages/kombu/entity.py", line 7, in <module> from .serialization import prepare_accept_content File ".../kallithea/venv/lib64/python3.7/site-packages/kombu/serialization.py", line 440, in <module> for ep, args in entrypoints('kombu.serializers'): # pragma: no cover File ".../kallithea/venv/lib64/python3.7/site-packages/kombu/utils/compat.py", line 82, in entrypoints for ep in importlib_metadata.entry_points().get(namespace, []) AttributeError: 'EntryPoints' object has no attribute 'get'
That made readthedocs builds fail, when it in the default web configuration used Python 3.7 .
Fixed by introducing an explicit dependency on importlib_metadata < 5.