tests: don't silence errors while creating test index
"Errors should never pass silently." For example, I had the problem, that a previous test did not clean up the lock file for the index_dir, which then caused the future tests to fail. Because the error has been silenced, it took a while until I found the reason.
docs: recommend --upgrade for all pip installations in a virtualenv
As suggested by Mads Kiilerich.
Don't recommend this practice when no virtualenv is used, i.e. root installation or --user installation: in this case upgrading existing packages may be too disruptive. The user can still add it manually if he so chooses.
tests: vcs: remove influence of user's hg settings
Certain settings in the user's hgrc file can cause test failures. For example, enabling a non-existent extension prints stderr messages like
*** failed to import extension journal: No module named journal
Tests that check stderr output for emptiness, like test_clone_hg_repo_by_admin, then fail.
Instead, avoid all influence of the user's settings, by setting HGRCPATH and HGPLAIN to empty.
In versions of Mercurial before 4.2, setting an empty HGRCPATH was generally enough to get expected behavior, but since 4.2 the behavior of some commands has changed, and a pager is now default. Setting HGPLAIN is a simple way of neutralizing even that.
tests: fix failures due to environment LANG settings (Issue #286)
When the environment in which the tests are run dictates a non-English language (e.g. via LANG=de_DE.UTF-8), certain tests started to fail because formencode validation messages where translated in that language.
A test failing this way is for example test_create_in_group_without_needed_permissions in kallithea/tests/functional/test_admin_repos.py .
This is a regression of commit 3b29103657df (i18n: remove explicit formencode language setting), but in fact reverting it is not the right solution as there is no problem except during test execution.
Instead, force the formencode language to 'empty' (meaning English) for test execution only.
docs/contributing: move installation of dev_requirements to default instructions
When intending to contribute to Kallithea, you are expected to run all tests, so installation of the dev_requirements are actually mandatory. Hence, update the instructions.
docs/contributing: make contribution information more accessible
Try to make the contribution information more accessible to new users, by giving a bit more detail about the different steps in making contributions.
Some rework of the original patch done by Thomas De Schampheleire: - don't give the impression that Bitbucket is mandatory - don't recommend creating feature branches for regular development - reword/cross-reference section on contribution flow
Some rework of the original patch done by Thomas De Schampheleire: - generalize to cover both Mercurial and Git - simplify instructions for importing via filesystem: don't impose a certain flow as it depends on different factors: are the repos already present on the machine, are they organized in a flat hierarchy or already in a more complex one, ... - invalidating cache while rescanning should not be needed when adding new repositories
git: improve performance working with git changesets
GitRepository._repo instantiates a new dulwich.repo.Repo on every usage, rather than once at initialization time of GitRepository. As this involves a lot of filesystem access, this is a costly operation.
Instead, let GitRepository.__init__ instantiate a dulwich.repo.Repo once, and let GitRepository._repo just return it.
This improves performance significantly. On test_graphmod_git, performance improves from 6.29 seconds median to 3.06 seconds median.
[Thomas De Schampheleire: extend improvement to _all_ usage of GitRepository._repo instead of only some. To limit the delta, retain the _repo property but simply return self.repo.]
pytest-benchmark is a benchmarking suite for individual functions. It presents itself as a pytest fixture, i.e. you accept it as a test function argument, and then can use it to benchmark a function.
Output is presented in the following form:
----------------------------------------- benchmark: 1 tests ---------------------------------------- Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations ----------------------------------------------------------------------------------------------------- test_graphmod_git 3.1122 3.1682 3.1295 0.0236 3.1196 0.0313 1;0 5 1 -----------------------------------------------------------------------------------------------------
This script will automate how the current attribution and copyright listings are generated, thus also enabling it to be maintained from repository history in the future.
This doesn't change anything about who has or hasn't contributed or deserves credit, it just does a best effort attempt at maintaining existing attribution we inherited and automating it going forward. The automation might expose existing problems - they should be fixed by adding or removing exceptions in the script.
email: don't crash on sending mails with unicode comments without appropriate environment configuration (Issue #275)
For example, on Linux, running `gearbox serve` with LANG=C, would crash in:
File ".../kallithea/lib/celerylib/tasks.py", line 307, in send_email % (' '.join(recipients), headers, subject, body, html_body)) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 224: ordinal not in range(128)
Replacing render with render_unicode works in this case ... but there might be other problems elsewhere.
journal: make "repository:" filtering condition work as expected (Issue #261)
Before this revision, journal filtering conditions like as below never match against any entry, even if there are corresponded repositories.
- repository:foo/bar - repository:foo-bar
Whoosh library, which is used to parse filtering condition, does:
- treat almost all non-alphanumeric characters as delimiter at parsing condition - join each conditions at filtering by "AND", by default
For example, filtering condition "repository:foo/bar" is translated as "repository:foo AND repository:bar". This combined condition never matches against any entry, because it is impossible that "repository" field in DBMS table "user_logs" has both "foo" and "bar" values at same time.
Using TEXT for "repository" of JOURNAL_SCHEMA causes this issue, because TEXT assumes tokenization at parsing.
In addition to it, using TEXT also causes unintentional ignorance of "stop words" in filtering conditions. For example, "this", "a", "you", and so on are ignored at parsing, because these are too generic words (from point of view of generic "text search").
To make "repository:" filtering condition work as expected, this revision uses ID instead of TEST for "repository" of JOURNAL_COLUMN. ID avoids both tokenization and removing "stop words".
This replacement should be safe with already existing DBMS instance, because:
- JOURNAL_SCHEMA is used only to parse filtering condition - DBMS table "user_logs" itself is defined by UserLog class (in kallithea/model/db.py)
BTW, using ID also avoids normalization by lowercase-ing. But this doesn't violate current case-insensitive search policy, because LOWER-ing in actual SQL query is achieved by get_filterion() or so in kallithea/controllers/admin/admin.py.
To simplify email filtering, add a header indicating the type of email being sent. The 'type_' value corresponds to one of the types defined in class Notification in kallithea/model/db.py, e.g. 'cs_comment', 'pull_request', 'pull_request_comment', ...
git: include an LF at the end of the service advertisement (Fixes #230)
This fixes hg-git/Dulwich and possibly other conservative Git clients, which do not ignore the absence of the LF.
The original comment was a guess based on reverse engineering the protocol not specified in the documentation yet at that moment. Now that the documentation exists and states this explicitly, just do as it says.
lock: fix for Mercurial 3.6+ - wrap hgweb to catch Locked exceptions from hooks
With Mercurial 3.6, the handling of WSGI responses changed. The hook exceptions are no longer raised directly when app(environ, start_response) is called so the 'except HTTPLockedRC as e' block in _handle_request (a few lines above ) does not work anymore because the exception happens later.
Therefore I created a wrapper class that can catch the exceptions.
This makes locking work again and fixes lock related tests like TestVCSOperations.test_clone_after_repo_was_locked_hg which expect certain output of the hg client in case of an HTTPLockedRC exception.
style: drop form-horizontal - our style is much closer to plain Bootstrap forms
form-horizontal is made for grid markup. It give form-groups a negative margin to break out of the grid ... but the way we use it for settings, we have to do weird things to undo that. The default styling for forms is much closer to what we want. It looks ok without our custom styling and is easier to style to our style.
If we want grid markup with form-horizontal, it would be correct to re-introduce both at once.
graph: align the dots with the middle of the first *visible* child element
The changelog graph will have invisible first elements when a range is selected. That caused an odd graph. Fixed by adding the :visible selector that was lost in 5258b66bf5d7.
docs: add documentation about internationalization from a user perspective
While there is already documentation about creating and updating translations (developer perspective), there was nothing for the user/admin of Kallithea. Since the handling in TurboGears2 is different, this is a good time to add it.
i18n: use TG2 compatible settings names i18n.lang and i18n.enabled
Following changes are made: - Remove list of available translations: severely outdated and hard to maintain. - Replace lang with i18n.lang, reasons: - Follow TurboGears2 conventions - Align with 'i18n.enabled' which cannot be shortened - Scoped settings provide grouping and are more clear The old 'lang' is still recognized though, so no upgrade issue. - Introduce/document i18n.enabled
Comments are intentionally kept concise in favor of the documentation.
Under TurboGears2, automatic user language detection based on browser settings is enabled by default. If a user has 'English' as preferred language in the browser, then the old English .mo file with a few dummy entries would be read and used first, before trying the language configured in the Kallithea .ini file (e.g. French). This renders a mix of languages in the UI ... which is unexpected.
The hiding of configured translation is fixed by removing the old english translationfile. It was preserved via commit 7d6c4bd58abd when removing all other .mo files. The next release will require major config changes anyway, and we don't try to be backwards compatible.
When the browser size has been changed, the dots may no longer be correctly aligned with the table rows. Currently, the graph will just overflow without changing size, but that can soon change.