validate-commits can be used to verify a series of commits before submitting/pushing it. It will create a virtualenv and run run-all-cleanup and pytest.
tests: fix assert rewriting in non-test modules like api_base.py
pytest rewrites assert statements in tests so it can print details about the values involved when the assert fails.
Since pytest 3.0.0, this was no longer the case for files/modules that are not discovered as test modules, i.e. starting with 'test_'. Examples are: api/api_base.py models/common.py base.py fixture.py
.coveragerc: fix reporting of coverage to match what is run
Without this change, the HTML report will show all lines specified in run.omit. As they have 0% coverage, this will negatively impact the overall coverage percentage calculated in the report.
Note: at this moment, we have an overall 74% test coverage. Test by installing pytest-cov and running: py.test --cov=kallithea --cov-config=.coveragerc --cov-report html and opening htmlcov/index.html in a browser.
In a fresh virtualenv on the stable branch, pastescript 3.0.0 is installed which depends on paste 3.0.x. Using this virtualenv to upgrade to the default branch, using 'pip install --upgrade -e .' fails because on the default branch, the paste version is restricted with '>= 2.0.3, < 3'. Following error occurs:
pastescript 3.0.0 has requirement Paste>=3.0, but you'll have paste 2.0.3 which is incompatible. ... Traceback (most recent call last): File "<string>", line 1, in <module> File ".../kallithea/kallithea-release/setup.py", line 160, in <module> """, File "/usr/lib64/python2.7/distutils/core.py", line 151, in setup dist.run_commands() File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File ".../kallithea/venv/kallithea-release/lib/python2.7/site-packages/setuptools/command/develop.py", line 36, in run self.install_for_development() File ".../kallithea/venv/kallithea-release/lib/python2.7/site-packages/setuptools/command/develop.py", line 117, in install_for_development self.run_command('egg_info') File "/usr/lib64/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File ".../kallithea/venv/kallithea-release/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 270, in run ep.require(installer=installer) File ".../kallithea/venv/kallithea-release/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2307, in require items = working_set.resolve(reqs, env, installer) File ".../kallithea/venv/kallithea-release/lib/python2.7/site-packages/pkg_resources/__init__.py", line 854, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.VersionConflict: (Paste 2.0.3 (.../kallithea/venv/kallithea-release/lib/python2.7/site-packages), Requirement.parse('Paste>=3.0'))
The '< 3' restriction is introduced with commit e1ab826131334150b1f003e26de3207c34fc6e67 in January 2017, at which point 2.0.3 was the latest version. Version 3.0.0 was introduced in October 2018.
lib: BaseRepoController: use webob.exc.HTTPNotFound if possible
In the entire code base, the use of 'paste' is very limited. In particular, 'paste.httpexceptions' is only still used in lib/base.py, in three occurrences: - two of them in class BasicAuth which derives from paste.auth.something. Here it probably makes sense to use paste.httpexceptions. - one in BaseRepoController, which has no specific relation to paste. This occurrence can be replaced with webob.exc like in the rest of the code base.
docs: upgrade: make upgrade instructions from version control more explicit
Instruct users to make a note of the orginal and new revision, both to help in any failure analysis, and also to let them realize when the 'hg update' command failed.
auth: move 'active' handling out of the individual auth modules
The 'active' flag in the Kallithea user database is very fundamental and should not be specific to auth modules. Modules should only care about whether the user is active in the external authentication system.
user_activation_state is thus removed, and 'hg.extern_activate.auto' is now consistently checked for all kinds of external authentication.
auth: drop active_from_extern from internal auth API
Modules should never auth a user if the auth source knows the user is inactive. Also, it is too late and unreliable to disable users when they try to log in. There is thus no need for this concept.
Only the crowd module had some traces of actual active_from_extern usage. The 'active' flag for crowd users was fully controlled from crowd. Now, Instead, just let crowd reject authentication of users that are inactive in crowd, and leave the internal Kallithea 'active' flag under admin control.
auth: change get_allowed_ips to be more resilient when operating on a cached default user
Before, random changes to how things are fetched and cached across database sessions could cause get_allowed_ips to fail with:
DetachedInstanceError: Instance <User> is not bound to a Session; attribute refresh operation cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
Instead, just check for user_id, using same pattern as a bit later in same function.
files: use the web browsers built-in js history instead of native.history.js
The history API is available in all web browsers we support.
window.history.pushState is called to register a state that we can go back/forward to. (But contrary to native.history.js, it doesn't do any immediate processing of the state and doesn't actually navigate to it.)
When navigation occurs, we get the popstate event and invoke load_state to actually load the state.
front-end: Move .less files to the front-end folder
The less directory was public - we prefer to have the source file outside the public folder. And we prefer to name the folder after the use of content instead of the primary file type of content.
The last remaining 3rd-party file, mergely.css, was still not used from that location and is just removed.
front-end: Use select2 from node_modules and stop bundling it
select2 3.5.4 was added in 304e83e9bcde ... but the latest npm release in the 3 series is 3.5.1, so we use that one instead. We should probably upgrade to the 4 series.
The select2 images were not in the location the generated css pointed - now we copy them from node_modules to the right location, next to the generated css.
Note: this will drop 190cb30841de "branches: fix performance of branch selectors with many branches - only show the first 200 results" ... but it should no longer be relevant now when we use server side filtering.
15e507047bae introduced select2-bootstrap.css - it is not clear what version was used, but we use the latest 1.4.6 which also is very old.
front-end: Store temporary files in a tmp directory
Avoid mixing temporary files with source files or have them in the generated output. But for now, keep them in the front-end directory where we need write access for node_modules anyway.
front-end: Introduce 'front-end' directory with source files for building the front-end
The top level with package.json is not included when installing with pip, and 'kallithea-cli front-end-build' and npm would thus fail.
Instead, introduce 'kallithea/front-end/'. It is under 'kallithea/' and is thus included when doing pip install, but it is just a source directory and not under "public". Most of the "less" stuff should probably move there. And probably also most things that now are checked in under "public", so a fully populated public front-end directory can be built anywhere, without write access to the Python installation directory.
search: better messaging when the index hasn't been built yet
Improve the situation around the "Please run whoosh indexer" message.
1. It is not a situation that justifies logging a stack trace. 2. Don't just say "whoosh indexer" - it actually has a name: kallithea-cli index-create . 3. Don't report server side commands in the user interface.
kallithea-cli: remind users to generate the front-end
Being in the habit of just generating a config file and possibly a database, users may forget to generate the front-end and get a crippled Kallithea experience at first set-up.
Try to avoid this by reminding users explicitly about front-end-build in these two steps of the set-up process.
model: move notification types from Notification to NotificationModel
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
As there is no database storage of notifications anymore, the Notification class will be removed. However, the notification type definitions are still used for email notifications, and need to live somewhere. As creating notifications is always passing via NotificationModel, it makes sense to move the types there.
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
templates: remove notification count from user profile button
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
This commit removes the notification count 'badge' next to the username top-right, and the count in the expanded field when clicking that username.
model: notification: don't round-trip via list if you want a set
'create' in NotificationModel starts from an empty list, appends entries to it, then converts it to a set. You could equally start with a set and add to it, avoiding the final conversion.
Make sure that the help text and error messages from Celery (e.g. from 'kallithea-cli celery-run -c my.ini -- --help' or '-- -xyz') contain a valid 'Usage:' string.
Without these changes, the usage string will use the arbitrary description from 2c3d30095d5e and a full path: Usage: kallithea celery worker .../bin/kallithea-cli [options]
With the changes, it becomes: Usage: kallithea-cli celery-run -c CONFIG_FILE -- [options]
Click will let optparse find 'kallithea-cli' from argv[0]. The command part 'celery-run -c CONFIG_FILE --' could perhaps be found with Click introspection, but it is simpler and perhaps equally reliable to make it explicit.
Kallithea is under the GPL license, and we can thus only distribute any generated code if we also ship the corresponding source.
We are moving towards a web front-end that use npm to download and compile various open source components. The components might not be GPL, but if we distribute any parts of their code (compiled or converted to other representation), then we also must distribute the corresponding source under the GPL.
It doesn't seem feasible for us to distribute the source of everything that npm downloads and includes when we are building. It thus also doesn't seem feasible for us to build and ship the compiled (possibly minified) front-end code. Instead, we have to make it as smooth as possible for our users to get up and running.
It doesn't seem feasible for us to ship or install npm. We must assume it is available. That requirement must be documented clearly, and we must recommend how to install npm for the most common platforms.
We could perhaps just document what manual steps to run. Kallithea doesn't work out of the box anyway - it has to be configured and initialized. Extra steps might not be a big problem.
Another approach is to call out to npm while pip is installing Kallithea and download the requirements and build the files. It can be done by customizing setuptools commands in setup.py. But: Python packaging is fragile. Even though we only support pip, it really isn't built for things like this. Custom output is muted and buffered and only shown if running with -v or the command fails. And pip and setup.py can be used to build and install in so many ways that we probably can't make it work reliably with all ways of installing Kallithea.
The approach implemented by this commit is to add a custom cli command 'front-end-build' to run the required commands. This single user-facing command can internally run various steps as needed. The only current requirement is the presence of npm and an internet connection.
For now, this will just create/update style.css ... but currently probably without any actual changes. The files created by npm (and the node_modules directory) must *not* be a part of the release package made with 'setup.py sdist'.
(Commit message is mostly written by Mads Kiilerich)
The conversion of celery_args (a tuple) into a list is mandatory, otherwise following error happens on celery-run:
Traceback (most recent call last): File ".../bin/kallithea-cli", line 11, in <module> load_entry_point('Kallithea', 'console_scripts', 'kallithea-cli')() File ".../lib/python2.7/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File ".../lib/python2.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File ".../lib/python2.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ".../lib/python2.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File ".../lib/python2.7/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File ".../bin/kallithea_cli_base.py", line 52, in runtime_wrapper return annotated(*args, **kwargs) File ".../bin/kallithea_cli_celery.py", line 39, in celery_run return cmd.run_from_argv('kallithea celery worker', celery_args) File ".../lib/python2.7/site-packages/celery/bin/worker.py", line 177, in run_from_argv *self.parse_options(prog_name, argv, command)) File ".../lib/python2.7/site-packages/celery/bin/base.py", line 412, in parse_options return self.parser.parse_args(arguments) File "/usr/lib64/python2.7/optparse.py", line 1404, in parse_args args = largs + rargs TypeError: can only concatenate list (not "tuple") to list
The problem was introduced in 1d539bb18165 in last minute changes made by Mads.
As changeset.create_comment is now only used from changeset.create_cs_pr_comment (used from pullrequests controller as well), there is no need to have it as a separate method.
controllers: changeset: always allow status changes
Don't disallow status changes on changesets that are part of a pull request and on which the last status change happened via the pull request.
This odd restriction was already previously highlighted by Mads Kiilerich as 'RLY?' in commit 7834f845505aec3086f525600c81209a40b495ef, so it seems fair to remove it.
When dont_allow_on_closed_pull_request no longer is set, StatusChangeOnClosedPullRequestError will no longer be raised.
Rename the oddly named method '_get_is_allowed_change_status' to '_is_allowed_to_change_status'. Not only does this read more easily, but also it is more in line with the variable to which the result is assigned to 'allowed_to_change_status'.