front-end: Use select2 from node_modules and stop bundling it
select2 3.5.4 was added in 304e83e9bcde ... but the latest npm release in the 3 series is 3.5.1, so we use that one instead. We should probably upgrade to the 4 series.
The select2 images were not in the location the generated css pointed - now we copy them from node_modules to the right location, next to the generated css.
Note: this will drop 190cb30841de "branches: fix performance of branch selectors with many branches - only show the first 200 results" ... but it should no longer be relevant now when we use server side filtering.
15e507047bae introduced select2-bootstrap.css - it is not clear what version was used, but we use the latest 1.4.6 which also is very old.
front-end: Store temporary files in a tmp directory
Avoid mixing temporary files with source files or have them in the generated output. But for now, keep them in the front-end directory where we need write access for node_modules anyway.
front-end: Introduce 'front-end' directory with source files for building the front-end
The top level with package.json is not included when installing with pip, and 'kallithea-cli front-end-build' and npm would thus fail.
Instead, introduce 'kallithea/front-end/'. It is under 'kallithea/' and is thus included when doing pip install, but it is just a source directory and not under "public". Most of the "less" stuff should probably move there. And probably also most things that now are checked in under "public", so a fully populated public front-end directory can be built anywhere, without write access to the Python installation directory.
search: better messaging when the index hasn't been built yet
Improve the situation around the "Please run whoosh indexer" message.
1. It is not a situation that justifies logging a stack trace. 2. Don't just say "whoosh indexer" - it actually has a name: kallithea-cli index-create . 3. Don't report server side commands in the user interface.
kallithea-cli: remind users to generate the front-end
Being in the habit of just generating a config file and possibly a database, users may forget to generate the front-end and get a crippled Kallithea experience at first set-up.
Try to avoid this by reminding users explicitly about front-end-build in these two steps of the set-up process.
model: move notification types from Notification to NotificationModel
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
As there is no database storage of notifications anymore, the Notification class will be removed. However, the notification type definitions are still used for email notifications, and need to live somewhere. As creating notifications is always passing via NotificationModel, it makes sense to move the types there.
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
templates: remove notification count from user profile button
This commit is part of the removal of the UI notification feature from Kallithea, which is not deemed useful in its current form. Only email notifications are preserved.
This commit removes the notification count 'badge' next to the username top-right, and the count in the expanded field when clicking that username.
model: notification: don't round-trip via list if you want a set
'create' in NotificationModel starts from an empty list, appends entries to it, then converts it to a set. You could equally start with a set and add to it, avoiding the final conversion.
Make sure that the help text and error messages from Celery (e.g. from 'kallithea-cli celery-run -c my.ini -- --help' or '-- -xyz') contain a valid 'Usage:' string.
Without these changes, the usage string will use the arbitrary description from 2c3d30095d5e and a full path: Usage: kallithea celery worker .../bin/kallithea-cli [options]
With the changes, it becomes: Usage: kallithea-cli celery-run -c CONFIG_FILE -- [options]
Click will let optparse find 'kallithea-cli' from argv[0]. The command part 'celery-run -c CONFIG_FILE --' could perhaps be found with Click introspection, but it is simpler and perhaps equally reliable to make it explicit.
Kallithea is under the GPL license, and we can thus only distribute any generated code if we also ship the corresponding source.
We are moving towards a web front-end that use npm to download and compile various open source components. The components might not be GPL, but if we distribute any parts of their code (compiled or converted to other representation), then we also must distribute the corresponding source under the GPL.
It doesn't seem feasible for us to distribute the source of everything that npm downloads and includes when we are building. It thus also doesn't seem feasible for us to build and ship the compiled (possibly minified) front-end code. Instead, we have to make it as smooth as possible for our users to get up and running.
It doesn't seem feasible for us to ship or install npm. We must assume it is available. That requirement must be documented clearly, and we must recommend how to install npm for the most common platforms.
We could perhaps just document what manual steps to run. Kallithea doesn't work out of the box anyway - it has to be configured and initialized. Extra steps might not be a big problem.
Another approach is to call out to npm while pip is installing Kallithea and download the requirements and build the files. It can be done by customizing setuptools commands in setup.py. But: Python packaging is fragile. Even though we only support pip, it really isn't built for things like this. Custom output is muted and buffered and only shown if running with -v or the command fails. And pip and setup.py can be used to build and install in so many ways that we probably can't make it work reliably with all ways of installing Kallithea.
The approach implemented by this commit is to add a custom cli command 'front-end-build' to run the required commands. This single user-facing command can internally run various steps as needed. The only current requirement is the presence of npm and an internet connection.
For now, this will just create/update style.css ... but currently probably without any actual changes. The files created by npm (and the node_modules directory) must *not* be a part of the release package made with 'setup.py sdist'.
(Commit message is mostly written by Mads Kiilerich)
The conversion of celery_args (a tuple) into a list is mandatory, otherwise following error happens on celery-run:
Traceback (most recent call last): File ".../bin/kallithea-cli", line 11, in <module> load_entry_point('Kallithea', 'console_scripts', 'kallithea-cli')() File ".../lib/python2.7/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File ".../lib/python2.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File ".../lib/python2.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ".../lib/python2.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File ".../lib/python2.7/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File ".../bin/kallithea_cli_base.py", line 52, in runtime_wrapper return annotated(*args, **kwargs) File ".../bin/kallithea_cli_celery.py", line 39, in celery_run return cmd.run_from_argv('kallithea celery worker', celery_args) File ".../lib/python2.7/site-packages/celery/bin/worker.py", line 177, in run_from_argv *self.parse_options(prog_name, argv, command)) File ".../lib/python2.7/site-packages/celery/bin/base.py", line 412, in parse_options return self.parser.parse_args(arguments) File "/usr/lib64/python2.7/optparse.py", line 1404, in parse_args args = largs + rargs TypeError: can only concatenate list (not "tuple") to list
The problem was introduced in 1d539bb18165 in last minute changes made by Mads.
As changeset.create_comment is now only used from changeset.create_cs_pr_comment (used from pullrequests controller as well), there is no need to have it as a separate method.
controllers: changeset: always allow status changes
Don't disallow status changes on changesets that are part of a pull request and on which the last status change happened via the pull request.
This odd restriction was already previously highlighted by Mads Kiilerich as 'RLY?' in commit 7834f845505aec3086f525600c81209a40b495ef, so it seems fair to remove it.
When dont_allow_on_closed_pull_request no longer is set, StatusChangeOnClosedPullRequestError will no longer be raised.
Rename the oddly named method '_get_is_allowed_change_status' to '_is_allowed_to_change_status'. Not only does this read more easily, but also it is more in line with the variable to which the result is assigned to 'allowed_to_change_status'.
controllers: forward pullrequests.delete_comment to changeset
Remove duplication between pullrequests and changeset. We move the code outside ChangesetController to make it callable from PullrequestsController.
Note: - instead of keeping the method pullrequests.delete_comment itself and letting it forward to changeset.delete_comment, an alternative solution would have been to change the routing table directly. However, the chosen solution makes it more explicit which operations are supported on each controller.
tests: small improvements to test_changeset_comments
- response checking is incorrectly placed under 'Test DB' comment - no response check on the comment text was done - related: use different text for the comments in different tests - commit_id should be comment_id - remove duplicate checking of comment count in DB
cli: convert 'gearbox celeryd' into 'kallithea-cli celery-run'
Note: - '--' is never explicitly present in the arguments when using Click. The click parser will take care of '--' as separator between dash-dash-arguments and positional arguments, following standard UNIX conventions.
cli: initial introduction of 'kallithea-cli' command
This commit adds a command 'kallithea-cli' that intends to replace the existing set of 'gearbox' commands that relate to setting up kallithea. Gearbox would still be used for 'gearbox serve', but other commands like 'make-config', 'setup-db', etc. would be converted to 'kallithea-cli' commands.
The python package 'Click' is used to generate the CLI. Using decorators, this package makes it very easy to generate a CLI out of simple methods. See: http://click.pocoo.org/7/
Using Gearbox for custom commands is possible, but documentation on this topic is limited. As the added value of Gearbox for that use case is not clear, we can well switch to something else if it seems better/easier.
graph: don't try to get obsolete status for fake parent revisions
Fake parents use negative revision numbers as internal placeholders. Previous Mercurial versions could handle negative non-existing revision numbers, but Mercurial 4.8 will fail on this invalid input when getting the obsolete status. Instead, make sure we only get obsolete status for actual revisions.
tests: fix vcs Hg helper functions get_user_name and get_user_email to actually use provided config_file
Running just test_get_user_name and test_get_user_email would fail when run as: py.test kallithea/tests/vcs/test_repository.py -k test_get_user
The config file was not used at all, and the tests would only pass when test_get_config_value had been run first and as an unfortunate side-effect had read the same file.
The icon of the comment bubble is slightly off-center vertically (too low). This problem can be solved by reducing the line height instead of using the default of 20px.
style: remove vertical scrollbar on hover of last line's comment bubble
When hovering the comment bubble of the last line of a diff, a vertical scrollbar appears. This behavior is not present on the stable branch.
Attempts to fix the problem by changing element sizes or padding did not solve the problem. As an alternative, this commit hides overflow - this makes no visual difference.
model/scm: fix incorrect reporting of pull from remote git repo
In issue #327 it was reported that the pull of a remote git repo causes the following inaccurate journal entry: "committed via Kallithea into repository"
The problem is caused by the following flow:
1. pull_changes calls _handle_push for git repos, passing action='push_remote'.
2. _handle_push calls _handle_rc_scm_extras, but does not pass 'action'
3. _handle_rc_scm_extras uses a default value of 'push_local' for action.
4. The action string 'push_local' will result in the mentioned journal entry, via action_map in action_parser() (kallithea/lib/helpers.py).
To get a more accurate log entry, _handle_push should pass the action that it got from pull_changes. In that case, the log entry will become "pulled from remote into repository"
Commit 39bdf0ab2862 has changed the Git hooks. When upgrading Kallithea to this commit, but not explicitly reinstalling the Git hooks, pushing to Git repos will fail.
Add a section to the documentation to explain that a reinstall of the hooks may be needed, and how to do it.
notification: don't repeat common actions for each email recipient
The emails sent upon PR creation or comments are identical for all recipients, except for the 'To:' header added by the send_email method. Calculation of subject, body, etc. can thus be moved outside of the loop.
Move the manipulation of the recipient list down to where it is used, for clarity.
it does not actually work. The '/repos' part is stripped off by the canonical_url method.
The 'host' entry in the arguments passed to routes.url does not strictly need to be a pure hostname. At least, the implementation does no validation of this fact, it is concatenated verbatim between the protocol and the rest of the URL.
As mapping Kallithea to a subpath of a base hostname is a valid implementation, the canonical_url feature should allow it.
setup: bump all upper pip dependency versions to minor updates of what currently is available and testable on pypi
Based on manual editing after: pip freeze | sed -n 's/==/ /gp' | while read p v; do sed -i -e "/\<$p[ \"]/s/\(\",\|$\)/, < $v\1/gi" setup.py dev_requirements.txt; done
These updates have been tested with automated tests and some amount of manual testing. Remaining problems will hopefully be caught by additional testing before the branch is declared stable.
The latest supported Celery version, 3.1.26.post2, is the latest in the whole 3 series. It works fine. But celery-4.0.0 doesn't work at all when testing. For now, just make it explicit that this is the limit.
`gearbox ishell` has an "optional" dependency on ipython as it fails with: Kallithea ishell requires the IPython Python package Before the previous change, after installing ipython < 4, it would however still fail the same way. With the previous change, it would fail in a slightly more helpful way with: ImportError: No module named traitlets.config.loader
With ipython < 4, after installing the missing traitlets (4.3.2) as hinted, ishell still failed with: TraitError: The 'config' trait of an InteractiveShellEmbed instance must be a Config or None, but a value of class 'traitlets.config.loader.Config' (i.e. {'InteractiveShellEmbed': {'confirm_exit': False}}) was specified.
With ipython >= 4, traitlets is installed as dependency, and ishell works. Tested with both ipython 4.0 and current latest version 5.8 .
Thus, just clarify that we only support ipython >= 4 .
auth: strip RFC4007 zone identifiers from IPv6 addresses before doing access control
If using IPv6, the request IP address might contain a '%' that the ipaddr module that is used for IP filtering can't handle.
https://tools.ietf.org/html/rfc4007#section-11 specifies how IPv6 addresses can have zone identifiers like trailing '%13' or '%eth0'. The zone identifier is used to help distinguish *if* the same address should be available on multiple interfaces. It *could* potentially have security implications in the odd case where the same address is different on different interfaces. The IP whitelist functionality does however not support zone filters, so there is no way users can expect the zone to be relevant for IP filtering. We can thus safely strip the zone index and only check for match on the other parts of the address.
Remove some unused imports, using 'autoflake' followed by manual redacting. find kallithea/lib/paster_commands/ -name "*.py" \ | xargs autoflake --remove-all-unused-imports -i
This same command could be applied gradually throughout the code base as areas are touched. Since autoflake may be too greedy in special cases, it is more difficult to do a big-bang action over all code.