__get_lem and __get_index_filenames has nothing in common with the rest of util2, neither with respects to implementation nor usage, and belongs in a dedicated module. Also, they are clearly not actually private, so shouldn't be named with leading underscores.
compare: ensure that repositories exist before proceeding
The index method on CompareController did not verify that other_repo existed, causing a rendering error if it wasn't.
Since neither controller method can proceed if either repository is non-existent, check existence and load Repository objects in __before__. Also perform type compatibility check up front while we're at it, remove redundant repository database lookups, and enable error message i18n.
paster: split paster specifics out of kallithea.lib.utils
BasePasterCommand and ask_ok are only useful in a Paster/command-line context, and can thus be removed from the already overly cluttered main utils module.
(The new common.py has been added to Mercurial as a copy of utils.py, preserving its file history, but creating a somewhat bewildering diff.)
First, find all calls to HasPermissionAll with only a single permission given, and convert to equivalent calls to HasPermissionAny.
Next, observe that it's hard to envision situations requiring multiple permissions (of the same scope: global/repo/repo group) to be satisfied. Sufficiently hard that there are actually no such examples in the code.
Finally, considering that (should it ever be needed) HasPermissionAll can be trivially built as a conjunction of HasPermissionAny calls (the decorators, too) with only a small performance impact, simply remove HasPermissionAll and related classes and functions.
A log level of WARNING meant that Alembic was entirely silent when nothing went wrong. With INFO, Alembic actually shows what migrations it is running.
kallithea/tests/models/test_dump_html_mails.ref.html is expected to contain trailing whitespace in the "-- " de-facto standard signature separator (as e.g. referenced in RFC 3676, section 4.3).
The subject line is used for mail threading in gmail and can thus not be changed without impacting users ... but now we do it.
* The tag '[Review]' is more spot-on than '[Added]'. * The subject should be short so it fits on one line, so abbreviate "pull request" to PR. * Add the PR owner - convenient for filtering comments on own PRs from comments on other PRs.
pullrequests: better handling of Mercurial pullrequests with missing revisions - don't crash
Trying to display a Mercurial PR with missing changesets could give a crash when trying to compute available updates after 3f646f7bac39 did that c.cs_ranges could be empty. That would normally not happen on Mercurial, but could happen when restoring an old filesystem backup ... or when using strip on the server.
hooks: always convert unicode to byte strings when passed to ui.status
Kallithea generally uses unicode strings internally, but ui.status follows the Mercurial convention and expects a byte string. Strings passed to ui.status should thus always by converted to byte strings. Do that explicitly with safe_str. (The alternative of using more byte strings internally seems less appealing.)
hooks: fix encoding problems of lock release ui messages
The vcs test test_push_unlocks_repository_hg would fail when waitress tried to append a unicode chunk to the byte stream. The unicode string came from the 'Released lock on repo' message passed to ui.status from the push hook.
ui.status do however follow the Mercurial convention and expects a byte string.
Changelog.rst is empty and just refer to the Mercurial logs. More readable release notes and other meta information is also available on the web site and in announcement mails.
templates: disable special mako error handler - ironically this gives better stack traces
Errors in templates could give truncated stack traces pointing at the mako error handler:
... File '.../kallithea/kallithea/controllers/files.py', line 202 in index return render('files/files.html') File '.../kallithea-venv/lib/python2.7/site-packages/pylons/templating.py', line 244 in render_mako cache_type=cache_type, cache_expire=cache_expire) File '.../kallithea-venv/lib/python2.7/site-packages/pylons/templating.py', line 219 in cached_template return render_func() File '.../kallithea-venv/lib/python2.7/site-packages/pylons/templating.py', line 241 in render_template return literal(template.render_unicode(**globs)) File '.../kallithea-venv/lib/python2.7/site-packages/mako/template.py', line 452 in render_unicode as_unicode=True) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 803 in _render **_kwargs_for_callable(callable_, data)) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 835 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 855 in _exec_template _render_error(template, context, compat.exception_as()) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 864 in _render_error result = template.error_handler(context, error) File '.../kallithea-venv/lib/python2.7/site-packages/pylons/error.py', line 22 in handle_mako_error raise (exc, None, sys.exc_info()[2]) AttributeError: 'tuple' object has no attribute 'node'
Without the mako error handler we get a full and useful stack trace - including calls in generated but readable .html.py files.
File '.../kallithea/kallithea/controllers/files.py', line 202 in index return render('files/files.html') File '.../kallithea-venv/lib/python2.7/site-packages/pylons/templating.py', line 244 in render_mako cache_type=cache_type, cache_expire=cache_expire) File '.../kallithea-venv/lib/python2.7/site-packages/pylons/templating.py', line 219 in cached_template return render_func() File '.../kallithea-venv/lib/python2.7/site-packages/pylons/templating.py', line 241 in render_template return literal(template.render_unicode(**globs)) File '.../kallithea-venv/lib/python2.7/site-packages/mako/template.py', line 452 in render_unicode as_unicode=True) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 803 in _render **_kwargs_for_callable(callable_, data)) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 835 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 860 in _exec_template callable_(context, *args, **kwargs) File '.../data/templates/base/root.html.py', line 219 in render_body __M_writer(escape(next.body())) File '.../data/templates/base/base.html.py', line 57 in render_body __M_writer(escape(next.main())) File '.../data/templates/files/files.html.py', line 121 in render_main runtime._include_file(context, u'files_ypjax.html', _template_uri) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 730 in _include_file callable_(ctx, **_kwargs_for_include(callable_, context._data, **kwargs)) File '.../data/templates/files/files_ypjax.html.py', line 57 in render_body runtime._include_file(context, u'files_source.html', _template_uri) File '.../kallithea-venv/lib/python2.7/site-packages/mako/runtime.py', line 730 in _include_file callable_(ctx, **_kwargs_for_include(callable_, context._data, **kwargs)) File '.../data/templates/files/files_source.html.py', line 117 in render_body __M_writer(escape(h.pygmentize_annotation(c.repo_name,c.file,linenos=True,anchorlinenos=True,lineanchors='L',cssclass="code-highlight"))) File '.../kallithea/kallithea/lib/helpers.py', line 360 in pygmentize_annotation return literal(markup_whitespace(annotate_highlight(filenode, url_func(repo_name), **kwargs))) File '.../kallithea/kallithea/lib/annotate.py', line 57 in annotate_highlight highlighted = highlight(filenode.content, lexer, formatter) File '.../kallithea-venv/lib/python2.7/site-packages/pygments/__init__.py', line 87 in highlight return format(lex(code, lexer), formatter, outfile) File '.../kallithea-venv/lib/python2.7/site-packages/pygments/__init__.py', line 66 in format formatter.format(tokens, realoutfile) File '.../kallithea-venv/lib/python2.7/site-packages/pygments/formatter.py', line 95 in format return self.format_unencoded(tokensource, outfile) File '.../kallithea-venv/lib/python2.7/site-packages/pygments/formatters/html.py', line 850 in format_unencoded for t, piece in source: File '.../kallithea/kallithea/lib/annotate.py', line 168 in _wrap_tablelinenos for el in self.filenode.annotate)) File '.../kallithea/kallithea/lib/annotate.py', line 167 in <genexpr> annotate = ''.join((self.annotate_from_changeset(el[2]()) File '.../kallithea/kallithea/lib/vcs/backends/hg/changeset.py', line 273 in get_file_annotate sha = hex(annotate_data[0].node())
setup: move test dependencies to dev_requirements.txt to make them optional
Remove the need for having test tools on production systems. Installing test dependencies is made an extra explicit step.
pip is the future, but doesn't have the same tests_require features as setuptools kind of has.
I don't like this way of handling it without setup.py support and with explicit naming of the ugly dev_requirements.txt ... but that seems to be the way to do it.
dbmigrate-test executes Alembic upgrade scripts between two Kallithea versions in a clean environment. There is no automated testing of whether the upgraded database is functional, only that the upgrade proceeds without errors.
Use Alembic to migrate database away from SQLAlchemy Migrate. This eliminates the last vestiges of SQLAlchemy Migrate. Since we drop the Migrate table (and its contents), it is not possible to revert this database change; however, downgrading to this Alembic schema revision (9358dc3d6828) will in practice be enough to ensure compatibility with all previous Kallithea versions.
(As always, the Alembic migration script is committed in the same revision as the database schema changes.)
db: enable use of main Kallithea config as Alembic config
Newly generated Kallithea config .ini files will be valid Alembic config files, eliminating the need for a separate alembic.ini config redundantly specifying the database connection string.
We reference the Alembic migration environment using kallithea:alembic, which should work independently of how Kallithea is installed.
We also configure a default 'alembic' log level of WARNING, to reduce the amount of clutter in the config file, reduce the changes needed to upgrade existing config files for use with Alembic, and allowing us to change the default Alembic log level for all users down the road.
(It makes sense to define Alembic logging in code, while all other loggers are configured in the configuration file, because Alembic is special: it runs on the command line, not as part of the web app.)
The command is placed kallithea.lib.dbmigrate:UpgradeDb (which was the location of the old command, too), to ensure that "paster upgrade-db" continues to work, even if Kallithea is installed in "editable" mode (setup.py develop/pip install -e) and package metadata has not been updated (and also to prevent issues caused by stale .pyc files).