db: drop unused _hash_key

A bit less md5 ...

diff: drop internal inconsistent use of '...' as context line numbers

as_html() did in some cases not check for '...' and would thus emit unnecessary
html.

diff: fix per file display of "Raw diff" and others

With py3, the matcher must be a list of byte strings.

ini: change .ini template to include sqlalchemy.url examples for all database types

Rely less on users creating their initial .ini "correctly" with the right
database choice.

hg: fix crash on displaying bookmarks - add lost hex conversion when computing bookmark property

The crash was seen for example when using "Switch To" with bookmarks.

The problem was introduced with py3 conversion in e7dbe089e10d.

comments: add changeset author in mail notification subject

In analogy to the handling of comments on pull requests, as changed in
commit d4061c6cc0e2.

lib/diffs: mark trailing tabs similar to trailing spaces

So far, spaces have not been marked up, but trailing spaces are followed by
'<i></i>' (newline symbol). Tabs have been marked up as '<u>\t</u>' (underlined
with icon), but trailing tabs didn't get the same "trailing whitespace" markup
as trailing spaces got.

Fix this unfairness by handling trailing tabs explicitly as '<u>\t</u><i></i>'
so they get both kinds of markup.

notifications: explicitly add author of pull request in invitation mail

When an author of a pull request does not get feedback from any of the
reviewers, he is left wondering whether they got the invitation mail at all.
By explicitly sending the invitation also to the author, he can at least know
if e-mail works at the server side and how reviewers see it.
The mail can also be forwarded as a 'reminder'.

logging: try to avoid using ANSI color codes when not logging to a terminal

Color on the console is nice, but it is annoying to get these codes when for
example redirecting to a file.

The formatter doesn't know exactly where the message will be used - it is fully
configurable. In the default configuration, the messages are passed to a
StreamHandler to sys.stderr .

As an approximate optmization, hardcode the color formatters to only emit color
formatting if logging to something that isatty.

setup: bump version numbers to current max

Based on 'pip freeze' before/after:

sed -i -e '/>=[^#]*$/s/, < [0-9.]\+//g' -e 's, == , >= ,g' setup.py dev_requirements.txt

Markdown 3.3 is a few months old could be bumped too ... but it change test
output due to
https://python-markdown.github.io/change_log/release-3.2/#codehilite-now-always-wraps-with-ltcodegt-tags
and can wait.

This is landing early in the development phase and will thus get good test
coverage.

permissions: drop hg.create.write_on_repogroup "Repository creation with group write access" setting

Simplify permissions system and get rid of some confusing tech debt.

Before, the global 'write_on_repogroup' setting controlled what write
permission on a repo group meant.

With this change, users can create repositories in a repo group if and only if
they have write access. Write access to a repo group will now mean the
permission to create repositories in it.

Write access to repo groups must be granted explicitly. There should not be any
other reason to grant write access than to allow users to create repos. There
is thus no upgrade concerns for this change.

An admin that doesn't want users to create repos in a repogroup should just not
give them write access.

These global settings might still exist in the database, but is ignored and no
longer used and do no harm.

db_manage: drop unused fix functions

The functions might be ok, but we don't know when or why to call them ... and
apparently we don't need them.

inifile: make it possible for expand() to comment out settings without assigning new value

For completeness, when we already can create and update values, also make it
possible to delete. This fits nicely into the implementation. Potentiallly
"nice to have" but not used yet.

The ini file is a text file and it only really makes sense to set string
values. Intuitively, it makes sense that setting something to None means that
the old value should be commented out but no new value should be set. If we
want to set a value of "None", then specify "None" - not None.

tg: include the Kallithea middleware wrapping of the TG WSGI application in application.py

This seems to make the architecture more clear than using hooks.

tg: refactor app_cfg blueprint configuration to align more with TG quickstart conventions

Drop prefer_toscawidgets2 and use_toscawidgets - they are not important and
already default to False.

lib: drop own asbool implementation and consistently use tg.support.converters as utils2.asbool

str2bool never reported error on odd input such as '' or '-1', but the tg
asbool behaviour of raising ValueError("String is not true/false: %r" % obj) in
that case seems fine.

setup: exclude celery 4.4.4 which is broken due to unexpressed dependency

Celery 4.4.4 introduced the use of the 'future' package but forgot to
express it in its dependencies.

We could add the missing dependency on 'future' in Kallithea, but since the
problem is already fixed upstream shortly after 4.4.4 was released [1], we can
be sure that the next release (presumably 4.4.5) will contain the fix.

[1] https://github.com/celery/celery/pull/6146

cleanup: drop last traces of kallithea-config

The kallithea-config sources were removed in commit
213085032127e941a3bd93d0e510828a9d87bf32 but an entry point was still
created by setup.py.
Moreover, the ini file still referenced this, instead of kallithea-cli
(config-create).

html: fix invalid html / mako structure

Fix bad nesting of tags and mako conditional and drop superfluous '}' outside
the mako block.

tests: actually test something useful in test_edit for gists (Issue #376)

Even though there was a test for editing gists, it did not catch the basic
loading problem reported in issue #376. In fact, the test just loaded the
edit page, but since no user was actually logged in, it just loaded the
login screen. As a result, no real gist editing code was tested at all.

Instead, explicitly check the redirection to a login screen, then proceed
with logging in and check that the edit page can be loaded.

Additionally, don't rely on the magic gist id '1' but create an actual gist
first.

Note: the test does not yet really edit a gist.

ini: explain how % must be escaped as %% (Issue #375)

Gearbox / paste would fail with:
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '('

The special handling of % is standard configparser behaviour, unless explicitly
fetching raw settings.

db: drop double specification of unique constraint (Issue #374)

Consistently use "unique=True" for single column constraints. Only use
UniqueConstraint for composite constraints.

MySQL did not like this double specification, while PostgreSQL and SQLite
did not care.

ini: put kallithea-config version in the ini file it creates (Issue #373)

Help keeping track of how the .ini was created.

docs: clarify the installation steps and how things fit together

Hint that create-db user information is for creating the initial local user
and that the database connection (which also might use some kind of user) is
something else.

tests: coverage of Git repo feeds

Coverage of the case reported in Issue #372.

feeds: fix failure getting feed for Git repos (Issue #372)

GitChangeset.diff() did
return ''.join(self.repository.get_diff(...))
even though get_diff returned a string. It worked, but was unnecessary and
inefficient.

That fails in py3: get_diff returns bytes ... and iterating doesn't give
characters but integers and we would get:
TypeError: sequence item 0: expected a bytes-like object, int found

Fixed by dropping the unnecessary iteration and joining.

This function is only used for feeds statistics.

auth: simplify handling of Crowd json response (Issue #370)

Correct error where Crowd authentication didn't work due to urllib.readlines()
returning bytes and thus failing to be joined with a string.

json.loads is however happy to take bytes directly.

Fix error handling to also handle bytes without crashing.

contributors: map different names of J. Lavoie onto one

Assuming the last used format is the right one.

i18n: updated translation for German

Currently translated at 68.9% (750 of 1087 strings)

i18n: updated translation for French

Currently translated at 98.9% (1076 of 1087 strings)

i18n: updated translation for French

Currently translated at 98.9% (1076 of 1087 strings)

i18n: updated translation for Russian

Currently translated at 100.0% (1087 of 1087 strings)

i18n: updated translation for French

Currently translated at 98.1% (1067 of 1087 strings)

i18n: updated translation for French

Currently translated at 98.0% (1066 of 1087 strings)

i18n: updated translation for Greek

Currently translated at 94.8% (1031 of 1087 strings)

i18n: updated translation for Luxembourgish

Currently translated at 6.1% (67 of 1087 strings)

issue: log error if still using issue_pat_XXX configuration after it was removed in d24051ce961c

Show a helpful:
found unsupported issue_prefix_pr = 'PR' - use issue_sub_pr instead
before bailing out with:
skipping incomplete issue pattern 'issue_pat_pr': '(?:PR\\s*)(\\d+)' -> 'https://kallithea-scm.org/repos/kallithea/pull-request/{id}' None

scripts/i18n: update i18n howto with recipe for use of scripts/i18n tooling

For now, just adding a new section. It should be integrated with the existing
documentation. Some steps can perhaps be simplified or automated so we need
less words.

inifile: add doctest coverage of comment corner cases for trailing space and value of None

Shows room for improvement - doesn't show any bugs.

The value of None is undefined behaviour.

tests: fix TestPermissions.test_inactive_user_group_does_not_affect_repo_group_permissions_inverse

Handle L1_new leak from test_rename_top_level_group_in_nested_setup .

tests: fix test_auth_ldap dependency on test order to avoid failure on uniqueness constraint

It would perhaps be even better to have each test clean up ... but let's only
aim for only avoiding collision on leaked test data.

tests: fix test_admin_settings test dependencies

Failure could be reproduced with
py.test kallithea/tests/functional/test_admin_settings.py -k x

uwsgi: enable reload-on-exception in configuration template

In case of severe failures in the stack (not handled by TG), it will thus
report "monitored exception detected, respawning worker ...".

hg: support Mercurial 5.4

rc0 has no release notes yet, but it seems to work and the test suite pass ...

ini: tweak template configuration for TG's handling of application errors

Mention what was learned while debugging request processing at WSGI/TG level
and the complexity of the TG stack gets in the way.

inifile: new implementation of setting updates to optimize reuse of comments and append location

Code comments should explain the algorithm, and test coverage suggest it works
correctly.

inifile: add doctest coverage of comment corner cases

Shows room for improvement - doesn't show any bugs.

uwsgi: clarify that --ini-paste-logged currently requires pastescript

This dependency is removed with uWSGI commit
https://github.com/unbit/uwsgi/commit/0807b3881f701a6cc1cc95fe872bb7ec2426ef3f
and should normally be included in the next release after 2.0.18.

uwsgi: slim down and tweak the default '[uwsgi]' configuration section

The goal is to have a basic working setup that show how and why to use uWSGI.
System administrators should check uWSGI documentation for further information
and general advice about operations.

Based on uWSGI documentation, experiments, and
https://www.techatbloomberg.com/blog/configuring-uwsgi-production-deployment/ .

uwsgi: drop unnecessary dependency of http module - just use http-socket directly

The http plugin has more advanced http functionality like https and load
balancing, duplicating what in many setups is handled by separate front-end
servers. In this simple template, just use the basic http-socket.

db: fix bad characters in __repr__

Introduced in effd091203ae.

i18n: fix dead code in Accept-Language workaround from 7c7d6b5c07c7

AppConfig is just providing defaults, and the .ini file has not been read yet.
There is thus no point in checking if i18n.lang has been set, and the code
would thus *always* set i18n.lang=en ... but that was fine, as it just is a
default.

email templates: fix missing translation of titles and buttons

The buttons and titles of email templates were not correctly translated. The
corresponding strings were not part of the i18n files because they were not
recognized by the extraction logic.

Note: this will generate new translation strings.

Reported-by: Jan Heylen <jan.heylen@nokia.com>