setup: limit WebTest version to <2.0

Running
pip install -e .
in a clean environment (or --upgrade), would cause:
"ContextualVersionConflict: (WebOb 1.1.1 (.../lib/python2.7/site-packages), Requirement.parse('WebOb>=1.2'), set(['WebTest']))"

setup.py required webob>=1.0.8,<=1.1.1 which pip resolved as WebOb==1.1.1.
setup.py required Pylons which requires WebTest>=1.3.1. Pip would thus install
the latest WebTest-2.0.23 ... which however has a WebOb>=1.2 dependency.
Pip only makes one pass at resolving dependencies and it would not resolve or
notice this inconsistency.

This problem came with dd676fdeda0f which moved pinning of WebTest 1.4.3 out of
setup.py.

Since Pylons require webtest anyway, there is no point in having it as a dev
requirement only - put it back in setup.py.

This might just be a temporary workaround until the WebOb version is updated.

Original patch by Dominik has been rewritten by Mads.

notifications: new email templates with html styling

- with heavy interfering and hacking by Mads Kiilerich.

mails: restructure notification mail content

* Mention "The Thing" in the header, link to online version
* Show The Thing
* A dense summary of the essentials of the context below it

The html indentation is odd in order to make the next diff smaller.

The text version is based on:

for a in kallithea/templates/email_templates/*.html; do sed -e 's,<\([^%/>]\|/[^%>]\)*>,,g' -e 's,\.html",\.txt",g' -e 's,^ *,,g' -e 's/}/|n,unicode}/g' $a > ${a%%.html}.txt ; done

compare: improve select2 performance with many branches

Like elsewhere for pure branch selectors.

branches: fix performance of branch selectors with many branches - only show the first 200 results

The way we use select2, it will cause browser performance problems when a
select list contains thousands of entries. The primary bottleneck is the DOM
creation, secondarily for the query to filter through the entries and decide
what to show. We thus primarily have to limit how many entries we put in the
drop-down, secondarily limit the iteration over data.

One tricky case is where the user specifies a short but full branch name (like
'trunk') but many other branches contains the same string (not necessarily at
the beginning, like 'for-trunk-next-week') which come before the perfect match
in the branch list. It is thus not a solution to just stop searching when a
fixed amount of matches have been found.

Instead, we limit the amount of ordinary query matches, but always show all
prefix matches. We thus always have to iterate through all entries, but we
start using the (presumably) cheaper prefix search when the limit has been
reached.

There is no filtering initially when there is no query term, so that case has
to be handled specially.

Upstream select2 is now at 4.x. Upgrading is not trivial, and getting this
fixed properly upstream is not a short term solution. Instead, we customize our
copy. The benefit from this patch is bigger than the overhead of "maintaining"
it locally.

diff: only highlight of difference between del and add line for one-liners

Comparing a single del line with the first of several add lines would often be
quite arbitrary and misleading.

Note: the non-gitdiff code paths in diffs.py are incomplete and unused and
unusable.

diff: tweak mergely side-by-side diff view

Expand diff pane to match view port and avoid extra scroll bars.

files: always show the requested version - not last changeset that touched the file

The last changeset to touch the file is rarely relevant - the whole annotate of
individual lines of code will give that kind of information in a much more
relevant way. The information is however available in the history drophown.

This also gets rid of some weird revision compare.

hg: drop pointless push_ssl configuration setting - if there is a risk push can be compromised, credentials can also easily be stolen for pull

Everybody should have a ssl-only setup now. Alternatively, there is a use case
for 'only anonymous traffic on ssl - all authentication and authenticated
traffic must be on ssl'. That can be done with proper web server configuration.

pullrequests: drop 'Closing.' comment when closing a PR - the semantic change is shown clearly everywhere

Things have changed since 9cfc66a665ae.

reposcan: don't report missing repos as removed if not using --delete-obsolete

(Odd name ... but whatever ...)

pullrequests: use monospace for hashes - columns of hashes look messy when they have different lenght

17f9f921a538 was insufficient.

css: disable unfortunate styling of disabled btn-danger - it would sometimes appear grey-in-grey

Seen for "Delete" on the files page.

files: use revision in Edit and Delete links

Branch names can contain '/' and can't generally be encoded in URLs.

model: refactor and simplify _get_instance

_get_instance is a BaseModel method, but never uses "self". Instead
it takes a class argument, which indicates that it's better suited as
a classmethod on said classes.

Also rename to something more descriptive, remove leading underscore
since it's not a private API, and refactor for readability.

model: drop BaseModel.cls

It no longer serves a purpose.

model: inline BaseModel.get_all

This is just needless indirection (and doesn't actually add any
abstraction on top of the database object), so inline all calls to it.

Don't touch PullRequestModel.get_all, though, since it is an entirely
unrelated method that just shadows the inherited get_all.

db: remove deprecated getAll method

Not sure why this wasn't done immediately when get_all was added. Maybe
search and replace was out of order at the time.

model: inline superfluous "helper" methods

__get_changeset_status wasn't even used and has simply been removed.

Notification.__get_notification reimplemented BaseModel._get_instance,
warts and all, and has been replaced by calls to that method.

The rest were just needless indirection, and have been inlined.

pygmentsutils: separate from util2

__get_lem and __get_index_filenames has nothing in common with the rest
of util2, neither with respects to implementation nor usage, and belongs
in a dedicated module. Also, they are clearly not actually private, so
shouldn't be named with leading underscores.

defaults: remove stub controllers

Remove empty controllers that seem to be left over from experiments with the
automatic 'routes' routing for 'resource'.

Also, remove test coverage of this. The tests didn't even test the empty
controllers and have been broken for a while.

controllers: remove pointless comments

These comments are mostly trivial and sometimes wrong. We don't want to
maintain or pretend we do.

tests: update mail template dump test

The combination of 552800808386 and ab29e4c39550 caused incorrect template content.

compare: ensure that repositories exist before proceeding

The index method on CompareController did not verify that other_repo
existed, causing a rendering error if it wasn't.

Since neither controller method can proceed if either repository is
non-existent, check existence and load Repository objects in __before__.
Also perform type compatibility check up front while we're at it, remove
redundant repository database lookups, and enable error message i18n.

paster: split paster specifics out of kallithea.lib.utils

BasePasterCommand and ask_ok are only useful in a Paster/command-line
context, and can thus be removed from the already overly cluttered main
utils module.

(The new common.py has been added to Mercurial as a copy of utils.py,
preserving its file history, but creating a somewhat bewildering diff.)

auth: clean up PermsFunction

Now shows scope in HasUserGroupPermissionAny instead of '?'.

auth: remove HasPermissionAll and variants

First, find all calls to HasPermissionAll with only a single permission
given, and convert to equivalent calls to HasPermissionAny.

Next, observe that it's hard to envision situations requiring multiple
permissions (of the same scope: global/repo/repo group) to be satisfied.
Sufficiently hard that there are actually no such examples in the code.

Finally, considering that (should it ever be needed) HasPermissionAll
can be trivially built as a conjunction of HasPermissionAny calls (the
decorators, too) with only a small performance impact, simply remove
HasPermissionAll and related classes and functions.

db: increase default Alembic log level to INFO

A log level of WARNING meant that Alembic was entirely silent when
nothing went wrong. With INFO, Alembic actually shows what migrations
it is running.

whitespacecleanup: exclude email test reference

kallithea/tests/models/test_dump_html_mails.ref.html is expected to
contain trailing whitespace in the "-- " de-facto standard signature
separator (as e.g. referenced in RFC 3676, section 4.3).

run-all-cleanup: fix incorrect interpreter

POSIX (and Debian) requires /bin/sh.

notifications: tweak PR mail subject lines

The subject line is used for mail threading in gmail and can thus not be
changed without impacting users ... but now we do it.

* The tag '[Review]' is more spot-on than '[Added]'.
* The subject should be short so it fits on one line, so abbreviate "pull
request" to PR.
* Add the PR owner - convenient for filtering comments on own PRs from comments
on other PRs.

email: add relevant title to subject of emails

For comment and pullrequest mails, add changeset/pullrequest title (shortened
to first 50 characters of first line)

notifications: make more template strings available for mails

To be used soon ...

tests: fix unicode issues in test_notifications mail template dump test

Other strings should perhaps be unicode too, but that is fixed silently for
pure ascii strings. Let's hope they are all pure.

update-repoinfo: pass command line repository names to the database as unicode

Fix bind SAWarning about not receiving unicode.

repo-scan: rewrite get_filesystem_repos to use os.walk instead of stupid recursion

I think this is more readable. It is also faster. Perhaps because the more
readable implementation makes it easier to optimize.

style: use monospace for hashes - columns of hashes look messy when they have different lenght

Especially 'f' was rendered narrow in the previous font.

We keep the default size (10.something px) instead of 12 px as the important
text on the line uses.

helpers: pass url generator function to Pager - don't rely on Pylons hack in webhelpers

This makes the code less magical and helps towards TurboGears2.

pullrequests: better handling of Mercurial pullrequests with missing revisions - don't crash

Trying to display a Mercurial PR with missing changesets could give a crash
when trying to compute available updates after 3f646f7bac39 did that
c.cs_ranges could be empty. That would normally not happen on Mercurial, but
could happen when restoring an old filesystem backup ... or when using strip on
the server.

pullrequests: better handling of revision range pullrequests with missing revisions - don't crash

Trying to display a revision range PR with missing changesets would give a
crash.

3f646f7bac39 solved a similar problem but caused another one when it did that
c.cs_ranges could be empty.

setup: support later waitress versions

Latest version (0.9.0) seems to work and has a nice-to-have fix for coverage.py
( https://github.com/Pylons/waitress/issues/107 ).

hooks: always convert unicode to byte strings when passed to ui.status

Kallithea generally uses unicode strings internally, but ui.status follows the
Mercurial convention and expects a byte string. Strings passed to ui.status
should thus always by converted to byte strings. Do that explicitly with
safe_str. (The alternative of using more byte strings internally seems less
appealing.)

hooks: fix encoding problems of lock release ui messages

The vcs test test_push_unlocks_repository_hg would fail when waitress tried to
append a unicode chunk to the byte stream. The unicode string came from the
'Released lock on repo' message passed to ui.status from the push hook.

ui.status do however follow the Mercurial convention and expects a byte string.

We solve the problem by applying safe_str.

Patch modified by Mads Kiilerich.

cache-keys: fix printing of unicode repo names - don't crash

LANG=C paster cache-keys my.ini --show
would crash with UnicodeEncodeError in print. That was a real problem when
running from crontab.

tests: move test data to top level 'data' folder - make grepping in kallithea/ easier

We like to have test.ini under kallithea/tests, but we don't want the temporary files there.