auth: make internal AuthUser methods private

These methods are never invoked from outside AuthUser; make this
explicit in the code.

Also remove '_instance' member, which is not referenced anywhere else
in the code.

auth: actually use _determine_auth_user argument

Fix silly mistake which slipped through the review. (We should not look
up the session cookie again, when it's passed as a function argument.)

admin: e-mail: don't repeat the e-mail settings from the .ini file

None of the settings from the .ini file are currently listed in the admin
interface, except for e-mail. There is no need for such an exception: either
we list all settings or none.

repos: fix abuse of dead 'repo' routing

The repo "show" controller didn't do anything and was unused. There was a
routing GET entry for it but it was only used for generating URLs for DELETE
and PUT operations that have separate controllers that happen to have the same
URL.

Use the right routing entries when generating URLs and drop the dead code.

cleanup: check for None object identity in cases where that is what the 'contract' says

Avoid applying bool() on complex objects - it might return something unexpected
such as the key (which might be 0 and thus be false). Checking for None is more
safe and faster.

ini: drop production.ini - everybody should create their own with unique and random keys

test.ini and development.ini are more relevant ... but could perhaps be
generated from templates on demand.

e-mail: clarify that smtp_use_ssl defaults to false

Commented settings in configuration files conventionally indicate the
default value.

e-mail: remove unused setting error_message from ini files

The ini files and templates contain a commented setting of 'error_message'
which does not seem to be used. It is referring to the error_message
variable in Paste, which has as description (Paste:ErrorMiddleware):

When debug mode is off, the error message to show to users.

However, setting this value apparently made no effect at all in Kallithea.

admin: remap/rescan: fix alignment of button

The 'Rescan repositories' button was not aligned with the checkboxes as
intended (like on the search settings page) due to a stray </div> tag.

validators: cleanup of message wording

NotReviewedRevisions was unused and could thus got the big cleanup ...

i18n: use plain strings on the english lookup side of translations

Unicode strings are unnecessarily complex and do not in any of our cases add
any value. It is thus better to consistently use plain strings.

scm: introduce AvailableRepoGroupChoices

Extract reusable code to reduce code duplication.

Note: groups_choices from the db module is only used from this function and
should perhaps also be extracted ... but it is also closely related to other
stuff that is (mis)placed in the db module so perhaps not ...

repos: refactor repo group handling - extract generic functionality that can be used consistently

By using the right repo group list in the form, it is no longer possible for
users to pick wrong repo creation locations ... and doing so anyway will be
caught earlier on with a more generic error message at the specific place.

remote: simplify clone_uri UI

The UI was a bit weird ... probably in an attempt of making it editable while
hiding passwords. Instead, just show the URL with password hidden, and only
save it back if it changed.

The UI only contains the clone_uri with passwords hidden. It will thus only be
saved when the form result is different from the value that was shown to the
user.

helpers: drop h.tooltip

h.tooltip did more magic in the past - now it just did a douple (or triple?)
escape of html.

c9cfaeb1cdfe removed most of the need for double escaping - and the places
where it is used, it must be 'tagged' with the safe-html-title class. Thus,
none of the remaining uses of h.tooltip are relevant (and might even be wrong);
normal automatic template escaping is just fine.

This is mostly:
sed -i 's,title="\${h.tooltip(\([^}]*\))}",title="${\1}",g' `hg mani`

helpers: cleanup of journal action_parser

rev will always have op and ref_name attributes if it is a BaseChangeset or
AttributeDict.

lazy_cs was undefined for the (apparently rarely hit changeset-not-found case)
- fix that by setting the default to False.

Title is not relevant when using lazy-cs so there is no need for this h.tooltip
invocation.

data_table: use simple 'escape' function instead of 'tooltip' function

The values will end up being double escaped when h.escape is used in template
expansion and turned into a string which then will be escaped in the template.

db: cleanup of repo group choices - better handling of root and better sorting

groups_choices always get a groups list so there is no reason have code for
creating a list.

All the responsibility of creating the tuples is moved to _generate_choice -
also for the magic top level entries.

Also sort the choices by all path elements - not just the top level element.

pullrequests: don't show '(Reviewer)' after each reviewer

PR author is still tagged with '(Owner)'.

changeset: make code more stable against unexpected comments

Avoid passing None to the template in comments. This makes the code more
correct and fixes a crash seen while hacking, probably never seen in the wild.

lib: allow ';' when recognizing URLs

URLs often contains '&' which often has been html encoded to '&amp;' before
urlification.

changelog: fix URL after submitting new page size via GET form

The URL ended up getting lots of crap. Workaround this by redirecting to a
clean URL.

archive: refactor archive handling

Use file descriptors in a more sane way.

BaseController: hide "Log out" link for external login sessions

If user is authorized by external means (API key or container auth),
Kallithea is not actually able to log the user out and should not show
the "Log out" link.

AuthUser: refactor AuthUser cookie/session serialization

* All serialization/deserialization logic is now in one place.

* CookieStoreWrapper is gone; any login sessions established in 2012 or
earlier will be terminated and users will have to log in again.

* The term "cookie store" has been dropped in favor of simply "cookie"
(the value is not a cookie "store" or data repository, but just the
actual cookie/session serialization of AuthUser).

Note: from_cookie_store was never used.

BaseController: enable container authentication on all pages

Previously, user had to visit the login page to log in using container
authentication; this now happens on every page load, unless user has an
existing login session.

The container authentication result is cached in session on success.

AuthSettingsController: don't validate options for disabled plugins

If the user disables a plugin, any submitted settings for that plugin
should be disregarded (neither validated nor stored in the database).

AuthSettingsController: refactor rendering

Instead of having one controller method ('auth_settings', which handles
POST) call another controller method ('index', which handles GET), have
both methods call a third, common '__render' method.

This makes it clear which arguments 'index' really takes (none). It's
also seen that the 'prefix_error' argument is always False, so it can
be removed. Finally, '__load_defaults' is no longer called twice when
POSTing invalid form values.

AuthSettingsController: don't add auth settings to tmpl_context

Do not import auth setting into the template context, since the values
are not actually used, except for 'auth_plugins'.

For 'auth_plugins', use the existing (and identical) 'enabled_plugins'
attribute instead.

AuthSettingsController: remove _defaults variable

Assign directly to c.defaults instead for simplicity.

AuthSettingsController: remove formglobals variable

Since the formglobals variable is used only to populate the template
context, just assign values directly to `c` instead. This does involve
dropping the (very large) debug log entry listing all the formglobals.

AuthSettingsController: rename dictionary key for consistency

Rename 'auth_plugins_shortnames' to 'plugin_shortnames' for consistency
with 'plugin_settings', and to emphasize that it's not one of the values
retrieved from the database using get_auth_settings (all of which start
with 'auth_').

AuthSettingsController: assume 'auth_plugins' option exists

In formglobals, the provided 'auth_plugins' default value is always
replaced with the real database value, returned along with other
plugin settings by Setting.get_auth_settings.

We can safely assume that the database value exists, since not only does
db_manage initialize the 'auth_plugins' option upon database creation,
but other authentication modules already assume its existence.

For instance, Settings.get_auth_plugins() is equivalent to

Settings.get_auth_settings()['auth_plugins']

except that get_auth_plugins will throw AttributeError, not KeyError,
if the 'auth_plugins' key does not exist in the database.

log_in_user: extract user session setup from LoginController

The next changeset will need to set up a user login session outside
LoginController. 'log_in_user' extracted and added as a top-level
function in kallithea.lib.base since the code doesn't need access to
the current controller.

Code refactored to take a User object instead of a username, to allow
callers flexibility in how the user should be looked up.

BaseController: refactor session cookie authentication

Untangle session cookie authentication. If no session cookie is set,
AuthUser constructor will be called with user_id set to None (the
argument default value), and will never raise a UserCreationError. Thus
that case can safely be moved to the end of _determine_auth_user and
outside the try-except block.

If a session cookie *is* set, but we get a UserCreationError, we fall
through to the "no cookie" case, which is also effectively the same
behavior as before. (Not sure what circumstances, if any, can actually
trigger a UserCreationError here, since the user is already logged in
and - presumably - created, plus the user is identified by the Kallithea
database ID, not user name, which would make it difficult to create a
new user... but judging from the existing code, it's possible.)

BaseController: refactor API key authentication

Untangle API key authentication. Creating an AuthUser from an API key
can leave the AuthUser authenticated or not, depending on key validity
and Kallithea configuration; but either way, _determine_auth_user will
not change this fact, and we can return early.

AuthUser: simplify check_ip_allowed and drop is_ip_allowed

check_ip_allowed is always called with user_id and inherit_from_default
arguments taken from the same User/AuthUser object, so just take that
object instead. This simplifies the is_ip_allowed method to the point
where it can be removed.

AuthUser: make get_perms method private

The get_perms method is used exactly once, by the permissions property.

A future, more thorough examination of the code may reveal if it should
stay or go; for now, just make it private. Also, enhance docstring
accuracy.

pullrequest: re-add username when adding reviewers

Commit 90e982b6bca8 removed the username when adding reviewers, to align the
way new and existing reviewers are displayed. However, when a user does not
have a name set, the list of reviewers becomes useless unless the username
is displayed too.

user: show 'Firstname Lastname (username)'

Additionally, clarify both full_name_or_username and full_name_and_username
with docstrings.

user: rename property username_and_name into full_name_and_username

In analogy of existing properties full_name and full_name_or_username,
rename username_and_name into full_name_and_username.

autocomplete: misc cleanup

Some miscellaneous changes that didn't really fit in the previous patches:
- clarification of variable names
- removal of unnecessary underscore in MembersAutoComplete
- avoid hardcoded div specifications

autocomplete: fix completion of repository owner

The owner field of a repository setting was supposed to be autocompletable,
but never really did (at least not in Kallithea, probably it once did in
Rhodecode).

Instead of making yet another 'OwnerAutoComplete', make a generic
SimpleUserAutoComplete that can be reused in other places that only need
completion of a text input field.

autocomplete: introduce function to set up basic autocomplete functionality

To set up an autocomplete instance, there is quite some boilerplate that is
with minimal differences between instances. Move this to a helper function
to reduce duplication, while still allowing each specific case to tweak
behavior.

autocomplete: remove redundant specification of responseSchema

YAHOO.util.DataSource.responseSchema determines which fields from the data
source are returned upon requests. In the case of user/group autocompletion,
the data source is a static JavaScript array containing only the relevant
fields. When not specifying a responseSchema, all fields of the selected
entry are returned, which is fine (in fact, the responseSchema currently
specified just mentions all fields, redundantly).

autocomplete: remove redundant return in MembersAutoComplete

Nobody is using these values; they were removed in the other autocompletes
before.

autocomplete: remove leftover 'cache' in PullRequestAutocomplete

The cache array is no longer used, only written to, and can thus be removed.

autocomplete: clean up handlers for itemSelectEvent

The handlers that execute when an autocompleted item is selected are really
specific and cannot be factored out. Nevertheless, some cleanup:

- rationalize indentation
- remove redundant copy/pasted comments
- remove redundant group handling in MentionsAutoComplete
- remove redundant checks on 'oData.nname != undefined', used to
differentiate user vs. group completion, in places that only allow user
autocompletion

autocomplete: factor out formatResult function

Note: remove an irrelevant 'mentionQuery' clause in the PullrequestAutocomplete

autocomplete: increase reuse of gravatar logic

By splitting autocompleteGravatar in a first part that returns the core
gravatar html code, we can remove yet some more duplication. Hooray!

autocomplete: factor out _gravatar function

(including rename of input arguments, and simpler return)

autocomplete: remove dead code

The three blocks of autocomplete code are clearly copy/pasted from one
another, with dead code remaining due to group-autocomplete not being
relevant for some cases.

tests: hide database setup queries with pytest

Unlike nosetest, pytest would show the database setup queries at the
beginning of the test run, which don't bring added value.
Hide them by disabling logging during this time.

Note that this does not provide an easy method of enabling the logging on
demand (what could be done with the -s switch in nosetest).

notification tests: delete notifications before (not after) tests

Don't clean notifications (and changeset comments) *after* the test,
but *before* the test. Other unit tests don't care if they leave
notifications in the database, and neither should these. Rather,
they should ensure their *own* preconditions before testing.

Admittedly, currently only one test leaves a notification in the
database, but more could come along at any time (and why worry?):
TestPullrequestsController.test_create_with_existing_reviewer

i18n: updated translation for Belarusian

Currently translated at 83.8% (935 of 1115 strings)

i18n: updated translation for Dutch (Belgium)

Currently translated at 0.7% (8 of 1116 strings)

AuthUser: Drop ip_addr field

None of the AuthUser consumers actually need to get the IP address from
the AuthUser object, so it's just redundant.

Also, AuthUser represents a user session, and should not be used as a
generic user + IP address data structure.

tests: improve robustness of notifications tests

The notifications tests make the assumption that there are no notifications
at the start of the test, which is explicitly asserted.

However, any other test, like the one introduced in commit 9a23b444a7fe
(pullrequests: detect invalid reviewers and raise HTTPBadRequest), could add
new notifications to the database, and thus fail these assertions.

Just like the notifications tests already cleaned all notifications at the
end of the test (tearDown), make sure to clean them at the start (setUp)
too.