style: use npm less to generate kallithea stylesheet style.css from style.less
This is a minimal change to introduce less as framework to generate our stylesheet. The input less file is currently the same as the original css stylesheet, so that the delta between the input style.less and the output style.css is zero.
To regenerate style.css from style.less, run:
npm --prefix kallithea/public/less install npm --prefix kallithea/public/less run less
style: change indentation of stylesheet to match future less output
Align the indentation of the stylesheet with the indentation that a less-compiled version of this stylesheet would use. This will allow easy comparison of the original and less-generated stylesheet in future commits.
style: remove empty lines in stylesheet to match future less output
Align the formatting of the stylesheet with the formatting that a less-compiled version of this stylesheet would use. This will allow easy comparison of the original and less-generated stylesheet in future commits.
This will also align the formatting of the stylesheet with the indentation that a less-compiled version of this stylesheet would use. This will allow easier comparison of the original and less-generated stylesheet in future commits.
diff: fix crash when displaying diff on a single file
File diff on an image file would fail on %if op in 'DM': because op was None: TypeError: 'in <string>' requires string as left operand, not NoneType
But really, if op is None, we also don't want to show invalid "Show images" links. Thus, guard the whole image display section with having an actual op.
_parse_gitdiff will never return op None, but wrapped_diff is more lazy and might do that. It could be considered a bug in wrapped_diff, and this change is just a bad workaround.
vcs: add support for Mercurial annotate in Mercurial 4.4
Mercurial fctx.annotate() returns tuples with two elements: info about the annotate line and the actual line. The code is changed to clarify that.
After Mercurial 4.4 with https://www.mercurial-scm.org/repo/hg/rev/2e32c6a31cc7 , the info is no longer a tuple but an attr object. Assume fctx is available as an attribute, but catch exceptions and fall back to indexing as before.
That can't easily be done in hgcompat, so we do it inline.
changelog: bring the right-aligned tags in a least to most common order
The goal is, to more often, have equal tags being display like a column, without giving each tag its own table cell, which would waste a lot of space. So move tags and branch, which come in singles move in front of phase, unstable, etc. which most of the time come in groups.
files: use same parent/child navigation as on changeset page
There are different reasons to do this. First, be consistent. Second, steping through the revisions by revision number is problematic. It means jumping between branches (I think even with the 'Follow current branch' checkbox this is not obvious to all the users). Also if we'd continue to use it, it'd require some special handling of hidden revisions.
files: simplify header, making it more like changeset
"Search file list" is moved out of the header to a separate div.
The "Follow current branch" functionality is dropped, and there is thus less need for passing the current URL around ... but it is generally still used for browser history navigation.
The removal of branch following is a temporary feature regression - that will be fixed next.
Binary diffs can make the diffs VERY big and cause MemoryError exceptions.
Before giving MemoryError, the system might start swapping, any process might fail when allocating memory, random processes might get killed, and our process might fail in other places. The proper fix would be to avoid the problem by not trying to process more data than we can handle - for example by not processing more than a certain amount of Git output.
Before, memory errors were shown to the user as a 500 Internal Server Error page.
Now, as long as we have no better/safer way get the diff, catch the MemoryError and show the page with a flash error message and no diff.
The error handling is placed in the diffs module to avoid leaking flash messages into the vcs lib.
config: tweak template http_server conditionals - don't leave an empty section when using UWSGI
The web servers run by 'gearbox serve' share the same basic configuration:
[server:main] host = ... port = ... use = egg:... ...
UWSGI doesn't use gearbox and doesn't use a [server:main] section but will read it's own section:
[uwsgi] http = ...:... ...
Before, 'make-config my.ini http_server=uwsgi' would create an empty [server:main] section, and only after 6a5fb5070765 it stopped putting unused host/port information there.
Now, let everything but UWSGI share the [server:main] template section and put host/port first. Everything UWSGI is handled in its own conditional template section.
- use jQuery instead of YUI in js code - since the old library used seconds and the new library uses ms we have to make timestamps 1000 times bigger - the new library uses a slightly different data format
auth: consume request body before responding 401 or 403 during authentication
In order to work correctly with reverse proxies like Apache, the application needs to consume the whole body before returning and closing the connection. Otherwise the reverse proxy may complain about a broken pipe.
For example, if the client sends a lot of data and kallithea doesn't read all that data before sending 401, the connection will be closed before the reverse proxy has sent all the data. In this case an apache reverse proxy will fail with a broken pipe error.
This is not necessary for all wsgi servers. Waitress automatically buffers (and therefore reads) all the data and uwsgi has a 'post-buffering' option to do the same. But AFAIK there is no way to push to a password protected hg repository when using gunicorn without this changeset.
commands: remove custom (re-)initializing of database model
The model has already beeen initialized through TurboGears after make_app_without_logging has been called.
Since kallithea.lib.utils2.engine_from_config no longer is used, remove it. All remaining references to engine_from_config are directly to the one from sqlalchemy.
diffs: move as_html and _safe_id from method to a pure function - avoid calling the method as function
as_html was sometimes used in a way where we actually don't want to use the whole DiffProcessor - we just created a dummy instance and passed custom input as parameter to the instance method.
Instead, make it the function we apparently want.
Make it clear that as_html not just returns a "diff" but that it is a html diff.
diffs: do check of cut_off_limit on the raw diff before any parsing and allocation
This redefines the exact meaning of cut_off_limit but it will still be approximately the same thing. It makes it more a limitation of what amount of work should be done, more than how much html should be outputted.
It could make sense to push the limit further back to vcs to also prevent computing or allocating memory for huge diffs.
diffs: inline prepare() into __init__ and make the result available as .parsed
Make it more clear what the DiffProcessor is: Something that works on a raw diff as input, mainly compute when initialized, and returns an object where the result is available in different ways.
diffs: avoid extra copy of diff when splitting into files
Instead of allocating memory for an extra copy of the whole raw diff, just use buffer() to give read-only string views of the relevant sections of the big diff string given as input.
diffs: cleanup of variable naming around cut_off_limit
A brief summary of this area:
The base controller sets self.cut_off_limit from config and is used for diffs, unless controllers are given a fulldiff query parameter. In a few cases, these are passed to templates as c.cut_off_limit or c.fulldiff . Also, if the diff function returns a LimitedDiffContainer, c.limited_diff is set so the UI can report the data set is partial.
templates: back out the DataTables paging disabling from c5bff92d5084
Lessons learned: * It was not sufficiently tested with gravatars. * It was not sufficiently tested on Chrome - Firefox seems to handle some big pages better. * It is unclear how big amounts of data we want to optimize for, but this regressed too much for some cases.
ini: drop insertion of header comments in generated ini files
The header comments were kind of redundant and could easily get out of sync. Also, we are moving towards just generating files and don't need this and don't want to maintain it.
ini: add support for adding extra entries to sections
When adding entries to an existing section, assume that comments and empty lines *before* a section header belongs to that section. That is an approximation; not always entirely correct, but the best we can come up with.
ini: drop redundant template update of [handler_console] level=DEBUG
After 05d09cec7b26, the handlers in the default template no longer had a 'level' setting the ini generator could override with DEBUG. We are apparently fine without it, so just remove the last traces.
templates: disable paging and page size controls in DataTables
Since all data is sent to the client side, it can just as well be rendered. PageDown or scrolling is more convenient for paging than paging buttons, so short of "infinite scrolling", showing all entries is an acceptable solution ... especially when it very rarely is hundres of entries.
There could perhaps be further changes to how DataTables is shown - this is one step.
index: show repositories and repository groups in the same table
Having two different tables with their own paging and search gave a bad UI. Instead, do like all other UIs that show directory content and show both "folders" and "files" in the same list.
The rendering of repo groups is changed to use js data instead of a taking data from an html table.
Repository groups are shoe-horned into the repository DataTable. The columns are no perfect match - some of the existing columns are thus given an empty default value.
repogroups: fix display of links to parent repo groups
Links inside panel-primary were shown with the same color as the panel background. Arguably, that's a bug in Bootstrap, but marking it as panel-title fixes that.
pygments: reimplement get_lem under the name get_extension_descriptions
The old implementation was cryptic and over-engineered. And reduce() is not a builtin in Python 3.
This function works on static input, and it was verified that this implementation returns exactly the same as the old one. And is simpler and slightly more readable. In my opinion.
py3: replace list comprehension with for-loop due to scope
The scope of the list comprehension variable 'x' will be limited to the list comprehension in python3. Thus switching to a full loop (without this scope restriction) in preparation for python3.
utils: remove Session after we are done using it in set_app_settings
When the Kallithea WSGI application or celeryd is started, TurboGears app_config.make_base_app calls Kallithea app_cfg.setup_configuration which runs utils.set_app_settings. That function will read settings from the database and store them in the global config. It uses a database session which is created on demand, but this session was not dismissed but left around for the next thing that asked for a database session. MySQL will by default close connections after 1 hour, so when celery tried to run a task after 1 hour of inactivity, it could fail because of the closed connection.
utils.set_app_settings must thus remove the Session after use, just like auth.set_available_permissions do.
This will thus fix for example some MySQL connection problems seen with Celery.
tests: use temporary copy of test.ini, possibly customized for TEST_DB
It was an undocumented feature that if setting the environment variable TEST_DB, that would be used for tests instead of the default kallithea/tests/test.ini sqlalchemy.url . That did however not work for Git hooks and tests would fail.
Instead, create a copy of test.ini in the temp folder and use that for testing. If TEST_DB is set, edit the file so the specified DB URL is used. This fixes Git hook related tests if TEST_DB is used.
Since this also changes the path of %(here)s to the temporary location, just use the default paths.
This also has the advantage that the data folders are now in the temp folder as well. Therefore a broken data folder, from a past run, can no longer influence a test.
templates: tweak file edit page markup to be slightly more correct and make it look slightly better
- use <label> for the commit message label - use .form-group - use form-inline only for the heading - use form-group-sm to make the <select> less bloated