Backport #15459
If an avatar is requested in a particular size ensure that /avatars also gets the size request
Fix#15453
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Backport #15519
It is possible that tag commits could be deleted or missing from repos. This causes
migration 156 to fail and breaks upgrade.
This PR simply logs the failure.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15452Fix#14793.
The previous implementation used the first return value of matcher.Match, which is the chosen language tag but may contain extensions such as de-DE-u-rg-chzzzz.
As mentioned in the documentation of language package, matcher.Match also returns the index of the supported tags, so I think it is better to use it rather than manipulate the returned language tag.
Co-authored-by: Naohisa Murakami <tiqwab.ch90@gmail.com>
Backport #15457
Some postgres users have logging which logs even failed transactions. So
just query the db before trying to insert.
Fix#15451
Signed-off-by: Andrew Thornton art27@cantab.net
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Backport #15482
It appears that there are significant performance problems with the pure git backend
on windows.
Therefore until we can sort this out - provide go-git backend builds.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15432
There is a weird gotcha with GetTagCommitID that because it uses git rev-list
can cause an ambiguous argument error.
This PR simply makes tags use the same code as branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15352
The issue is that the TestPatch will reset the PR MergeBase - and it is possible for TestPatch to update the MergeBase whilst a merge is ongoing. The ensuing merge will then complete but it doesn't re-set the MergeBase it used to merge the PR.
Fixes the intermittent error in git test.
Signed-off-by: Andrew Thornton art27@cantab.net
Backport #15338
More recent versions of git have increased support for detection of renames meaning
that a rename with diff changes is now supported.
Although ParsePatch supports this - our templates do not and the simplest solution
is simply to show the diff.
Fix#15335
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Backport #15323
It appears that there is a slight bug in the handling of the data of logout event -
the javascript should be testing the data field of the data field for the logout
instruction.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15275
A previous commit that sent unauthorized if the user is unauthorized
simply leads to the repeated reopening of the eventsource. #
This PR changes the event returned to tell the client to close the
eventsource and thus prevents the repeated reopening.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR proposes an alternative solution to #15255 - just add the size to the
save function. Yes it is less apparently clean but it may be more correct.
Close#15255Fix#15253
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Backport #15213
`enry.IsVendor` is kinda slow as it simply iterates across all regexps.
This PR ajdusts the regexps to combine them to make this process a
little quicker.
Related #15143
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15199
I do not understand how this can happen or why.
There is an apparent possibility for a comment.Patch to be missing a hunk header
- this should not happen and do not understand how. But it appears to happen on
1.13 at least in some case.
This PR will simply add a new section if the cursection is empty
thus preventing the NPE.
Fix#15198
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Backport #15179
/api/v1/repos/issues/search is a highly inefficient search which is unfortunately
the basis for our dependency searching algorithm. In particular it currently loads
all of the repositories and their owners and their primary coding language all of
which is immediately thrown away.
This PR makes one simple change - just get the IDs.
Related #14560
Related #12827
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15210
Unfortunately #14623 changed from the deprecated IsInteractiveSession to
IsWindowsService without recognising that they are the complement of
each other.
This means that Windows SVC control is not working correctly. This PR
adds some Tracing statements but also fixes the bug.
Fix#15159
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15131
Whilst creating a test for v176 in the migrations_test PR
it has become clear that this was still wrong.
This is now fixed. Genuinely.
Also fix repo transfer
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15110
There is a serious issue with the v176 migration where there is a mistaken missing
label_id selection.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #15085
There are a few recurrent issues with comment as diff reporting panics that are resistant to fixing due to the fact that the panic occurs in the template render and is swallowed by the template renderer.
This PR just adds some logging to force the panic to properly logged and re-propagates back up to the template renderer so we can actually detect what the issue is.
Signed-off-by: Andrew Thornton art27@cantab.net
* Never add labels not from this repository or organisation and remove org labels on transfer
Prevent the addition of labels from outside of the repository or
organisation and remove organisation labels on transfer.
Related #14908
Signed-off-by: Andrew Thornton <art27@cantab.net>
* switch to use sql
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove AS
Signed-off-by: Andrew Thornton <art27@cantab.net>
* subquery alias
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Give me some AS?
Signed-off-by: Andrew Thornton <art27@cantab.net>
* double AS
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
* once more around the merry go round
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix api problem
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add outside label consistency check into doctor
This PR adds another consistency check into doctor in order to detect
labels that have been added from outside of repositories and organisations
Fix#14908
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix migration
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for merge
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix postgres ID sequences broken by recreate-table
Unfortunately there is a subtle problem with recreatetable on postgres which
leads to the sequences not being renamed and not being left at 0.
Fix#14725
Signed-off-by: Andrew Thornton <art27@cantab.net>
* let us try information_schema instead
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Prevent 404 on new pull request button on forked fork owned by the owner
of the root repository. Also ensure that the names make sense.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix an issue with panics related to attributes
* Wrap goldmark render in a recovery function
* Reduce memory use in render emoji
* Use a pipe for rendering goldmark - still needs more work and a limiter
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Make sure sibling images get a link too
Due a problem with the ast.Walker in the our transformer in goldmark
an image with a sibling image will not be transformed to gain a parent
link. This PR fixes this.
Fix#12925
Signed-off-by: Andrew Thornton <art27@cantab.net>
If there is a problem uploading to LFS it is possible for the raw
endpoint to return a very large file when a pointer file is expected
This will then cause the drone logs to fill up unnecessarily with
the contents of the very large file.
If the file returned from raw is of the incorrect size we should
therefore not test it see if it contains the pointer file
and just declare that it is incorrect.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Never add labels not from this repository or organisation and remove org labels on transfer
Prevent the addition of labels from outside of the repository or
organisation and remove organisation labels on transfer.
Related #14908
* switch to use sql
* subquery alias
* once more around the merry go round
* fix api problem
When editing forked repos using the API the BaseRepository needs to loaded
in order to check its visibility otherwise there will be NPE panic.
Fix#14956
Signed-off-by: Andrew Thornton <art27@cantab.net>
This is an alternative PR to #13687.
Add `[ui.explore]` settings to allow restricting the
explore pages to logged in users only and to disable the users explore page.
The two proposed settings are:
- `REQUIRE_SIGNIN_VIEW`: Only allows access to the explore pages if the
user is signed in. Also restricts
- `/api/v1/user/search`
- `/api/v1/users/{username}`
- `/api/v1/users/{username}/repos`
- but does not restrict `/api/v1/users/{username}/heatmap`
- `DISABLE_USERS_PAGE`: Disables the /explore/users page
Fix#2908Close#13687
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
PR #13610 unfortunately disabled importing repositories from local paths.
This PR restores this functionality.
Fix#14700
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make SSH server host key path configurable
* make it possible to have multiple keys
* Make gitea.rsa the default key
* Add some more logging
Signed-off-by: Andrew Thornton <art27@cantab.net>
Closed milestones and issues should only be marked overdue if they were
closed after their deadline.
Fix: #14536
Signed-off-by: Andrew Thornton <art27@cantab.net>
Add SameSite setting for cookies and rationalise the cookie setting code. Switches SameSite to Lax by default.
There is a possible future extension of differentiating which cookies could be set at Strict by default but that is for a future PR.
Fix#5583
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR adds a simple check to only test the MilestoneID if it is not empty.
Fix#14906
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Continuing on from #14888
The previous implementation has race whereby an incomplete upload or
hash mismatch upload can end up in the ContentStore. This PR moves the
validation into the reader so that if there is a hash error or size
mismatch the reader will return with an error instead of an io.EOF
causing the storage to abort the storage.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Extract out the common cat-file batch calls
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move bleve and elastic indexers to use a common cat-file --batch when indexing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move catfilebatch to batch_reader and rename to batch_reader.go
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Unfortunately `git log revision ... --skip=x -- path` skips the number of commits
not the number of commits relating to the path.
This PR changes the function to have a reader that reads and skips the
necessary number of commits by hand instead.
Fix#8716
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Most DBs apart from SQLite will use a default Collation that is not case insensitive.
This means that SearchIssuesByKeyword becomes case sensitive for db indexing - in
contrast to the bleve and elastic indexers.
This PR simply uses UPPER(...) to do the LIKE - and although it may be more efficient
to change collations this would be a non-trivial task.
Fix#13663
Signed-off-by: Andrew Thornton <art27@cantab.net>
When using HCaptcha on link account pages the site key needs to be passed
in. This PR ensures that HCaptchaSiteKey is set in the data.
Fix#14766
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
MakeAbsoluteAssetURL should just url join the static url prefix on to appurl
if it is not an absolute path - this is because StaticURLPrefix is an absolute
prefix not a relative prefix to the app sub url.
Fix#14422
Signed-off-by: Andrew Thornton <art27@cantab.net>
* CutDiffAroundLine makes the incorrect assumption that `---` and `+++` always represent part of the header of a diff.
This PR adds a flag to its parsing to prevent this problem and adds a streaming parsing technique to CutDiffAroundLine using an io.pipe instead of just sending data to an unbounded buffer.
Fix#14711
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle unquoted comment patch files
When making comment patches unfortunately the patch does not always quote the filename
This makes the diff --git header ambiguous again.
This PR finally adds handling for ambiguity in to parse patch
Fix#14812
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add in testing for no error
There is no way currently for CutDiffAroundLine in this test to cause an
error however, it should still be tested.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Instead of causing a log.Fatal, we should handle broken OAuth2
providers by disabling them.
Fix#8930
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Since the chi upgrade if the templates are missing an endless loop will occur if
status/500.tmpl is missing.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Move the stopwatches to the eventsource stream
Use the /user/events eventsource to update the stopwatches
instead of polling /api/v1/user/stopwatches if the eventsource
is enabled.
Signed-off-by: Andrew Thornton <art27@cantab.net>
PostProcess is supposed to be parsing and handling HTML
fragments, but on fuzzing it appears that there is a weird
issue with NUL elements that could cause a memory address
error in downstream libraries.
The simplest solution is to strip out the weird NULs - they
should not be there in any case and would be stripped out
anyway.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reduce calls to git cat-file -s
There are multiple places where there are repeated calls to git cat-file
-s due to the blobs not being created with their size.
Through judicious use of git ls-tree -l and slight adjustments to the
indexer code we can avoid a lot of these calls.
* simplify by always expecting the long format
* Also always set the sized field and tell the indexer the update is sized
* Use cat-file --batch in GetLanguageStats
This PR moves to using a single cat-file --batch in GetLanguageStats
significantly reducing the number of processes spawned during language stat
processing.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/git/repo_language_stats_nogogit.go
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Somehow the test for detecting if branches are equal broke
this PR restores this functionality.
Fix#14502
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR adds `fullTextSearch: 'exact'` to most dropdown
invocations meaning that if there is a search box for the
dropdown it will automatically do a fullTextSearch looking
for the provided fragment instead of starting at the beginning
We should consider changing other places that use
`fullTextSearch: true` to `'exact'` because these will be using a
fuzzy-textual search that doesn't necessarily return the
expected results.
Fix#14689
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create Xorm session provider
This PR creates a Xorm session provider which creates
the appropriate Session table for macaron/session.
Fix#7137
Signed-off-by: Andrew Thornton <art27@cantab.net>
* extraneous l
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use key instead of ID to be compatible with go-macaron/session
Signed-off-by: Andrew Thornton <art27@cantab.net>
* And change the migration too.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update spacing of imports
Co-authored-by: 6543 <6543@obermui.de>
* Update modules/session/xorm.go
Co-authored-by: techknowlogick <matti@mdranta.net>
* add xorm provider to the virtual provider
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for master merge
* prep for merge master
* As per @lunny
* move migration out of the way
* Move to call this db session as per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Gitea runs diff on highlighted code fragment for each line in order to provide
code highlight diffs. Unfortunately this diff algorithm is not aware that span tags
and entities are atomic and cannot be split.
The current fixup code makes some attempt to fix these broken tags however, it cannot
handle situations where a tag is split over multiple blocks.
This PR provides a more algorithmic fixup mechanism whereby spans and entities are
completely coalesced into their respective blocks.
This may result in a incompletely reduced diff but - it will definitely prevent the
broken entities and spans that are currently possible.
As a result of this fixup several inconsistencies were discovered in our testcases
and these were also fixed.
Fix#14231
Signed-off-by: Andrew Thornton <art27@cantab.net>
When there is a panic during template rendering unrolled/render
will automatically render the error. This leads to the
panic being displayed in the page and not a 500 page
Fix#14467Fix#14525
Signed-off-by: Andrew Thornton <art27@cantab.net>
There is potentially a race with a slow starting internal
queue causing a NPE if Has is checked before the internal
queue has been setup.
This PR adds a lock on the Has() fn.
Fix#14311
Signed-off-by: Andrew Thornton <art27@cantab.net>
GetCommit() assumes that all 40 char strings are SHA1s. This leads to an
error if you try to do a PR on a branch which is 40 characters long.
This PR attempts the SHA first - and if it fails will switch to using rev-parse.
Fix#14470
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR improves HasPreviousCommit to prevent the automatic and recursive loading
of previous commits using git merge-base --is-ancestor and git rev-list
Fix#13684
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent adding nil label to .AddedLabels or .RemovedLabels
There are possibly a few old databases out there with malmigrated data that can
cause panics with empty labels being migrated.
This PR adds a few tests to prevent nil labels being added.
Fix#14466
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add doctor command to remove the broken label comments
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Memcached TTL cannot be > 30 days and if it is attempted the TTL is interpreted as
a unix timestamp.
This PR ensures that the TTL is switched to a unix timestamp in those cases.
Fix#14571
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Exclude the current dump file from the dump
Always prevent the current file from being added to the dump.
Fix#13618
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add skip custom directory option
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Breaking the pipe is a valid way of killing a piped command and any error from
a broken cat-file batch command should be passed back up to the writer any way
therefore specifically logging it is unnecessary.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Migrations currently uses the default Xorm mapper which is
not the same as the mapper Gitea actually uses.
This means that there is a difference between the struct
parsing and mapping to database tables in migrations as
compared to normal Sync2.
This was the cause for the catastrophic problem in v168 -
untagged fields are not mapped in the same way in migrations
as compared to outside of migrations.
This is also likely the cause of some weird subtle failures
in other migrations as any untagged field may not be being
mapped exactly the same way.
This PR suggests that we ensure that the mapper is set at
the start of the migrations code - but also enforces a strict
clean mapper between each migration.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent panic on fuzzer provided string
The fuzzer has found that providing a <body> tag with an attribute to
PostProcess causes a panic. This PR removes any rendered html or body
tags from the output.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Placate lint
* placate lint again
Signed-off-by: Andrew Thornton <art27@cantab.net>
* minor cleanup
Signed-off-by: Andrew Thornton <art27@cantab.net>
The incorrect use of filepath instead of path means that
it is possible to cause a stackoverflow on Windows
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Ensure that schema search path is set with every connection on postgres
Unfortunately every connection to postgres requires that the search path is
set appropriately.
This PR shadows the postgres driver to ensure that as soon as a connection
is open, the search_path is set appropriately.
Fix#14088
Signed-off-by: Andrew Thornton <art27@cantab.net>
* no golangci-lint that is not a helpful suggestion
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use Execer if available
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Disable SSH key addition and deletion when externally managed
When a user has a login source which has SSH key management
key addition and deletion using the UI should be disabled.
Fix#13983
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make only externally managed keys disabled
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
