Backport #19773
Our character detection algorithm can potentially incorrectly detect utf-8 as iso-8859-x
if there is a truncated character at the end of the partially read file.
This PR changes the detection algorithm to truncated utf8 characters at the end of the
buffer.
Fix#19743
Signed-off-by: Andrew Thornton <art27@cantab.net>
It may be prudent to add runtime finalizers to the git.Repository and
git.blobReader objects to absolutely ensure that these are both properly
cancelled, cleaned and closed out.
This commit is a backport of an extract from #19448
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #19537
There is a potential rare race possible whereby the c.running channel could
be closed twice. Looking at the code I do not see a need for this c.running
channel and therefore I think we can remove this. (I think the c.running
might have been some attempt to prevent a hang but the use of os.Pipes should
prevent that.)
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Backport #19531
- Doing 64-bit atomic operations on 32-bit machines is a bit tricky by golang, as they can only be done under certain set of conditions(https://pkg.go.dev/sync/atomic#pkg-note-BUG).
- This PR fixes such case whereby the conditions weren't met, it moves the int64 to the first field of the struct, which will 64-bit operations happening on this property on 32-bit machines.
- Resolves#19518
Backport #19516
Within doArchive there is a service goroutine that performs the
archiving function. This goroutine reports its error using a `chan
error` called `done`. Prior to this PR this channel had 0 capacity
meaning that the goroutine would block until the `done` channel was
cleared - however there are a couple of ways in which this channel might
not be read.
The simplest solution is to add a single space of capacity to the
goroutine which will mean that the goroutine will always complete and
even if the `done` channel is not read it will be simply garbage
collected away.
(The PR also contains two other places when setting up the indexers
which do not leak but where the blocking of the sending goroutine is
also unnecessary and so we should just add a small amount of capacity
and let the sending goroutine complete as soon as it can.)
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <6543@obermui.de>
If an `os/exec.Command` is passed non `*os.File` as an input/output, go
will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If
the code following this is responsible for closing `io.Pipe`s or other
handlers then on process death from context cancellation the `Wait` can
hang.
There are two possible solutions:
1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these.
2. create a goroutine waiting on the context cancellation that will close the inputs.
This PR provides the second option - which is a simpler change that can
be more easily backported.
Closes#19448
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
- Backport #19457
- When 3-way merge is enabled for conflict checking, it has a new interesting behavior that it doesn't return any error when it found a conflict, so we change the condition to not check for the error, but instead check if conflictedfiles is populated, this fixes a issue whereby PR status wasn't correctly on conflicted PR's.
- Refactor the mergeable property(which was incorrectly set and lead me this bug) to be more maintainable.
- Add a dedicated test for conflicting checking, so it should prevent future issues with this.
- Ref: Fix the latest error for https://gitea.com/gitea/go-sdk/pulls/579
Co-authored-by: zeripath <art27@cantab.net>
Older git clients need uploadpack.allowAnySHA1InWant if partial cloning is allowed.
Fix#19118
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Backport #19429
When a mirror repo interval is updated by the UI it is rescheduled with that interval
however the API does not do this. The API also lacks the enable_prune option.
This PR adds this functionality in to the API Edit Repo endpoint.
Signed-off-by: Andrew Thornton <art27@cantab.net>
The service worker causes a lot of issues with JS errors after instance
upgrades while not bringing any real performance gain over regular HTTP
caching.
Disable it by default for this reason. Maybe later we can remove it
completely, as I simply see no benefit in having it.
There was a bug that the CSRF token wouldn't in 24h. This fix just does what the CSRF function comment says: If this request is a GET request, it will generate a new token. Then the CSRF token can be kept up-to-date.
* Revert the minimal golang version requirement from 1.17 to 1.16 and add a warning in Makefile
* Apply suggestions from code review
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* 1.16
* Update modules/util/net.go
Co-authored-by: Gusted <williamzijl7@hotmail.com>
* correct bool conditional
yay tests for catching this :)
* Update hostmatcher.go
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
* Use full output of git show-ref --tags to get tags for PushUpdateAddTag (#19235)
Strangely #19038 appears to relate to an issue whereby a tag appears to
be listed in `git show-ref --tags` but then does not appear when `git
show-ref --tags -- short_name` is called.
As a solution though I propose to stop the second call as it is
unnecessary and only likely to cause problems.
I've also noticed that the tags calls are wildly inefficient and aren't using the common cat-files - so these have been added.
I've also noticed that the git commit-graph is not being written on mirroring - so I've also added writing this to the migration which should improve mirror rendering somewhat.
Fix#19038
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* fix rebase relict
Co-authored-by: 6543 <6543@obermui.de>
Backport #19225
Backport #19229
The RepoIndexerTest is failing with considerable frequency due to a race inherrent in
its design. This PR adjust this test to avoid the reliance on waiting for the populate
repo indexer to run and forcibly adds the repo to the queue. It then flushes the queue.
It may be worth separating out the tests somewhat by testing the Index function
directly away from the queue however, this forceful method should solve the current
problem.
Fix#19162
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Hide sensitive content on admin panel progress monitor (#19218)
Sanitize urls within git process descriptions.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Do not include global arguments in process manager (#19226)
Backport #19226
The git command by default adds a number of global arguments. These are not
helpful to be displayed in the process manager and so should be skipped for
default process descriptions.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Backport #19175
Unhelpfully Locations starting with `/\` will be converted by the
browser to `//` because ... well I do not fully understand. Certainly
the RFCs and MDN do not indicate that this would be expected. Providing
"compatibility" with the (mis)behaviour of a certain proprietary OS is
my suspicion. However, we clearly have to protect against this.
Therefore we should reject redirection locations that match the regular
expression: `^/[\\\\/]+`
Reference #9678
Signed-off-by: Andrew Thornton <art27@cantab.net>
Unfortunately #19169 causing a panic at startup in prod mode. This was hidden by dev
mode because the templates are compiled dynamically there. The issue is that DotEscape
is not in the original FuncMap at the time of compilation which causes a panic.
Ref #19169
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #19010
Use DeployKeyID to replace the IsDeployKey, then CanWriteCode uses the DeployKeyID to check the write permission.
Fix#19009
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Backport #19169
Unfortunately many email readers will (helpfully) detect url or url-like names and
automatically create links to them, even in HTML emails. This is not ideal when
usernames can have dots in them.
This PR tries to prevent this behaviour by sticking ZWJ characters between dots and
also set the meta tag to prevent format detection.
Not every email template has been changed in this way - just the activation emails but
it may be that we should be setting the above meta tag in all of our emails too.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #19124
* Clean paths when looking in Storage
Ensure paths are clean for minio aswell as local storage.
Use url.Path not RequestURI/EscapedPath in storageHandler.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
Co-authored-by: Lauris BH <lauris@nix.lv>
Backport #19132
Make SKIP_TLS_VERIFY apply to git data migrations too through adding the `-c http.sslVerify=false` option to the git clone command.
Fix#18998
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #19049
Unfortunately #18642 does not work because a `*net.OpError` does not implement
the `Is` interface to make `errors.Is` work correctly - thus leading to the
irritating conclusion that a `*net.OpError` is not a `*net.OpError`.
Here we keep the `errors.Is` because presumably this will be fixed at
some point in the golang main source code but also we add a simply type
cast to also check.
Fix#18629
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #19045
Yet another issue has come up where the logging from SyncMirrors does not provide
enough context. This PR adds more context to these logging events.
Related #19038
Signed-off-by: Andrew Thornton <art27@cantab.net>
Backport #19040
The PAM module has previously only checked the results of the authentication module.
However, in normal PAM practice most users will expect account module authorization to also be checked. Without doing this check in almost every configuration expired accounts and accounts with expired passwords will still be able to login.
This is likely to represent a significant gotcha in most configurations and cause most users configurations to be potentially insecure. Therefore we should add in the account authorization check.
## ⚠️ **BREAKING** ⚠️
Users of the PAM module who rely on account modules not being checked will need to change their PAM configuration.
However, as it is likely that the vast majority of users of PAM will be expecting account authorization to be checked in addition to authentication we should make this breaking change to make the default behaviour correct for the majority.
---
I suggest we backport this despite the BREAKING nature because of the surprising nature of this.
Thanks to @ysf for bringing this to our attention.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: ysf <34326+ysf@users.noreply.github.com>
Only pass `status_filter` on admin page
Use a more general method to pass query parameters, remove hard-coded keys
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
